Observibility and Infrastructure tools

My neighbor is the CEO of a startup in the observibility space. Before forming his own company he worked at Google and Facebook using their homegrown observability tools and then went into a successful start up where part of his job was choosing the engineering and cloud infrastructure tools the company used including their observability tools. I’m being deliberately vague about the name of the company he is the CEO at as well as the startup he worked at.

His company used NEWR. He said their tools were pretty good but their sales and support were a disaster. THeir company experienced really rapid growth so they quickly outgrew the contract that they signed with NEWR. NEWR never notified his company that their use had exceeded what they paid for. Instead NEWR THREW AWAY all their data that exceeded their contract. He said they looked at multiple other solutions and ended up switching to DDOG. They paid DDOG the equivalent of 5 engineering salaries but he said they would have needed to pay 10x that number of people to roll their own solution. He also said DDOG had a better solution than NEWR. He said that even those DDOG is expensive they offer an incredible ROI.

I asked why not ESTC, he said that ESTC is basically almost equivalent to rolling your own solution. Their codebase requires a huge amount of work, lots of engineering, breaks often and is too much of a hassle. He was very dismissive of them.

He had a few other comments that I found very interesting. Observibility is a hard problem as the workloads it is being applied to becomes more and more heterogeneous. The industry becomes more complex year by year and nobody has won yet. Of all the generalizable solutions he said DDOG is currently the best that he has used but still pales in comparison to the homegrown solutions at the big companies like google and facebook but he said that DDOG is solving a way harder problem because their solutions has to work across many different scenarios where facebook and google have much more defined workloads. He thinks the observibility space is still wide open.

We ran out of time but at the end i asked him about cloudflare, fsly, and AWS.

Cloudflare -great products and marketing, doesn’t have the reputation for a great engineering culture.

FSLY - great engineering culture.

AWS - Has one name but it is made up of hundreds of businesses that don’t talk to each other, sometimes compete with each other and don’t necessarily work well together. Some solutions are really good, some are really bad.

I hope everyone is staying safe between the fires, pandemics, hurricanes and general 2020 craziness. Stay safe and be kind to each other!

-Ethan

21 Likes

Great post, Ethan. From my own perspective I can confirm much of what your neighbor says. Some additional color:

• With Elastic you are rolling your own solution, but you get to start with a decent base. Elastic has a list of available integrations here: https://www.elastic.co/integrations?solution=observability Note that integrations with companies like Cloudflare and Fastly are missing, and that other integrations are for “actions” only, not monitoring (looks for “Logs” and “Metrics”). Although they have branched out, I still feel Elastic is more around great search than anything else.

• Homegrown solutions enable you to stitch in monitoring/logging logic at any point in your application. At its simplest form it’s just printf’s to log files (coders will know what that means), and all the tools, including Elastic, analyze log files, but when you’re rolling your own you get to stick logic into your application that can greatly assist monitoring and debugging. Above all, designing your cloud/web application for observability is a best practice, no matter what external tools you’re using.

• I’m not sure what the AWS comment means. AWS is from one company - Amazon, and there are hundreds, if not thousands, of companies that offer products that run on AWS and have varying degrees of integration with each other. AWS is really just a platform on which to run cloud services. And no matter whether you’re in a public cloud or private cloud or just an on-premise web server, you’re likely running a multitude of products from different companies.

• The battle between DataDog and New Relic goes on. This one (https://www.upguard.com/blog/datadog-vs-newrelic ) from UpGuard likes New Relic a bit better. Gartner (https://www.gartner.com/reviews/market/application-performan… ) has them tied. I don’t have personal experience with either, so I’m in the dark.

20 Likes

Sumo Logic is one of the tools my company uses, I am personally a daily user. I have toyed with DDOG and find the two solutions very competitive and very, very similar in many ways. Netflix, Lyft and other notable names use Sumo Logic.

Sumo Logic is planning their IPO shortly: https://seekingalpha.com/article/4373325-ipo-update-sumo-log…

https://pulse2.com/sumo-logic-plans-to-raise-over-310-millio…

It will be interesting to see how Sumo with IPO funding affects DDOG’s market share. They are in a growing and highly innovative space, so there is probably plenty of market for all these companies, but I would think some of these younger startups are ripe for consolidation into some of the more established legacy companies.

4 Likes

Ethan: AWS - Has one name but it is made up of hundreds of businesses that don’t talk to each other, sometimes compete with each other and don’t necessarily work well together. Some solutions are really good, some are really bad.

Smorg: I’m not sure what the AWS comment means. AWS is from one company - Amazon, and there are hundreds, if not thousands, of companies that offer products that run on AWS and have varying degrees of integration with each other. AWS is really just a platform on which to run cloud services. And no matter whether you’re in a public cloud or private cloud or just an on-premise web server, you’re likely running a multitude of products from different companies.

Not sure what about Ethan’s friend’s comment was confusing. He is solely talking about the umbrella of products under AWS not other platforms built upon their IaaS (by AWS customers).

AWS itself has hundreds of products, across 25+ categories. https://aws.amazon.com/products/

A select few are ground breaking new cloud-native features that have become paradigm shifts (Lambda), but the rest fall somewhere between good, okay and outright bad. Some are in competition with each other (DocumentDB vs DynamoDB, or RDS vs Aurora) - AWS doesn’t care as long as one of them sells. Many don’t talk to each other particularly well, which is a massive oversight on their part (dev teams aren’t working together). Many are turnkey infrastructure, while others are just shells over some infrastructure scripting and require a lot of sysadmin handholding.

The cloud team I work with stays away from new products for the first year, to let AWS get the kinks out. Greengrass (IoT) was one product avoided like that.

There is no cohesive vision for any of it. AWS sees some margin available to steal from an infrastructure platform, and they make a product to rob THEIR CUSTOMER (the company building solutions on top of their IaaS) of revenue so that they can be the direct host of the feature. See MongoDB and Elastic as prime examples. This is just like Amazon does to retail sellers on its platform.

Some folks are getting pretty fed up with this strategy – hence the rise of multi-cloud infrastructure platforms (Snowflake, MongoDB, Elastic), which help enable their customers to adopt multi-cloud strategies (no lock-in, so gaining leverage over the cloud provider to compete on cost & features). Customers can take their services with them to another platform, if it comes down to it.

So “AWS is really just a platform on which to run cloud services” is not true at all. AWS is building products on top of themselves, in an attempt to steal business from their IaaS customers that have success in these directions. GCP and Azure are clearer in their partnerships to not do this sort of thing (co-opting margin at the expense of customer success).

-muji

51 Likes

So “AWS is really just a platform on which to run cloud services” is not true at all.

I stand 100% behind it. None of the Amazon database services you mention are bundled free with AWS - you have to pay additional for their use. Similarly with Microsoft, which also offers, for instance, add-on database products that compete with 3rd party database products on the Azure platform.

AWS sees some margin available to steal from an infrastructure platform, and they make a product to rob THEIR CUSTOMER (the company building solutions on top of their IaaS) of revenue so that they can be the direct host of the feature. See MongoDB and Elastic as prime examples. This is just like Amazon does to retail sellers on its platform.
Some folks are getting pretty fed up with this strategy – hence the rise of multi-cloud infrastructure platforms (Snowflake, MongoDB, Elastic), which help enable their customers to adopt multi-cloud strategies (no lock-in, so gaining leverage over the cloud provider to compete on cost & features).

I think you and Amazon disagree on who THEIR CUSTOMERs are. Amazon’s customers are not third parties like MongoDB, Elastic or Snowflake, they are the enterprises that host solutions using AWS and add-on services, which may be from Amazon or third parties like MongoDB or Snowflake. Microsoft’s Partner program is quite different than what Amazon does, but it’s arguable which benefits the real customers more.

I also believe you have the motivation for what drives third party providers wrong. They don’t offer products that work across different clouds because Amazon is competing with them (especially since Snowflake came after RedShift), but because THEIR CUSTOMERs don’t want to be tied into a single cloud platform like AWS or Azure. it becomes a selling point that enterprises can move solutions from Amazon to Azure to GCP to OCP, etc.

Of course, an enterprise that chooses Snowflake, for instance, becomes locked in to Snowflake’s data structures and API, and so if Snowflake changes their pricing or level of support those enterprises have to consider the work they have to move to a different product from a different provider.

Interestingly, decades ago a number of Standards Organizations came about to standardize things like FORTRAN (https://wg5-fortran.org/ ), HTML (https://www.w3.org/TR/html52/ ), SQL (https://www.iso.org/standard/45498.html ), etc. Those always took years to reach agreement, but today the technologies are moving so quickly that by the time a world-wide committee might be convened, an even newer technology has already replaced it. Unless the technology is provided by an Open Source group (eg Apache Spark), there probably won’t be a defacto standard in today’s fast-moving world for many things.

7 Likes

I asked why not ESTC, he said that ESTC is basically almost equivalent to rolling your own solution. Their codebase requires a huge amount of work, lots of engineering, breaks often and is too much of a hassle. He was very dismissive of them.

I will be interested to know when his experience with ESTC was. ESTC rolls out features rapidly so what might have been true in 2018 may not be the case now. Recently, they announced 1-click data ingest which Shay thinks highly of and is part of their goal to increase the ease of use of their platform. Also, the fact that people are building on it (i.e. spending time) increases their stickiness over Datadog. While Ddog may be the best observability solution now, Poffringa thinks it is very easy to switch between observability tools.

I stand 100% behind it. None of the Amazon database services you mention are bundled free with AWS - you have to pay additional for their use.

I am befuddled to hear someone so immersed in the tech space say this, and now realize I have no idea what the heck you think AWS actually is. Are you saying the cloud is some one-size-fits-all product called “AWS” and other things like DynamoDB are completely separate?

FACT 1:

AWS (Amazon Web Services subsidiary of Amazon-com) is a conglomeration of 100s of infrastructure-as-a-service products. https://aws.amazon.com/products/# Native products directly within the AWS umbrella include EC2, S3, RedShift, DynamoDB, Lambda, EBS, EMR, Kinesis, Cloudfront, RDS, Route53, and many many more managed services covering a wide range of services over compute and storage and networking infrastructure. Each of these is a separate product you can sign up for as an AWS customer, and have some type of usage-based pricing (based on compute costs, transfer fees, etc).

Beyond that is AWS Marketplace, https://aws.amazon.com/marketplace where other companies offer add’l infrastructure services that can be signed up for.

(My entire point before was that the solutions in AWS Marketplace are all frenemies of Amazon - they would rather side-step them and make native products within AWS to capture all the margin for themselves.)

Amazon’s customers are not third parties like MongoDB, Elastic or Snowflake, they are the enterprises that host solutions using AWS and add-on services,

Again, completely befuddled that you think this. ALL of those parties are customers of AWS services. And back to point #1 - what the heck do you mean by “AWS and add-on services”? “AWS” is not a product! EVERY PART OF AWS is an add-on service.

FACT 2:

MongoDB, Elastic, and Snowflake ARE ABSOLUTELY CUSTOMERS OF AWS, major ones in fact. In fact nearly all of the SaaS companies we own here are (Okta, Crowdstrike, Datadog, et al). But you just stated “third-party” SaaS providers are NOT customers? What? They all buy infrastructure upon AWS infrastructure-as-a-service (IaaS) to host their SaaS platforms, and they pay AWS heavily for that.

Snowflake’s clusters on AWS use EC2 & S3 extensively, and Mongo and Elastic’s clusters on AWS are probably EC2 or EKS (Kubernetes) and some storage services are used to run their Atlas and Elastic Cloud SaaS stacks (hosting managed dbs). These database services in particular are all what I call cloud-within-a-cloud. They buy the underlying AWS cloud infrastructure to host their platform, and write software running upon it that allows customers to do something in that platform, where customers can scale up and down their usage or amount of data. Their platform in turn scales up and down in AWS as needed to handle that load.

Other AWS customers - enterprise and SMB business - come along and sign up for native AWS services as well as non-native services like Snowflake, MongoDB and Elastic in the AWS Marketplace when they need them. Customers using MongoDB Atlas in AWS is paying MongoDB, who in turn pay AWS for the cloud infrastructure ultimately utilized.

I have no idea of the numbers but would have to think that SaaS Providers compose a huge amount of AWS’s revenue.

No need to respond. This is getting way off topic, you are diving way too deep into technical topics (what does FORTRAN standards body have to do with anything?) and it is all completely irrelevant to the companies at hand that Ethan started this thread on (DDOG et al). … But I am utterly confused as to what you think AWS is and had to say something – I strongly suggest you revisit your understanding of them.

-muji

32 Likes

I have no idea what the heck you think AWS actually is

I’ll quote Amazon: Amazon Web Services (AWS) is the world’s most comprehensive and broadly adopted cloud platform
https://aws.amazon.com/what-is-aws/ (emphasis added)

At a broad level, you can think of Cloud Platforms like one thinks of Operating Systems for PCs and phones.

MongoDB, Elastic, and Snowflake ARE ABSOLUTELY CUSTOMERS OF AWS, major ones in fact.

If you talk with people at Amazon like I have (yes, I’m speaking from experience), they’ll tell you that they view the ultimate end customer as their customer, even when that customer pays indirectly through a third party. Those customers often get to tell Snowflake or MongoDb which cloud platform (see above) they want the service to run on, so it’s important to Amazon to be the most appealing cloud platform. It’s not just about making Snowflake or MongoDb happy, it’s making their joint customers happy, at least in Amazon’s eyes.

I’m guessing Microsoft might think differently, but I have much less exposure there, except that it appears Microsoft is geared around its partner program and so it might be treating its partners as its customers under the assumption that the partners get to choose on which cloud platform they’ll run for their customers.

They all buy infrastructure upon AWS infrastructure-as-a-service (IaaS) to host their SaaS platforms, and they pay AWS heavily for that.

Yes, if one chooses MongoDB’s Atlas then one is paying Mongo, who then has to pay Amazon for usage. But, one can also choose to install MongoDB’s Enterprise Advanced on one’s own AWS EC2 instance (https://aws.amazon.com/marketplace/pp/B07GH68DSS?qid=1600144… ), in which case one is paying both Mongo and Amazon directly. As that page actually says: This listing is only for MongoDB Enterprise Advanced licenses. It does not give you access to any compute and storage resources. You must purchase the underlying infrastructure separately from AWS.

Elastic offers Elastic Cloud as both managed and self-managed services (https://www.elastic.co/pricing/ ), as well as Elastic Stack, which is self-managed. The self-managed services require the customer to pay Amazon directly for usage.

The choice comes down to customer sophistication and control desires, not to mention pricing. Large usage customers get discounted pricing from Amazon and may want to control deployments themselves, especially when multiple services from multiple vendors are to be integrated. But, my point that regardless of who is paying whom Amazon considers the end customer as its customer stands. And that’s probably why Amazon doesn’t view competing with third party vendors as hurting its customers.

No need to respond.

Sorry, you can’t post on AWS and then say AWS is off-topic, especially when Ethan’s first post included some thoughts on AWS. As we see, many of the companies we discuss run on AWS and other cloud platforms, so I think it’s useful to understand what a cloud platform is, as well as understand the different service models our companies provide.

It’s also important to understand the relationship Amazon has with third parties providing services on top of the AWS cloud platform, as well as what customers end up choosing and why. For instance, in early 2019 Amazon came out with DocumentDB when many here were invested in MDB. You think that’s Amazon trying to take away MongoDB customers, but I suspect Amazon was thinking that traditional MongoDb instances were hard to deploy on AWS and while Atlas had been around for a few years, it didn’t appear to Amazon that Mongo was making things easy enough for customers.

At any rate, DocumentDB’s introduction caused a lot of investor consternation, with many people selling MDB without fully understanding the limitations of Amazon’s offering or how MDB would respond. By mid-year, however, the Amazon threat was better understood and MongoDB’s Atlas offering continued to grow in usage and revenue. As Beth Kindig remarked: The financial markets guessed wrong about AWS’s ability to compete with MongoDB. We see very little evidence that AWS’s DocumentDB has been a success with Amazon changing its tone at a recent software developer conference. https://beth.technology/mongodb-atlas/

For all we know, Amazon was happy in the end to have forced Mongo to step up its Atlas support on AWS (they added Atlas Data Lake, Atlas Full-Text Search and MongoDB Charts in response, for instance) and thus make their joint customers happier. Because otherwise customers may have gone to other cloud platforms.

BTW, since you missed it, the discussion where I brought up the Fortran standards body was for perspective around how quickly technology moves these days compared to last century, as a result how we’ve moved from official standards to Open Source code/APIs as defacto standards instead. This is driven partially as a result of technology changing so quickly that by the time a standard would be approved, the technology behind it would be obsolete. And, as some of our companies use or publish Open Source code and/or APIs it’s useful to understand the importance of Open Source and why enterprises might still pay for additional services/features/support even when the base is free.

26 Likes

Muji: “It’s obviously 1 foot.”

Smorg: “It’s 12 inches.”

I love both of you, but I really think you’re both describing the same thing in two different ways.

I would say the most important thing isn’t even how either of you look at AWS, or even what it is in the most precise sense, but rather the perspective we should care about is that of Ethan’s friend, a potential customer of DDOG, NEWR, ESTC, and probably many other companies we discuss here.

Even if I’m wrong about that, this discussion really isn’t really helping me (and likely not the other non-techies on the board, either). Please take it offline.

Thanks,
Bear
Assistant board manager

36 Likes

I did not see this as a technical discussion but rather a business discussion that puts the technical market place relationships into perspective. As such I personally see it critical to making SaaS cloud stock choices and decisions. I do not think that gearheads Smorg or Muji think for one second that this was a technical discussion. Double emoji thumbs up to both Smorg and Muji! Thanks.

-zane

11 Likes