DDOG, ESTC, MDB: perspectives

[othalan]

Reading through the recent long discussion on Data Dog (DDOG), Elastic (ESTC), and MongoDB (MDB) has been very interesting. We all know the numbers by now and I won’t repeat them. I want to instead offer a few bits of perspective from outside the discussion combined with a little bit of my own personal experience.

MDB: My experience as a user

I am a software engineer by trade. While I have retired from that profession, I do still sometimes take on a small project or develop an app for my own personal use. As I am invested in MDB I thought I would give it a try in a personal project and compare other available options. My use case is trivial compared to the capabilities of the database: a simple data store for a music player.

Conclusion: MongoDB is amazing to work with and has no viable direct alternative.

There are many NoSQL database options available. I looked through the capabilities of every option I could find and created tests for working with a few of the most promising. For the type of capability MongoDB offers (document based NoSQL database), I found no other option which I felt comes close to the capabilities of MongoDB. I did find a few which were better in specific use cases, however even then those alternatives often did not offer the ability to scale up to large size as is possible with MongoDB. MongoDB is not perfect. The interface is at times irritating. Yet I don’t see any realistic alternative I would choose if I needed a large scale document based NoSQL database.

As an investor, I find this hopeful for my investment. It helps me understand that there is good reason MongoDB customers love the database and why there is a lot of room in the industry to attract new customers. It also gives me faith that customers are not going to be leaving MongoDB any time soon.

I see a very real possibility for MDB becoming an industry juggernaut on par with CSCO.

My one concern is that MDB does not seem concerned about moving towards profitability as is shown by the negative FCF which is getting worse (as pointed out by GauchoChris looking at TTM FCF). This will probably have me reducing my investment though not selling out in the near future.

ESTC: Not so compelling

I also investigated Elastic (ESTC) from a software engineer’s perspective. Note that I did not use their product so take this with a grain of salt. It seems to me there is nothing uniquely compelling about Elastic Search comparable to what I encountered right away with MongoDB. Why couldn’t a company replace ESTC’s search engine with a different product? I don’t see the path to becoming an industry juggernaut as I do for MDB. I also distrust the pure open source model as an investment.

I am not invested in ESTC and am happy with that decision. Maybe it will be a great investment but I am not comfortable holding it it my own portfolio.

DDOG: A viral technology?

No personal evaluation of DDOG as I have no use case yet I felt I had to understand why the company has such a high valuation. So I looked at everything I could find on their web site.

My conclusion: DataDog has gone viral.

My (perhaps naïve) take on what DataDog offers the world:

The internet is now filled with large scale cloud based applications and networks of services working together. While a lot of information on these applications and services is available, viewing them all in a way that gives a clear picture of what is happening has become anywhere from challenging to impossible because the data is coming in from so many diverse locations with no unified way of visualizing what is happening. In short, information overload. DataDog to the rescue. Large quantities of data are easily organized into a variety of views that are easily understood both from a system wide perspective and in the individual details.

It seems to me that there has been such a pent up need for what DataDog offers, that the technology has gone viral.

As an investor, I like everything about DataDog except the valuation. I am reminded of the quote by Warren Buffett: “Be Fearful When Others Are Greedy and Greedy When Others Are Fearful.” I am very reassured that the discussions on this board are not universally in favor of DDOG. Yet I am concerned that there is so much excitement about DDOG that the price will at some point crash once the novelty has worn off (in the way of everything viral that gets forgotten). I have no clue when that will happen or how far the price will fall, yet I am concerned enough over that happening that I will likely limit my investment in DDOG more than I might otherwise given the apparent prospects for the company.

[captainccs]

There were a lot of posts about MDB and ESTC at the NPI board and the gist about ESTC was early enthusiasm which faded rather quickly. Not so about MDB for which early enthusiasm grew with the announcement of v4 with transaction processing and later Atlas.

I have zero experience with DataDog but monitoring my own websites became a major issue for me after I discovered bugs that had made some of the features inoperative for months before I discovered the problem. I love writing code so I created The Ultimate PHP Error Reporter over eight years ago and I have been updating it on occasion ever since, adding features. A robust monitoring system is an ABSOLUTE must for anyone doing business on the WWW, no ifs or buts about it. No wonder DataDog went viral.

Denny Schlesinger

June 6, 2011
The Ultimate PHP Error Reporter
Keeping track of the health of websites is more critical than ever.

https://softwaretimes.com/downloads/the+ultimate+php+error+r…

[Darthtaco]

Why couldn’t a company replace ESTC’s search engine with a different product?

I’ll ask the obvious? What’s the competing product that companies are/would replace Elastic with?

[sarksnz]

Hi othalan,

Thanks for your view. I’m curious as another Geek, why you couldn’t replace mongo with any ol’ database? Mongo (imo) is not a better database, but they’ve done extremely well to position it as the default for web apps. The challenges still remain, but mongo let’s you kick the can down the road, and the ability to scale without thinking too much about it (for a while) comes out of the box. I know one of their big users is looking to move away to Amazon because of issues with MDB, whatever that means.

Mongo recently announced lucene integration (the basis of elastic search) so the overlap between with ESTC is growing.

ESTC as a tech is not directly comparable to MDB. You use MDB as your primary data store, whereas ESTC is a secondary data store. Which it does extremely well and I don’t know of another ‘search engine’ you could replace it with. AFAIK, if you want search, elastic search is definitely on your radar (Normally you’d want search to not be operating on your primary data store).

This is referring to elasticsearch specifically, rather than kibana or their Siem, etc.

Why hasn’t ESTC gone anywhere? My theory is that there’s a bunch of potential, but no obvious impact on revenue. A bit like TWLO and Flex. sure, ESTC could knock it out of the park with SIEM, but… We have to wait and see.

Cheers
Greg

[Darthtaco]

Those interested in this topic with DDOG and Elasticsearch May want to check out this video from a “meet up” with some DDOG engineers and others about Elastic. It’s from 2016, so could be completely different now, who knows? DDOG starts at 1:10.

https://youtu.be/w_ryhDJbsTY

Slide deck:

https://drive.google.com/file/d/0B5MjeRRcIi97NHh4emh1aHFmUEU…

To say the least, the Dog engineers love them some Elastic. It may be the only thing that could work for them to scale. A couple of other data stores are mentioned but shelves for Elastic. They went from collecting a billion events to a trillion. All shipped to Elastic clusters for indexing and search. When you open up your DataDog browser you are looking at events in Elasticsearch. Dog app is essentially their own version of Kibana.

They started with a cluster per AZ and by the time of this video, 20+. Was listening for any clues on if they are using any pay features. May not be. Alerting and notifications would be an area where they might have been. The first presenter is Prelert. Bought by Elastic shortly after this video, this is the ML and anomaly company that brought that capability to the Elastic platform. Haven’t watched their portion of the video yet.

As the company says Elastic is what feeds the DataDog applications. And they use it extensively internally. It can also be used for e-commerce sites and recommender feeds for Twitter. “Elastic can do what ever you want it to”.

Darth

[bly2k]

I’ll ask the obvious? What’s the competing product that companies are/would replace Elastic with?

It depends on your definition of Elastic. If you are talking about Elasticsearch (the core search capability/App Search/Site Search), you can go to a managed cloud vendor solution (example: Google CloudSearch, Amazon CloudSearch, MarkLogic, Endeca/Oracle), or you can go open-source solution (example: Lucene, SOLR, Sphinx). Heck depending on use-case you can even use just a standard NOSQL database to replace Elasticsearch if your needs are not that complex (example: MongoDB, Couch, Postgres) but of course you’ll have to a little bit more legwork to get there. Those are just a few examples.

If you are talking about the higher level Elastic products:

• Elastic APM - you can use any other APM product like DataDog, New Relic, etc
• Elastic SIEM - you can use any other SIEM product like Splunk, Solarwinds
• Elastic Logs - you can use any other logging/oberservability product like DataDog, Loggly
• Elastic Endpoint Security - lot of options here from big and small security companies

One thing I like to mention is Elastic as a search solution is no doubt one of the best in the industry. The fact that you can get people to use an open-source product with low touch sales speaks to the quality of the stuff that they put out. This is something that is difficult to grasp and understand unless you are part of the open-source community.

As an investment, it is a little harder to rationalize since the company’s DNA is really open-source. It is intrinsic to the company and it is the reason how they got to today.

This is straight from their CEO:

https://www.elastic.co/blog/on-open-distros-open-source-and-…

I can see what they are trying to do, which is get people hooked on the capability of the underlying technology (Elasticsearch) and then expand to other use cases using the other products that they build (APM, SIEM, Logs, etc) all under the same umbrella. Whether or not it will work as an investment, that is up to you/me to decide. But I can tell you as a long time user of Elasticsearch and some of their products, they are excellent, people love them, and people realize and understand the real value of what they provide, and the best part is they listen to their customers and evolve/improve their products extremely fast (i.e. they will not just sit around and wait for the competition).

Which leads me to another point. There are some posts here that repeatedly mention that you can take ES and provide it to others and not pay Elastic a dime (just like Amazon). Although that is technically true, it is not practically realistic. Elasticsearch is a fairly complex offering that 1) evolves very fast and 2) requires some level of complexity with management of your deployment. So let’s take the case of Amazon for example: they take a version of Elastic, fork it, put lipstick on it, and then “resell it”. Guess what, 1 year later Elasticsearch has morphed and significantly improved the product with many newer capabilities, fixed discovered issues (security, enterprise, whatever) that the market demands to such a point that the the Amazon “offering” is seriously outdated and behind (and potentially with unfixed security issues, for example). So what is Amazon to do - they can either continue to run the outdated “offering”, or try to keep up with Elastic’s offering, or go in a different direction with their fork. Think about that for a second. Is Amazon really going to keep up with that?

The second point about other companies just using Elasticsearch (self-hosted) underneath and not paying them a dime. Again this is technically true. However one important thing it misses is not all Elasticsearch features are available in the basic/free tier. If you go to their website (https://www.elastic.co/subscriptions), you can clearly see differentiation in their subscription tiers and what capabilities are not available in the basic/free tier. What I am trying to say is if you have a product of serious volume/complexity/usage, you are likely using some of the more advanced features of Elastic which makes you a paying subscriber. This is very important as this is part of their business model. Throwing the statement “Elastic is free and you don’t need to pay them anything” is correct, but it is practically misleading.

Anyway, I am not here to persuade anybody to consider Elastic as an investment. This is just to clarify and dispel some of the stuff regarding Elasticsearch as a company and their technology offerings.

[othalan]

Thanks for your view. I’m curious as another Geek, why you couldn’t replace mongo with any ol’ database?

To some extent in software, anything can be replaced. Add an abstraction layer and you are done. At least in theory. In reality, most developers are not careful enough to do this easily and abstraction layers slow down the application and are often less than perfect. Amazon found this when with its MongoDB compatibility.

Also, replace it with what? Amazon? The limitations in that were discussed on this board back when Amazon announced its compatibility layer. A new product? Mongo may have its bugs and limitations, but that will be even more true of something new. Especially when large data volumes are thrown at it. I don’t see a good alternative that offers what MDB offers. Especially with Atlas added in!

Eventually there will be a competitor. But that will take time. Some day when Mongo is too large a company to innovate rapidly a new company will come along and start taking their customers with a better offering. But that will take time.

Mongo (imo) is not a better database, but they’ve done extremely well to position it as the default for web apps.

I agree. Is Mongo a better database? Nope. Not even close. To be honest it is horrible in many ways. However, it is absolutely brilliant at solving an old problem: Storing and retrieving data which is similar yet non-homogenous.

As an investor, I don’t need MongoDB to be a “better database”. Just better than the competition at what it is specifically targeted at doing. Just enough better to be attracting new customers looking for a solution that is not as easily available elsewhere.

ESTC as a tech is not directly comparable to MDB. You use MDB as your primary data store, whereas ESTC is a secondary data store. Which it does extremely well and I don’t know of another ‘search engine’ you could replace it with. AFAIK, if you want search, elastic search is definitely on your radar (Normally you’d want search to not be operating on your primary data store).

Never meant to imply MDB and ESTC are in any way comparable. They are not.

Is ElasticSearch (and their other services) “on your radar”? Of course. But unlike MDB, I don’t see a compelling reason why they should remain the #1 solution for a large segment of users. Too many other search options in development (some from the same open source code as ESTC). Also I don’t see how it is radically changing business profitability compared to some of my other SaaS investments. But perhaps I’m missing something.

Useful tool? Unquestionably. Good investment? Yes. As good as my other investments? I’m not convinced.

Perhaps I am wrong. Like Saul, I don’t need to be right about companies I am not invested in.

[Darthtaco]

Thanks bly2k. Yes I was referring to the context of the original question. Elastic as the search engine data store.

The point is not that there aren’t other solutions, but what would you replace Elastic with? And you seemed to answer that in your excellent post.

Nothing.

Who knows what kind of license DDOG uses for their Elastic clusters. The video and other presentations we have discussed were from 2016 & 2017. Elastic has evolved much since then. Both the product and the license. One area for them that would be nice to know. How do they do the field and document level authorization and authentication that would be required to have individual customers to access their data and only their data stored on the Dog clusters?

Darth

[bly2k]

One area for them that would be nice to know. How do they do the field and document level authorization and authentication that would be required to have individual customers to access their data and only their data stored on the Dog clusters?

Document and field level security has been in the product for a while now (as well as their entire security features: encryption, RBAC, etc). You can read about it here (if technically inclined):

https://www.elastic.co/guide/en/elasticsearch/reference/7.4/…

https://www.elastic.co/guide/en/elasticsearch/reference/curr…

I believe the security stuff was one of the most requested features a while ago that enabled them to break into the enterprise/bigger customers.

[Darthtaco]

Document and field level security has been in the product for a while now

Correct. And it was formerly only in x-pack. Now only in platinum.

And the only other known source is Search-guard, which has been DMCA taken down on Github and Sonatype as Elastic is suing them for copyright infringement.

Question for you then as it seems you know more than I do on how this works. Would DataDog be able accomplish this without paying Elastic for the license? Is there another source. Could they build their own API?

[XMFBreakerTinker]

I cannot find anything on whether Datadog pays Elastic anything or not, but in looking came upon this interesting topic of Datadog monitoring Elastic (instead of Elastic monitoring Elastic, which one would think would be more than adequate, but apparently not.

https://www.datadoghq.com/blog/monitor-elasticsearch-datadog…

Datadog has its own proprietary Kibana like products, etc. I don’t see why they could not have created their own security as well.

Either way, whether they are paying Elastic or not paying Elastic, they are making more money, with higher margins, than Elastic is. I doubt Datadog would have gross margins this high if they had to pay an Elastic tax. In fact, Datadog would have to be, perhaps (probably) Elastic’s largest customer and it would be in the millions, perhaps tens of millions of dollars. It would be very material for both Datadog and Elastic yet neither company report any such connection between the companies.

I’m not putting Elastic down here, as Elastic management understands they are open source and they are building a suite of proprietary software (like Datadog has done)to run on top of the open source portion, and using the open source aspects serve them.

Elastic has a lot of new products, momentum, etc., but it is in motion and the business momentum this creates is to be seen. Is the stock price not following the business momentum? That is conjecture. I have no information with real data to show that Elastic is economically accelerating or not from the new products. If their hosted product hits 100%+ growth and acceleration is guided, then the stock price will take off.

However (if valuation were not an issue, and all things are equal) there is no way anyone in their right mind would choose Elastic as their company over Datadog at this point in time. The Datadog numbers are as good as any you will find.

Many will say, this leaves more upside for Elastic, as Elastic has more upside surprise potential. Fair enough and we have seen it before. I like Elastic. I do not fully understand this market with Elastic.

One thing I have seen, like with Zscaler and Palo Alto and Blue Coat, analysts also ask Elastic competitors if the “unknown” vendor is having an impact on their business. Splunk, as an example, this was a prominent analyst question.

Tinker

[bly2k]

Would DataDog be able accomplish this without paying Elastic for the license? Is there another source. Could they build their own API?

If you are asking about securing Elasticsearch at a high level, you can use anything (proxy, whatever) that secures at the IP/port level. If you are specifically asking about document/field level security, you can probably pull it from the open distro that Amazon puts out (I have no experience with that personally), or technically I don’t see what’s stopping you from rolling your own if so inclined (assuming there are no legal limitations here from ES). I think at some point, you have to realize is it better go that way, or does it make more sense to be an Elastic subscriber at the right level and get all of the stuff you need out of the way so that you can spend more time on your product. There is a cost either way. It’s just a matter of what makes sense long term for you and your business.

I cannot speak for sure as to what Datadog does or even if they use such feature.

[truewinner]

What about Elastic’s special geospatial data store and search capabilities? They tout customers such as Uber and Tinder that fundamentally require location data search. This has always stood out to me as Elastic’s key differentiation in terms of store/search. If you run an app that requires location data and related analytics, is there a good alternative?

[bly2k]

What about Elastic’s special geospatial data store and search capabilities? They tout customers such as Uber and Tinder that fundamentally require location data search. This has always stood out to me as Elastic’s key differentiation in terms of store/search. If you run an app that requires location data and related analytics, is there a good alternative?

Geo search (and indexing) is fairly common these days. You can get that in almost any modern database (SQL or NOSQL). Elastic does have very good support for geo search and they spent a lot of time honing and evolving it over the years. Although geo search itself is not a key differentiator, the ability to store (and optimize) geo data and search on it with any level of accuracy as well as speed is something that Elasticsearch is very good at. It is the reason why it is typically used for things like Uber which is sensitive to speed and scale for these types of searches (and even more so when done in combination with other types of searches). In addition they also use this underlying capability to do maps in Kibana for other uses like Elastic Logging and Endpoint.

[Tamhas]

Mongo has geospatial datatype and searches
https://docs.mongodb.com/manual/geospatial-queries/

[doppelg36]

othalan, great write-up!

Did you have any thoughts on how much competition DDOG faces? I feel like there are plenty of APM and IT monitoring solutions in the marketplace. Obviously DDOG’s revenue growth indicates they are offering a great solution… but I have some nagging doubts that even a great DDOG offering won’t be able to differentiate itself from competitors like MDB can.

I am long MDB and AYX because I see them as having more business value with more lock-in. I’m actually slightly short DDOG at this time