Based on their recent job posting, Snowflake will soon be venturing into the observability and APM (Application Performance Monitoring) space.
https://careers.snowflake.com/us/en/job/5487583002?gh_jid=54…
“We’re hiring talented Senior Software Engineers to help us build the next generation of observability and alerting product, enabling our customers to use time series data, define alerts and receive notifications on their data as well as the operation of their pipelines in Snowflake. Having such a comprehensive mechanism will help our customers build a more comprehensive continuous data processing pipeline.”
The space is getting crowded, and here’s an idea of the significant current players in the market based on Gartner’s Magic Quadrant for 2021: https://www.dynatrace.com/news/blog/dynatrace-again-named-a-…
14 Likes
Great discovery!
My personal guess is that what they are going into is very different from the APM observability (DDOG, Dynatrace, New Relic) or the security monitoring (CRWD, NET)
Based on the job description and recent Snowflake product news, I’m guessing this has more to do with “who is using what part of the data warehouse, when, and how much?” and “Is my data drifting from what I expect?”
In the data world today, with the rise of machine learning models, there’s an increasing demand of monitoring that the data you receive is what you expect to see and you want to be alerted when things look out of whack.
For example, if you run a hospital, you want to receive alert when your ER visits in the last week spiked 3X. This insight should be extracted from data warehouse with very little effort. But the reality is that there’s a lack of good tools that does this on the buy side vs very fragmented landscape on the build side. You need to spend a lot of time building a reliable system that can do this because most databases treat this as an afterthought.
Datadog and other APM observability tools track the real time health of backend of internet services and help you answer questions like “is my checkout system’s speed normal right now? Is one of our POS down?” and in my opinion it is different from Snowflake’s proposed value.
As a developer that uses Snowflake daily this is very exciting but I fail to see the connection between SNOW and DDOG as competitors.
As an investor this is a natural step for SNOW to go into as there’s very little competition from other mature players in this field but is a major problem for data teams and can be a very profitable add-on module for Snowflake.
54 Likes
I agree with chang88.
Quote from the job description “build the next generation of observability and alerting product, enabling our customers to use time series data, define alerts and receive notifications on their data as well as the operation of their pipelines in Snowflake.Having such a comprehensive mechanism will help our customers build a more comprehensive continuous data processing pipeline.”
For time-series data, event detection and the analysis of event timing (clustering, autocorrelation…etc) are the most important features to be extracted. Most of time the patterns of events have to be customized by users for specific type of datasets (physiological signals, financial charts…etc). If snowflake can develop an algorithm to simplify the process in the pipeline or even automated the process, I think it will make snowflake more sticky for customers.
5 Likes
Completely agree. What Snowflake is going after is building a tool automatically detects anomalies in data and also allows for custom alerting.
This is exactly what Monte Carlo does (https://www.montecarlodata.com/about-us/), which is likely the market leader in this.
From Monte Carlo website : “In software engineering, every team has a solution like New Relic, DataDog, or PagerDuty to measure the health of applications and ensure reliability. How come data teams are flying blind?”
I have worked with Monte Carlo at my current job (the tool works great); what Snowflake is doing seems like a natural extension of their business.
7 Likes