Thank you ExponentialDave for your great post on Snowflake!
I’ve encountered an interesting personal experience with Snowflake which I believe might be of interest to the members of this board [https://jmarquesdatabeyond.medium.com/what-makes-snowflake-s…
"My Personal Experience with Snowflake
Since I first used Snowflake back in 2016, I have been an enthusiastic advocate for its qualities.
Back then, I was implementing a greenfield data platform for a travel company, and when asked to try this new database we hadn’t heard of before, we were highly suspicious of it. It was a time when NoSQL databases were promising an easy world for everyone, and here it was, yet another “tool” claiming to take all the hassle away, no tuning needed, linear scaling…. too good to be true, but still we decided to give it a chance and do a side by side comparison with our preferred tool, AWS Redshift.
For the test, we used the hardest job we had: “shredding” hundreds of millions of click-stream data XML… after a lot of testing, we found that for the same price point, Snowflake won by a nice margin. After some more tests with typical query patterns, we saw that Snowflake was just a better tool for the job, and we went with it.
After little more than one year into the project, I was absolutely sold… let me share an anecdotal story:
I had just found a bug in the shredding process that deemed all our shredded data incomplete. Here I was, 11am on a Friday and my last day before a two week break, how lucky of me… Shredding hundreds of millions of these chunky (some of them 1MB+) XMLs would take forever! Unless…
I call the Client and explain the situation: I needed to start a very big cluster to get this over the finish line in time… And so I did, and the whole thing was done in less than a hour. I went on holidays at 6PM that day, with the work done and worry-free.
The best part? Because it scales linearly, waiting hours for it to run on a small cluster would have cost the same as doing it quickly in a large cluster. This was probably the big turning point for me: I now loved Snowflake.
Conclusions
Snowflake is just a great productivity enabler. It transforms all the expensive waiting around into productive work time: Data Engineers and Analysts are much more expensive than licensing, and this is before we even consider time-to-market. Cherry on top, no more “database administrators” needed, there is really nothing for them here…
I have worked with more than one client handling multi-billion records a day, and I know it can be done with other technologies. But this is the only technology that makes me breathe a sight of relief…"
Lemat