User notes on SNOW

Hi: Long time lurker, and only occasional poster. I have been intending to post some of my experiences with Snowflake as a user of their service, because it is one of the only ways I can think of contributing to this great board.

I word for a small to medium sized manufacturing company. 40 years ago it was an edgy startup, and now it is a matron in its industry. I have been here a little over 20 years and in that time it has slowly changed from a skunkworks to much more conventional. Along the way I drifted into computers and databases. We’ve built lots of in-house systems to do things that would have otherwise been impossible for a company our size – either because the existing software was too expensive, or it simply didn’t exist. Some of the systems we’ve built are (ahem) best in class. About 5 years ago a youngster I work with came across Snowflake, and talked me into giving it a try. We are a microsoft shop, with an increasingly rigid IT department, and we were having difficulties getting the right data to the right people within those constraints. I was skeptical of snowflake, but decided to give it a try.

User notes: it has a learning curve. ETL is very finicky, but once you finally get it set up, it pretty much runs unattended. SNOW’s claims regarding stability and speed are entirely justified – the speed part is incredible, actually. Our barriers to using it even more are 1) IT, and 2) limits on our staff to build things using it.

A couple of other things the board might find interesting. At a certain point our data we hosted on snowflake started to get more popular among users, so more people started to notice it was a bit ‘slow’ (very relative term…). Snowflake makes it very very easy to increase the horsepower on your applications, and it is in terms of 2x, 4x, 8x and maybe now 16x or more. Each of those costs 2x the tier below, so moving from the slowest rate doubled our cost per mile (so to speak). It is still usage based, but the rate is now double. You can turn it up or down any time, for any reason, and billing is immediately reflected. It was therefore painless to double our payments to Snowflake! I didn’t give it a second thought when I upped our horsepower until the IT supervisor asked me if anything had changed, because we were now paying snowflake double what we paid a month or two ago! Very clever of snowflake.

I mentioned in another post that one of our vendors has used Snowflake to build their application on. It was an offhand comment one person made in a conversation, and I was so surprised that I scheduled a call with their technical director to get more information on how they made it work. Among other things, they have a large team of high powered developers in various countries working on it… Nevertheless, the fact that they have used a software platform like Snowflake to build their app (it is a route accounting software – fairly low tech, and a warehouse management system – substantially higher requirements) was remarkable. I am sure they are only one of MANY. It is a use of Snowflake that I would have never imagined. That vendor works with many companies substantially larger than ours, and when we got to talking about the nuts and bolts of data architectures etc., he mentioned in an offhand way that many snowflake users have dispensed entirely with the old data warehouse architectures, and simply dump their data up into snowflake in one very (very) large table. For the db folks out there, that is unimaginable. He said he knew of one user that had some umpteen terabyte table (singular) with all their data.

So, while this is all very anecdotal, I know, it seems to me that snowflake quite likely is on to something, and even they don’t really know where it is all heading.

Long snow (not much, and bought at the peak, but still)

57 Likes

Thanks so much for posting from users’ perspective. It’s very helpful! When your team decided on deploying Snowflake, did you do any research on other alternative options to Snowflake? I know Snowflake’s product is definitely superb but am curious whether there was any other solutions ever worth your consideration as well.

Best,
Luffy

1 Like

Luffy asked: “When your team decided on deploying Snowflake, did you do any research on other alternative options to Snowflake?”

That’s the funny part. We never really ‘decided’ to deploy snowflake. We started using it as a test, and before we knew it, it was in production. If I recall, they had (have?) some kind of test mode where you can use it for free for a month or something. We had it running and liked it by then, so never turned it off. The lowest horsepower setting was still an order of magnitude (at least) faster to run our queries, and the cost was so low that it didn’t even cause a blip on the radar.

25 Likes

SO, now, do you do your ETL on the Snowflake server? (Move raw data, then massage it there instead of massaging the raw data at home, then sending?)