“AQUA provides a new distributed and hardware accelerated cache that brings compute to the storage layer for Amazon Redshift and delivers up to 10x faster query performance than other cloud data warehouses.”
So if this project greatly improves Redshift’s performance by 10x…and Redshift is Amazon’s competitor to Snowflake…then shutting down AQUA could potentially harm Redshift’s value proposition against Snowflake? If so, this would only be beneficial to SNOW in capturing more customers.
I work at a large Fortune 500 in financial services. We are about to do a major change to our tech stack, building out a proper cloud analytics platform (from on-prem). We are going with BOTH Databricks and Snowflake. I know of 3 other large firms that are about to move to both as well.
With respect to your comment about AQUA, here is what I can say about our experience with going with Snowflake:
Amazon is involved (co-selling). Our CTO and Architecture team reviewed options and decided Snowflake best met our needs, but AWS and Azure are involved almost from the outset, and the sentiment and mood is very positive. There is no pushing their own products, they are more than happy for Snowflake to get the primary business while they reap the rewards as well. For large enterprises that are multi-cloud, it seems Snowflake has an edge, and the data marketplace is a huge factor for us as many of the partners we wish to share data with are already on Snowflake.
The hyperscalers will make their share of revenue/profit with the proliferation and explosion of data, and the partnerships with Snowflake, Databricks, etc will only continue to get stronger.
I will try to add something to the discussion. Thanks for all the posts and insights you have been doing lately.
It seems that the main difference between RedShift and Snowflake is that Snowflake separates the storage and compute costs. This blog post explains very well why this is an advantage to Snowflake.
“The big difference in cost between Snowflake and Redshift is the fact that Snowflake was built to decouple computing and storage costs.” " His main point when it comes to Snowflake is that its ability to quickly and cheaply spin up virtual warehouses is a game-changer. He plays out the following scenario:
If you have a query to run on a small machine (i.e., virtual warehouse) it might cost $3 an hour.
It could take an hour and will cost $3.
Or, with Snowflake, you spin up a machine that’s 10x bigger and costs $30 an hour.
Then, it could take you 6 minutes and still cost $3.
Alessandro goes on to say, “I can take these monster machines and 5-hour queries and make them run in 10 minutes. And I pay the same.” Redshift can’t do this. Even if it could, the fact that its most commonly used version can’t decouple storage and compute pricing, running a larger machine would break the bank. You’re forced to move slower."
Amazon introduced Redshift RA3 to provide an option to separate compute and storage. But it was a low performance option. Then AQUA was introduced as a way to offer high performance in RA3. See this post for more information.
“To recap, AQUA sits between an Amazon Redshift RA3 cluster’s compute and storage, and runs with Amazon Redshift RA3 instances; these are the compute instances that are designed for separating compute from storage.”
CONCLUSION
It seems AQUA was developed to try and compete with solutions like Snowflake that offer the flexibility of separating storage and compute while still having high performance. It would seem that Amazon discontinuing AQUA is a good thing for Snowflake. One interesting thing from the ZDNET post above is that AQUA requires specialized hardware to operate. I would assume that makes it expensive to deploy. I think the only reason to cancel the product are because it is not profitable or it does not work well.
AQUA is already released and available today, and it comes at no extra charge when you purchase Redshift. AQUA is special hardware (based on custom chips) that accelerates analytics queries that are executed on RA3 instances. The RA3 instance decouples storage from compute, just like Snowflake does.
Here’s the documentation on AQUA:
[AQUA] uses acceleration based on a field-programmable gate array (FPGA) to push as much computation as possible into the storage layer. These components are connected together in a unique way, allowing data scanning without using a traditional CPU. At the same time, these components allow intermediate results to be aggregated in high-speed memory. AQUA is a data cache and maintains high-speed connections to Redshift managed storage.
Redshift and Snowflake have different strengths, and customers pick one or the other solution based on their requirements. That being said, Snowflake continues to be an important partner for AWS.