Cockroach Labs: AMD outperforms Intel…

This is really interesting – Cockroach Labs is behind CockroachDB, an extremely resilient, globally distributed OLTP database comparable to Google Spanner. Really interesting company – if I were in NYC I’d try to work for them :slight_smile: I find it strange that they didn’t review the Arm CPUs from Amazon, though. Maybe the timing was wrong – Graviton 3 is just rolling out but not that widely available, Graviton 2 may be stalled a bit because of Graviton 3’s debut.

Cockroach Labs recently released their annual cloud report which evaluates the performance of AWS, Microsoft Azure and Google Cloud for common OLTP workloads. Differently from the past, this year’s report does not indicate a best overall provider, but concludes that AMD instances outperform Intel ones. ARM instances were not covered in the tests.

According to the report, all the main three cloud providers offer similar price-competitive options for critical applications and workloads. Performing over 3000 benchmark runs on 56 different instance types and 107 discrete configurations, the price-for-performance leaders were AMD-based instances running Milan processors. Keith McClellan, director of partner solutions engineering at Cockroach Labs and the lead author of the Cloud Report, writes:

It’s like the late 1990s again, with AMD beating out Intel on pure performance… I think the last time this happened AMD was still selling chips called “Athlon”!

The lack of coverage for ARM instances and the most recent instance types, leaves questions unanswered for use cases where for example Graviton instances on AWS could be a better fit. McClellan acknowledges it:

ARM is going to be making a comeback in the 2023 Cloud Report - unfortunately we didn’t have supported ARM binaries for CockroachDB ready in time for the report this year.

Not every analysis agrees on the benefits of AMD instances for database workloads. The benchmark “Economical Comparison of AWS CPUs for MySQL (ARM vs Intel vs AMD)” performed by Percona, concludes that Graviton is cheaper in most cases and usually Intel performs better than AMD on AWS.

According to Cockroach Labs, small instances outperform large ones. In both the OLTP and CPU benchmarks, the report shows a per-vCPU performance advantage to running on smaller instances of a particular instance type, regardless of CPU platform, cloud, or instance type.

Processors and CPU benchmarks were not the only focus. The report stresses the importance of the storage and transfer costs and how they have a significant impact on total cost to operate:

For even relatively small amounts of persistent block storage, the cost of running a particular workload is much more influenced by the cost of the storage than it is the cost of the instance. For persistent workloads, it is extremely important to optimize cost calculations based on this consideration.