Cutting To The Front Of The Server CPU Line: NextPlatform

CPUs are in a kind of tight loop with the hyperscalers and cloud builders, who not only get to help co-design the CPUs and their SKU stacks, but they ensure they get the best (meaning lowest) pricing, the first availability, and as many vendor coffee cups as they need to make sure their server CPU competition knows the royal treatment they are getting. This shift from HPC to hyperscalers and clouds makes perfect sense given the HPC centers acquire a new machine every three years with several thousand nodes, a volume that is several orders of magnitude lower than what the hyperscalers and cloud builders buy each quarter.

In many ways, particularly when it comes to the overall profitability of a CPU line over time, it sucks to be a CPU maker. Everything is exactly backwards.

Now, on the front-end of any cycle, the profits are relatively low because of the huge volume price discounts – on the order of 50 percent over the years, depending on supply, which changes that equation in ways no one talks about publicly – that the hyperscalers and cloud builders normally command. And CPU makers have to rely on the other half of the volume ramp with enterprises and telcos and other service providers – who buy in much lower volumes and therefore pay higher unit prices against lowering chip manufacturing costs as yields improve.

In the old days, only the HPC centers and the most desperate large enterprises dove into a new CPU first, and the volumes were relatively low and the price CPU makers could command was high and in direct proportion to the cost of the chip. As the chip production ramped, the chip got cheaper at the same time customers started looking for deals – particularly in an earlier era when Xeons were competing against Opterons. The revenue and profit curves sort of mapped.

And now, they are inverted. Except when AMD has a little bit of leverage, as it most certainly does when Intel’s “Sapphire Rapids” Xeon SP processors are ramping their volume only now and are not expected to be formally launched now until January 10 of next year. This against a “Milan” Epyc 7003 processor that has been in the field since March 2021 and a “Genoa” kicker with a lot more oomph due to be launched on November 10 of this year. (And of course already shipping to the hyperscalers and cloud builders.) This is the one time that being a CPU vendor in the datacenter most certainly does not suck.

Even with higher operating expenses as AMD pushes its Epyc CPU roadmap and covers the cost of ramp of the 5 nanometer Genoa Epyc 9004s, operating income at the Data Center group rose faster than revenue. Part of that, we think, is that Milan is mature and still competitive with anything Intel can make and sell, so the lowering costs of Milan manufacturing are also translating to more profits as not just hyperscalers and clouds adopt this processor, but also the rest of the IT market.

…

Drilling into datacenter sales, this was the 10th straight quarter of record server CPU sales for AMD, which was a mix of initial Genoa shipments and what we presume was a lot of Milan shipments. As a kind of leading indicator, Lisa Su, chief executive officer at AMD, said sales to the hyperscalers and cloud builders more than doubled year on year in Q3 and also increased sequentially from Q2, with more than 70 new instances based on AMD Epyc CPUs coming out from Amazon Web Services, Microsoft Azure, Tencent, Baidu, which she called out by name, and others, which she did not. Enterprise, service provider/telco, government, and academia accounts – which are all lumped into the same “enterprise” category – are being more cautious about spending at the moment, according to Su. And she added that spending among enterprises in particular for the datacenter group were down. That means sales of Epyc CPUs for enterprises were down.

Aaron Rakers, the semiconductor analyst at Wells Fargo, estimates that the hyperscalers and cloud builders account for at least 70 percent of Epyc CPU sales. If you do some math on that, extracting out some nominal GPU sales and even more nominal DPU sales, then we reckon a few things. To make cloud go up and enterprise go down, hyperscalers and cloud builders needed to account for at least 80 percent of Epyc sales in Q3 2022. That implies hyperscalers and clouds spent $1.23 billion on Epyc CPUs, up 105 percent year on year, and thus all other kinds of organizations together (again, that broader “enterprise” category) spent $307 million on Epyc CPUs, down only 2.6 percent over the period.

Su added that sales of Instinct datacenter GPU compute engines were “down significantly” compared to Q3 2021, when it was shipping the Instinct MI250X accelerators based on the “Aldebaran” GPU for the “Frontier” exascale-class supercomputer at Oak Ridge National Laboratories. Our model from last year before Xilinx and Pensando were added to AMD pegged datacenter GPU sales at $164 million in Q3 2021 and $148 million in Q4 2021, and then it returned to its normal run rate of around $70 to $75 million in Q1 2022 and beyond. But this is just intelligent guesswork. We think that over time and as the AMD ROCm software ecosystem gets more traction, datacenter GPU sales will do better. But it is a long way from taking a big bite out of Nvidia in AI, even though AMD GPUs are certainly being adopted for very powerful supercomputers at HPC centers.

4 Likes

A Genoa kicker?
Normally a kicker is the same design as a previous chip with a few tweaks, higher clocks, etc.

Wikipedia says it has the new Zen 4 cores and an increase from 64 to 96 of them.
I guess kicker means to kick out the competition!

image

Mike

1 Like

Actually, Milan-X is/was the kicker. There will be customers who see little or no benefit from Milan-X. Most, however, would/will see a substantially higher clock speed even if they only replaced the current core number. For many customers with currently less than 64 cores per CPU, Milan-X will reduce the main memory reads enough to say, go from 24 to 32 cores and see a significant improvement.

I don’t know how many of you have worked a 2-second response time. Yes, that includes all of the database queries, etc. The end–user (end purchaser?) won’t care whether it takes adding tables to the database, specifying a faster computer, more cores or whatever. The usual is either 80% response within two seconds for users (Administrators are excluded. They tend to start back-ups or updates from an account and check in the morning to make sure it completed. :wink: In the case of cloud conversions customers will want the percentage to be higher than they are getting now from the hardware to be retired.