Zen 4/4c plus an AI accelerator, and making it all work

Interesting – again, Intel overreached. A mix of P and E cores that didn’t support the same instruction set was just too hard to pull off. I don’t think I’d fully understood that. A sign of a culture problem still not fully worked out.

During Computex 2023, I had a chance to visit AMD’s towering offices in Taipei, Taiwan, to see the company’s Ryzen AI demo and speak with David McAfee, the Corporate VP and GM of the Client Channel Business. Most of our conversation centered on AMD’s efforts in the consumer AI space, but I also squeezed in a few questions about AMD’s take on hybrid CPUs. McAfee told me AMD has a different vision of hybrid processors than Intel that would avoid the complexity that forced Intel to remove AVX-512 support from its chips.

I interviewed AMD CTO Mark Papermaster two weeks ago in Antwerp, Belgium. He told me that we would “see high-performance cores mixed with power-efficient cores mixed with acceleration” in future AMD client [consumer] processors, signaling that, like Intel before it, AMD would adopt a hybrid CPU execution core design in the future. That wasn’t too surprising – we saw the first signs of two different CPU core types in AMD’s software manuals months ago. Besides, AMD is already laying the foundation with its coming EPYC Bergamo chips with dense Zen 4c cores akin to efficiency cores.

AMD’s current Ryzen 7040 laptop chips already feature a hybrid design, but not with two different types of CPU cores. Instead, the Ryzen 7040 has just one type of CPU core paired with an in-built AI accelerator engine that operates independently of the CPU and GPU cores. This engine provides advantages for certain types of AI inference workloads, but the CPU and GPU cores are better for other types of inference. So, the trick is to direct the different AI workloads to the correct type of cores to extract the best performance and power efficiency.

Throwing separate performance and efficiency CPU cores into that mix would introduce yet another compute option for AI inference workloads, and I asked McAfee if, conceptually, it would be feasible that efficiency cores would be better for AI than a dedicated piece of silicon (the AI engine). McAfee explained that the AI engines’ strict focus on AI-specific operations would give it an efficiency advantage over any general-purpose CPU compute – even an efficiency core.

Then we shifted to discussing Intel’s hybrid chips, which have two types of cores, each with its own unique microarchitecture. That’s created interesting problems: Intel’s performance cores support AVX-512, but the smaller efficiency cores do not. That led Intel to disable AVX-512 support entirely (forcibly in the end), thus de-featuring its own chip and wasting precious die area.

I asked McAfee how AMD felt about that approach to hybrid designs.

“What I will say is this, I think the way that we think about it, the approach of two very different performance and efficiency cores with very different ISA support and IPC and capability is not necessarily the right approach,” McAfee responded. "I think it invites far more complexity around what can execute where, and as we’ve looked at different options for core design, that’s not the approach that we’re taking.