AMD’s AI strategy

ANALYSIS After re-establishing itself in the datacenter over the past few years, AMD is now hoping to become a big player in the AI compute space with an expanded portfolio of chips that cover everything from the edge to the cloud.

It’s quite an ambitious goal, given Nvidia’s dominance in the space with its GPUs and the CUDA programming model, plus the increasing competition from Intel and several other companies.

But as executives laid out during AMD’s Financial Analyst Day 2022 event last week, the resurgent chip designer believes it has the right silicon and software coming into place to pursue the wider AI space.

At last week’s event, AMD executives said they have started to see some early traction in the AI compute market with the company’s Epyc server chips being used for inference applications and its Instinct datacenter GPUs being deployed for AI model training.

For instance, multiple cloud service providers are already using AMD’s software optimizations via its ZenDNN library to provide a “very nice performance uplift” on recommendation engines using the company’s Epyc CPUs, according to Dan McNamara, the head of AMD’s Epyc business.

Short for Zen Deep Neural Network, ZenDNN is integrated with the popular TensorFlow and PyTorch frameworks as well as ONNXRT, and it’s supported by the second and third generation of Epyc chips.

“I think it’s really important to say that a large percentage of the inference is happening in CPUs, and we expect that to continue going forward,” he said.

In the near future, AMD is looking to introduce more AI capabilities into CPUs at the hardware level.

This includes the AVX-512 VNNI instruction, which will be introduced to accelerate neural network processing in the next-generation Epyc chips, code-named Genoa, coming out later this year.

Since this capability is being implemented in Genoa’s Zen 4 architecture, VNNI will also be present in the company’s Ryzen 7000 desktop chips that are also due by the end of the year.

AMD plans to expand the AI capabilities of its CPUs future by making use of the AI engine technology from its $49 billion acquisition of FPGA designer Xilinx, which closed earlier this year.

The AI engine, which falls under AMD’s newly named XDNA banner of “adaptive architecture” building blocks, will be incorporated into several new products across the company’s portfolio in the future.

After making its debut in Xilinx’s Versal adaptive chip in 2018, the AI engine will be integrated in two future generations of Ryzen laptop chips. The first is code-named Phoenix Point and will arrive in 2023 while the second is code-named Strix Point and will arrive in 2024. The AI engine will also be used in a future generation of Epyc server chips, though AMD didn’t say when that would happen.

In 2024, AMD expects to debut the first chips using its next-generation Zen 5 architecture, which will include new optimizations for AI and machine learning workloads.

Big AI training ambitions with ‘first datacenter APU’
As for GPUs, AMD has made some headway in the AI training space with its most recent generation of Instinct GPUs, the MI200 series, and it’s hoping to make even more progress in the near future with new silicon and software improvements.

For instance, in the latest version of its ROCm GPU compute software, AMD has added optimizations for training and inference workloads running on frameworks like PyTorch and TensorFlow.

The company has also expanded ROCm support to its consumer-focused Radeon GPUs that use the RDNA architecture, according to David Wang, the head of AMD’s GPU business.

“And lastly, we’re developing SDKs with pre-optimized models to ease the development and deployment of AI applications,” he said.

AMD’s coverage in the AI compute space was mainly in cloud datacenters with its Epyc and Instinct chips, at enterprises with its Epyc and Ryzen pro chips, and at homes with its Ryzen and Radeon chips.

But with Xilinx’s portfolio now under the AMD banner, the chip designer has much broader coverage in the AI market. This is because Xilinx’s Zynq adaptive chips are used in a variety of industries, including health care and life sciences, transportation, smart retail, smart cities, and intelligent factories. Xilinx’s Versal adaptive chips, on the other hand, are used by telecommunications providers. Xilinx also has Alveo accelerators and Kintex FPGAs that are used in cloud datacenters too.