AMD Custom Makes CPUs for Azure: 88 "Zen 4" Cores and HBM3 Memory

[AleksandarK]

Microsoft has announced its new Azure HBv5 virtual machines, featuring unique custom hardware made by AMD. CEO Satya Nadella made the announcement during Microsoft Ignite, introducing a custom-designed AMD processor solution that achieves remarkable performance metrics. The new HBv5 virtual machines deliver an extraordinary 6.9 TB/s of memory bandwidth, utilizing four specialized AMD processors equipped with HBM3 technology. This represents an eightfold improvement over existing cloud alternatives and a staggering 20-fold increase compared to previous Azure HBv3 configurations. Each HBv5 virtual machine boasts impressive specifications, including up to 352 AMD EPYC "Zen4" CPU cores capable of reaching 4 GHz peak frequencies. The system provides users with 400-450 GB of HBM3 RAM and features doubled Infinity Fabric bandwidth compared to any previous AMD EPYC server platform. Given that each VM had four CPUs, this yields 88 Zen 4 cores per CPU socket, with 9 GB of memory per core.

The architecture includes 800 Gb/s of NVIDIA Quantum-2 InfiniBand connectivity and 14 TB of local NVMe SSD storage. The development marks a strategic shift in addressing memory performance limitations, which Microsoft identifies as a critical bottleneck in HPC applications. This custom design particularly benefits sectors requiring intensive computational resources, including automotive design, aerospace simulation, weather modeling, and energy research. While the CPU appears custom-designed for Microsoft's needs, it bears similarities to previously rumored AMD processors, suggesting a possible connection to the speculated MI300C chip architecture. The system's design choices, including disabled SMT and single-tenant configuration, clearly focus on optimizing performance for specific HPC workloads. If readers can recall, Intel also made customized Xeons for AWS and their needs, which is normal in the hyperscaler space, given they drive most of the revenue.



View at TechPowerUp Main Site | Source

 

[Aquinus]

with 9 GB of memory per core.

That's what I'm talking about! Now if only we could see this kind of integration for consumer level CPUs. I doubt we'll ever see it, but one can dream.

 

[AleksandarK]

That's what I'm talking about! Now if only we could see this kind of integration for consumer level CPUs. I doubt we'll ever see it, but one can dream.

It's my dream... I wouldn't know what to do with it, but I wouldn't let it sit idle

 

[Daven]

Companies like MS are so big now that they can dictate CPU architectures now and not just have to buy what’s on the shelf.

And that’s two custom processors for MS from AMD: this one and the one in the Xbox. It seems weird that MS went with Qualcomm instead of another AMD custom chip in their Surface series.

 

[ymdhis]

Do a desktop APU with 32GB integrated HBM and a beefy iGPU, I'd buy that even if it costs a lot. It would be an SFF dream cpu.

 

[Darmok N Jalad]

Companies like MS are so big now that they can dictate CPU architectures now and not just have to buy what’s on the shelf.

And that’s two custom processors for MS from AMD: this one and the one in the Xbox. It seems weird that MS went with Qualcomm instead of another AMD custom chip in their Surface series.

Because one of AMD's specialties is making custom configurations. Their designs are built to offer this kind of flexibility, so that's an advantage to leverage.

 

[bug]

Companies like MS are so big now that they can dictate CPU architectures now and not just have to buy what’s on the shelf.

And that’s two custom processors for MS from AMD: this one and the one in the Xbox. It seems weird that MS went with Qualcomm instead of another AMD custom chip in their Surface series.

Problems are a little different at that scale. So the solutions will also be a little different.
Everybody does it, only custom designs are usually around ARM instead of x86.

 

[Caring1]

Upgrading their side before the big push for Windows as a service?

 

[Aquinus]

It's my dream... I wouldn't know what to do with it, but I wouldn't let it sit idle

Although with that said, HBM tends to be very power efficient even if you were. I've always felt that this kind of integration would make sense for mobile devices where space and power are at a premium, but you still want to produce a high performance device.

 

[AnarchoPrimitiv]

Do a desktop APU with 32GB integrated HBM and a beefy iGPU, I'd buy that even if it costs a lot. It would be an SFF dream cpu.

Yes!!! I've been extremely vocal for the past 6 years about how I desperately want AMD to create a powerhouse APU with at least 4GB of on-package HBM, but preferably at least 8GB, though 16GB would really be amazing.

Could you imagine what Strix Halo would be like if it had 16GB of integrated HBM3 in ADDITION to between 32GB - 128GB of soldered LPDDR5X at 7500+MT/s....it'd be crazy...and expensive.....unfortunately however, because AI has an insatiable thirst for HBM, it'll never be feasible to use it in a consumer product for the foreseeable future.

 

[igormp]

Upgrading their side before the big push for Windows as a service?

Those Azure instances are often used for HPC, were memory bandwidth is a major bottleneck. The previous iteration (v4) used Genoa-X for that extra L3 cache that does wonders for such workloads.

ServeTheHome has actually posted about this:

This is the Microsoft Azure HBv5 and AMD MI300C

At SC24 we saw the custom 96 core AMD MI300C HBM3 CPU and four-socket system powering the Microsoft Azure HBv5 instances

www.servethehome.com

MI300C CPU, basically a MI300A with the GPU die replaced with more cores, for a total of 96 cores (with SMT disabled since that's better for HPC).

So neither that Azure service nor that CPU will end up being directly used by mainstream consumers in any way.

FWIW, Intel has a similar product with their Xeon MAX lineup, albeit this one has way less HBM memory than the CPU at hand.

 

[mrnagant]

Companies like MS are so big now that they can dictate CPU architectures now and not just have to buy what’s on the shelf.

And that’s two custom processors for MS from AMD: this one and the one in the Xbox. It seems weird that MS went with Qualcomm instead of another AMD custom chip in their Surface series.

Xbox One has two AMD custom chips, and the Xbox Series is two custom chips. The Steamdeck chip was initially a custom chip for Microsoft.

 

[hsew]

Can’t wait until THESE wind up on eBay!

 

[thesmokingman]

This reads like its going to be a monster.

 

[Patriot]

Companies like MS are so big now that they can dictate CPU architectures now and not just have to buy what’s on the shelf.

And that’s two custom processors for MS from AMD: this one and the one in the Xbox. It seems weird that MS went with Qualcomm instead of another AMD custom chip in their Surface series.

It's because of separation of divisions inside MS, they used amd Zen+ 4c and Zen2 8c at 15w and they were less than stellar with the vega igp. I imagine that left a sour taste in their mouth.

 

[Kaotik]

It's not really custom, it's MI300C that never got released. It's exclusive to MS though.

 

[Visible Noise]

AMD CPU’s, using Nvidia interconnects. Says nothing good about AMD interconnect.

 

[Patriot]

AMD CPU’s, using Nvidia interconnects. Says nothing good about AMD interconnect.

These cpus are using infinity fabric, the networking external to the servers are Mellanox, an NVidia acquired company.
The Mellanox magic and currently the only 800gbit InfiniBand on the market.

AMD is part of the ultra eth consortium but the first products are not quite out yet and MS has its own nics. Odd choice but not unexpected to just use some Mellanox IB.

 

[Visible Noise]

These cpus are using infinity fabric, the networking external to the servers are Mellanox, an NVidia acquired company.
The Mellanox magic and currently the only 800gbit InfiniBand on the market.

AMD is part of the ultra eth consortium but the first products are not quite out yet and MS has its own nics. Odd choice but not unexpected to just use some Mellanox IB.

Read the the Microsoft post, not the summary here. Anything that goes off chip is over the Nvidia interconnect. It’s literally part of the socket.

 

[Aquinus]

Read the the Microsoft post, not the summary here. Anything that goes off chip is over the Nvidia interconnect. It’s literally part of the socket.

I would suggest what @Patriot said. The only mention about Infiniband is:

• 800 Gb/s of NVIDIA Quantum-2 InfiniBand, balanced as 200 Gb/s per CPU SoC

That does not mean that the interconnect between the CPUs is using Infiniband. It's saying only 200Gb/s is allocated to each CPU. For what it's worth, I was unable (with a quick Google search,) to find any implementation of Infiniband that isn't between servers. None of it is on a single PCB. On top of that, it also said:

• 2X total Infinity Fabric bandwidth among CPUs as any AMD EPYC™ server platform to date

If you're correct, then why is Infinity Fabric even implemented? Oh wait, it's for CPU-CPU communication... like every other EPYC CPU.

 

[Patriot]

the the Microsoft post, not the summary here. Anything that goes off chip is over the Nvidia interconnect. It’s literally part of the socket.

Sorry bud, the socket to socket is infinity fabric, this is a very traditional mi300A server with cpu only chiplets instead of a mix.
Each APU (now just cpu) has 16 gen5 lanes for a nic, so that there is symmetric loading and scaling.
Hence each socket gets its own nic. They don't talk to each other over that nic, they have their own fabric.

You are reading things you do not understand and arguing with those that do lol.
Try another article that might enhance your understanding of the system.

This is the Microsoft Azure HBv5 and AMD MI300C

At SC24 we saw the custom 96 core AMD MI300C HBM3 CPU and four-socket system powering the Microsoft Azure HBv5 instances

www.servethehome.com