NVIDIA Grace CPU Specs Remind Us Why Intel Never Shared x86 with the Green Team

[btarunr]

NVIDIA designed the Grace CPU, a processor in the classical sense, to replace the Intel Xeon or AMD EPYC processors it was having to cram into its pre-built HPC compute servers for serial-processing roles, and mainly because those half-a-dozen GPU HPC processors need to be interconnected by a CPU. The company studied the CPU-level limitations and bottlenecks not just with I/O, but also the machine-architecture, and realized its compute servers need a CPU purpose-built for the role, with an architecture that's heavily optimized for NVIDIA's APIs. This, the NVIDIA Grace CPU was born.

This is NVIDIA's first outing with a CPU with a processing footprint rivaling server processors from Intel and AMD. Built on the TSMC N4 (4 nm EUV) silicon fabrication process, it is a monolithic chip that's deployed standalone with an H100 HPC processor on a single board that NVIDIA calls a "Superchip." A board with a Grace and an H100, makes up a "Grace Hopper" Superchip. A board with two Grace CPUs makes a Grace CPU Superchip. Each Grace CPU contains a 900 GB/s switching fabric, a coherent interface, which has seven times the bandwidth of PCI-Express 5.0 x16. This is key to connecting the companion H100 processor, or neighboring Superchips on the node, with coherent memory access.



Serial processing muscle on the NVIDIA Grace CPU is care of a 72-core Arm v9 64-bit CPU. A Superchip would contain 144 cores. The main memory interface is LPDDR5x, with each "socket" having a maximum memory bandwidth of 1 TB/s (or rivaling that of over 24 channels of DDR5). This includes ECC. A key serial-IO interface is PCI-Express Gen 5, with 68 lanes on offer. These are mainly to wire out NVMe storage devices. The chip has a TDP rating of 500 W peak.

The Grace CPU demonstrates the engineering muscle of NVIDIA at designing large multi-core processors for enterprise and HPC applications. With Arm achieving near-parity with x86-64 in performance, efficiency, and IPC, we're beginning to understand why NVIDIA couldn't become an x86 licensee. It would have achieved a winning enterprise processor rivaling Intel's much before. Future generations of NVIDIA's DGX compute nodes, as well as pre-built workstations and servers, spanning a multitude of applications, could see NVIDIA wean away from x86-based CPUs, replacing them with Grace and its successors.

View at TechPowerUp Main Site | Source

 

[Daven]

This is why Intel can never go home again. The total addressable market (TAM) spanning tablet, laptop, desktop and server has too many players for there ever to be one dominant company. At its peak, Intel commanded over 90% of the TAM. Now the company is destined to drop well below 50% and we the customers will be the ultimate winners. Go competition! Die too big too fail or fall!

 

[Assimilator]

68 PCIe 5.0 lanes while ADL offers 16 and yet-to-be-launched Zen 4 can only manage a meagre 24...

edit:

My point => .












The heads of all the people telling me that this is a completely different market segment, which I am well aware of: O O O

 

[Daven]

68 PCIe 5.0 lanes while ADL offers 16 and yet-to-be-launched Zen 4 can only manage a meagre 24...

This is an HPC CPU not a desktop CPU. You need to compare to Epyc and Xeon. Epyc Zen 4 will have over 128 PCIe lanes. But I’m not sure about Sapphire Rapids.

edit: ok it looks like sapphire rapids will have over 80 PCIe lanes so Nvidia’s solution is behind on this spec.

 

[ncrs]

68 PCIe 5.0 lanes while ADL offers 16 and yet-to-be-launched Zen 4 can only manage a meagre 24...

You're comparing specialized server chips to desktops
Zen 4 EPYC will have 128 PCIe 5.0 lanes if you want apples-to-apples.

 

[Tom Yum]

68 PCIe 5.0 lanes while ADL offers 16 and yet-to-be-launched Zen 4 can only manage a meagre 24...

This doesn't compete with ADL or Zen 3/4, this competes with Epyc which has 128 PCIE lanes at PCIe 5 (in Genoa form)....

 

[HenrySomeone]

Look at all the redsters immediately ready to defend the honor of their company, lmao!

 

[ncrs]

Look at all the redsters immediately ready to defend the honor of their company, lmao!

There's the Intel Defense Forces!!111111one


And yes, the above is a joke.

 

[Daven]

The heads of all the people telling me that this is a completely different market segment, which I am well aware of: O O O

Then your comment really doesn’t make sense unless you were joking or being sarcastic.

Anyway, an Nvidia desktop CPU would also be welcome. Of course it won’t have or need as many PCIe lanes as 68 but more choices are always nice.

 

[bug]

68 PCIe 5.0 lanes while ADL offers 16 and yet-to-be-launched Zen 4 can only manage a meagre 24...

And that's a good thing. All that bandwidth would be wasted on a desktop PC, while adding significantly to the cost of the CPU and the motherboard

 

[dj-electric]

Sign for things to come.
NVIDIA is fully invested in x86 replacement chips based on ARM - on all segments.

 

[bug]

Sign for things to come.
NVIDIA is fully invested in x86 replacement chips based on ARM - on all segments.

It depends. Right now things move towards ARM, but RISC-V is also up-and-coming, which is royalty-free.

 

[trsttte]

The Grace CPU demonstrates the engineering muscle of NVIDIA at designing large multi-core processors for enterprise and HPC applications. With Arm achieving near-parity with x86-64 in performance, efficiency, and IPC, we're beginning to understand why NVIDIA couldn't become an x86 licensee. It would have achieved a winning enterprise processor rivaling Intel's much before. Future generations of NVIDIA's DGX compute nodes, as well as pre-built workstations and servers, spanning a multitude of applications, could see NVIDIA wean away from x86-based CPUs, replacing them with Grace and its successors.

I don't think it as simple as that but the general idea is absolutely true, Intel and AMD should not have been allowed to lock down the PC market into a duopoly. ARM support still has a lot to grow and it needs the standardization that make x86 able to boot anything, but otherwise will be a great thing to displace the current duopoly where when either Intel or AMD fall behind the market stagnates.

 

[bug]

I don't think it as simple as that but the general idea is absolutely true, Intel and AMD should not have been allowed to lock down the PC market into a duopoly. ARM support still has a lot to grow and it needs the standardization that make x86 able to boot anything, but otherwise will be a great thing to displace the current duopoly where when either Intel or AMD fall behind the market stagnates.

How would you have prevented that? At some point we also had Cyrix, VIA and many others building x86 chips. They either couldn't compete or decided to shift to something else. Would you have forced them to stay in the business instead?

 

[TechLurker]

This is probably why NVIDIA wanted ARM so badly; they could have made a stronger takeover of the general ARM space with biased deals offering their design over other competitors. Instead, they now have to innovate to complete alongside other ARM licensees. As an aside, their desire to homogenize their HPC systems reminds me of Apple and their walled garden; looking to also replace Intel out of their systems (if they haven't already), although they still use Radeon GPUs here and there.

That said, it looks like the competition will really be between NVIDIA and AMD. AMD added Xlinx to their portfolio, also own an ARM license, and are jointly working with Samsung to integrate RDNA with elements of ARM (via Exynos), which would help them combat NVIDIA across all platforms too. This is assuming NVIDIA also ports elements of this CPU down into their next-gen gaming tablets (and the next-gen Switch, assuming Nintendo sticks with NVIDIA), and even some gaming laptops running either Steam OS or Windows ARM.

Meanwhile, Intel, despite all their recent acquisitions, haven't really gotten anything to show for it, aside from Foveros, and it'll be awhile longer before their own compute and gaming GPUs can prove reliable enough in the high-value markets. Kind of wild to see such a dramatic shift the last 5 years.

 

[Nanochip]

Perhaps nVidia never wanted to become an x86 licensee, given that its primary business used to be GPUs... Also, using ARM, if you can scale it to x86 peformance levels like the Ampere Altra 128-core chip is a great way to save on power ($$).

 

[AnotherReader]

I'm not sure Nvidia would have provided better competition than AMD. Their previous CPUs, despite some innovative ideas, were lackluster in both performance and performance per watt. That is why they are using ARM's designs now. This is a highly specialized CPU and would be totally unsuited to tasks that a Xeon or Epyc would do. Don't be impressed by specint_rate; it's highly correlated to memory bandwidth and has little correlation to most tasks that server CPUs are used for. That being said, I suspect it's very suitable to the task that it has been designed to do; feed the GPU and augment its memory capacity by virtue of a high speed CPU <--> GPU link.

 

[bug]

Perhaps nVidia never wanted to become an x86 licensee, given that its primary business used to be GPUs... Also, using ARM, if you can scale it to x86 peformance levels like the Ampere Altra 128-core chip is a great way to save on power ($$).

Idk if Nvidia ever had a business case for x86 CPUs, but they did build chipsets for both AMD and Intel. Pretty good ones, too. Plus, regardless of your primary business, you still want a series of secondaries to fall back to.

 

[john_]

NVIDIA Grace CPU Specs Remind Us Why Intel Never Shared x86 with the Green Team

I think this was on everyone's mind who knew that Intel rejected giving a license to Nvidia.

Nvidia did a mistake to not REALLY concentrate on ARM sooner and produce products like Grace long ago. Not this kind of huge processors for servers from the beginning maybe, but SOCs for laptops and desktops, or if not desktops, at least mini PCs, running Windows on ARM, or Linux, or Android, or all of them. Qualcomm is a sleeping, boring, failure in that area.

They'll probably start accelerating in the ARM platform now. They lost time waiting to see if they can first have the absolute control of ARM. No one wanted them, so it's good to see that their pride and arrogance - which is part of their business mentality, sometimes helps them, mosts times, it doesn't - is not becoming an obstacle to their plans to start developing CPUs also.

As much as Intel needs GPUs for it's future, the same Nvidia needs CPUs for it's future. We all saw what happened to Nvidia's financials this quarter, because they only stand on one foot. GPUs. Hit that foot and the whole company trembles.

 

[ncrs]

I think this was on everyone's mind who knew that Intel rejected giving a license to Nvidia.

Nvidia did a mistake to not REALLY concentrate on ARM sooner and produce products like Grace long ago. Not this kind of huge processors for servers from the beginning maybe, but SOCs for laptops and desktops, or if not desktops, at least mini PCs, running Windows on ARM, or Linux, or Android, or all of them. Qualcomm is a sleeping, boring, failure in that area.

They'll probably start accelerating in the ARM platform now. They lost time waiting to see if they can first have the absolute control of ARM. No one wanted them, so it's good to see that their pride and arrogance - which is part of their business mentality, sometimes helps them, mosts times, it doesn't - is not becoming an obstacle to their plans to start developing CPUs also.

As much as Intel needs GPUs for it's future, the same Nvidia needs CPUs for it's future. We all saw what happened to Nvidia's financials this quarter, because they only stand on one foot. GPUs. Hit that foot and the whole company trembles.

NVIDIA has a long history with ARM. Looks like you're missing the fact that they did build multiple ARM SoCs and even designed their own cores instead of licensing ARM designs for some generations

 

[Assimilator]

I'm not sure Nvidia would have provided better competition than AMD. Their previous CPUs, despite some innovative ideas, were lackluster in both performance and performance per watt. That is why they are using ARM's designs now. This is a highly specialized CPU and would be totally unsuited to tasks that a Xeon or Epyc would do. Don't be impressed by specint_rate; it's highly correlated to memory bandwidth and has little correlation to most tasks that server CPUs are used for. That being said, I suspect it's very suitable to the task that it has been designed to do; feed the GPU and augment its memory capacity by virtue of a high speed CPU <--> GPU link.

It's just btarunr making up clickbait as usual, ignore him, that's basically his job here.

Idk if Nvidia ever had a business case for x86 CPUs, but they did build chipsets for both AMD and Intel. Pretty good ones, too. Plus, regardless of your primary business, you still want a series of secondaries to fall back to.

Of course they had a business case, it's called making money. Even though Arm CPUs are in everything and the kitchen sink nowadays, I'd wager the x86 market cap still exceeds every other CPU architecture type combined.

NVIDIA has a long history with ARM. Looks like you're missing the fact that they did build multiple ARM SoCs and even designed their own cores instead of licensing ARM designs for some generations

Yeah, I don't know WTF he's talking about... after NVIDIA realised they wouldn't get an x86 license, they went all-in on Arm and at this point have been producing highly specialised Arm CPUs of various flavours for 14 years.

 

[Aquinus]

it is a monolithic chip

You'd think that they would have learned by now that MCM is the future for high density computing.

The chip has a TDP rating of 500 W peak.

Typical nVidia these days. Lame. Might have gone better with a non-monolithic solution.

 

[Fouquin]

Perhaps nVidia never wanted to become an x86 licensee, given that its primary business used to be GPUs... Also, using ARM, if you can scale it to x86 peformance levels like the Ampere Altra 128-core chip is a great way to save on power ($$).

nVidia bought Transmeta specifically to get their x86 license. They were shot down by Intel when they tried to use it.

 

[john_]

NVIDIA has a long history with ARM. Looks like you're missing the fact that they did build multiple ARM SoCs and even designed their own cores instead of licensing ARM designs for some generations

I am not. I am talking about mass production targeting markets with a huge user base. Other than Switch, that probably just happened, Nvidia was using ARM for very specific applications.
I haven't miss something.

Yeah, I don't know WTF he's talking about... after NVIDIA realised they wouldn't get an x86 license, they went all-in on Arm and at this point have been producing highly specialised Arm CPUs of various flavours for 14 years.

All in on ARM? Really? Tell me some products that where made for mass production and availability to the general public. Except the obvious mention to Switch, what else is there? Shield tablet?

You both missed my point.

It's just btarunr making up clickbait as usual, ignore him, that's basically his job here.

No, he is right. Nvidia had always better vision than AMD, it was more ambitious, and was in better position to utilize the hardware it was making, thanks to it's software and better promote it's products thanks to the much more aggressive marketing.
The only time in history when AMD did a bold move, that eventually saved it, was when it bought ATI.

 

[ncrs]

I am not. I am talking about mass production targeting markets with a huge user base. Other than Switch, that probably just happened, Nvidia was using ARM for very specific applications.
I haven't miss something.

All in on ARM? Really? Tell me some products that where made for mass production and availability to the general public. Except the obvious mention to Switch, what else is there? Shield tablet?

You both missed my point.

The article I linked has lists of products using their chips. Not only from lesser-known OEMs, but from HTC, Motorola, LG, Samsung, Acer, Sony, Dell, Toshiba, ASUS, Microsoft, Google, Xiaomi, Lenovo and Tesla. Huge ranges of products as well.