NVIDIA Supercharges Hopper, the World's Leading AI Computing Platform

[GFreeman]

NVIDIA today announced it has supercharged the world's leading AI computing platform with the introduction of the NVIDIA HGX H200. Based on NVIDIA Hopper architecture, the platform features the NVIDIA H200 Tensor Core GPU with advanced memory to handle massive amounts of data for generative AI and high performance computing workloads.

The NVIDIA H200 is the first GPU to offer HBM3e - faster, larger memory to fuel the acceleration of generative AI and large language models, while advancing scientific computing for HPC workloads. With HBM3e, the NVIDIA H200 delivers 141 GB of memory at 4.8 terabytes per second, nearly double the capacity and 2.4x more bandwidth compared with its predecessor, the NVIDIA A100. H200-powered systems from the world's leading server manufacturers and cloud service providers are expected to begin shipping in the second quarter of 2024.



"To create intelligence with generative AI and HPC applications, vast amounts of data must be efficiently processed at high speed using large, fast GPU memory," said Ian Buck, vice president of hyperscale and HPC at NVIDIA. "With NVIDIA H200, the industry's leading end-to-end AI supercomputing platform just got faster to solve some of the world's most important challenges."

Perpetual Innovation, Perpetual Performance Leaps
The NVIDIA Hopper architecture delivers an unprecedented performance leap over its predecessor and continues to raise the bar through ongoing software enhancements with H100, including the recent release of powerful open-source libraries like NVIDIA TensorRT -LLM.

The introduction of H200 will lead to further performance leaps, including nearly doubling inference speed on Llama 2, a 70 billion-parameter LLM, compared to the H100. Additional performance leadership and improvements with H200 are expected with future software updates.

NVIDIA H200 Form Factors
NVIDIA H200 will be available in NVIDIA HGX H200 server boards with four- and eight-way configurations, which are compatible with both the hardware and software of HGX H100 systems. It is also available in the NVIDIA GH200 Grace Hopper Superchip with HBM3e, announced in August.

With these options, H200 can be deployed in every type of data center, including on premises, cloud, hybrid-cloud and edge. NVIDIA's global ecosystem of partner server makers - including ASRock Rack, ASUS, Dell Technologies, Eviden, GIGABYTE, Hewlett Packard Enterprise, Ingrasys, Lenovo, QCT, Supermicro, Wistron and Wiwynn - can update their existing systems with an H200.

Amazon Web Services, Google Cloud, Microsoft Azure and Oracle Cloud Infrastructure will be among the first cloud service providers to deploy H200-based instances starting next year, in addition to CoreWeave, Lambda and Vultr.

Powered by NVIDIA NVLink and NVSwitch high-speed interconnects, HGX H200 provides the highest performance on various application workloads, including LLM training and inference for the largest models beyond 175 billion parameters.

An eight-way HGX H200 provides over 32 petaflops of FP8 deep learning compute and 1.1 TB of aggregate high-bandwidth memory for the highest performance in generative AI and HPC applications.

When paired with NVIDIA Grace CPUs with an ultra-fast NVLink-C2C interconnect, the H200 creates the GH200 Grace Hopper Superchip with HBM3e - an integrated module designed to serve giant-scale HPC and AI applications.

Accelerate AI With NVIDIA Full-Stack Software
NVIDIA's accelerated computing platform is supported by powerful software tools that enable developers and enterprises to build and accelerate production-ready applications from AI to HPC. This includes the NVIDIA AI Enterprise suite of software for workloads such as speech, recommender systems and hyperscale inference.

Availability
The NVIDIA H200 will be available from global system manufacturers and cloud service providers starting in the second quarter of 2024.

View at TechPowerUp Main Site | Source

 

[AnarchoPrimitiv]

Nvidia's enterprise products are confusing...there's some that start with "L" like the L40, some that start with "A" like the A100, some that start with "H", some that start with "T", then it gets more confusing when you consider that an A40 has more cores than an A100, so you cant even assume that a larger model# equates to increased performance....can anyone quickly explain the difference between these? I'm having difficulty finding a guide online.

 

[seventy]

Nvidia's enterprise products are confusing...there's some that start with "L" like the L40, some that start with "A" like the A100, some that start with "H", some that start with "T", then it gets more confusing when you consider that an A40 has more cores than an A100, so you cant even assume that a larger model# equates to increased performance....can anyone quickly explain the difference between these? I'm having difficulty finding a guide online.

Letter is the architecture.
L for Lovelace
A for Ampere
H for Hopper
T for Turing

then it gets more confusing when you consider that an A40 has more cores than an A100

It follows the "bigger number = better" rule.
A100 is twice as fast, it's based on the cut GA100 chip (which only goes up to 8192) with HBM memory. A40 is full GA102 with GDDR.

 

[thesmokingman]

Reaction to AMD much... lol.

 

[Anymal]

Reaction to AMD much... lol.

What AMD?

 

[thesmokingman]

What AMD?

Preliminary data has AMD accelerators beating Nvidia in perf and specs. This release here is trying to close the specs gap before AMD hits the market.

 

[Anymal]

With what?