To enable professionals worldwide to build and run AI applications right from their desktops, NVIDIA and AMD are powering a new line of workstations equipped with NVIDIA RTX Ada Generation GPUs and AMD Ryzen Threadripper PRO 7000 WX-Series CPUs. Bringing together the highest levels of AI computing, rendering and simulation capabilities, these new platforms enable professionals to efficiently tackle the most resource-intensive, large-scale AI workflows locally.

Bringing AI Innovation to the Desktop
Advanced AI tasks typically require data-center-level performance. Training a large language model with a trillion parameters, for example, takes thousands of GPUs running for weeks, though research is underway to reduce model size and enable model training on smaller systems while still maintaining high levels of AI model accuracy. The new NVIDIA RTX GPU and AMD CPU-powered AI workstations provide the power and performance required for training such smaller models, as well as local fine-tuning, and helping to offload data center and cloud resources for AI development tasks. The devices let users select single- or multi-GPU configurations as required for their workloads.

In an event at the White House today, NVIDIA announced support for voluntary commitments that the Biden Administration developed to ensure advanced AI systems are safe, secure and trustworthy. The news came the same day NVIDIA's chief scientist, Bill Dally, testified before a U.S. Senate subcommittee seeking input on potential legislation covering generative AI. Separately, NVIDIA founder and CEO Jensen Huang will join other industry leaders in a closed-door meeting on AI Wednesday with the full Senate.

Seven companies including Adobe, IBM, Palantir and Salesforce joined NVIDIA in supporting the eight agreements the Biden-Harris administration released in July with support from Amazon, Anthropic, Google, Inflection, Meta, Microsoft and OpenAI.

In its debut on the MLPerf industry benchmarks, the NVIDIA GH200 Grace Hopper Superchip ran all data center inference tests, extending the leading performance of NVIDIA H100 Tensor Core GPUs. The overall results showed the exceptional performance and versatility of the NVIDIA AI platform from the cloud to the network's edge. Separately, NVIDIA announced inference software that will give users leaps in performance, energy efficiency and total cost of ownership.

GH200 Superchips Shine in MLPerf
The GH200 links a Hopper GPU with a Grace CPU in one superchip. The combination provides more memory, bandwidth and the ability to automatically shift power between the CPU and GPU to optimize performance. Separately, NVIDIA HGX H100 systems that pack eight H100 GPUs delivered the highest throughput on every MLPerf Inference test in this round. Grace Hopper Superchips and H100 GPUs led across all MLPerf's data center tests, including inference for computer vision, speech recognition and medical imaging, in addition to the more demanding use cases of recommendation systems and the large language models (LLMs) used in generative AI.

Underscoring NVIDIA's growing relationship with the global technology superpower, Indian Prime Minister Narendra Modi met with NVIDIA founder and CEO Jensen Huang Monday evening. The meeting at 7 Lok Kalyan Marg—as the Prime Minister's official residence in New Delhi is known—comes as Modi prepares to host a gathering of leaders from the G20 group of the world's largest economies, including U.S. President Joe Biden, later this week.

"Had an excellent meeting with Mr. Jensen Huang, the CEO of NVIDIA," Modi said in a social media post. "We talked at length about the rich potential India offers in the world of AI." The event marks the second meeting between Modi and Huang, highlighting NVIDIA's role in the country's fast-growing technology industry.

NVIDIA's latest financial report for FY2Q24 reveals that its data center business reached US$10.32 billion—a QoQ growth of 141% and YoY increase of 171%. The company remains optimistic about its future growth. TrendForce believes that the primary driver behind NVIDIA's robust revenue growth stems from its data center's AI server-related solutions. Key products include AI-accelerated GPUs and AI server HGX reference architecture, which serve as the foundational AI infrastructure for large data centers.

TrendForce further anticipates that NVIDIA will integrate its software and hardware resources. Utilizing a refined approach, NVIDIA will align its high-end, mid-tier, and entry-level GPU AI accelerator chips with various ODMs and OEMs, establishing a collaborative system certification model. Beyond accelerating the deployment of CSP cloud AI server infrastructures, NVIDIA is also partnering with entities like VMware on solutions including the Private AI Foundation. This strategy extends NVIDIA's reach into the edge enterprise AI server market, underpinning steady growth in its data center business for the next two years.

PNY Technologies today announced it is now offering the latest NVIDIA RTX Ada Generation GPUs - the NVIDIA RTX 5000, NVIDIA RTX 4500 and NVIDIA RTX 4000 high-performance workstation graphics cards and the NVIDIA L40S GPU for data centers. These new GPUs are now available to order from PNY.

Joining the NVIDIA RTX 6000 Ada Generation and NVIDIA RTX 4000 SFF Ada Generation, the NVIDIA RTX 5000, NVIDIA RTX 4500 and NVIDIA RTX 4000 high-performance GPUs are based on the powerful and ultra-efficient NVIDIA Ada Lovelace architecture, making them ideal for real-time ray tracing, physically accurate simulation, neural graphics, and generative AI. These GPUs combine the latest-gen RT Cores, Tensor Cores, and CUDA cores with large GPU memory to offer unprecedented performance for creators and professionals, empowering them to unleash their imagination while maximizing productivity. Turnkey HW + Sync bundles are also available (NVIDIA RTX 5000 + HW Sync, NVIDIA RTX 4500 + HW Sync, NVIDIA RTX 4000 + HW Sync).

NVIDIA and global manufacturers today announced powerful new NVIDIA RTX workstations designed for development and content creation in the age of generative AI and digitalization. The systems, including those from BOXX, Dell Technologies, HP and Lenovo, are based on NVIDIA RTX 6000 Ada Generation GPUs and incorporate NVIDIA AI Enterprise and NVIDIA Omniverse Enterprise software.

Separately, NVIDIA also released three new desktop workstation Ada Generation GPUs - the NVIDIA RTX 5000, RTX 4500 and RTX 4000 - to deliver the latest AI, graphics and real-time rendering technology to professionals worldwide. "Few workloads are as challenging as generative AI and digitalization applications, which require a full-stack approach to computing," said Bob Pette, vice president of professional visualization at NVIDIA. "Professionals can now tackle these on a desktop with the latest NVIDIA-powered RTX workstations, enabling them to build vast, digitalized worlds in the new age of generative AI."

NVIDIA GeForce GTX 1650 was released more than four years ago. With its TU117 graphics processor, it features 896 CUDA cores, 56 texture mapping units, and 32 ROPs. NVIDIA has paired 4 GB GDDR5 memory with the GeForce GTX 1650, which are connected using a 128-bit memory interface. Interestingly, according to the latest Steam Hardware Survey results, this GPU still remains the most popular choice among gamers. While the total addressable market is unknown with the exact number, it is fair to assume that a large group participates every month. The latest numbers for June 2023 indicate that the GeForce GTX 1650 is still the number one GPU, with 5.50% of the users having that GPU. The second closest one was GeForce RTX 3060, with 4.60%.

Other information in the survey remains similar, with CPUs mostly ranging from 2.3 GHz to 2.69 GHz in frequency and with six cores and twelve threads. Storage also recorded a small bump with capacity over 1 TB surging 1.48%, indicating that gamers are buying larger drives as game sizes get bigger.

NVIDIA's H100 Hopper GPU is a device designed for pure AI and other compute workloads, with the least amount of consideration for gaming workloads that involve graphics processing. However, it is still interesting to see how this 30,000 USD GPU fairs in comparison to other gaming GPUs and whether it is even possible to run games on it. It turns out that it is technically feasible but not making much sense, as the Chinese YouTube channel Geekerwan notes. Based on the GH100 GPU SKU with 14,592 CUDA, the H100 PCIe version tested here can achieve 204.9 TeraFLOPS at FP16, 51.22 TeraFLOPS at FP32, and 25.61 TeraFLOPS at FP64, with its natural power laying in accelerating AI workloads.

However, how does it fare in gaming benchmarks? Not very well, as the testing shows. It scored 2681 points in 3DMark Time Spy, which is lower than AMD's integrated Radeon 680M, which managed to score 2710 points. Interestingly, the GH100 has only 24 ROPs (render output units), while the gaming-oriented GA102 (highest-end gaming GPU SKU) has 112 ROPs. This is self-explanatory and provides a clear picture as to why the H100 GPU is used for computing only. Since it doesn't have any display outputs, the system needed another regular GPU to provide the picture, while the computation happened on the H100 GPU.

GIGABYTE TECHNOLOGY Co. Ltd, a leading manufacturer of premium gaming hardware, today launched the top-grade water-cooled external graphics - AORUS RTX 4090 GAMING BOX. AORUS RTX 4090 GAMING BOX is equipped with the most powerful NVIDIA Ada Lovelace architecture - GeForce RTX 4090 graphics card and the Thunderbolt 3 high-speed transmission interface. It endows ultrabooks with 3D computational performance beyond imagination, transforming ultrabooks into game platforms with full ray tracing and becoming a reliable assistant for creators, creating an unprecedented work efficiency experience. In addition, the AORUS WATERFORCE Cooling System is the only solution that combines performance and comfort, allowing users to enjoy a quiet and comfortable environment while handling heavy work.

The AORUS RTX 4090 GAMING BOX is the top-of-the-line water-cooled external graphics box in the market. It enables users to enjoy top-level GeForce RTX 4090 performance with independent high-wattage and stable power supply, while enjoying a quiet and comfortable environment. AORUS has minimized the size of the GAMING BOX, taking up minimal desktop space, making it the ideal companion for ultrabooks.

Hardware tipster kopite7kimi has learned from insider sources that a variant of NVIDIA's GeForce RTX 4070 graphic card could be lined up with a different GPU - the AD103 instead of the currently utilized AD104-derived AD104-250-A1. The Ada Lovelace-based architecture is a staple across the RTX 40-series of graphics cards, but a fully unlocked AD103 is not yet attached to any product on the market - it will be a strange move for NVIDIA to refresh or expand the mid-range RTX 4070 lineup with a much larger GPU, albeit in a reduced form. A cut down variant of the AD103 is currently housed within NVIDIA's GeForce RTX 4080 graphics card - its AD103-300-A1 GPU has 9728 CUDA Cores and Team Green's engineers have chosen to disable 5% of the full article's capabilities.

The hardware boffins will need to do a lot of pruning if the larger GPU ends up on the rumored RTX 4070 sort-of upgrade - the SKU's 5,888 CUDA core count spec would require a 42% reduction in GPU potency. It is somewhat curious that the RTX 4070 Ti has not been mentioned by the tipster - you would think that the more powerful card (than the standard 4070) would be the logical and immediate candidate for this type of treatment. In theory NVIDIA could be re-purposing dies that do not meet RTX 4080-level standards, thus salvaging rejected material and repurposing it for step down card models.

NVIDIA's rumored RTX 5000 Ada Generation GPU has been outed once again, according to VideoCardz - the cited source being a keen-eyed member posting information dumps on a laptop discussion forum. Team Green has released new driver documentation that makes mention of hardware ID "26B2" under an entry for a now supported device: "NVIDIA RTX 5000 Ada Generation." Forum admin StefanG3D posted the small discovery on their favored forum in the small hours of Sunday morning (April 23).

As reported last month, the NVIDIA RTX 5000 Ada is destined to sit between existing sibling workstation GPUs - the AD102-based RTX 6000 and AD104-based RTX 4000 SFF. Hardware tipster kopite7kimi has learned enough to theorize that the NVIDIA RTX 5000 Ada Generation workstation graphics card will feature 15,360 CUDA cores and 32 GB of GDDR6 memory. The AD102 GPU is expected to sit at the heart of this unannounced card.

At the turn of the 1980s, most PC adventure games were played using only the keyboard. In those days, adventure games didn't use action menus like more modern games, but simply presented the player with a command line where they could freely input text to decide the actions that characters would take and proceed through the story. Free text input systems like these allowed players to feel a great deal of freedom. However, they did come with one common source of frustration: players knowing what action they wanted to perform but being unable to do so because they could not find the right wording. This problem was caused by the limitations of PC performance and NLP technology of the time.

40 years have passed since then, and PC performance has drastically improved, as have the capabilities of NLP technology. Using "The Portopia Serial Murder Case" as a test case, we'd like to show you the capabilities of modern NLP and the impact it can have on adventure games, as well as deepen your understanding of NLP technologies.

NVIDIA has begun selling its compact RTX 4000 Ada Lovelace graphics card, offering GeForce RTX 3070-like performance at a mere 70 W power consumption, allowing it to fit in almost all desktop PCs. The low-profile, dual-slot board is priced higher than the RTX 4080 as it targets professional users, but it can still be used in a regular gaming computer. PNY's RTX 4000 Ada generation graphics card is the first to reach consumer shelves, currently available for $1,444 at ShopBLT, a retailer known for obtaining hardware before its competitors. The card comes with four Mini-DisplayPort connectors, so an additional mDP-DP or mDP-HDMI adapter must be factored into the cost.

The NVIDIA RTX 4000 SFF Ada generation board features an AD104 GPU with 6,144 CUDA cores, 20 GB of GDDR6 ECC memory, and a 160-bit interface. With a fixed boost frequency floating around 1560 MHz to reduce overall board power consumption, the GPU is rated for just 70 Watts of power. To emphasize the efficiency, this card requires no external PCIe power connector, as all the juice is fed through the PCIe slot. The GA104 graphics processor in this configuration delivers a peak FP32 performance of 19.2 TFLOPS, comparable to the GeForce RTX 3070. The 20 GB of memory makes the card more valuable for professionals and AI researchers needing compact solutions. Although the card's performance is overshadowed by the recently launched GeForce RTX 4070, the RTX 4000 SFF Ada's professional drivers, support for professional software ISVs, and additional features make it a strong contender in the semi-professional market. Availability and pricing are expected to improve in the coming weeks as the card becomes more widely accessible.More images, along with specification table, follow.

AMD has published an exciting development for its Radeon Open Compute Ecosystem (ROCm) users today. Now, ROCm is coming to the Windows operating system, and the company has extended ROCm support for consumer graphics cards instead of only supporting professional-grade GPUs. This development milestone is essential for making AMD's GPU family more competent with NVIDIA and its CUDA-accelerated GPUs. For those unaware, AMD ROCm is a software stack designed for GPU programming. Similarly to NVIDIA's CUDA, ROCm is designed for AMD GPUs and was historically limited to Linux-based OSes and GFX9, CDNA, and professional-grade RDNA GPUs.

However, according to documents obtained by Tom's Hardware (which are behind a login wall), AMD has brought support for ROCm to Radeon RX 6900 XT, Radeon RX 6600, and R9 Fury GPU. What is interesting is not the inclusion of RX 6900 XT and RX 6600 but the support for R9 Fury, an eight-year-old graphics card. Also, what is interesting is that out of these three GPUs, only R9 Fury has full ROCm support, the RX 6900 XT has HIP SDK support, and RX 6600 has only HIP runtime support. And to make matters even more complicated, the consumer-grade R9 Fury GPU has full ROCm support only on Linux and not Windows. The reason for this strange selection of support has yet to be discovered. However, it is a step in the right direction, as AMD has yet to enable more functionality on Windows and more consumer GPUs to compete with NVIDIA.

ADLINK Technology Inc., a global leader in edge computing, today launched Pocket AI - the first ever ultra-portable GPU accelerator to offer exceptional power at a cost-effective price point. With hardware and software compatibility, Pocket AI is the perfect tool to boost performance and productivity. It provides plug-and-play scalability from development to deployment for AI developers, professional graphics users and embedded industrial applications.

Pocket AI is a simple, reliable route to impressive GPU acceleration at a fraction of the cost of a laptop with equivalent GPU power. Its many benefits include a perfect power/performance balance from the NVIDIA RTX A500 GPU; high functionality driven by NVIDIA CUDA X and accelerated libraries; quick, easy delivery/power via Thunderbolt 3 interface and USB PD; and compatibility supported by NVIDIA developer tools. For the ultimate portability, the Pocket AI is compact and lightweight - est. 106 x 72 x 25 mm and 250 grams.

NVIDIA has killed SLI on its graphics cards, disabling the possibility of connecting two or more GPUs to harness their power for gaming and other workloads. However, SLI is making a reincarnation today in the form of a new H100 GPU model that spots higher memory capacity and higher performance. Called the H100 NVL, the GPU is a unique edition design based on the regular H100 PCIe version. What makes the H100 HVL version so special is the boost in memory capacity, now up from 80 GB in the standard model to 94 GB in the NVL edition SKU, for a total of 188 GB of HMB3 memory, running on a 6144-bit bus. Being a special edition SKU, it is sold only in pairs, as these H100 NVL GPUs are paired together and are connected by three NVLink connectors on top. Installation requires two PCIe slots, separated by dual-slot spacing.

The performance differences between the H100 PCIe version and the H100 SXM version are now matched with the new H100 NVL, as the card features a boost in the TDP with up to 400 Watts per card, which is configurable. The H100 NVL uses the same Tensor and CUDA core configuration as the SXM edition, except it is placed on a PCIe slot and connected to another card. Being sold in pairs, OEMs can outfit their systems with either two or four pairs per certified system. You can see the specification table below, with information filled out by AnandTech. As NVIDIA says, the need for this special edition SKU is the emergence of Large Language Models (LLMs) that require significant computational power to run. "Servers equipped with H100 NVL GPUs increase GPT-175B model performance up to 12X over NVIDIA DGX A100 systems while maintaining low latency in power-constrained data center environments," noted the company.

ASUS today announced its participation in NVIDIA GTC, a developer conference for the era of AI and the metaverse. ASUS will offer comprehensive NVIDIA-certified server solutions that support the latest NVIDIA L4 Tensor Core GPU—which accelerates real-time video AI and generative AI—as well as the NVIDIA BlueField -3 DPU, igniting unprecedented innovation for supercomputing infrastructure. ASUS will also launch the new ProArt Studiobook Pro 16 OLED laptop with the NVIDIA RTX 3000 Ada Generation Laptop GPU for mobile creative professionals.

Purpose-built GPU servers for generative AI
Generative AI applications enable businesses to develop better products and services, and deliver original content tailored to the unique needs of customers and audiences. ASUS ESC8000 and ESC4000 are fully certified NVIDIA servers that support up to eight NVIDIA L4 Tensor Core GPUs, which deliver universal acceleration and energy efficiency for AI with up to 2.7X more generative AI performance than the previous GPU generation. ASUS ESC and RS series servers are engineered for HPC workloads, with support for the NVIDIA Bluefield-3 DPU to transform data center infrastructure, as well as NVIDIA AI Enterprise applications for streamlined AI workflows and deployment.

NVIDIA today announced six new NVIDIA RTX Ada Lovelace architecture GPUs for laptops and desktops, which enable creators, engineers and data scientists to meet the demands of the new era of AI, design and the metaverse. Using the new NVIDIA RTX GPUs with NVIDIA Omniverse, a platform for building and operating metaverse applications, designers can simulate a concept before making it a reality, planners can visualize an entire factory before it is built and engineers can evaluate their designs in real time.

The NVIDIA RTX 5000, RTX 4000, RTX 3500, RTX 3000 and RTX 2000 Ada Generation laptop GPUs deliver breakthrough performance and up to 2x the efficiency of the previous generation to tackle the most demanding workflows. For the desktop, the NVIDIA RTX 4000 Small Form Factor (SFF) Ada Generation GPU features new RT Cores, Tensor Cores and CUDA cores with 20 GB of graphics memory to deliver incredible performance in a compact card.

Microsoft, Tencent and Baidu are adopting NVIDIA CV-CUDA for computer vision AI. NVIDIA CEO Jensen Huang highlighted work in content understanding, visual search and deep learning Tuesday as he announced the beta release for NVIDIA's CV-CUDA—an open-source, GPU-accelerated library for computer vision at cloud scale. "Eighty percent of internet traffic is video, user-generated video content is driving significant growth and consuming massive amounts of power," said Huang in his keynote at NVIDIA's GTC technology conference. "We should accelerate all video processing and reclaim the power."

CV-CUDA promises to help companies across the world build and scale end-to-end, AI-based computer vision and image processing pipelines on GPUs. The majority of internet traffic is video and image data, driving incredible scale in applications such as content creation, visual search and recommendation, and mapping. These applications use a specialized, recurring set of computer vision and image-processing algorithms to process image and video data before and after they're processed by neural networks.

Benchmarks and specifications of an alleged NVIDIA AD106 GPU have tipped up on Chiphell, although the original poster has since removed all the details. Thanks to @harukaze5719 on Twitter, who posted the details, we still get an insight into what we might be able to expect from NVIDIA's upcoming mid-range cards. All these details should be taken as is, as the original source isn't exactly what we'd call trustworthy. Based on the data in the TPU GPU database, the GPU in question should be the GeForce RTX 4070 Mobile with much higher clock speeds or an equivalent desktop part that offers more CUDA cores than the RTX 4060 Ti. Whatever the specific AD106 GPU is, it's being compared to the GeForce RTX 2080 Super and the RTX 3070 Ti.

The GPU was tested in AIDA64 and 3DMark and it beats the RTX 2080 Super in all of the tests, while drawing some 55 W less power at the same time. In some of the benchmarks the wins are within the margin of testing error, for example when it comes to the memory performance in AIDA64. However, we're looking at a GPU connected to only half the memory bandwidth here, as the AD106 GPU only has a 128-bit memory bus, compared to 256-bit for the RTX 2080 Super, although the memory clocks are much higher, but the overall memory bandwidth is still nearly 36 percent higher in the RTX 2080 Super. Yet, the AD106 GPU manages to beat the RTX 2080 Super in all of the memory benchmarks in AIDA64.

VideoCardz has obtained images and specifications of PNY's two upcoming GeForce RTX 4070 Ti models. According to the latest leak, these GPUs are equipped with 7680 CUDA cores and 12 GB of GDDR6X memory. This configuration resembles the canceled GeForce RTX 4080 12 GB edition card, which confirms that NVIDIA will rebrand it under the RTX 4070 Ti naming scheme. PNY has prepared GeForce RTX 4070 Ti XLR8 VERTO and VERTO GPUs, with the difference in cooler design and applied factory overclocking. The XLR8 version bears the same cooler as the RTX 4080 XRL8 card, adapted for the RTX 4070 Ti GPU SKU. This design should naturally offer greater overclocking performance than the regular VERTO SKU.

The render below looks like the 16-pin 12VHPWR power connector remains on these cards and that PNY has not swapped it for another solution. We expect to hear more about these cards on January 5th, when NVIDIA plans to launch.

While NVIDIA has launched high-end GeForce RTX 4090 and RTX 4080 GPUs from its Ada Lovelace family, middle and lower-end products are brewing to satisfy the entire consumer market. Today, according to the kopite7kimi, a well-known leaker, we have potential information about the configuration of the upcoming GeForce RTX 4060 Ti graphics card. Featuring 4352 FP32 CUDA cores, the GPU is powered by an AD106-350-A1 die. On the die, there is 32 MB of L2 cache. To pair, it has 8 GB of GDDR6 18 Gbps memory, which should be enough to power games at 1440p resolution, which this card is aiming for.

The design of the cards reference PG190 PCB is supposedly very short, making it ideal for ITX-sized designs we could see from NVIDIA's AIB partners. Interestingly, with a TDP of 220 Watts, the reference card is powered by the infamous 16-pin 12VHPWR connector, capable of supplying 600 Watts of power. This choice of connector is unclear; however, it could be NVIDIA's push to standardize its usage across all products in the Ada Lovelace family stack. While the card should not need the full potential of the connector, it signals that the company could only be using this type of connector for all of its future designs.

According to the well-known hardware leaker @momomo_us, HP is preparing the launch of its next-generation Omen 17 gaming laptops. And with a new generation of chips coming to consumers, HP accidentally made some information about laptop SKUs public. Four models are listed, and they represent a combination of Intel's 13th-generation Raptor Lake mobile processors with NVIDIA's Ada Lovelace RTX 40 series graphics cards for the mobile/laptop sector. The four SKUs are: CM2007NQ/CM2005NQ with Core i7-13700HX & RTX 4060 8 GB; CM2001NQ with Core i7-13700HX & RTX 4070 8 GB; CK2007NQ/CK2004NQ with Core i7-13700HX & RTX 4080 12 GB; CK2001NQ with Core i7-13700HX & RTX 4090 16 GB.

The most exciting find here is the appearance of the xx90 series in the mobile/laptop form factor, which has not been the case before. The GeForce RTX 4090 laptop edition is supposedly equipped with 16 GB of VRAM, and the GPU SKU should be a cut-down version of AD102 GPU adjusted for power and clock constraints so it can run within a reasonable TDP. With NVIDIA seemingly giving its clients an RTX 4090 SKU option, we have to wait and see what the CUDA core counts are and how clocks scale in a more restricted laptop environment.

NVIDIA's high-performance computing hardware stack is now equipped with the top-of-the-line Hopper H100 GPU. It features 16896 or 14592 CUDA cores, developing if it comes in SXM5 of PCIe variant, with the former being more powerful. Both variants come with a 5120-bit interface, with the SXM5 version using HBM3 memory running at 3.0 Gbps speed and the PCIe version using HBM2E memory running at 2.0 Gbps. Both versions use the same capacity capped at 80 GBs. However, that could soon change with the latest rumor suggesting that NVIDIA could be preparing a PCIe version of Hopper H100 GPU with 120 GBs of an unknown type of memory installed.

According to the Chinese website "s-ss.cc" the 120 GB variant of the H100 PCIe card will feature an entire GH100 chip with everything unlocked. As the site suggests, this version will improve memory capacity and performance over the regular H100 PCIe SKU. With HPC workloads increasing in size and complexity, more significant memory allocation is needed for better performance. With the recent advances in Large Language Models (LLMs), AI workloads use trillions of parameters for tranining, most of which is done on GPUs like NVIDIA H100.