Tuesday, February 27th 2024
Samsung Develops Industry-First 36GB HBM3E 12H DRAM
Samsung Electronics, a world leader in advanced memory technology, today announced that it has developed HBM3E 12H, the industry's first 12-stack HBM3E DRAM and the highest-capacity HBM product to date. Samsung's HBM3E 12H provides an all-time high bandwidth of up to 1,280 gigabytes per second (GB/s) and an industry-leading capacity of 36 gigabytes (GB). In comparison to the 8-stack HBM3 8H, both aspects have improved by more than 50%.
"The industry's AI service providers are increasingly requiring HBM with higher capacity, and our new HBM3E 12H product has been designed to answer that need," said Yongcheol Bae, Executive Vice President of Memory Product Planning at Samsung Electronics. "This new memory solution forms part of our drive toward developing core technologies for high-stack HBM and providing technological leadership for the high-capacity HBM market in the AI era."
The HBM3E 12H applies advanced thermal compression non-conductive film (TC NCF), allowing the 12-layer products to have the same height specification as 8-layer ones to meet current HBM package requirements. The technology is anticipated to have added benefits especially with higher stacks as the industry seeks to mitigate chip die warping that come with thinner die. Samsung has continued to lower the thickness of its NCF material and achieved the industry's smallest gap between chips at seven micrometers (µm), while also eliminating voids between layers. These efforts result in enhanced vertical density by over 20% compared to its HBM3 8H product.
Samsung's advanced TC NCF also improves thermal properties of the HBM by enabling the use of bumps in various sizes between the chips. During the chip bonding process, smaller bumps are used in areas for signaling and larger ones are placed in spots that require heat dissipation. This method also helps with higher product yield.
As AI applications grow exponentially, the HBM3E 12H is expected to be an optimal solution for future systems that require more memory. Its higher performance and capacity will especially allow customers to manage their resources more flexibly and reduce total cost of ownership (TCO) for datacenters. When used in AI applications, it is estimated that, in comparison to adopting HBM3 8H, the average speed for AI training can be increased by 34% while the number of simultaneous users of inference services can be expanded more than 11.5 times.
Samsung has begun sampling its HBM3E 12H to customers and mass production is slated for the first half of this year.
Source:
Samsung
"The industry's AI service providers are increasingly requiring HBM with higher capacity, and our new HBM3E 12H product has been designed to answer that need," said Yongcheol Bae, Executive Vice President of Memory Product Planning at Samsung Electronics. "This new memory solution forms part of our drive toward developing core technologies for high-stack HBM and providing technological leadership for the high-capacity HBM market in the AI era."
Samsung's advanced TC NCF also improves thermal properties of the HBM by enabling the use of bumps in various sizes between the chips. During the chip bonding process, smaller bumps are used in areas for signaling and larger ones are placed in spots that require heat dissipation. This method also helps with higher product yield.
As AI applications grow exponentially, the HBM3E 12H is expected to be an optimal solution for future systems that require more memory. Its higher performance and capacity will especially allow customers to manage their resources more flexibly and reduce total cost of ownership (TCO) for datacenters. When used in AI applications, it is estimated that, in comparison to adopting HBM3 8H, the average speed for AI training can be increased by 34% while the number of simultaneous users of inference services can be expanded more than 11.5 times.
Samsung has begun sampling its HBM3E 12H to customers and mass production is slated for the first half of this year.
12 Comments on Samsung Develops Industry-First 36GB HBM3E 12H DRAM
AMD launched some products with minimal profit margins during the first and second generations of HBM. Normally, the current generation (HBM3) would be about five times more expensive. However, due to AI-created scarcities, prices should have doubled or tripled.
If the AI/MI 'push' and demand for GPGPU/NPU power is as strong as it seems, why can't us plebian consumers get the left overs?
Graphics GPUs don't need the latest HBM, that's complete overkill. There isn't a single consumer GPU with 32GB in total, let alone one that needs 32GB per HBM stack like the HBM3e discussed in the article.
Speaking Towards (old gen)HBM:
My friend and I were just recently comparing his old FuryX vs. my, RX580X and Vega64 in Avatar:FoP.
The FuryX performed very similarly (in avg) to the RX580X. However, the HBM-equipped FuryX had very stable minimum framerates.
OToH, the FuryX performed almost precisely 'half as well' as the Vega 64 did; both, having very steady minimum framerates.
(In my Jaded opinion)
I suspect us consumers don't get HBM anymore because, it makes products last too long.
In the money-making AI/MI (and 'Big Data') spaces, once a product has been superseded in performance, it's replaced.
More often than not, those retired devices are (semi-)proprietary and only re-useable by a smaller firm in a similar field.
'Individual consumers' and 'in-built obsolescence' both become a non-issue when, your primary market(s) have every profit-motive for buying The New Hotness (and, in bulk qty.).
Newer faster GDDR6(X) cards may beat older HBM cards but, the (lower) avg framerate is steadier, and in some cases (moreso w/ Vega 20) flat-out sustains higher 0.1% and 1% lows.Yes. For-profit 'Big Data' and 'AI/MI' eat up the supply of HBM, and have been the driving-force in manufacturing/developing the stuff.
Undeniable, and even sensible from my tiny little PoV.
Regardless, those stated benefits of HBM have wide (consumer) applications.
Mobile SoCs, GPUs; really, anything that benefits from compact, power efficient, ultra-low-latency DRAM.
Which, is pretty much every flavor of personal computing device imaginable. IMO, there's merely not yet been-allowed opportunity for older, well-developed process HBMs (HBMe HBM2e, HBM3, etc.) to find places in consumer-facing products.
IMO, (If press releases are to be believed :laugh:) HBM's teething issues (overall), are mostly-solved. The issues inherit in each new generation (+extreme demand) are currently cost-driving.
Admittedly, The technology itself, complicates 'integration' just willy-nilly...Still, in-theme w/ Wirko's point: rather than deal w/ 'HBM shenanigans', they designed a best effort replacement-of-function.
Which, in relation to (GP)GPUs, reminds me...
Personally, I'd even be happy w/ a return to phatass memory busses, like with ATI R600, nVidia G80, AMD Hawaii, nVidia GT200(B), etc.
As I recall, wide membus GPUs saw similar raw performance benefits as we'd later see exemplified-further in 1024bit - 4096bit on-die/package HBM. Still, nothing can replace HBM's extreme latency benefits of being wide and fast RAM, sitting next/atop the ASIC.
Problem: Some of the main purposes for both AMD's InfinityCache and Both's use of HBM, is to circumvent the issues and complications (typically/historically) seen with wide memory bus (GP)GPUs.
Though today, I could see an MCM-MCH design mitigating some those problems while allowing lots of cheaper (G)DRAM.