ThunderIs there a possibility that we will see HBM in Desktop GPUs one day?

ThunderIs there a possibility that we will see HBM in Desktop GPUs one day?

ThunderIs there a possibility that we will see HBM in Desktop GPUs one day?

unwind-protectI'm more interested in having this on a CPU.

I would have applications that need every bit of core speed and don't need much memory.

igormpApart from costs, is there much of a point?
The 5090 with GDDR7 at 512-bit manages 1.8TB/s, which is higher than the A100 40GB PCIe (1.6TB/s) and pretty near the A100 80GB SXM/H100 80GB PCIe (2TB/s), all of which use HBM2e, and even the H100 SXM 64GB (2TB/s, HBM3).

To reach such high bandwidth you'd need enough stacks, which would both be hella expensive, and also give a consumer GPU way too much memory that's only usually meant for enterprise offerings.

ThunderIs there a possibility that we will see HBM in Desktop GPUs one day?

AnotherReaderIt's very unlikely; HBM is far too expensive for anything short of a 5090.

igormpThe 5090 with GDDR7 at 512-bit manages 1.8TB/s, which is higher than the A100 40GB PCIe (1.6TB/s) and pretty near the A100 80GB SXM/H100 80GB PCIe (2TB/s), all of which use HBM2e, and even the H100 SXM 64GB (2TB/s, HBM3).

igormpTo reach such high bandwidth you'd need enough stacks, which would both be hella expensive, and also give a consumer GPU way too much memory that's only usually meant for enterprise offerings.

AnotherReaderThe cost is prohibitive for most GPUs. However, given how much a 5090 costs, increasing prices by $500 to cover the HBM's cost shouldn't impact expected sales.

AnotherReaderA bigger factor is capacity constraints; TSMC was capacity constrained for COWOS so it makes sense to use that limited capacity for higher margin datacenter GPUs rather than gaming GPUs. Given the explosion in interest in machine learning, the capacity constraint might be even worse now despite TSMC's investments in ameliorating it.

AnotherReaderAs for the advantages, HBM is far more power efficient than GDDR of the same generation. One stack of HBM4 would offer 89% of the bandwidth of the 5090's GDDR7 at a fraction of the power.

AnotherReaderAlternatively, two stacks of HBM3e would exceed that bandwidth and increase capacity. HBM PHYs also require less area than GDDR PHYs so you could either have a smaller die or increase the number of SMXs to take advantage of the saved area and power.

TomorrowIf by one day you mean one day again, then yes. Absolutely.

Which version. AMD was able to release a consumer card with 16GB HBM2 six years ago for 700. Even if we assume doubling of capacity and upping to HBM3e that cards costing four figures (5080 and up) the cost is not the biggest issue. I suspect supply would be much more a problem.

That's only one side of the equation. There's also the power and the size on card. G7 may offer these things but it requires equally complex multiplayer PCB to support 512bit and G7 still requires 16 separate chips on the PCB.

Consumer cards dont need more than two stacks of HBM4 to easily surpass G7 in capacity, speed, power efficiency and space savings.

My point exactly. With 5090 costing 2000+ the argument of "expensive" HBM seems more and more silly.

That's what im thinking too. Right now all HBM is sold to data center cards for much higher margins. Until the AI boom pops this wont change.
I believe this was also the reason why RDNA4 multi.chiplet high end versions were canned.

The lowest 4-Hi stack is 16GB using 4GB layers. So two 16GB stacks would offer 32GB with 3.2TB/s of speed.

And HBM3e is cheaper as it's not the latest and greatest.

TomorrowConsumer cards dont need more than two stacks of HBM4 to easily surpass G7 in capacity, speed, power efficiency and space savings.

WirkoHere's one of the problems (if you're a consumer, that is). Space savings are costly and don't benefit you if all you're buying is one or two GPUs. But if you're trying to compress 100 kilowatts' worth of processors in one rack, small memory footprint is crucial.