AMD Radeon RX 9070 XT Could Get a 32 GB GDDR6 Upgrade

[Makaveli]

With the 5090, NV has released a 32GB VRAM consumer GPU, ofc AMD is going to do the same (wasn't it the same with 24GB VRAM consumer GPUs?). The difference is the 9070 (XT) is based on a 256-bit chip using GDDR6 ~600 GB/s vs 512-bit GDDR7 1792 GB/s for the 5090. Still fast enough.

AFAIK, only modded games may require more than 24GB VRAM in 4K right now, but 32GB are nice for fully offloading/hosting big-ish LLMs locally.

Regarding CUDA/ML stack, indeed, I think of AMD GPUs only in terms of running/inferencing LLMs, not training/finetuning, but I read it's still possible and supposedly got easier over the last years, but CUDA is tier agnostic and supports consumer GPUs, workstation GPUs and enterprise cards. To improve this, UDNA (U for unified) will replace RDNA at some point.

5090' idle power consumption unfortunately increased to 30W (4090 22W), but it's still not too bad (it's more than linear in video playback: 54W 5090 vs 26W 4090) considering there are 16 2GB modules (linear increase: 22W[4090]/12[GDDR6X]*16[GDDR7] = 29.33W).

For me to consider this RDNA4 32GB GPU (in no particular order):

• DLSS 2-like upscaling quality improvement
• Fix HDMI 2.1 48GB/s, aka HDMI 2.1a on Linux
• Back to good power scaling like in RDNA2
• Low idle power consumption, linear increase with the amount of VRAM compared to the 16GB VRAM, at the worst
• Just like the 5090, 9070 (XT) 32GB also must be a consumer GPU, so that the price increase is minimal

So, AMD, it's 48GB VRAM consumer GPUs for the UDNA arch after RDNA4 then as well? Would allow to fully offload `Llama-3.3-70B-Instruct-Q4_K_M.gguf` (42.5GB) (by then we will have a different and more capable 70B LLM, ofc), or allow for much higher context.


Yes, 24GB VRAM can't fit a e.g 27GB `Qwen2.5-32B-Instruct-Q6_K.gguf` SOTA LLM, but the .gguf format allows to offload the rest of the LLM layers to RAM, but it will run much slower. The tokens per second speed increases exponentially the more layers are offloaded to the GPU, I did some testing:
View attachment 384684

I'm not sure they will do 48GB on consumer just yet as that will give the W7800 and W7900 workstation gpu's some competition but we shall see.

Right now this model gives me the best performance on a 24GB VRAM gpu

Doing about 28 tok / sec

Looks like this rumor is false.

 

[bitsandboots]

AFAIK, only modded games may require more than 24GB VRAM in 4K right now, but 32GB are nice for fully offloading/hosting big-ish LLMs locally.

Regarding CUDA/ML stack, indeed, I think of AMD GPUs only in terms of running/inferencing LLMs, not training/finetuning, but I read it's still possible and supposedly got easier over the last years, but CUDA is tier agnostic and supports consumer GPUs, workstation GPUs and enterprise cards. To improve this, UDNA (U for unified) will replace RDNA at some point.

5090' idle power consumption unfortunately increased to 30W (4090 22W), but it's still not too bad (it's more than linear in video playback: 54W 5090 vs 26W 4090) considering there are 16 2GB modules (linear increase: 22W[4090]/12[GDDR6X]*16[GDDR7] = 29.33W).

For me to consider this RDNA4 32GB GPU (in no particular order):

• DLSS 2-like upscaling quality improvement
• Fix HDMI 2.1 48GB/s, aka HDMI 2.1a on Linux
• Back to good power scaling like in RDNA2
• Low idle power consumption, linear increase with the amount of VRAM compared to the 16GB VRAM, at the worst
• Just like the 5090, 9070 (XT) 32GB also must be a consumer GPU, so that the price increase is minimal

So, AMD, it's 48GB VRAM consumer GPUs for the UDNA arch after RDNA4 then as well? Would allow to fully offload `Llama-3.3-70B-Instruct-Q4_K_M.gguf` (42.5GB) (by then we will have a different and more capable 70B LLM, ofc), or allow for much higher context.


Yes, 24GB VRAM can't fit a e.g 27GB `Qwen2.5-32B-Instruct-Q6_K.gguf` SOTA LLM, but the .gguf format allows to offload the rest of the LLM layers to RAM, but it will run much slower. The tokens per second speed increases exponentially the more layers are offloaded to the GPU, I did some testing:

Yeah spot on.
Regarding VRAM and games: I think its hard to get games to use above 16GB but the one exception similar to what you said with mods, is VRChat. This game is quite unusual in that you're seeing user uploaded unity assets and as a result the optimization is horrible, a tragedy of the commons situation in which not enough individuals optimize their assets so each avatar could be as bad as 500MB of vram... so if you go to a populated room, there is no real upper bound for how much VRAM you'd like! And, as that's a VR game, displayport 2 is a must for future proof because current-gen VR headsets already saturate what DP 1.4 can do.

Regarding VRAM and AI: If you train a LoRA for SD XL, you'll already cross the 16GB boundary. SD 3 and Flux are going to be worse. Training speed isn't really an issue here, just vram.
Inference as you say, doesn't need hardly as much vram, so the 9070xt will shine with even less VRAM.

But as a competitor to say a 5080, the 5080 just doesnt have enough vram. I think 24GB is enough, 32 is a bonus, but 16 just isn't enough for these admittedly obscure tasks.

 

[Bomby569]

not sure that cards need it, it's mostly to serve the AI crowd, grab more money, less gpus available for gamers

 

[HD64G]

Smart move if they manage to offer that 32GB iteration as a workstation GPU in order to lighten gamers' GPUs series demand from whoever (non-gamer) needs more RAM for apps. And they will be able to sell them for higher profit margins that should allow AMD to keep the gamers' GPUs in normal pricing.

 

[AusWolf]

Look at the update, guys. There's no 32 GB gamer card, but maybe a Radeon Pro workstation card coming later.