Upcoming GDDR7

[Vayra86]

As others have said VRAM allocated isn't the same thing as VRAM needed. That hit home to me about 10 years ago on a review of COD: Advanced Warfare here. The game was running fine on 3 GB cards at all resolutions and it was even stated in the review that the engine was just loading up the VRAM on cards with more VRAM just because it was there without actually needing it and one member here running a Maxwell Titan X said his card was showing 11 GB used.

Highly game specific statement, definitely not applicable as a blanket statement.

You can monitor the actual VRAM in use these days with RTSS. Do it. Its very interesting to see the differences. You will find some games absolutely do use almost everything they allocated, and if you allocate less, something's gonna give somewhere. Its not always stutter, these days, its also reduced texture quality, more pop in, etc. You can even sometimes detect post processing effects to generate less quickly each frame.

 

[chrcoluk]

To backup Vayra's point, those of us who have moaned about VRAM problems, we dont do it just because a tool reports high usage, its usually because we notice symptoms. I have experienced all of the following in games I played on my RTX 3080.

Game crashing due to running out of VRAM.
Texture pop in's.
Delayed texture loading due to constant flushes to make room.
Reduced LOD textures.
Stutters.
Glitches.
Textures that dont load in at all, completely missing.
Excessive texture unloading, so e.g. cant do a 360 in game whilst keeping everything in view loaded.

Many years back games would just routinely crash when no resources left, now days things are more dynamic and so people get the impression there is no problem. An example might be the game will let you pick ultra textures, and instead of crashing it will make you think you using ultra but instead it dynamically lowers the quality to keep the game running.

 

[londiste]

Now the only question is the VRAM size. GDDR7 will still be produced using 2 GB memory chips, and later on with 3 GB in size. Maybe that's the reason Nvidia pushed back its next gen to 2025. Only way to increase the VRAM size without increasing bus width while still using 2 GB chips, would be to use socalled clamshell mode where memory chips are stacked at each side of the PCB, this however doesn't offer any memory bandwidth improvement, just doubles the memory size, and it's also cheaper for production. I believe that high end GPUs will use 3 GB chip size on a 384-bit (36 GB) wide bus, while mid-to-high tier will be on 2 GB size (24 GB) using 384-bit wide bus (something like a RTX 5080 perhaps).

I bet this is exactly why Nvidia postponed. There has been a lot of complaining about memory sizes and they need to do something. At least Micron seems to have 24Gbit GDDR7 in production so waiting for these would make sense. It would allow basically to increase memory capacity to the next size:
128bit bus, 4 chips - 8GB with 2GB chips, 12GB with 3GB chips
192bit bus, 6 chips - 12GB with 2GB chips, 18GB with 3GB chips
256bit bus, 8 chips - 16GB with 2GB chips, 24GB with 3GB chips
384bit bus, 12 chips - 24GB with 2GB chips, 36GB with 3GB chips

I don't think anything mid-high tier would get 386bit bus, not even 5080. At least not when 3GB GDDR7 is available. like you calculated, essentially 4090 bandwidth and 24GB will end up quote optimal there. And clamshell is not something a manufacturer wants to do. Costs more, is a bit more complex, needs cooling on the backside. GDDR6X was pretty hot and 3090s with clamshell VRAM showed that pretty convincingly.

Its funny how Nvidia prioritise bandwidth over capacity. Whilst also fighting tooth and nail to make the bus width as small as possible, causing their weird 6/10/12 gig setups.

Pushing bus width down was AMDs playbook, they did it a generation earlier. Same with large cache to mitigate this change. What bit Nvidia back was probably them expecting 2GB GDDR6X to be available sooner.

Looks like there are going to be even faster speeds for GDDR7.

Samsung to showcase 37 Gbps GDDR7 memory for next-gen GPUs at ISSCC - VideoCardz.com

GDDR7 at 37 Gbps to enable up to 1.7 TB/s bandwidth for 384-bit cards Samsung is preparing 37 GB/s graphics memory based on GDDR7 technology. The upcoming 2024 ISSCC program outlines plans for high-speed memory, featuring 37 Gb/s and 35.4 Gb/s variants based on Samsung and SK Hynix’s GDDR7...

videocardz.com

GDDR7 spec sees up to 48Gbps and up to 8GB chips. Whether and when manufacturers will be able to build something like that we will see. Micron is currently at 2GB and 3GB sizes and modest speeds. Samsung announcing they have built faster stuff has been tricky before, it took years for them to get fast GDDR6 from announcement to production.

 

[iameatingjam]

We learned last year that GDDR7 will be running at around 32 Gbps, quite an improvement of the current GDDR6X. I did some research and came up with some interesting numbers regarding bandwidth, bus width and memory clocks for the upcoming memory. I may be wrong, so please do correct me if I'm mistaken.

Most importantly, GDDR7 will be using PAM3 and NRZ signaling, while current GDDR6X uses PAM4 signaling. NRZ (non return to zero) signaling can transfer 1 bit of data per cycle, PAM4 is transfering 2 bits of data per cycle, while PAM3 will be transfering 1,5 bits per cycle. NRZ still exists in GDDR7 spec because it lowers power consumption, and will be used instead of PAM3 when there is low bandwidth need. GDDR6X clocks slower compared to other memory types because of PAM4 signaling.

So as we're currently calculating the memory effective speed on GDDR6X:

Speed = memory clock * (bit rise/fall) * QDR (quad data rate, since GDDR5) * bits per cycle

So RTX 4080 memory speed would be: 1,4 * 2 * 4 * 2 (uses PAM4) = ~22,4 Gbps for example.

Announced 32 Gbps would equal to something like: 2667 MHz memory clock, or: 2,667 * 2 * 4 * 1,5 = ~32 Gbps

A lot of people are speculating that a larger than 384-bit wide bus will be used for the next gen high end GPUs. If we take a look at memory bandwidth on a 384-bit bus, coupled with 32 Gbps effective speed, that would equal to more than 1,5 TB/s of memory bandwidth! Memory bandwidth = memory effective speed * bus width / 8 (8 bits make a byte), so 32 * 384 / 8 = 1,536 TB/s. Seems like a perfect choice for the next high end GPUs.
Even if the 256-bit bus interface will be used coupled with ~32 Gbps, it will have a memory bandwidth of 1,02 TB/s, practically the same as current RTX 4090.

Now the only question is the VRAM size. GDDR7 will still be produced using 2 GB memory chips, and later on with 3 GB in size. Maybe that's the reason Nvidia pushed back its next gen to 2025. Only way to increase the VRAM size without increasing bus width while still using 2 GB chips, would be to use socalled clamshell mode where memory chips are stacked at each side of the PCB, this however doesn't offer any memory bandwidth improvement, just doubles the memory size, and it's also cheaper for production. I believe that high end GPUs will use 3 GB chip size on a 384-bit (36 GB) wide bus, while mid-to-high tier will be on 2 GB size (24 GB) using 384-bit wide bus (something like a RTX 5080 perhaps).

Any thoughts?

When you say high end you mean like the 5090? You think it will have 36gb? Wont that endanger its enterprise lineup? Then again if its expensive enough I suppose it wont matter that much.

 

[igormp]

You can monitor the actual VRAM in use these days with RTSS.

You cannot. It only shows dedicated vs shared memory. To actually see the real, in-use VRAM usage you'd need to use a debugger like Renderdoc.

 

[AusWolf]

I'm on a "we'll see when it comes out" approach as (nearly) always.

 

[DaemonForce]

A lot of that constant mesh and texture load/unload cycling and LoD is desktop.
In VR it wants ALL of it loaded into memory, especially in some common engines that don't optimize the experience.
So for those of us doing both, we need way more vram than looks necessary or we get really bad glitches and slowdowns.

 

[Zach_01]

“Many games manage memory usage differently, automatically dialing down quality settings or removing textures altogether.”

How Much VRAM is Enough for PC Gaming?

Graphics card memory, or VRAM, has been a hot topic of discussion as of late because we're not getting enough of it. Here's our latest testing of...

www.techspot.com

“With the exception of entry-level options, you shouldn't be buying 8GB graphics cards anymore. 12GB is now the bare minimum, and 16GB is the ideal target.”

 

[Raysterize]

Personally, I think the 5090 will retain the 384-bit bus and employ the G7 memory. High throughput and adequate capacity - makes for a balanced configuration. It also keeps the cost relatively down compared to using a 512-bit bus and 16 memory devices. If the 6900 XT and the RTX 4080 can teach people a lesson is that sometimes more is not better, both Navi 21 and AD103 make for harmonious utilization of the hardware, you can extract pretty much everything out of these cards.

>24 GB is currently entirely unnecessary targeting 4K or 8K DLSS gaming, and as the 7900 XTX clearly shows, it does not benefit from having 24 GB as there is virtually no situation where it is a boon over the RTX 4080's 16 GB. Whenever the XTX pans out to be faster, it is usually in compute-heavy, D3D12 or Vulkan games (where its drivers don't get in the way), it is not because of its memory, or even bandwidth, but rather because that scenario favors its architecture.

Truth be told, if they wanted to hit it out of the park, they should release another graphics card like the RTX 3090 Ti: fully enabled core, fastest memory bin, no nonsense.

My 7900XTX was using 19GB VRAM on Warzone 4K on the Ultra Preset... Sold it and went back to Xbox Series X as got banned 3 times for 2 weeks at a time! (Like going from a Ferrari to a Robin Reliant!).

 

[xtreemchaos]

My God i've just taken in GDDR6, not again my brain canny not take it .

 

[chrcoluk]

I still have my 3080 (and my 9900k with 32 gigs of ram, so a reasonable base system too), so now have the luxury of comparing 10 gigs to 16 gigs in games, and have tested a couple and will test some more after a games purchasing spree last night, results are satisfying in terms of my 4080 purchase.

 

[londiste]

I still have my 3080 (and my 9900k with 32 gigs of ram, so a reasonable base system too), so now have the luxury of comparing 10 gigs to 16 gigs in games, and have tested a couple and will test some more after a games purchasing spree last night, results are satisfying in terms of my 4080 purchase.

VRAM aside, 4080 is 45-50% faster

 

[chrcoluk]

VRAM aside, 4080 is 45-50% faster

Yeah, I am not comparing framerates, I am checking VRAM specific behaviour, texture quality issues, pop in, game stability, that sort of thing.

 

[iameatingjam]

I still have my 3080 (and my 9900k with 32 gigs of ram, so a reasonable base system too), so now have the luxury of comparing 10 gigs to 16 gigs in games, and have tested a couple and will test some more after a games purchasing spree last night, results are satisfying in terms of my 4080 purchase.

You know recently I was wondering if anybody had created a consumer facing utility to eat xxx amount of vram. It would be really helpful when trying to help find solutions for people on low vram cards. When you have a big pool of vram you never quite know if a game is caching things because it knows its has the room to do so, or if its just being used irresponsibly. Also yeah ' idk works fine on my 4090' is not a popular phrase.

Really though, I like to try and be helpful on steam and being able to limit to 8 or 12 would be super useful. Sometimes it feels like a fool's errand though. Especially when you're talking to people who are say used to being console gamers where things just work and don't really know about troubleshooting. You also got to be careful when using terminology that you forget an average gamer might not know. Sure its easy enough to say, something like 'reduce all your cores by 400mhz and turn xmp off and if the error no longer appears, then the cpu is most likely degraded.' I think I've said some variation of that maybe 5 times and let me tell you it is not something people like to hear even though it literally takes less than a couple minutes to set up. Instead I usually get some variant of 'but it worked before' Well yes, things generally work before they don't. I've even offered to walk people through it. But idk. They still respond with hostility.

Why do I bother? Idk. I guess I just want people to enjoy the games that I enjoy....

I feel bad for the developers who basically get put into tech support roles due to stuff like this.

 

[igormp]

You know recently I was wondering if anybody had created a consumer facing utility to eat xxx amount of vram.

It's easily doable, but idk if there's such an app for windows.
See this video of mine I did years ago (I did it as an example of allocated vs actually used VRAM):

http://imgur.com/Zc0flqt

 

[iameatingjam]

It's easily doable, but idk if there's such an app for windows.
See this video of mine I did years ago (I did it as an example of allocated vs actually used VRAM):

http://imgur.com/Zc0flqt

You've posted this somewhere else haven't you? I've searched for such a tool and I am 99% sure I have seen this at one point.

 

[Dr. Dro]

You've posted this image somewhere else haven't you? I've searched for such a tool and I am 99% sure I saw that photo at one point.

Was here on TPU on a thread some time ago. Neat software, that one.

 

[iameatingjam]

I bet this is exactly why Nvidia postponed. There has been a lot of complaining about memory sizes and they need to do something. At least Micron seems to have 24Gbit GDDR7 in production so waiting for these would make sense. It would allow basically to increase memory capacity to the next size:
128bit bus, 4 chips - 8GB with 2GB chips, 12GB with 3GB chips
192bit bus, 6 chips - 12GB with 2GB chips, 18GB with 3GB chips
256bit bus, 8 chips - 16GB with 2GB chips, 24GB with 3GB chips
384bit bus, 12 chips - 24GB with 2GB chips, 36GB with 3GB chips

I don't think anything mid-high tier would get 386bit bus, not even 5080. At least not when 3GB GDDR7 is available. like you calculated, essentially 4090 bandwidth and 24GB will end up quote optimal there. And clamshell is not something a manufacturer wants to do. Costs more, is a bit more complex, needs cooling on the backside. GDDR6X was pretty hot and 3090s with clamshell VRAM showed that pretty convincingly.

Pushing bus width down was AMDs playbook, they did it a generation earlier. Same with large cache to mitigate this change. What bit Nvidia back was probably them expecting 2GB GDDR6X to be available sooner.

GDDR7 spec sees up to 48Gbps and up to 8GB chips. Whether and when manufacturers will be able to build something like that we will see. Micron is currently at 2GB and 3GB sizes and modest speeds. Samsung announcing they have built faster stuff has been tricky before, it took years for them to get fast GDDR6 from announcement to production.

Recently I've been worrying they might end up going 128 bit for the 5070 too. With 3gb chips thats 12GB with a respectable amount of bandwidth. I mean I hope its more but we know how nvidia rolls. Fastest, newest, ram modules but bare minimum bus, bare minimum capacity unless its a halo card or a very weak card that has no chance to compete with enterprise cards. Mid-range, or even high end 80 series cards still seem to get the bare minimum. I think the 3080 was the biggest insult. Have to go all the way to 3080 ti to get the same vram as the 3060??? And it was similar with the 40 series. Sure you can have a 16GB 4060 ti. But if you want vram and power, you gotta pay $1200 for a 4080. Mind you, its a little better now after the refresh, 4070 ti super can get you 16gb for $800. But thats still... a lot of money.

 

[igormp]

You've posted this somewhere else haven't you? I've searched for such a tool and I am 99% sure I have seen this at one point.

Likely, I've posted this in many forums as an example of allocating ram without using it haha

 

[londiste]

It's easily doable, but idk if there's such an app for windows.
See this video of mine I did years ago (I did it as an example of allocated vs actually used VRAM):

http://imgur.com/Zc0flqt

That's basically a virtual disk on VRAM, right?

Cool. I remember back during the GTX970 VRAM problems when trying to look into VRAM usage patterns and trying to force it into the 3.5-4GB range there were some solutions around to consume VRAM but they were not all too reliable. Simply allocating clearly did not work, filling the allocated range did help but there was some weirdness in that as well. Basically it seemed that user did not really get complete control over the actual VRAM management and drivers or firmware had their own say in what actually happened. Both on Nvidia and AMD cards. And yeah, worse on Windows of course (more handholding from what it looked like). Making a virtual drive out of it and copying actual files onto that does seem like a more reliable solution

 

[igormp]

That's basically a virtual disk on VRAM, right?

Yup.

Basically it seemed that user did not really get complete control over the actual VRAM management and drivers or firmware had their own say in what actually happened. Both on Nvidia and AMD cards. And yeah, worse on Windows of course (more handholding from what it looked like). Making a virtual drive out of it and copying actual files onto that does seem like a more reliable solution

I think that even just copying files like I did would not really achieve what you want. Maybe manually doing some cudaMalloc for many regions, getting their addresses and freeing up the ones not in your desired range could work, but nothing stops the memory manager to just move those pages around.