NVIDIA's Frame Generation Technology Could Come to GeForce RTX 30 Series

[AleksandarK]

NVIDIA's deep learning super sampling (DLSS) has undergone many iterations to the current version 4 with the transformer model, delivering new technologies such as DLSS Multi Frame Generation, predicting multiple frames in advance to generate the upcoming frame, and increasing the frame output per second. However, not every NVIDIA GPU generation supports these more modern DLSS technologies. In an interview with Digital Foundry, Bryan Catanzaro, VP of Applied Deep Learning Research at NVIDIA, commented on trickling down some DLSS technologies to older GPU generations. For example, DLSS Ray Reconstruction, Super Resolution, and Deep Learning Anti-Aliasing (DLAA) work on NVIDIA GeForce RTX 20/30/40/50 series GPUs. However, the RTX 40 series carries an exclusive DLSS Frame Generation, and the newest RTX 50 series carries the DLSS Multi Frame Generation as an exclusive feature.

However, there is hope for older hardware. "I think this is primarily a question of optimization and also engineering and then the ultimate user experience. We're launching this Frame Generation, the best Multi Frame Generation technology, with the 50 Series, and we'll see what we're able to squeeze out of older hardware in the future." So, frame generation will most likely arrive on the older RTX 30 series, with even a slight possibility of the RTX 20 series getting the DLSS frame generation. Due to compute budget constraints, the multi-frame generation will most likely stay an RTX 50 series exclusive as it has more raw computing power to handle this technology.




Bryan Catanzaro also shared some notes about the DLSS development. "When we built NVIDIA DLSS 3 Frame Generation, we absolutely needed hardware acceleration to compute Optical Flow. We didn't have enough Tensor Cores and we didn't have an Optical Flow algorithm that was good enough. We hadn't developed a real-time Optical Flow algorithm that ran on Tensor Cores that could fit our compute budget. We had the Optical Flow accelerator, which NVIDIA had been building for years as an evolution of our video encoder technology, and it's also been a part of our automotive computer vision acceleration for self-driving cars.

It made sense for us to use that for NVIDIA DLSS 3 Frame Generation. But the difficult part about any sort of hardware implementation of an algorithm like Optical Flow is that it's really difficult to improve it. It is kind of what it is and the failures that arose from that hardware Optical Flow, we couldn't undo them with a smarter neural network until we decided to just replace it and go with a fully AI-based solution, so that's what we've done for Frame Generation in DLSS 4."

View at TechPowerUp Main Site | Source

 

[AusWolf]

I knew this would happen even back during the Ada launch, but everybody was like "nah, it's got this super-duper advanced high tech optical flow thingy, so it's not possible". Yeah, right.

So c'mon lads, get your piping hot RTX 50 series GPUs today, because DLSS 4 definitely won't run on older hardware, pinkie promise.

 

[Bjorn_Of_Iceland]

lol pretty sure they will add multi frame generation for 40 series later on as well . They need to sell these 50 series so bad.


Timed exclusivity

 

[Daven]

Backwards compatible tech breaks Nvidia’s corporate rule: the more you buy the more you save (aka planned obsolescence).

 

[pavle]

It will be interesting to see whether RTX 3xxx series handle the optical flow.

 

[Legacy-ZA]

Yes, but... the VRAM capacity makes it a big problem for the RTX2000/3000 series at 1440p+ though, it could be beneficial on 1080p displays.

Anyways... what did I say? They just locked it to the 4000 series because it was a massive selling point while driving up the prices. People fell for it, bulls saw that leather jacket fly and they stormed for it. ;P

 

[AusWolf]

It will be interesting to see whether RTX 3xxx series handle the optical flow.

I wouldn't be surprised to see that there's very little (if any) difference between RTX 20, 30, 40, and even 50 Tensor and RT cores. Nvidia doesn't share the technical details, after all, so who knows.

 

[konga]

DLSS4 frame gen no longer uses optical flow hardware at all. They analyze motion purely with machine learning now, which is the context for this development. So now that they are using an AI-based solution instead of the optical flow accelerators, they can run the model on older hardware that does not have optical flow accelerators. The catch is that the model is going to be somewhat heavy on cards that don't have great AI acceleration. Nvidia may not want it to even be an option if it isn't actually going to improve your frame rate much.

In short, the reason this is an option now when it wasn't before is because they completely changed the model so it no longer uses hardware that was exclusive to the newer cards. But the Nvidia guy was also very specific in his wording, it's only a vague possibility currently because they aren't sure if they can get the model to be performant enough on older hardware to be worth using.

 

[remekra]

Maybe they can talk to AMD to help them, I mean their Optical Flow doesn't need accelerator nor AI and yet it works. And as much as I agree that FSR upscaling looks worse than DLSS then FG is a different story, don't see much difference between them.

But from that interview something caught my attention more. Their new Transformer model seems to be a lot better but it's also requires more compute. And while Blackwell will be fine, I'm wondering what will be the performance hit on 2xxx, 3xxx and 4xxx. That is also the reason why in Cyberpunk demo you can select which model you want, because the new one is heavier.

 

[AusWolf]

DLSS4 frame gen no longer uses optical flow hardware at all. They analyze motion purely with machine learning now, which is the context for this development. So now that they are using an AI-based solution instead of the optical flow accelerators, they can run the model on older hardware that does not have optical flow accelerators. The catch is that the model is going to be somewhat heavy on cards that don't have great AI acceleration. Nvidia may not want it to even be an option if it isn't actually going to improve your frame rate much.

Optical flow, machine learning, AI... these words don't mean much for the mere mortal, so Nvidia can throw them around however they want to just to mask their planned obsolescence.

 

[Vayra86]

It will be interesting to see whether RTX 3xxx series handle the optical flow.

The question is how credible its performance really is, when we get it.

Nvidia can tell us anything. Most of it is a convenient half-truth, if not an outright lie. In the end it just comes down to how they manage their development, and the first aim is always to sell more GPUs. Not to bring features to older gens; that comes after the sales are made, or to fuel a new round of sales.

 

[AusWolf]

The question is how credible its performance really is, when we get it.

Nvidia can tell us anything. Most of it is a convenient half-truth, if not an outright lie. In the end it just comes down to how they manage their development, and the first aim is always to sell more GPUs. Not to bring features to older gens; that comes after the sales are made, or to fuel a new round of sales.

Unless it's an artificial lock just like the mining limiter on LHR Ampere cards, hidden behind some half-true technobabble. Without knowing the true specs of the RT and Tensor cores, we'll never know. Personally, I don't feel inclined to believe anything at this point.

 

[chrcoluk]

I knew this would happen even back during the Ada launch, but everybody was like "nah, it's got this super-duper advanced high tech optical flow thingy, so it's not possible". Yeah, right.

So c'mon lads, get your piping hot RTX 50 series GPUs today, because DLSS 4 definitely won't run on older hardware, pinkie promise.

Yeah, the short answer is, we will back port when its no longer the latest gen model. Multi frame gen on 40 series after 60 series launch.

Latest gen is now early access to software features.

 

[dyonoctis]

Optical flow, machine learning, AI... these words don't mean much for the mere mortal, so Nvidia can throw them around however they want to just to mask their planned obsolescence.

I mean, AMD is pretty much using the same kind of argument about FSR 4, so either everybody is lying, or there's some truth to what they are saying. Techies do seems to love a conspiracy story , and ever since ML got involved into the mix, it feels like GPU tech became more of a black box to anyone not interested/savvy with machine learning

 

[chrcoluk]

I think its at least a half truth on launch, but then later they optimise, but just dont release the optimisation deliberately until there is a newer gen of cards and new exclusive feature on those cards.

 

[shk021051]

 

[AusWolf]

I mean, AMD is pretty much using the same kind of argument about FSR 4, so either everybody is lying, or there's some truth to what they are saying. Techies do seems to love a conspiracy story , and ever since ML got involved into the mix, it feels like GPU tech became more of a black box to anyone not interested/savvy with machine learning

View attachment 380701

Well, RDNA 3's AI cores were heavily advertised, and we still haven't seen them do anything.

 

[Macro Device]

Now that's the typo. I quit my pants laughing at it.

 

[AusWolf]

View attachment 380703
Now that's the typo. I quit my pants laughing at it.

They should have gone all-in and typed arquetechture. Would have been real classy.

 

[JollyRoger96]

Will it come or no, stop giving people false hope.

 

[Prima.Vera]

Another big middle finger to nGreedia.
I am using AMD's Frame Generation without any issues, there is also a freeware application available that does this, and now nGreedia is telling us that they could bring the DLSS Frame Gen to the 30x0 generation too.
Well, F**K YOU very much nVidia!!!

 

[john_]

I mean, AMD is pretty much using the same kind of argument about FSR 4, so either everybody is lying, or there's some truth to what they are saying.

The thing is that FSR 3.1 and even Lossless Scaling proved that it is possible. Nvidia can argue that quality wasn't going to be at the level they wanted, but if I am not mistaken FSR 3.1 Frame Generation works fine.