Tuesday, September 20th 2022
NVIDIA Introduces DLSS 3 With Breakthrough AI-Powered Frame Generation for up to 4x Performance
NVIDIA today announced NVIDIA DLSS 3, an AI-powered performance multiplier that kicks off a new era of NVIDIA RTX neural rendering for games and applications. DLSS 3 builds on the company's lead in AI-accelerated super-resolution techniques to deliver outstanding image quality and up to 4x the performance of brute-force rendering, plus the quick responsiveness that defines a comprehensive gaming experience.
The technology debuted today during the GeForce Beyond: Special Broadcast at GTC, which also introduced GeForce RTX 40 Series GPUs based on the new NVIDIA Ada Lovelace architecture. "DLSS is one of our best inventions and has made real-time ray tracing possible. DLSS 3 is another quantum leap for gamers and creators," said Jensen Huang, founder and CEO of NVIDIA. "Our pioneering work in RTX neural rendering has opened a new universe of possibilities where AI plays a central role in the creation of virtual worlds."
DLSS 3 is already receiving widespread support from the gaming ecosystem, with over 35 games and applications integrating the technology.
DLSS 3: The Performance Multiplier, Powered by AI
The combination of ray tracing and AI technologies has revolutionized video games by simultaneously delivering dramatic improvements in image quality along with massive uplifts in performance—a feat unheard of before GeForce RTX.
Powered by new fourth-generation Tensor Cores and a new Optical Flow Accelerator on GeForce RTX 40 Series GPUs, DLSS 3 is the latest iteration of the company's critically acclaimed Deep Learning Super Sampling technology and introduces a new capability called Optical Multi Frame Generation.
Optical Multi Frame Generation generates entirely new frames, rather than just pixels, delivering astounding performance boosts. The new Optical Flow Accelerator incorporated into the NVIDIA Ada Lovelace architecture analyzes two sequential in-game images and calculates motion vector data for objects and elements that appear in the frame, but are not modeled by traditional game engine motion vectors. This dramatically reduces visual anomalies when AI renders elements such as particles, reflections, shadows and lighting.
Pairs of super-resolution frames from the game, along with both engine and optical flow motion vectors, are then fed into a convolutional neural network that analyzes the data and automatically generates an additional frame for each game-rendered frame—a first for real-time game rendering. Combining the DLSS-generated frames with the DLSS super-resolution frames enables DLSS 3 to reconstruct seven-eighths of the displayed pixels with AI, boosting frame rates by up to 4x compared to without DLSS.
Because DLSS Frame Generation executes as a post-process on the GPU, it can boost frame rates even when the game is bottlenecked by the CPU. For CPU-limited games, such as those that are physics-heavy or involve large worlds, DLSS 3 allows the GeForce RTX 40 Series GPUs to render the game at up to twice the frame rate than the CPU is able to compute the game.
DLSS 3 integrations also incorporate NVIDIA Reflex, which synchronizes the GPU and CPU, ensuring optimum responsiveness and low system latency.
Game Developers Embrace DLSS 3, Over 35 Games, Applications Coming Soon
The revolutionary power of DLSS 3 is a boon for game developers who want to express their artistic vision. The technology is coming to the world's most popular game engines, such as Unity and Unreal Engine.
DLSS 3 has also received support from many of the world's leading game developers, with more than 35 games and applications announcing support, including:
Since DLSS 3 builds on top of DLSS 2 integrations, game developers can quickly enable it in existing titles that already support DLSS 2 or NVIDIA Streamline.
DLSS 3 Coming Oct. 12.
DLSS 3 is supported in GeForce RTX 40 Series GPUs and will debut on Oct. 12 with the availability of GeForce RTX 4090 GPUs. More details are available on GeForce.com or NVIDIA.com, including details on GeForce RTX 40 Series GPUs and NVIDIA DLSS technology.
The technology debuted today during the GeForce Beyond: Special Broadcast at GTC, which also introduced GeForce RTX 40 Series GPUs based on the new NVIDIA Ada Lovelace architecture. "DLSS is one of our best inventions and has made real-time ray tracing possible. DLSS 3 is another quantum leap for gamers and creators," said Jensen Huang, founder and CEO of NVIDIA. "Our pioneering work in RTX neural rendering has opened a new universe of possibilities where AI plays a central role in the creation of virtual worlds."
DLSS 3: The Performance Multiplier, Powered by AI
The combination of ray tracing and AI technologies has revolutionized video games by simultaneously delivering dramatic improvements in image quality along with massive uplifts in performance—a feat unheard of before GeForce RTX.
Powered by new fourth-generation Tensor Cores and a new Optical Flow Accelerator on GeForce RTX 40 Series GPUs, DLSS 3 is the latest iteration of the company's critically acclaimed Deep Learning Super Sampling technology and introduces a new capability called Optical Multi Frame Generation.
Optical Multi Frame Generation generates entirely new frames, rather than just pixels, delivering astounding performance boosts. The new Optical Flow Accelerator incorporated into the NVIDIA Ada Lovelace architecture analyzes two sequential in-game images and calculates motion vector data for objects and elements that appear in the frame, but are not modeled by traditional game engine motion vectors. This dramatically reduces visual anomalies when AI renders elements such as particles, reflections, shadows and lighting.
Pairs of super-resolution frames from the game, along with both engine and optical flow motion vectors, are then fed into a convolutional neural network that analyzes the data and automatically generates an additional frame for each game-rendered frame—a first for real-time game rendering. Combining the DLSS-generated frames with the DLSS super-resolution frames enables DLSS 3 to reconstruct seven-eighths of the displayed pixels with AI, boosting frame rates by up to 4x compared to without DLSS.
Because DLSS Frame Generation executes as a post-process on the GPU, it can boost frame rates even when the game is bottlenecked by the CPU. For CPU-limited games, such as those that are physics-heavy or involve large worlds, DLSS 3 allows the GeForce RTX 40 Series GPUs to render the game at up to twice the frame rate than the CPU is able to compute the game.
DLSS 3 integrations also incorporate NVIDIA Reflex, which synchronizes the GPU and CPU, ensuring optimum responsiveness and low system latency.
Game Developers Embrace DLSS 3, Over 35 Games, Applications Coming Soon
The revolutionary power of DLSS 3 is a boon for game developers who want to express their artistic vision. The technology is coming to the world's most popular game engines, such as Unity and Unreal Engine.
DLSS 3 has also received support from many of the world's leading game developers, with more than 35 games and applications announcing support, including:
DLSS 3 Coming Oct. 12.
DLSS 3 is supported in GeForce RTX 40 Series GPUs and will debut on Oct. 12 with the availability of GeForce RTX 4090 GPUs. More details are available on GeForce.com or NVIDIA.com, including details on GeForce RTX 40 Series GPUs and NVIDIA DLSS technology.
31 Comments on NVIDIA Introduces DLSS 3 With Breakthrough AI-Powered Frame Generation for up to 4x Performance
Be not afraid!
As for the AI created frame insertion itself, we'll have to see if it's any good. I've personally never seen artificial frame insertion that didn't make me feel sick watching it.
Should it be called DLSS 2.5+AIFI? Yeah, that really flows off the tongue, just like 802.11ac, IEEE-1394, and h.264.
Pedants and a handful of programmers might argue this into the wee hours but it really doesn't matter to Joe Gamer.
I don't suppose you work in marketing.
So for a 30fps game, running ”60fps” with frame insertion, will have an added input delay of 17ms plus whatever time it takes to calculate the additional frame. Also power consumption will go up, possibly even lowering the ’base’ framerate. Also, motion vector based interpolation won’t be able to interpolate curved motion well, meaning that mouse input in games may feel off.
a 30 fps game with DLSS 3.0 running at 60 fps will look like it's running at 60 fps but it will feel like a 30 fps game. Most people don't want higher fps for just the visual gain, but to get lower latency and it won't help there unlike DLSS 2.x where you can generate simpler frame.
And some people already saw that some of the generated frame look weird (nothing that future version can fix but still.
It might also lead to few artefact when an action or an event is in the next frame but in between the game generate frame that aren't aware at all of that events.
If the AI continues to insert frames using the same set of motion vectors until the server sends data updating the client and then receives new vectors saying "no actually this player stopped 38ms ago and went the opposite direction" you are going to see a lot of situations where the character model of other players will jump a few pixels. This would be especially noticeable in any game with giggle peaking or AD spam.
This is problematic because those frames are artificial. The game server doesn't actually see the player at that position. As a result this can cause no regs which are not the fault of the game or server but this new Nvidia tech.
I don't really see how you could fix this issue as it's a fundemental downside to having an AI predict motion and insert frames. That is, at least for online games. For single player games it's going to depend on how often the game engine is feeding those motion vectors and any associated latency with that. The average reaction time may be 250ms but humans can notice differences in motion and motion consistency down to single digit ms.
Technically, DLSS is something different and should be tagged as something different. But they want to surf on the good name DLSS have.
In a game you should have DLSS presets then DLSS 3.0 frame insertion settings. Or something like that. They should have called it something like DLFI instead.
meet Project SVP with Opti-Flow
Not that I wouldn't put it past Nvidia but ending support for current gen DLSS before the generation even ends is extremely anti-consumer. The bare minimum they should do is make the frame insertion only limited to 4000 series cards. 2000 and 3000 series users should still be able to use DLSS 3.0 and future iterations.
But many people choose to purchase a 20xx/30xx card Because they had DLSS. It is really indeed a sucker move for Nvidia to lock it to the 40xx series but the people that bought the previous card showed they were fine buying locked features.
Nvidia already claiming that people who bought into ampere spend 300$ more, so the new pricing was made with that into consideration. People vote with their wallet and they voted for overpriced GPU consuming a lot of power with locked in features.
And this is what Nvidia is giving them.
nvidia/comments/xjcqut
That AI frame generation to double the framerate, even in CPU limited situations seems like another black magic step, holy crap the potential there if it's good (ie, looks like regular 2.x upscaled frames) is insane.