Tuesday, August 22nd 2023
NVIDIA Announces DLSS 3.5 Ray Reconstruction Technology, Works on GeForce 20 and Newer
At this year's Gamescom 2023, NVIDIA will be releasing several new updates for the GeForce Gaming crowd. These are led by the announcement of the new DLSS 3.5 Ray Reconstruction feature, available this Fall for all GeForce RTX GPUs (RTX 20-series and later). DLSS 3.5 introduces a new feature called "Ray Reconstruction Technology" that's specifically designed to improve the way ray traced elements look in games. While traditional rasterization calculates every single pixel, for each frame, real-time ray tracing cannot do that, for performance reasons. During rendering, only few rays are shot in a coarse grid, which leaves empty "black" gaps in-between the ray outputs. To fill those in, a denoiser is used that runs various algorithms to literally fill in the blanks.
With DLSS 3.5, NVIDIA introduces a new denoiser that's optimized to work hand-in-hand with DLSS 2 upscaling, to provide better image quality results that are more correct at the same time. This feature relies on the Tensor Cores (not the RT cores, we asked), so it is available on all GeForce RTX graphics cards (Turing and newer).
The picture below shows the traditional way to do RT effects. Please note that DLSS 2 upscaling is enabled here—the image is composited at low resolution first and then scaled to native size.
In a first step, the engine creates the geometry and materials, but without any shading. This information is used to create the BVH acceleration structure for ray tracing, which helps to determine where rays intersect with world geometry. Next, a number of rays is cast and their path is traced, to calculate intersections, possibly let them bounce, maybe even several times. These results are now fed to the denoiser, which turns the individual pixels into a continuous image that looks like a ray traced reflection, shadow, lighting or ambient occlusion. With upscaling enabled, the denoiser generates output at the lower render resolution, not the final native output—the denoiser isn't even aware of the final resolution. On top of that, another problem is that the upscaler doesn't know anything about rays, it just sees the pixel output from the denoiser—all the original ray tracing values are lost at that stage.
The biggest problem with denoisers is that they rely on previous frames, to "collect" enough pixel data for the final image. This means that the RT output is an average of several previous frames. The slide above details such problematic cases. For example, the mirror on a moving car gets combined throughout several frames, which results in ghosting artifacts. Another problem is with subtle illumination effects and reflections that just look smeared out.
NVIDIA's innovation with DLSS 3.5 is that they are combining both the denoising and the upscaling steps into a single combined step that has more information available, which promises a higher-quality output image. The low-res output is combined with the output from rasterization, the ray tracing steps and the motion vectors, and everything is painted directly into a high-res output image, 4K in this case. The DLSS 3.5 algorithm also takes into account previous frames (temporal feedback), just like DLSS 2. Once upscaling is completed, another pass is made for the DLSS 3 Frame Generation feature (when enabled).
Here's some results provided by NVIDIA that show how DLSS 3.5 Ray Reconstruction promises to enhance the RT fidelity over classic denoising techniques.
Ray Reconstruction has negligible performance cost of its own, on frame-rate comparisons NVIDIA showed taken on an RTX 40-series GPU, DLSS 3.5 RR offers marginally higher frame-rates than DLSS 3 FG. NVIDIA made it clear that DLSS 3.5 is not a performance enhancing feature, but the focus is on image quality. Depending on the scene, the performance will be virtually identical, slightly better or slightly worse. In theory it is possible that game developers reduce the number of rays when DLSS 3.5 is enabled, which would lower the RT performance hit, and improve framerates—still with improved image quality. There's no handholding for that though, this is purely a game dev feature and out of the scope of NVIDIA's DLSS 3.5 implementation.
DLSS 3.5 will not only be available in games, but also in NVIDIA's professional D5 renderer, where it will enable real-time previews of stunning detail.
When it releases this Fall, DLSS 3.5 will be enabled on all GeForce RTX GPUs through a driver update. You now have three distinct subsets of DLSS—Super Resolution (SR), or the core image upscaling tech; Frame Generation (FG) introduced with DLSS 3, which doubles frame-rates by generating alternate frames using AI; and now the new Ray Reconstruction (RR) feature. DLSS 3.5 RR will work with all RTX GPUs, as all generations include tensor cores. On older RTX 20-series "Turing" and RTX 30-series "Ampere," DLSS 3.5 will work exactly like it does on the latest RTX 40-series "Ada," but FG won't be available. Games with support for Ray Reconstruction will have an additional checkbox "enable Ray Reconstruction", just like there's a checkbox "enable Frame Generation". We confirmed with NVIDIA that running DLAA with Ray Reconstruction is supported—you don't have to use the upscaler at all times.
While the naming is a bit confusing, it's great to see that NVIDIA is constantly improving their technology. There's no news yet regarding AMD's FSR 3; perhaps an announcement might come at Gamescom. However, from a technical standpoint, we'd classify Ray Reconstruction as "DLSS 2.5", because it has absolutely nothing to do with DLSS 3 Frame Generation, and is closely interlinked with DLSS 2 upscaling. It seems NVIDIA is now releasing all similar technologies under their established "DLSS" brand name, which is further segregated by feature. For example, "DLSS 3 Frame Generation" is only supported on GeForce 40—this announcement does not change that. The new "DLSS 3 Ray Reconstruction" works on GeForce 20 and newer though, just like "DLSS 2 Upscaling" works on GeForce 20, too.
With DLSS 3.5, NVIDIA introduces a new denoiser that's optimized to work hand-in-hand with DLSS 2 upscaling, to provide better image quality results that are more correct at the same time. This feature relies on the Tensor Cores (not the RT cores, we asked), so it is available on all GeForce RTX graphics cards (Turing and newer).
While the naming is a bit confusing, it's great to see that NVIDIA is constantly improving their technology. There's no news yet regarding AMD's FSR 3; perhaps an announcement might come at Gamescom. However, from a technical standpoint, we'd classify Ray Reconstruction as "DLSS 2.5", because it has absolutely nothing to do with DLSS 3 Frame Generation, and is closely interlinked with DLSS 2 upscaling. It seems NVIDIA is now releasing all similar technologies under their established "DLSS" brand name, which is further segregated by feature. For example, "DLSS 3 Frame Generation" is only supported on GeForce 40—this announcement does not change that. The new "DLSS 3 Ray Reconstruction" works on GeForce 20 and newer though, just like "DLSS 2 Upscaling" works on GeForce 20, too.
89 Comments on NVIDIA Announces DLSS 3.5 Ray Reconstruction Technology, Works on GeForce 20 and Newer
It's actually more like DLSS 3 for old RTX edition that you're gonna get.
DLSS 3 Frame Gen is only supported by 4000 series.
The naming is completely rubbish.
The Ray reconstruction is useful for cards that have the oomph to run games with RT on. For lower end RTX cards, it does not offer anything.
In cyberpunk, yes, the reflections will be improved. And it would be great to see it in control where the noise can clearly be seen in many surfaces.
Another way of saying it, is that with DLSS 3.5 Nvidia will finally no longer be lieing to us; DLSS 1.0 launched for RTX cards 5 years ago with the promise that it used the AI tensor cores in RTX cards to do the denoising for raytracing effects. After a few months that turned out to be a lie, since other GPUs without tensor cores were eventually supported. To backtrack on their initial lie, Nvidia simply claimed that when they said "AI cores" they meant they were using using deep learning AI back at Nvidia HQ to pre-bake tuned denoising algorithms into the game-ready driver profiles for each supported title. Now, they're claiming it was hand-tuned all along, not AI deep learning at all.
So, five years after launch, Nvidia is finally using AI as promised in its raytracing and DLSS. Better late than never - even if it is little more than a tech-demo for the first couple of years until game developers bother to implement it in significant numbers.
From what I gathered from the nebulous and misleading marketing, it seems to just be an AI enhanced denoiser.
Which is just fine, but I fail to see what it's actually going to make better. The intentionally f'd up images with half finished denoising as the "former example" have convinced me that this is a mostly moot upgrade since they had to make the current good-looking denoised images intentionally ugly. This marketing basically took a beautiful girl's face, smeared her with shit, and then cleaned her up and added some makeup and took the smeared face and made up face and went "look how beautiful she is with our makeup!". Ye, but without the smearing she was beautiful already.
The current denoising already works, it makes images already good looking with RT on.
So what does this AI denoising actually do? Enhance speed? Enhance some detail that you only notice when you zoom in 15x? Does it have an actual proper A/B comparison that doesn't require smearing some shit on the image?
The one image in Cyberpunk that compared a small pathway under lights honestly didn't make me feel like I was looking at anything different. And no, having to zoom in on the image doesn't mean crap. I don't use a lens when I game, nor do I randomly zoom in on things. I just play. If I can't see anything different on the fullscreen, then I don't see the point.
(watch the Nvidia cultists come and scream that if I don't see the Glory of DLSS I'm just not worth their time and that everyone should just naturally admire the Glory without any questions)
And by that we will finally have fake frame (dlss3) based on partial image (dlss4+3.5) all mashed up from a low res image (dlss2). Great.
Can't wait to get to dlss10, where you only need one click on the mouse and than the whole game rander and play itself automatically based on your past gaming profile.
The more you play, the higher your fps.
Nvidia has only idiots so they make pointlessly complex crap to impress them and then make superbly vapid crap to make them buy.
Idiots admire Nvidia, idiots wait for AMD to deliver, geniuses just play Factorio - me"If you don't see the Glory of DLSS, that's your problem"
Ah good to see that the cult mentality is doing as good as ever.Well, everything in games is fake. A lot in life is fake.
We live in a world of convincing fakeness.
This isn't even the most extreme example but still ,if you can't see the differences between these screenshots then it's totally you my man. Nvidia CANNOT fix blindness.
The worst is, this kind of talk works on most idiots.
Someone needs to already understand how the rt works and why there is a latency on the SR+FG image while on the last one everything is lit correctly.
You should have uploaded an image where the reflection resolution is better with RR on.
Not everyone understands what RT does on the scene, let alone the negatives that come with it.
As for Nvidia's screenshot, the last picture seems to be missing a few white light sources around that purple neon thing, that's why the tone of the whole image is more purple. That's the only difference I see. Could that be the reason for the higher FPS?
But the cultists must believe what the Master says...
Eventually they'll release some explanation of what they're actually doing that's different. AI-ifying the denoising process can't hurt much, and should bring some speed upgrades/faster RT. That's the only advantage I can think of.
DLSS 2 shouldn't be generating that many frames. Uhm, it just renders faster?May be transistors stopped shrinking since 10 years ago.
Now they're shrinking our wallet by unlocking what they have been hiding 10 years ago.
I marked the light sources that I was talking about in my previous post, and their reflections on the floor. The last picture seems to be missing them. It's easy to produce more frames per second with less detail.
Those flares on top were like icing on the cake. Sad to see the new DLSS punched them out.
Here's the same frame with DLSS 6. Nvidia finally managed to animate each frame. :rockout:
When you walk in the game, you will definitely notice that the light bounces on the surfaces have latency. It's ridiculously obvious.
In this image it's more than obvious that the pavement does not reflect the lights above because it's some msecs behind in time.
The same happens with the reflections. The resolution is low or low and later gets better after it is rendered.
The problem is people do not realize how difficult it is for a hardware to process this in msecs and post bullshXts about RT.
A good CPU can render one, just one, frame in minutes while a GPU has to render 60+ frames in ONE second.
How can you say that nVidia, or even AMD, are not good enough for delivering RT?
Their GPUs, both radeon and geforce, are multiple years ahead of the CPUs.
If it weren't about RT, we would get the same oily shXtty plasticky graphics like all the console ports we got this year.
Increasing the texture resolution is NOT better graphics.