Ray Reconstruction Technology

With DLSS 3.5, NVIDIA introduces a new ray tracing denoiser that's optimized to work hand-in-hand with DLSS 2 upscaling, to provide better image quality results that are more correct at the same time. This feature relies on the Tensor Cores (not the RT cores, we asked), so it is available on all GeForce RTX graphics cards (Turing and newer).


In modern game engines, ray tracing works in the following way: In a first step, the engine creates the geometry and materials, but without any shading. This information is used to create the BVH acceleration structure for ray tracing, which helps to determine where rays intersect with world geometry. Next, a number of rays is cast and their path is traced, to calculate intersections, possibly let them bounce, maybe even several times. These results are now fed to the denoiser, which turns the individual pixels into a continuous image that looks like a ray traced reflection, shadow, lighting or ambient occlusion. With upscaling enabled, the denoiser generates output at the lower render resolution, not the final native output—the denoiser isn't even aware of the final resolution. On top of that, another problem is that the upscaler doesn't know anything about rays, it just sees the pixel output from the denoiser—all the original ray tracing values are lost at that stage.


The biggest problem with denoisers is that they rely on previous frames, to "collect" enough pixel data for the final image. This means that the RT output is an average of several previous frames. The slide above details such problematic cases. For example, the mirror on a moving car gets combined throughout several frames, which results in ghosting artifacts. Another problem is with subtle illumination effects and reflections that just look smeared out.


NVIDIA's innovation with DLSS 3.5 is that they are combining both the denoising and the upscaling steps into a single combined step that has more information available, which promises a higher-quality output image. The low-res output is combined with the output from rasterization, the ray tracing steps and the motion vectors, and everything is painted directly into a high-res output image, 4K in this case. The DLSS 3.5 algorithm also takes into account previous frames (temporal feedback), just like DLSS 2. Once upscaling is completed, another pass is made for the DLSS 3 Frame Generation feature (when enabled).


Here's a slide that explains how lighting effects can look better with Ray Reconstruction.