Thursday, May 20th 2021
AMD's 2019 "Gaming Super Resolution" Patent Could be the Blueprint for FidelityFX Super Resolution?
As AMD's upcoming feature that rivals NVIDIA DLSS nears release, reportedly this June, a USPTO patent application sheds some light on its inner workings. In its November 2019 application, AMD describes the feature as "Gaming Super Resolution." The abstract points to what the feature essentially does—downscaling or upscaling images using various methods, with the goal of improving performance, without much loss in fidelity. "The present application provides devices and methods for efficiently super-resolving an image, which preserves the original information of the image while upscaling the image and improving fidelity. The devices and methods utilize linear and non-linear up-sampling in a wholly learned environment," the application reads.
The application emphasizes on using a combination of linear and non-linear upscaling methods to improve the fidelity of the lower-resolution render output to the user's display resolution, including leveraging a "wholly learned" AI deep-learning network. This would be a DNN that relies on ground-truth information to reconstruct some fidelity to the upscaled image. "The devices and methods include a gaming super resolution (GSR) network architecture which efficiently super resolves images in a convolutional and generalizable manner.""The GSR architecture employs image condensation and a combination of linear and nonlinear operations to accelerate the process to gaming viable levels. GSR renders images at a low quality scale to create high quality image approximations and achieve high framerates. High quality reference images are approximated by applying a specific configuration of convolutional layers and activation functions to a low quality reference image. The GSR network approximates more generalized problems more accurately and efficiently than conventional super resolution techniques by training the weights of the convolutional layers with a corpus of images."
A lot of water has flown under the bridge since November 2019. DLSS 2.0, introduced in March 2020, was a runaway success as a feature. It doesn't rely on wholly-learned DNNs, and uses temporal feedback techniques to reconstruct details in the final image. The FidelityFX Super Sampling feature that does come out next month, could hence be vastly different, as AMD is taking its own sweet time testing and validating the technology with game developers.
Sources:
Beyond3D Forums, VideoCardz, USPTO
The application emphasizes on using a combination of linear and non-linear upscaling methods to improve the fidelity of the lower-resolution render output to the user's display resolution, including leveraging a "wholly learned" AI deep-learning network. This would be a DNN that relies on ground-truth information to reconstruct some fidelity to the upscaled image. "The devices and methods include a gaming super resolution (GSR) network architecture which efficiently super resolves images in a convolutional and generalizable manner.""The GSR architecture employs image condensation and a combination of linear and nonlinear operations to accelerate the process to gaming viable levels. GSR renders images at a low quality scale to create high quality image approximations and achieve high framerates. High quality reference images are approximated by applying a specific configuration of convolutional layers and activation functions to a low quality reference image. The GSR network approximates more generalized problems more accurately and efficiently than conventional super resolution techniques by training the weights of the convolutional layers with a corpus of images."
A lot of water has flown under the bridge since November 2019. DLSS 2.0, introduced in March 2020, was a runaway success as a feature. It doesn't rely on wholly-learned DNNs, and uses temporal feedback techniques to reconstruct details in the final image. The FidelityFX Super Sampling feature that does come out next month, could hence be vastly different, as AMD is taking its own sweet time testing and validating the technology with game developers.
28 Comments on AMD's 2019 "Gaming Super Resolution" Patent Could be the Blueprint for FidelityFX Super Resolution?
The talk of ground-truth for DNN sounds like AMD might be 'resolution spiking' every x amount of frames to provide reference data back to the DNN, which I'm intrigued to see if so what impact it may have on frame time.
i was making fun of the title wording. I'm fully aware the patent is more specific.