NVIDIA Brings the Benefits of DirectStorage 1.1 to Vulkan Under its RTX-IO Brand

[btarunr]

NVIDIA dusted off its RTX-IO technology moniker which we thought it retired in the wake of the now-standardized DirectStorage API, in an attempt to bring its benefits to games powered by the Vulkan API. Team Green was the first to introduce such a technology to the PC platform, something functionally-similar existed with game consoles, where it plays a key role in speeding up game loading times. DirectStorage enables a means for the GPU to directly communicate with a storage device, with no round-trips to the CPU cores or main memory. This enables a quicker way for a game to transfer its assets to the video memory. NVIDIA introduces this as part of its latest GeForce 526.98 WHQL drivers. The same drivers also introduce official DirectStorage 1.1 support.

With DirectStorage 1.1, Microsoft went a step ahead and introduced GPU-accelerated game asset decompression. Game assets (such as textures) are stored on your disk in compressed form, and are decompressed as needed when your game loads. This involves the CPU cores, and tends to be slower when compared to getting the same job done by a GPU when not rendering 3D graphics. NVIDIA even developed a file-compression format optimized for highly-parallelized decompression hardware such as GPUs. The standardization by Microsoft extends this feature to other brands of GPUs (such as AMD and Intel, which are confirmed to be implementing it); but games powered by the Vulkan API were left out in the lurch. NVIDIA developed a Vulkan version of the original RTX-IO tech (which would go on to develop into DirectStorage), so now game developers with engines primarily designed for Vulkan (such as idTech), can speed up game load times.



View at TechPowerUp Main Site

 

[Chrispy_]

Heck, is this Nvidia contributing to an open standard for a change instead of selfishly keeping it proprietary?

The rapture must be nigh...

 

[CyberCT]

What games were announced that support direct storage, thus far? Any?

 

[defaultluser]

What games were announced that support direct storage, thus far? Any?

lazy river RT Experience (TM)

 

[isvelte]

What games were announced that support direct storage, thus far? Any?

I only know forspoken which has been delayed couple of times

 

[HisDivineOrder]

Remember that time when Nvidia launched the 30 series cards with the promise of DirectStorage improvements? And then remember when Microsoft sat on that and didn't do a thing with them for the last two years since then?

Thanks, Microsoft. The next generation of console will be out before you give PC gaming feature parity on something you announced before the Series X even came out.

 

[ryao]

DirectStorage enables a means for the GPU to directly communicate with a storage device, with no round-trips to the CPU cores or main memory.

This is completely false. The name DirectStorage might give that impression, but it is just a marketing name like DirectX, where nobody thinks that X is being made direct (Windows does not even use X). The Direct is just a marketing prefix so it sounds better than "storage API". As far as I have heard from Valve's developers, the initial DirectStorage library not only did everything on the CPU, but also works in Wine. DirectStorage 1.1 added a compute shader to do on-GPU decompression, but it is definitely not talking directly to storage devices. This is unlikely to ever change and does not need to change.

Talking about there being no-round trips to the CPU cores or memory is silly. A round trip would suggest that the GPU not only had the assets loaded, but then sent a copy of them back, which is unlikely to ever happen. Computers do not work like public libraries where you need to return the copy of data you borrowed.

 

[SunMaster]

This is completely false. The name DirectStorage might give that impression, but it is just a marketing name like DirectX, where nobody thinks that X is being made direct (Windows does not even use X). The Direct is just a marketing prefix so it sounds better than "storage API". As far as I have heard from Valve's developers, the initial DirectStorage library not only did everything on the CPU, but also works in Wine. DirectStorage 1.1 added a compute shader to do on-GPU decompression, but it is definitely not talking directly to storage devices. This is unlikely to ever change and does not need to change.

Are you sure? With info like https://docs.nvidia.com/cuda/gpudirect-rdma/index.html and https://www.theregister.com/2022/03/14/nvidia_gpu_data/ it certainly seems like it's viable.

 

[trsttte]

Heck, is this Nvidia contributing to an open standard for a change instead of selfishly keeping it proprietary?

The rapture must be nigh...

I don't see any mentions in the article that it will work on other GPUs

They developed the feature for Vulkan but are advertising it as RTX IO so maybe it will be nvidia exclusive until someone else shares an implemention with Kronos?

 

[Kaotik]

"Team Green was the first to introduce such a technology to the PC platform, something functionally-similar existed with game consoles, where it plays a key role in speeding up game loading times."

Radeon SSG, no matter how limited it was, says hi and calls BS.

 

[RedBear]

I don't see any mentions in the article that it will work on other GPUs

They developed the feature for Vulkan but are advertising it as RTX IO so maybe it will be nvidia exclusive until someone else shares an implemention with Kronos?

The compression format (GDeflate) is said to be open source. Of course you'll need to wait for AMD and Intel to implement this extension in their own drivers, which might take some time.

 

[londiste]

Heck, is this Nvidia contributing to an open standard for a change instead of selfishly keeping it proprietary?
The rapture must be nigh...

From other recent-ish examples - DXR and vulkan ray tracing
Besides, RTX-IO is just an implementation of the DirectStorage. Although probably they were part of DirectStorage itself as well.

 

[ryao]

Are you sure? With info like https://docs.nvidia.com/cuda/gpudirect-rdma/index.html and https://www.theregister.com/2022/03/14/nvidia_gpu_data/ it certainly seems like it's viable.

I am sure. I am this ryao:

Contributors to openzfs/zfs

OpenZFS on Linux and FreeBSD. Contribute to openzfs/zfs development by creating an account on GitHub.

github.com

When I first heard about DirectStorage, I looked into it since it piqued my curiosity as a filesystem developer. DirectStorage does not speak directly to hardware. It is just a marketing term for a convenience library on top of the existing OS APIs that does on-GPU decompression via a compute shader.

As for Nvidia's GPUDirect, that is potentially able to talk directly to NVMe, although you can expect to give up plenty of modern things such as software RAID, logical volume management, transparent encryption, etcetera if you are to use it. However, that is as related to DirectStorage as Chinese is related to English (i.e. they have a few superficial similarities, but that is all). It is also largely unnecessary since the CPU is the one telling the GPU what to do and it is very easy for the CPU to schedule reads into GPU VRAM. The only thing that their approach does is move the interrupt handler from the CPU to the GPU, but I have not heard anyone claim that is a bottleneck for graphics.

 

[Chrispy_]

From other recent-ish examples - DXR and vulkan ray tracing
Besides, RTX-IO is just an implementation of the DirectStorage. Although probably they were part of DirectStorage itself as well.

I can't cite any sources, but I believe Sony+AMD invented the concept of DirectStorage for the PS5's development. Since AMD make GPUs for Microsoft too, the next logical step was to match the PS5's new feature for the Xbox, and inevitably, it was added to DirectX for Windows and opened to Nvidia prior to Ampere's launch.

I don't honestly know which came first, but the PS5 technology reveal was demonstrated in realtime on ES, near-production silicon and the OS already had full software/API support for the feature way back in early 2020, before Nvidia even anounced Ampere. Clearly, Ampere silicon was already in pre-production at that point, but by then they were already a whole year, perhaps even 18-months behind Sony who were at the full trilogy of hardware/software/developer completion and demoing the finished technology to the public.

Nvidia's adoption and contribution to DirectStorage with RTX-IO was likely them jumping in to avoid FOMO, but I'm reasonably confident it was a feature born for consoles and driven by Sony/AMD for the PS5 first.

 

[AusWolf]

When you thought Nvidia kept showing off their own technologies... here they are, showing off something from Microsoft! The audacity!

 

[isvelte]

I can't cite any sources, but I believe Sony+AMD invented the concept of DirectStorage for the PS5's development. Since AMD make GPUs for Microsoft too, the next logical step was to match the PS5's new feature for the Xbox, and inevitably, it was added to DirectX for Windows and opened to Nvidia prior to Ampere's launch.

Pretty sure Nvidia has been talking about something like this since kepler days on their workstation product seminars, I think back in the day they were calling it gpu direct, or something like that. So i dont think amd+sony came up with it lol.

 

[Chrispy_]

Pretty sure Nvidia has been talking about something like this since kepler days on their workstation product seminars, I think back in the day they were calling it gpu direct, or something like that. So i dont think amd+sony came up with it lol.

Regardless of who came up with the idea first, Sony was showing it to the public before Nvidia even announced it.
"Action speaks louder than words" - Mark Twain

 

[ryao]

I can't cite any sources, but I believe Sony+AMD invented the concept of DirectStorage for the PS5's development. Since AMD make GPUs for Microsoft too, the next logical step was to match the PS5's new feature for the Xbox, and inevitably, it was added to DirectX for Windows and opened to Nvidia prior to Ampere's launch.

I don't honestly know which came first, but the PS5 technology reveal was demonstrated in realtime on ES, near-production silicon and the OS already had full software/API support for the feature way back in early 2020, before Nvidia even anounced Ampere. Clearly, Ampere silicon was already in pre-production at that point, but by then they were already a whole year, perhaps even 18-months behind Sony who were at the full trilogy of hardware/software/developer completion and demoing the finished technology to the public.

Nvidia's adoption and contribution to DirectStorage with RTX-IO was likely them jumping in to avoid FOMO, but I'm reasonably confident it was a feature born for consoles and driven by Sony/AMD for the PS5 first.

What do you consider the concept of DirectStorage to be? If you envision it to be a shader on the GPU talking to NVMe and loading data from the NVMe storage, you will find that only Nvidia has implemented that. If you envision it being that the CPU tells NVMe to read into GPU memory (and optionally decompress via a GPU shader), then that is nothing particularly new, although the industry seems to have only started talking about that recently. There is a 2017 USENIX paper on it:

https://www.usenix.org/system/files/conference/atc17/atc17-bergman.pdf

That paper itself does nothing more than use existing OS operations to achieve DMA and software has been able to do this for more than a decade.

What Sony did was implement a special hardware controller that does decompression. The CPU is still initiating these IO operations and handling the interrupt from the device saying that the IO has completed. The unified memory of the PS5 means that a copy into CPU memory is a copy into GPU memory, so Sony does not need to do anything special to make sure that the data is read into a GPU mapped buffer.

That said, I was able to confirm that Sony is not having the GPU do IO to NVMe by watching this presentation that Sony gave on the PS5:

The smoking gun is when they say that the game developer does not need to know about any of this and can do things as usual.

An engineer at Nvidia with whom I am acquainted disagreed. He thought Sony did have the GPU talk to NVMe. I asked him to watch that presentation and think about whether the description they give is consistent with the GPU talking to storage. His conclusion was that Sony was not having the GPU talk to NVMe, despite initially thinking that Sony had done the same thing Nvidia did based on the hype. He was rather disappointed, since he likes the idea of having the GPU talk to NVMe directly.

As for how it works, my understanding is that the I/O complex is doing compression transparently and hides that from the OS. That is why despite the compression being done, you cannot store more than the amount of data that the NAND flash is rated to store (unlike ZFS, which allows storage to exceed the raw storage capacity via compression). While the flash itself might be limited to 5.5GB/sec, the transparent decompression allows speeds to reach 9GB/sec, which is presumably the limit of their custom Kraken decompressor.

 

[Chrispy_]

What do you consider the concept of DirectStorage to be?

I think the answer to that question depends heavily on when we're talking about; The answer isn't even completely clear right now, when specifically talking about the Microsoft DirectStorage API as it's evolving even now despite still being effectively pre-release in terms of real-world examples taking advantage of it.

If we're talking about when this feature (which has been around since pre-2017) had its first real-world release, It's clear Sony/AMD kickstarted the process with the PS5's Kraken Engine which keeps data compressed both in VRAM and on NVMe, which is not how Microsoft's implementation works at the moment. The distinction is kind of unimportant, IMO. What Sony did is woke up Microsoft who realised that if Sony can do asset compression via the GPU and read compressed assets direct from NVMe with the AMD hardware in the PS5, the incredibly similar hardware in the Xbox should be able to do it too. DirectStorage (aka Xbox Velocity Architecture) was a copy of Sony's solution, born at around the time Microsoft learned of Sony's implementation. RTX-I/O uses shaders to depcompress assets read from NVMe whilst the PS5 and Xbox hardware has dedicated GPU decode blocks dedicated to the storage.

DirectStorage, RTX-I/O, XBox Velocity Architecture, and Sony's PS5 Kraken Engine are not 1:1 directly comparable. The two most closely-related are Velocity and Kraken simply because they use very similar silicon with heavily-overlapping capabilities. RTX-I/O uses shaders not tensor cores, so in theory it's not limited to RTX hardware at all, and I suspect Vulkan will get a GPU-agnostic equivalent in due-course.

 

[ryao]

PS5's Kraken Engine which keeps data compressed both in VRAM and on NVMe

My understanding of Sony's presentation here is that it does not keep data in VRAM (actually RAM) compressed. It is only compressed on NVMe.

DirectStorage, RTX-I/O, XBox Velocity Architecture, and Sony's PS5 Kraken Engine are not 1:1 directly comparable.

I would not compare any of them.

The two most closely-related are Velocity and Kraken simply because they use very similar silicon with heavily-overlapping capabilities.

The Xbox has a hardware decompression module. People have discussed using these to accelerate storage IO for years. There is a talk from 2014 that discusses doing the same in ZFS:

https://openzfs.org/w/images/6/63/Lightning_Talk-Zacodi_Labs-Maxim_Martynov.pdf

I suspect Vulkan will get a GPU-agnostic equivalent in due-course.

DirectStorage is just a library. Anyone can write one that does what it does. These guys did:

https://www.usenix.org/system/files/conference/atc17/atc17-bergman.pdf

I am not sure why people are overexcited about these things. Faster hardware has always been on its way. There is no radical overhaul needed for it to work. Making better use of OS APIs has always been better too and there is nothing stopping people from doing that right now without any special libraries.

 

[Chrispy_]

The smoking gun is when they say that the game developer does not need to know about any of this and can do things as usual.

Yup. At Ampere's launch I said it would be a generation or two before we saw any real-world use from DirectStorage on PC. So far that looks to be on track - The entire Ampere generation passed with zero results, and since any games starting to look into it now won't be release for 18-36 months, Lovelace will probably see it skipped too.

 

[hexaae]

Still nobody mentioned GPU requirements for RTX-IO... Will Pascal (1070/1080) cards support it for example or you're giving for granted everybody today has a RTX GPU?
AFAIK DirectStorage requirements are just: Win11 + SSD/NVMe + GPU DX12 and Shader Model 6.0 compatible, but I'm not sure about greedy NVidiaware...

EDIT: from a brief search on the net it seems will require GTX 3000+ at least.... which would be usual NVidia shame

 

[AusWolf]

Still nobody mentioned GPU requirements for RTX-IO... Will Pascal (1070/1080) cards support it for example or you're giving for granted everybody today has a RTX GPU?
AFAIK DirectStorage requirements are just: Win11 + SSD/NVMe + GPU DX12 and Shader Model 6.0 compatible, but I'm not sure about greedy NVidiaware...

EDIT: from a brief search on the net it seems will require GTX 3000+ at least.... which would be usual NVidia shame

That sounds like DLSS 3.0. All Nvidia cards from RTX 2000 series have the necessary hardware, but nVidia made sure it only runs on RTX 4000 series to make sure you buy one, even though the feature is only supported by a handful of games and is totally unnecessary 99% of the time.

 

[londiste]

That sounds like DLSS 3.0. All Nvidia cards from RTX 2000 series have the necessary hardware, but nVidia made sure it only runs on RTX 4000 series to make sure you buy one, even though the feature is only supported by a handful of games and is totally unnecessary 99% of the time.

"Have the necessary hardware" might not be sufficient. As an example GTX cards got DXR support (and older Radeons never did despite initial indications they would).

EDIT: from a brief search on the net it seems will require GTX 3000+ at least.... which would be usual NVidia shame

RTX IO is designed to be open source. It works across various operating systems and graphics APIs, computer-accelerated APIs, and SDKs.

We will have to wait and see I guess.

 

[Chrispy_]

As an example GTX cards got DXR support (and older Radeons never did despite initial indications they would).

I'd forgotten that the 5700-series was expecting DXR support at one point in time.

Perhaps the absolute dumpster-fire of RTX support at launch was enough to abandon that idea. I remember Nvidia promising dozens on titles with raytracing when the 20-series launched, and 18 months later it was basically still only enough titles to count on your fingers, the performance hit being crippling for all but the two best cards, and a few implementations like SOTR being almost indistinguishable from the raster version, apart from the massive hit to framerate.

Despite buying into RTX at launch, I don't think any games were truly worth using RTX on, other than enabling it briefly out of curiousity, until the next generation of games and the next generation of GPUs came along. 2018 and 2019, and most of 2020 were dark days for Raytraced games.