Microsoft DirectStorage API Available, but Without GPU-accelerated Decompression

[btarunr]

Microsoft officially launched the DirectStorage API on the Windows PC platform, on Monday. The API enables direct data interactions between the GPU, graphics memory, and a storage device, so games have a more direct path to stream game assets to the graphics hardware. The API is compatible both with Windows 10 and Windows 11, although Microsoft is recommending the latter, for "in-built storage optimizations." Also, unlike previously reported, you don't necessarily need an NVMe-based storage device, such as an M.2 SSD with PCIe/NVMe interface. Even a SATA SSD with AHCI protocol will do. Microsoft is, however, recommending the use of an NVMe SSD for the best performance.

There is, however, a wrinkle. Microsoft isn't launching a killer feature of the DirectStorage API yet, which is GPU-accelerated asset decompression. This feature allowed GPUs to use compute shaders to decompress game assets that are stored in compressed asset libraries on the disk. Most games store their local assets this way, to conserve their disk footprint. Without this feature, unless there's special game code from the developer's end to utilize GPGPU for asset decompression; compressed game assets still have to rope in the CPU, and lengthen the pipeline. Microsoft stated that enabling GPU-accelerated decompression is "next on their roadmap."



View at TechPowerUp Main Site

 

[DeathtoGnomes]

It sounds promising, but how will it affect GPU performance? Putting the burden of decompression on the GPU could take up resource best used elsewhere. Having said that, I wanna predict that IF ( big if ) this gains traction with developers, this could see a change in gpu architecture, down the road, to accommodate the extra load. Games today, more often, use close to 100% gpu cycles and memory, I gotta wonder how much memeory this api needs to be efficient and not interfere with game performance.

On the pother hand, it also makes me wonder if this will change how developers will use this with the existing environment. Will new games with this function use less resources or not.

This API could also mean no CPU bottle necks, but that doesnt mean developers wont still use the CPU.

 

[Metroid]

At least something, now wait more 3 years minimum to see first games using it.

 

[nguyen]

okay so this would make NVME noticeably faster than SATA SSD at loading game at least? because right now there aren't much incentive to buy NVME.

 

[wolf]

From the presentation I watched about DirectStorage and how it worked, iirc it seemed like GPU decompression was the killer feature where most of the benefits can come from in terms of faster asset streaming and vastly reduced CPU/RAM overhead.

What are the benefits in the current state? faster loading than no directstorage at all? lowered VRAM requirements? it's been a while.

 

[Chris_Ramseyer]

At least something, now wait more 3 years minimum to see first games using it.

Try a few weeks.

 

[nguyen]

From the presentation I watched about DirectStorage and how it worked, iirc it seemed like GPU decompression was the killer feature where most of the benefits can come from in terms of faster asset streaming and vastly reduced CPU/RAM overhead.

What are the benefits in the current state? faster loading than no directstorage at all? lowered VRAM requirements? it's been a while.

Yeah I guess the current benefit of Directstorage is that it can leverage the higher throughput of NVME.

On related news NVME are now pretty cheap, they can match SATA SSD on GB/dollar, so yeah no point buying new SATA SSD now

 

[Chomiq]

At least something, now wait more 3 years minimum to see first games using it.

Technically we already had games using it, only on Xbox Series consoles instead of PC.

 

[GoldenX]

So, NVMe and a DirectX 12 Ultimate device.
If Nvidia doesn't allow the GTX 1600 series, I'm rioting.

 

[wolf]

If Nvidia doesn't allow the GTX 1600 series

Pretty sure only RTX series are DX12 Ultimate

 

[Legacy-ZA]

A shame, I upgraded my OS just so I can use this feature for my game load times, I was hoping for day one patches on massive games like Cyberpunk etc.

I guess now we wait some more.

 

[GoldenX]

Pretty sure only RTX series are DX12 Ultimate

The only difference between a 2000 or 3000 series and a 1600 series is RT support.
Mesh shaders, variable rate shaders, every other aspect of D3D12U is included, so denying support would be a dick move.

 

[Calmmo]

Feels like its been 3 years since they started talking directstorage and it's not even complete yet..

 

[Valantar]

Feels like its been 3 years since they started talking directstorage and it's not even complete yet..

It's been on Xbox Series consoles since day one. PC is another thing.

It sounds promising, but how will it affect GPU performance? Putting the burden of decompression on the GPU could take up resource best used elsewhere. Having said that, I wanna predict that IF ( big if ) this gains traction with developers, this could see a change in gpu architecture, down the road, to accommodate the extra load. Games today, more often, use close to 100% gpu cycles and memory, I gotta wonder how much memeory this api needs to be efficient and not interfere with game performance.

On the pother hand, it also makes me wonder if this will change how developers will use this with the existing environment. Will new games with this function use less resources or not.

This API could also mean no CPU bottle necks, but that doesnt mean developers wont still use the CPU.

It honestly shouldn't take much for a modern GPU to decompress even a few GB/s of data - these aren't particularly complex compression algorithms, and most decompression scales well in parallel. And at least RDNA2 is designed with this feature from the get-go - it's in the Xbox Series consoles, after all. AFAIK Ampere also has the same thing (IDK about Turing). It's a DX12 Ultimate feature after all - they have to support it to get the sticker. I wouldn't expect a noticeable drop in GPU throughput - and it'll definitely be less noticeable than the very noticeable lag in current systems from streaming in new assets from disk. It also has the potential to drastically lower system RAM usage from games, as there's no longer a need to cache decompressed assets there in wait for the GPU to need them.

As for effects of this: in the short term, it should reduce VRAM usage. Why? Because a lot of, if not most VRAM "usage" in games today is pre-caching of data that might be useful in a few seconds. If you're going down a hallway with a left, centre and right door at the end, the game pre-caches assets for going through all three rooms, and then ejects whatever was behind the two doors you didn't enter after you've made your choice. The shortening of the streaming pipeline should allow for much faster, more responsive, more on-the-fly streaming of assets, reducing the need for aggressive pre-caching, thus reducing VRAM usage. In the long term it should then allow for higher quality assets in the same memory footprint, or reduced pop-in as assets can be more efficiently fetched "just-in-time" - assuming the rest of the system is able to keep up. But overall, this is a much more efficient and smart way of handling the feeding of assets to the GPU, and is a big step forward in making the inner workings of PC gaming a tad less dumb.

From the presentation I watched about DirectStorage and how it worked, iirc it seemed like GPU decompression was the killer feature where most of the benefits can come from in terms of faster asset streaming and vastly reduced CPU/RAM overhead.

What are the benefits in the current state? faster loading than no directstorage at all? lowered VRAM requirements? it's been a while.

Yeah, I have the same impression. I guess the game could preemptively use the CPU to decode assets, cache them on the SSD, and then stream them in directly? That seems inefficient and wasteful though. IMO, without GPU decompression this has near zero utility.

Try a few weeks.

That ... seems overly optimistic. Even if code from console games can be ported over directly (which it can't, given the lack of GPU decompression), it'll take time to test, validate, and ensure fallbacks to non-DS setups work properly across various hardware and software combinations.

 

[stimpy88]

So how are they using this on the newest X-Box then? This sounds like some kind of BS.

 

[Chomiq]

So how are they using this on the newest X-Box then? This sounds like some kind of BS.

They are using it, compare load times of FH5 on PC with any of the Xbox Series consoles. They shaved off at least 50 seconds.

 

[stimpy88]

They are using it, compare load times of FH5 on PC with any of the Xbox Series consoles. They shaved off at least 50 seconds.

So why is it so difficult to bring it to Windows, which the X-Box already uses? Something about this seems off to me.

Microsoft makes Windows
AMD and Microsoft makes the display driver for the XBox

I can understand if nGreedia has an issue, and is finding it difficult, but not the other two. Maybe this is the holdup... It's the only thing that makes any kind of sense to me.

 

[Chomiq]

So why is it so difficult to bring it to Windows, which the X-Box already uses? Something about this seems off to me.

Xbox console use their own OS, not Windows (although they share same core).
In case of W10/W11 for PC use you're not dealing with closed off console with two sets of hardware configurations.
Also, MS is just being MS.

 

[stimpy88]

Xbox console use their own OS, not Windows.
In case of W10/W11 for PC use you're not dealing with closed off console with two sets of hardware configurations.
Also, MS is just being MS.

They use Windows, a customised version, but still windows. And the last time I checked, Microsoft makes Windows. So again, why the years extra to bring a feature from XBox to PC, when the code is the same, and Windows 11 has zero excuse, as I could see maybe some API change or addition that has not been brought to Windows 10 yet, could hold things up. But I still don't get why this is so hard for Microsoft to do.

Totally agree with you saying MS is just being MS! This and the nGreedia arguments are the only ones I can see as a reason for the holdup.

 

[Chomiq]

But I still don't get why this is so hard for Microsoft to do.

Go and ask them, while you're at it also ask them why they still use GUI elements dating back to Windows 95.

 

[stimpy88]

Go and ask them, while you're at it also ask them why they still use GUI elements dating back to Windows 95.

Yeah! I'd love to! MS is a joke these days.

 

[Valantar]

So how are they using this on the newest X-Box then? This sounds like some kind of BS.

So why is it so difficult to bring it to Windows, which the X-Box already uses? Something about this seems off to me.

Microsoft makes Windows
AMD and Microsoft makes the display driver for the XBox

I can understand if nGreedia has an issue, and is finding it difficult, but not the other two. Maybe this is the holdup... It's the only thing that makes any kind of sense to me.

Xbox is a closed software and hardware ecosystem, where MS has complete control over the hardware, and very tight control over all software. They are also paid licencing fees for all software sold for the platform, and thus can afford to provide a lot more engineering support to developers. PCs are an open hardware and software ecosystem, so the best MS (as the OS vendor) can do is to provide APIs and try and make them work across as wide a range of hardware as is feasible - and the hardware differences make for millions of possible combinations. The degree of complexity for getting this to work on PCs is drastically higher than on consoles, even if the OS core is the same and the hardware is partially shared.

 

[looniam]

Microsoft isn't launching a killer feature of the DirectStorage API yet, which is GPU-accelerated asset decompression.

hopefully they will next week at gdc.

if not, i won't hold my breath.

 

[bug]

It sounds promising, but how will it affect GPU performance? Putting the burden of decompression on the GPU could take up resource best used elsewhere. Having said that, I wanna predict that IF ( big if ) this gains traction with developers, this could see a change in gpu architecture, down the road, to accommodate the extra load. Games today, more often, use close to 100% gpu cycles and memory, I gotta wonder how much memeory this api needs to be efficient and not interfere with game performance.

On the pother hand, it also makes me wonder if this will change how developers will use this with the existing environment. Will new games with this function use less resources or not.

This API could also mean no CPU bottle necks, but that doesnt mean developers wont still use the CPU.

I think this is a great way to alleviate the pressure on the CPU, shifting the work to the GPU. In a world where the CPU is not nearly a bottleneck for high-res gaming.

 

[pavle]

The big part of it (DS API) gets delayed, not acceptable really. Besides there should be no concern about misusing GPU/not being available for rendering of said content because it's mainly loaded before showing anything (game level) on the screen except for game lobby so to speak.

Go and ask them, while you're at it also ask them why they still use GUI elements dating back to Windows 95.

It's called legacy.