AMD Develops ROCm-based Solution to Run Unmodified NVIDIA's CUDA Binaries on AMD Graphics

[AleksandarK]

AMD has quietly funded an effort over the past two years to enable binary compatibility for NVIDIA CUDA applications on their ROCm stack. This allows CUDA software to run on AMD Radeon GPUs without adapting the source code. The project responsible is ZLUDA, which was initially developed to provide CUDA support on Intel graphics. The developer behind ZLUDA, Andrzej Janik, was contracted by AMD in 2022 to adapt his project for use on Radeon GPUs with HIP/ROCm. He spent two years bringing functional CUDA support to AMD's platform, allowing many real-world CUDA workloads to run without modification. AMD decided not to productize this effort for unknown reasons but did open-source it once funding ended per their agreement. Over at Phoronix, there were several benchmarks testing AMD's ZLUDA implementation over a wide variety of benchmarks.

Benchmarks found that proprietary CUDA renderers and software worked on Radeon GPUs out-of-the-box with the drop-in ZLUDA library replacements. CUDA-optimized Blender 4.0 rendering now runs faster on AMD Radeon GPUs than the native ROCm/HIP port, reducing render times by around 10-20%, depending on the scene. The implementation is surprisingly robust, considering it was a single-developer project. However, there are some limitations—OptiX and PTX assembly codes still need to be fully supported. Overall, though, testing showed very promising results. Over the generic OpenCL runtimes in Geekbench, CUDA-optimized binaries produce up to 75% better results. With the ZLUDA libraries handling API translation, unmodified CUDA binaries can now run directly on top of ROCm and Radeon GPUs. Strangely, the ZLUDA port targets AMD ROCm 5.7, not the newest 6.x versions. Only time will tell if AMD continues investing in this approach to simplify porting of CUDA software. However, the open-sourced project now enables anyone to contribute and help improve compatibility. For a complete review, check out Phoronix tests.



View at TechPowerUp Main Site | Source

 

[WonkoTheSaneUK]

Nvidia lawsuit arriving in 3...2...1...

 

[ThomasK]

Nvidia lawsuit arriving in 3...2...1...

I wouldn't be surprised, judging by the amount of nvidia lawyers I see in many posts' comment section.

 

[remekra]

It's written on github that AMD also abandoned the project. That's why he is making it open source as it was part of the contract.
One interesting tidbit for us here is:

• What's the future of the project?
  With neither Intel nor AMD interested, we've run out of GPU companies. I'm open though to any offers of that could move the project forward.
  Realistically, it's now abandoned and will only possibly receive updates to run workloads I am personally interested in (DLSS)

So who knows, DLSS on AMD GPUs soon?

 

[Beermotor]

Nvidia lawsuit arriving in 3...2...1...

Possibly. The individual is just a contractor so they'd go after AMD, Intel, and possibly Microsoft (Github) for hosting it. They're kind of asking for the FTC to get involved in an anti-trust action if they get too uppity.

It's written on github that AMD also abandoned the project. That's why he is making it open source as it was part of the contract.
One interesting tidbit for us here is:

So who knows, DLSS on AMD GPUs soon?

Remember that big Nvidia leak where the DLSS source got leaked? AMD RDNA2 GPUs could reportedly run DLSS at a hardware-level but nothing ever came of it. Nobody in their right mind would expose themselves to that kind of civil legal jeaopardy.

Nvidia's lawyers could easily construe the software to be based on stolen source code, even if it was a clean-room implementation like ZLUDA.

 

[remekra]

Possibly. The individual is just a contractor so they'd go after AMD, Intel, and possibly Microsoft (Github) for hosting it. They're kind of asking for the FTC to get involved in an anti-trust action if they get too uppity.


Remember that big Nvidia leak where the DLSS source got leaked? AMD RDNA2 GPUs could reportedly run DLSS at a hardware-level but nothing ever came of it. Nobody in their right mind would expose themselves to that kind of civil legal jeaopardy.

Nvidia's lawyers could easily construe the software to be based on stolen source code, even if it was a clean-room implementation like ZLUDA.

Yeah I'm not holding my breath. Considering that DLSS basically became their most important feature that is always mentioned when someone is asking whether to go AMD or nvidia if there suddenly was a way to run DLSS on AMD GPU, nvidia would sue everybody that probably even downloaded that

 

[dragontamer5788]

OptiX and PTX assembly codes still need to be fully supported.

Um... OptiX compiles down into NVidia-specific hardware. PTX is similarly NVidia-specific pseudo-assembly (PTX compiles into SASS / true NVidia assembly), so... I'm not sure if anything could ever truly support all PTX code.

But a subset of PTX (ex: add, multiply, and other common operations) probably could be built for AMD. But no one writing PTX would do the common + or * commands of CUDA/C++. The only reason anyone would dip down to PTX assembly is to take advantage of hardware-specific features and/or hardware-specific performance characteristics.

Neither would ever get done IMO. So the project as it stands probably has hit an end. CUDA/C++ cross-compiled binaries running on AMD/Intel sounds "good enough" to me... while later stages of this coding project sound like a lot of work without much gains...

 

[Dirt Chip]

The end of NV monopol in the professional, CUDA world?

 

[Frank_100]

I think it would be to nvidia's advantage to have such a project allowing anyone to use cudu.

Once your using their software you are halfway to being a customer.

Camel nose -> tent.

 

[progste]

Could be big!
I wonder if it would be possible to make some old physx games run on AMD, would be nice to have it for the Batman Arkham games.

 

[Dr. Dro]

I wouldn't be surprised, judging by the amount of nvidia lawyers I see in many posts' comment section.

So anyone that doesn't have a strictly negative view of "the super evil green monster" is not only a shill, but also a lawyer for Nvidia? I wish it was that easy.

I think it would be to nvidia's advantage to have such a project allowing anyone to use cudu.

Once your using their software you are halfway to being a customer.

Camel nose -> tent.

Everyone already uses CUDA regardless, AMD's market share has always been near irrelevant for the compute market and the absence of CUDA compatibility a massive negative blemish in the Radeon business. It's only been recently with extremely powerful CDNA hardware and software being developed for it that Instinct began making a dent.

The simple fact that even this translation layer outperforms AMD's native OpenCL stack speaks volumes of how ridiculously bad their compute system has been. This will need to be maintained in order to continue to keep up with newer versions of CUDA, and the funding for that will likely have to come from AMD, but it's a massive step in the right direction.

 

[john_]

AMD decided not to productize this effort for unknown reasons but did open-source it once funding ended per their agreement.

Probably AMD doesn't want to be seen as supporting Nvidia's CUDA officially. So they funded the project and then did what they usually do. Open source it.
To be fair, if AMD wants to see CUDA going away, the last thing they need to do is make CUDA run on their GPUs officially. If they support it, then they will be like supporting the only reason today Nvidia makes billions every quarter.

 

[AnotherReader]

So anyone that doesn't have a strictly negative view of "the super evil green monster" is not only a shill, but also a lawyer for Nvidia? I wish it was that easy.



Everyone already uses CUDA regardless, AMD's market share has always been near irrelevant for the compute market and the absence of CUDA compatibility a massive negative blemish in the Radeon business. It's only been recently with extremely powerful CDNA hardware and software being developed for it that Instinct began making a dent.

The simple fact that even this translation layer outperforms AMD's native OpenCL stack speaks volumes of how ridiculously bad their compute system has been. This will need to be maintained in order to continue to keep up with newer versions of CUDA, and the funding for that will likely have to come from AMD, but it's a massive step in the right direction.

There's a more likely explanation for the performance difference in many tests. Due to CUDA's dominance, more effort has been poured by developers into optimizing their applications for it.

 

[john_]

Could be big!
I wonder if it would be possible to make some old physx games run on AMD, would be nice to have it for the Batman Arkham games.

Nvidia's drivers are lock free for a number of years now. Buy a small Nvidia GPU, add it as a secondary GPU and CUDA/PhysX would be enabled and free to run on that GPU even if you have an AMD as primary. No patches needed anymore.

 

[Dr. Dro]

There's a more likely explanation for the performance difference in many tests. Due to CUDA's dominance, more effort has been poured by developers into optimizing their applications for it.

Indeed, but there is still a translation layer involved. It's not the first time we've seen translation layers outperform AMD's native solutions either, DXVK had widely been considered a way to fix performance on DX11 games until the new PAL abstraction layer UMD was developed. In some cases, still is.

 

[Denver]

I wouldn't be surprised, judging by the amount of nvidia lawyers I see in many posts' comment section.

Well, If I'm not mistaken, in the European Union there were discussions about CUDA being an anti-competitive monopoly tactic, and they should by law force Nvidia to open the ecosystem to competitors.

@topic, I find it strange as hell that this gimmick runs faster than AMD's own HIP, it's the same as saying that the company doesn't know its own hardware to create optimized code, in other words, they're underutilizing its capabilities. Incompetence? Lack of investment? What's it lol

 

[Nordic]

Possibly. The individual is just a contractor so they'd go after AMD, Intel, and possibly Microsoft (Github) for hosting it. They're kind of asking for the FTC to get involved in an anti-trust action if they get too uppity.

AMD hired the same contractor that Intel had hired to get Cuda running on Intel gpu's. If Nvidia was going to sue they would have done so already.

 

[ThomasK]

Well, If I'm not mistaken, in the European Union there were discussions about CUDA being an anti-competitive monopoly tactic, and they should by law force Nvidia to open the ecosystem to competitors.

Not gonna complain about the EU pushing for a more open-standard market. Look at what they did to apple, enforcing them to adopt USB-C, worldwide.

 

[Count von Schwalbe]

@topic, I find it strange as hell that this gimmick runs faster than AMD's own HIP, it's the same as saying that the company doesn't know its own hardware to create optimized code, in other words, they're underutilizing its capabilities. Incompetence? Lack of investment? What's it lol

OpenCL is a serious afterthought in most applications. OpenCL/CUDA performance difference is a major thing even on Nvidia.

Imagine a racecar with a slightly worse transmission (translation layer) and the same engine (GPU) but weighing much less (application optimization). Still faster than one with a better transmission.

 

[Vya Domus]

If Nvidia was going to sue they would have done so already.

If AMD is a tiny name on the map Intel is virtually nonexistent. No one cares if you can run CUDA on Intel GPUs, but they might if you can run it on these new MI300s.

But it doesn't matter because as far as I can tell this isn't actually backed by AMD.

 

[lexluthermiester]

Nvidia lawsuit arriving in 3...2...1...

I wouldn't be surprised, judging by the amount of nvidia lawyers I see in many posts' comment section.

Possibly. The individual is just a contractor so they'd go after AMD, Intel, and possibly Microsoft (Github) for hosting it.

NVidia's legal dept can do nothing about this. Every bit of it is covered by fair-use. NVidia would be fools to take any legal action whatsoever.

They likely don't care.

 

[LabRat 891]

AMD hired the same contractor that Intel had hired to get Cuda running on Intel gpu's. If Nvidia was going to sue they would have done so already.

NGL, I feel awful for the team/individual that worked on all this.
I might be overstating the complexity involved, but I see this as almost John Carmack tier,
and both Intel and AMD left them in the dust.



Looks like Intel and AMD both merely wanted a proof of concept.

Hopefully, this will end up like MCM chips. Where, (eventually) companies pool resources together for a new platform/company-agnostic standard(s).

Esp. w/ Intel (GP)GPUs and AMD MI Accelerators in Datacenters, even 'the industry' might support efforts for a more-agnostic CUDA-descendant.

 

[Minus Infinity]

ZLUDA is done and dusted. There will be no more development. Neither Intel or AMD saw any upside in getting CUDA to run on their hardware. They would rather compete head-on with oneAPI, ROCm/HIP.

ZLUDA

 

[ratirt]

I think it might be hard for NV to sue anyone. They are not creating CUDA cores, they just use blender with optimized CUDA and simulate them on any give GPU.
For the CUDA itself, these are not necessary to run apps with it. It is not hardware limited as advertised. Probably most NV stuff isn't but is being advertised as such and blocked so that you dont use it with other products. You all know why. I can bet same can be done to all those DLLS and FG etc.
If so, that means NV is charging extra for something that can work on any GPU. I really find hard to believe that a 3090Ti cant run FG or any given 3000 series card can't because of some hardware limitation.

 

[AusWolf]

For the CUDA itself, these are not necessary to run apps with it. It is not hardware limited as advertised. Probably most NV stuff isn't but is being advertised as such and blocked so that you dont use it with other products. You all know why. I can bet same can be done to all those DLLS and FG etc.

The same has been done on other GPUs - there's FSR. One can argue the minor differences in quality, but regardless of one's personal opinion, the technology is there, and it doesn't need no AI magic to run.