Thursday, June 20th 2024
New AMD ROCm 6.1 Software for Radeon Release Offers More Choices to AI Developers
AMD has unveiled the latest release of its open software, AMD ROCm 6.1.3, marking the next step in its strategy to make ROCm software broadly available across its GPU portfolio, including AMD Radeon desktop GPUs. The new release gives developers broader support for Radeon GPUs to run ROCm AI workloads. "The new AMD ROCm release extends functional parity from data center to desktops, enabling AI research and development on readily available and accessible platforms," said Andrej Zdravkovic, senior vice president at AMD.
Key feature enhancements in this release focus on improving compatibility, accessibility, and scalability, and include:
ROCm 6.1.3 now supports up to four qualified Radeon RX Series or Radeon PRO GPUs, allowing users to leverage configurations with data parallelism where each GPU independently computes inference and outputs the response. This enables client-based multi-user configurations powered by AMD ROCm software and Radeon GPUs.
With ROCm 6.1.3, we are making it even easier to develop for AI with Beta-level support for Windows Subsystem for Linux, also known as WSL 2. This means you can now run Linux-based AI tools on a Windows system. WSL 2 eliminates the need for a dedicated Linux system or a dual-boot setup.
Furthermore, we are announcing qualification of the TensorFlow framework, providing another choice for AI development aside from PyTorch and ONNX which were already supported, extending our robust open ecosystem of frameworks and libraries.
In addition, we are extending our client-based AI development offering with the introduction of the AMD Radeon PRO W7900 Dual Slot card which packs 192 AI accelerators and 48 GB of fast GDDR6 memory into a compact form factor for higher, system-level density.
AMD AI workstations equipped with a Radeon PRO W7900 GPU represent a new solution to fine-tune and run inference on large language models (LLMs) with high precision. For example, LLaMA-2 or LLaMA-3 with 70B parameters quantized at INT4 require at least 35 GB of local GPU memory, making the Radeon PRO W7900 GPU with 48 GB of fast on-board memory the right choice for workflows that make use of these models.
Generative AI for natural language processing (NLP) using such LLMs can help enterprises tailor interaction with customers, assist with development operations (DevOps), and improve the process of managing data and documents.
AMD is committed to building a highly scalable and open ecosystem with ROCm software. With the latest release of ROCm 6.1 software for Radeon desktop GPUs, AMD empowers system builders to take full advantage of our enhanced solution stack to create on-prem systems that add powerful AI performance to any IT infrastructure, making them ideal for mission-critical or low-latency projects - and allowing organizations to keep their sensitive data in-house.
With AMD ROCm 6.1 and the new AMD Radeon PRO W7900 Dual Slot card, which is now shipping, AI on desktops opens new opportunities for developers and enterprises to increase productivity, creativity, and innovation with performance and ease of use.
Source:
AMD
Key feature enhancements in this release focus on improving compatibility, accessibility, and scalability, and include:
- Multi-GPU support to enable building scalable AI desktops for multi-serving, multi-user solutions.
- Beta-level support for Windows Subsystem for Linux, allowing these solutions to work with ROCm on a Windows OS-based system.
- TensorFlow Framework support offering more choice for AI development.
With ROCm 6.1.3, we are making it even easier to develop for AI with Beta-level support for Windows Subsystem for Linux, also known as WSL 2. This means you can now run Linux-based AI tools on a Windows system. WSL 2 eliminates the need for a dedicated Linux system or a dual-boot setup.
Furthermore, we are announcing qualification of the TensorFlow framework, providing another choice for AI development aside from PyTorch and ONNX which were already supported, extending our robust open ecosystem of frameworks and libraries.
In addition, we are extending our client-based AI development offering with the introduction of the AMD Radeon PRO W7900 Dual Slot card which packs 192 AI accelerators and 48 GB of fast GDDR6 memory into a compact form factor for higher, system-level density.
AMD AI workstations equipped with a Radeon PRO W7900 GPU represent a new solution to fine-tune and run inference on large language models (LLMs) with high precision. For example, LLaMA-2 or LLaMA-3 with 70B parameters quantized at INT4 require at least 35 GB of local GPU memory, making the Radeon PRO W7900 GPU with 48 GB of fast on-board memory the right choice for workflows that make use of these models.
AMD is committed to building a highly scalable and open ecosystem with ROCm software. With the latest release of ROCm 6.1 software for Radeon desktop GPUs, AMD empowers system builders to take full advantage of our enhanced solution stack to create on-prem systems that add powerful AI performance to any IT infrastructure, making them ideal for mission-critical or low-latency projects - and allowing organizations to keep their sensitive data in-house.
With AMD ROCm 6.1 and the new AMD Radeon PRO W7900 Dual Slot card, which is now shipping, AI on desktops opens new opportunities for developers and enterprises to increase productivity, creativity, and innovation with performance and ease of use.
10 Comments on New AMD ROCm 6.1 Software for Radeon Release Offers More Choices to AI Developers
Am I crazy pilling takes?
So their datacenter/enterprise customers have been leaving what, 95% of the hardware sitting unused?
Consumer < prosumer < workstation/pro < datacenter/enterprise
They are bringing it to pro. It was already in datacenter.
The issue is performance. Stable Diffusion on Linux is about twice as fast on AMD cards as compared to Windows and really the AMD cards need that boost to be competitive price / performance wise. That might change now that ROCm supports WSL. There's more hoops you have to jump through to configure SD for AMD cards on windows as well. It's getting better but there's still a lot of lot to be done to optimize performance and config.
btw, there are
lotsa few distros picking up support for it that aren't on AMDs official list.rocm.docs.amd.com/projects/install-on-linux/en/latest/reference/system-requirements.html
If you keep to the base package and never touch extensions, sure, it works. As in, technically something gets generated.
However, a lot of SD workflows rely on extensions, which vary from "instantly bricks your entire install" to "doesn't work but doesn't affect anything else" to "runs but 20-200x slower than on an equivalent nvidia GPU"... And they tend to have very little documentation/discussion around AMD compatibility, so you might not find out until something gets bricked.
As of now, fiddling around with unofficial ZLUDA is your best bet, and that tends to work better on the high end 7000 series.
If you install the IP adapter extension for A1111 (webui) and your GPU is running inference via DirectML, the IP adapter would be running on directML as well. Functionality and stability should be equivalent.
Drawing the line at extensions seems arbitrary given that many stable diffusion front ends will have a lot of features rolled into them. SD.Next for example supports AMD GPUs via DirectML and has a ton of features rolled into the base software. You could install zero extensions and that would more than cover the vast majority of generation workloads.