T0@st
News Editor
- Joined
- Mar 7, 2023
- Messages
- 2,697 (3.63/day)
- Location
- South East, UK
| System Name | The TPU Typewriter |
|---|---|
| Processor | AMD Ryzen 5 5600 (non-X) |
| Motherboard | GIGABYTE B550M DS3H Micro ATX |
| Cooling | DeepCool AS500 |
| Memory | Kingston Fury Renegade RGB 32 GB (2 x 16 GB) DDR4-3600 CL16 |
| Video Card(s) | PowerColor Radeon RX 7800 XT 16 GB Hellhound OC |
| Storage | Samsung 980 Pro 1 TB M.2-2280 PCIe 4.0 X4 NVME SSD |
| Display(s) | Lenovo Legion Y27q-20 27" QHD IPS monitor |
| Case | GameMax Spark M-ATX (re-badged Jonsbo D30) |
| Audio Device(s) | FiiO K7 Desktop DAC/Amp + Philips Fidelio X3 headphones, or ARTTI T10 Planar IEMs |
| Power Supply | ADATA XPG CORE Reactor 650 W 80+ Gold ATX |
| Mouse | Roccat Kone Pro Air |
| Keyboard | Cooler Master MasterKeys Pro L |
| Software | Windows 10 64-bit Home Edition |
Large language models (LLMs) love large quantities of memory—so much so, in fact, that AI enthusiasts are turning to multi-GPU setups to make even more VRAM available for their AI apps. But since many current LLMs are extremely large, even this approach has its limits. At times, the GPU will decide to make use of CPU processing power for this data, and when it does, the performance of your CPU cache and DRAM comes into play. All this means that when it comes to the performance of AI applications, it's not just the GPU that matters, but the entire pathway that connects the GPU to the CPU to the I/O die to the DRAM modules. It stands to reason, then, that there are opportunities to boost AI performance by optimizing these elements.
That's exactly what we've found as we've spent time in our R&D labs with the latest AMD Ryzen CPUs. AMD just launched two new Ryzen CPUs with AMD 3D V-Cache Technology, the AMD Ryzen 9 9950X3D and Ryzen 9 9900X3D, pushing the series into new performance territory. After testing a wide range of optimizations in a variety of workloads, we uncovered a range of settings that offer tangible benefits for AI enthusiasts. Now, we're ready to share these optimizations with you through a new BIOS feature: AI Cache Boost. Available through an ASUS AMD 800 Series motherboard and our most recent firmware update, AI Cache Boost can accelerate performance up to 12.75% when you're working with massive LLMs.
Ready for AMD Ryzen CPUs with AMD 3D V-Cache Technology
To access AI Cache Boost, you'll need an ASUS AMD 800 Series motherboard and an AMD Ryzen 9000 Series Granite Ridge CPU. Open the UEFI BIOS utility and navigate to the Extreme Tweaker tab. Toggle AI Cache Boost to "Enabled" and you're off to the races. Activating AI Cache Boost engages a variety of optimizations, including overlocking the Infinity Fabric clock (FCLK) to 2100 MHz. Boosting FCLK is an important piece of the puzzle here, since it boosts the bandwidth of data transfers between CPU cores, cache, and memory—and those transfers matter when working with massive LLMs.
A brief aside for anyone who uses the same PC for both AI applications and gaming. Enabling AI Cache Boost doesn't significantly affect gaming performance, so you should feel free to leave the setting enabled even as you switch from one type of workload to the other. For enhanced stability in gaming, you might prefer to manually lower FCLK to a more conservative setting after enabling AI Cache Boost.
The performance uplift of AI Cache Boost
For our test system, we started with the ROG Crosshair X870E Hero motherboard and equipped it with an NVIDIA GeForce RTX 5090 graphics card and 2 x 16 GB of DDR5 RAM. We tested a variety of AMD Ryzen CPUs with AMD 3D V-Cache Technology, including the AMD Ryzen 9 9950X3D, 9900X3D, and 9800X3D. Our first test looks at Geekbench AI with the ONNX DirectML framework and the benchmark set to measure GPU performance.
The performance uplift we observed ranged from roughly 4% to 8%. That's significant for anyone running AI workflows for a good portion of their day—especially considering how easy it is to enable AI Cache Boost. Since it's early days for NVIDIA 50 Series graphics cards, we also checked the performance of AI Cache Boost on previous-gen hardware by swapping out the graphics card in our testing system for an NVIDIA RTX 4090.
Across the Geekbench AI test results, AI Cache Boost provides a noticeable performance bump, ranging up to 7.57% with the AMD Ryzen 7 9800X3D. Intriguingly, it's the AMD Ryzen 7 9800X3D that posts the highest numbers with AI Cache Boost enabled.
We see a similar pattern with the UL Procyon AI Computer Vision Benchmark, with AI Cache Boost providing performance uplift across the board. The AMD Ryzen 7 9800X3D enjoys a double-digit improvement to its score with AI Cache Boost—an increase of 12.75%, to be exact.
Bringing Turbo Game Mode into the mix
The outstanding performance of the AMD Ryzen 7 9800X3D in both the Geekbench AI and UL Procyon benchmarks suggests that we're looking at a scenario where the AMD 3D V-Cache technology is making more of a difference than the increased core counts available with the AMD Ryzen 9 9950X3D and 9900X3D. To investigate this further, we ran some additional tests in which we not only activate AI Cache Boost, but also Turbo Game Mode.
This BIOS feature, already available on ASUS AM5 motherboards, optimizes performance in up to two ways. The first only applies to those with a high-end AMD Ryzen CPU with two Core Complex Dies (CCD). The second CCD allows the chip to bring a stunning wealth of CPU cores and threads to bear on multithreaded workflows, but apps don't always make use of all those resources. Turbo Game Mode disables the second CCD, allowing the CPU to operate more efficiently in certain scenarios.
Second, Turbo Game Mode disables simultaneous multithreading (SMT). With SMT enabled, each core in your AMD Ryzen CPU can operate two sets of instructions at the same time, kicking your multithreaded performance into high gear. This is invaluable for many users, but if you're looking to optimize performance for a workflow that doesn't require that many CPU threads, you can see higher performance with SMT disabled. Disabling a CCD and SMT has a significant impact on multithreaded performance, so we recommend that users only enable Turbo Game Mode when it benefits their current workflow.
With both AI Cache Boost and Turbo Game Mode activated, the AMD Ryzen 9 9950X3D takes the lead in this benchmark over the other test configurations, posting a whopping 19.85% increase over default settings. The outstanding performance uplift that we're seeing here suggests that AI enthusiasts might find it valuable to enable both AI Cache Boost and Turbo Game Mode in circumstances in which their workflow doesn't take full advantage of their CPU's massive thread count. Both options are readily available in the Extreme Tweaker BIOS tab, so interested users won't find it hard at all to explore the benefits of toggling both BIOS features simultaneously.
Kick your AI ambitions up a notch with ASUS motherboards and AI Cache Boost
The new AMD Ryzen 9 9950X3D and AMD Ryzen 9 9900X3D offer an intriguing value proposition to PC enthusiasts of all kinds. Equipped with 32 and 24 threads, respectively, they're primed for fantastic multithreaded performance right out of the gate—and they offer the benefits of AMD 3D V-Cache Technology at the same time.
ASUS AMD 800 Series motherboards help ensure that you get the best experience with these new chips through optional BIOS settings that help optimize your experience. For AI workflows that involve LLMs, in particular, we've found that AI Cache Boost has a lot to offer—and it's incredibly easy to enable. To make sure that you're getting the most out of your new AMD CPU, browse our full selection of AM5 motherboards ranging from top-end X870E and X870 models to value-oriented B850 and B840 options.
View at TechPowerUp Main Site | Source
That's exactly what we've found as we've spent time in our R&D labs with the latest AMD Ryzen CPUs. AMD just launched two new Ryzen CPUs with AMD 3D V-Cache Technology, the AMD Ryzen 9 9950X3D and Ryzen 9 9900X3D, pushing the series into new performance territory. After testing a wide range of optimizations in a variety of workloads, we uncovered a range of settings that offer tangible benefits for AI enthusiasts. Now, we're ready to share these optimizations with you through a new BIOS feature: AI Cache Boost. Available through an ASUS AMD 800 Series motherboard and our most recent firmware update, AI Cache Boost can accelerate performance up to 12.75% when you're working with massive LLMs.
Ready for AMD Ryzen CPUs with AMD 3D V-Cache Technology
To access AI Cache Boost, you'll need an ASUS AMD 800 Series motherboard and an AMD Ryzen 9000 Series Granite Ridge CPU. Open the UEFI BIOS utility and navigate to the Extreme Tweaker tab. Toggle AI Cache Boost to "Enabled" and you're off to the races. Activating AI Cache Boost engages a variety of optimizations, including overlocking the Infinity Fabric clock (FCLK) to 2100 MHz. Boosting FCLK is an important piece of the puzzle here, since it boosts the bandwidth of data transfers between CPU cores, cache, and memory—and those transfers matter when working with massive LLMs.
A brief aside for anyone who uses the same PC for both AI applications and gaming. Enabling AI Cache Boost doesn't significantly affect gaming performance, so you should feel free to leave the setting enabled even as you switch from one type of workload to the other. For enhanced stability in gaming, you might prefer to manually lower FCLK to a more conservative setting after enabling AI Cache Boost.
The performance uplift of AI Cache Boost
For our test system, we started with the ROG Crosshair X870E Hero motherboard and equipped it with an NVIDIA GeForce RTX 5090 graphics card and 2 x 16 GB of DDR5 RAM. We tested a variety of AMD Ryzen CPUs with AMD 3D V-Cache Technology, including the AMD Ryzen 9 9950X3D, 9900X3D, and 9800X3D. Our first test looks at Geekbench AI with the ONNX DirectML framework and the benchmark set to measure GPU performance.
The performance uplift we observed ranged from roughly 4% to 8%. That's significant for anyone running AI workflows for a good portion of their day—especially considering how easy it is to enable AI Cache Boost. Since it's early days for NVIDIA 50 Series graphics cards, we also checked the performance of AI Cache Boost on previous-gen hardware by swapping out the graphics card in our testing system for an NVIDIA RTX 4090.
Across the Geekbench AI test results, AI Cache Boost provides a noticeable performance bump, ranging up to 7.57% with the AMD Ryzen 7 9800X3D. Intriguingly, it's the AMD Ryzen 7 9800X3D that posts the highest numbers with AI Cache Boost enabled.
We see a similar pattern with the UL Procyon AI Computer Vision Benchmark, with AI Cache Boost providing performance uplift across the board. The AMD Ryzen 7 9800X3D enjoys a double-digit improvement to its score with AI Cache Boost—an increase of 12.75%, to be exact.
Bringing Turbo Game Mode into the mix
The outstanding performance of the AMD Ryzen 7 9800X3D in both the Geekbench AI and UL Procyon benchmarks suggests that we're looking at a scenario where the AMD 3D V-Cache technology is making more of a difference than the increased core counts available with the AMD Ryzen 9 9950X3D and 9900X3D. To investigate this further, we ran some additional tests in which we not only activate AI Cache Boost, but also Turbo Game Mode.
This BIOS feature, already available on ASUS AM5 motherboards, optimizes performance in up to two ways. The first only applies to those with a high-end AMD Ryzen CPU with two Core Complex Dies (CCD). The second CCD allows the chip to bring a stunning wealth of CPU cores and threads to bear on multithreaded workflows, but apps don't always make use of all those resources. Turbo Game Mode disables the second CCD, allowing the CPU to operate more efficiently in certain scenarios.
Second, Turbo Game Mode disables simultaneous multithreading (SMT). With SMT enabled, each core in your AMD Ryzen CPU can operate two sets of instructions at the same time, kicking your multithreaded performance into high gear. This is invaluable for many users, but if you're looking to optimize performance for a workflow that doesn't require that many CPU threads, you can see higher performance with SMT disabled. Disabling a CCD and SMT has a significant impact on multithreaded performance, so we recommend that users only enable Turbo Game Mode when it benefits their current workflow.
With both AI Cache Boost and Turbo Game Mode activated, the AMD Ryzen 9 9950X3D takes the lead in this benchmark over the other test configurations, posting a whopping 19.85% increase over default settings. The outstanding performance uplift that we're seeing here suggests that AI enthusiasts might find it valuable to enable both AI Cache Boost and Turbo Game Mode in circumstances in which their workflow doesn't take full advantage of their CPU's massive thread count. Both options are readily available in the Extreme Tweaker BIOS tab, so interested users won't find it hard at all to explore the benefits of toggling both BIOS features simultaneously.
Kick your AI ambitions up a notch with ASUS motherboards and AI Cache Boost
The new AMD Ryzen 9 9950X3D and AMD Ryzen 9 9900X3D offer an intriguing value proposition to PC enthusiasts of all kinds. Equipped with 32 and 24 threads, respectively, they're primed for fantastic multithreaded performance right out of the gate—and they offer the benefits of AMD 3D V-Cache Technology at the same time.
ASUS AMD 800 Series motherboards help ensure that you get the best experience with these new chips through optional BIOS settings that help optimize your experience. For AI workflows that involve LLMs, in particular, we've found that AI Cache Boost has a lot to offer—and it's incredibly easy to enable. To make sure that you're getting the most out of your new AMD CPU, browse our full selection of AM5 motherboards ranging from top-end X870E and X870 models to value-oriented B850 and B840 options.
View at TechPowerUp Main Site | Source
Can the tweaks AI mode applies be manually applied and verified or is there some undocumented "special sauce" going on there as well?