Infinity Fabric bandwidth vs RAM bandwidth

[teck]

Inside cIOD the infinity fabric writes/reads 32 Bytes / cycle (256bits/cycle) to the Unified Memory Controller which in turns writes/reads at most 128 bits / cycles in dual memory channel to the RAM.

If everything runs at 1:1:1, how is it possible to write/read 256 bits / cycle to RAM which is only 128 bits / cycle at most in dual memory channel ?

The big Ryzen 7000 Memory and OC Tuning Guide - Infinity Fabric, EXPO, Dual-Rank, Samsung and Hynix DDR5 in Practice test with Benchmarks and Recommendations | igor´sLAB

AMD’s new Ryzen 7000 desktop CPUs, based on the Zen4 micro-architecture, still use the same chiplet design as their predecessors, with a few small but not negligible changes. Igor already gave details…

www.igorslab.de

https://www.igorslab.de/wp-content/uploads/2022/09/zen4guide_slide_4.jpg

 

[ShrimpBrime]

The new chips will run 1/2 to 1 or 1:2. Just like Intel. Which is great for bandwidth. Makes it easier to accomplish those really high DRAM speeds and maximize throughput.

The IF (AMDs *re-named?* version of Hyper Transport) controller even at 1:2 has enough bandwidth, so they are going to take advantage of that. The IF frequency will no longer have much impact on performance when you change it's frequency. Negligible I think is the word used here.

In fact, if I didn't know better, the IF has enough bandwidth to cover multi-socketed boards at the same frequency as single socketed motherboards.

 

[JWNoctis]

It does not.

On Zen 4 non-APU desktop, AFAICT (and someone, please, correct me if I got it wrong) fclk:uclk:memclk typically runs at 2:3:3. That means at 2000MHz fclk typical for the DDR5-6000 sweet spot, the absolute maximum IF bandwidth is 64GB/s read and 32GB/s/chiplet write. This number would stay the same however faster the memory itself would run, so long as fclk stays the same.

The new chips will run 1/2 to 1 or 1:2. Just like Intel. Which is great for bandwidth. Makes it easier to accomplish those really high DRAM speeds and maximize throughput.

The IF (AMDs *re-named?* version of Hyper Transport) controller even at 1:2 has enough bandwidth, so they are going to take advantage of that. The IF frequency will no longer have much impact on performance when you change it's frequency. Negligible I think is the word used here.

In fact, if I didn't know better, the IF has enough bandwidth to cover multi-socketed boards at the same frequency as single socketed motherboards.

Does it really have enough bandwidth, though? Admittedly actual impact is minor except for memory bandwidth bottlenecked applications, if my understanding is at all correct.

 

[ShrimpBrime]

It does not.

On Zen 4 non-APU desktop, AFAICT (and someone, please, correct me if I got it wrong) fclk:uclk:memclk typically runs at 2:3:3. That means at 2000MHz fclk typical for the DDR5-6000 sweet spot, the absolute maximum IF bandwidth is 64GB/s read and 32GB/s/chiplet write. This number would stay the same however faster the memory itself would run, so long as fclk stays the same.


Does it really have enough bandwidth, though? Admittedly actual impact is minor except for memory bandwidth bottlenecked applications, if my understanding is at all correct.

Bandwidth, I would hope to think so! I'm just curious if there's going to be a major sacrifice to latency.

But since the dawn of DDR... 1:1 was always the best performance. Would have the best latency.

Nope, you're right. 2:3:3. I was stuck on AM4 in my head cause that's what I have currently.

 

[TumbleGeorge]

There is definitely a need for AMD to work more on these parts of the architecture for its next generation.

 

[ShrimpBrime]

There is definitely a need for AMD to work more on these parts of the architecture for its next generation.

Well like the article said, the 2000mhz IF 1:1 never really happened. Was a total flop (imo), they had to loosen up for DDR5, but I think that's the nature of gaining frequency. Clock cycles matter, you have to be able to feed processors the data at the rate of consumption or we'd have horrible bottle-necking. I suppose pushing past 100GB/s, this is the route we have to go.

DDR5 8000Mhz sure does sound appealing. I easily run 7000mhz with Samsung chips on the competitor, would be nice to see AMD doing the same.

 

[teck]

But how is it possible that the infinity fabric writes 256 bits to the memory controller and the memory controller write only 64bit or 128 bit (dual memory channel) to RAM?

 

[ShrimpBrime]

But how is it possible that the infinity fabric writes 256 bits to the memory controller and the memory controller write only 64bit or 128 bit (dual memory channel) to RAM?

256b x2 actually. Well according to this TPU article, seems to explain it well.

AMD Strix Point SoC "Zen 5" and "Zen 5c" CPU Cores Have 256-bit FPU Datapaths

AMD in its architecture deep-dive Q&A session with the press, confirmed that the "Zen 5" and "Zen 5c" cores on the "Strix Point" silicon only feature 256-bit wide FPU data-paths, unlike the "Zen 5" cores in the "Granite Ridge" Ryzen 9000 desktop processors. "The Zen 5c used in Strix has a...

www.techpowerup.com

 

[JWNoctis]

256b x2 actually. Well according to this TPU article, seems to explain it well.

AMD Strix Point SoC "Zen 5" and "Zen 5c" CPU Cores Have 256-bit FPU Datapaths

AMD in its architecture deep-dive Q&A session with the press, confirmed that the "Zen 5" and "Zen 5c" cores on the "Strix Point" silicon only feature 256-bit wide FPU data-paths, unlike the "Zen 5" cores in the "Granite Ridge" Ryzen 9000 desktop processors. "The Zen 5c used in Strix has a...

www.techpowerup.com

Seems like one is IF and memory controller, the other is internal datapath of the processor cores themselves. Different things.

But how is it possible that the infinity fabric writes 256 bits to the memory controller and the memory controller write only 64bit or 128 bit (dual memory channel) to RAM?

One'd guess since DDR means the RAM accesses two bits per clock per line, it's effectively doubled.

 

[teck]

I asked chatgpt

how does amd infinity fabric writes 256 bit of data to ram?

short answer: data segmentation / the data is splitted

Step-by-Step Process​

1. Data Generation: The CPU cores generate a 256-bit block of data that needs to be written to RAM.
2. Infinity Fabric Transmission:
   • The CPU cores pass the 256-bit data block to the Infinity Fabric.
   • The Infinity Fabric acts as a high-speed conduit, transferring data between CPU cores, caches, and the memory controller.
   • The Fabric ensures data integrity and manages the flow to prevent bottlenecks.
3. Memory Controller Preparation:
   • The memory controller receives the 256-bit data block from the Infinity Fabric.
   • It prepares the data for transmission to the RAM by handling tasks such as addressing, timing, and synchronization.
   • The memory controller breaks down the 256-bit data into smaller chunks if necessary, matching the width of the data bus (e.g., 64-bit chunks for a 64-bit data bus).
4. Data Bus Transmission:
   • The prepared data chunks are transmitted over the data bus to the RAM modules.
   • Modern systems use multiple memory channels (e.g., dual-channel, quad-channel) to increase throughput, allowing simultaneous transfers.
   • Each channel can handle a portion of the 256-bit data, speeding up the overall transfer process.
5. DDR Memory Storage:
   • The RAM modules receive the data via the data bus.
   • DDR (Double Data Rate) memory transfers data on both the rising and falling edges of the clock signal, effectively doubling the data transfer rate.
   • The memory controller ensures that data is written to the correct memory address.

Example of 256-bit Data Transfer​

• Segmentation: The 256-bit data might be divided into four 64-bit chunks if the data bus is 64 bits wide.
• Parallel Channels: If using dual-channel memory, each 64-bit chunk could be sent simultaneously over two channels, speeding up the process.
• DDR Mechanism: With DDR memory, each 64-bit chunk could be written in two cycles (one on the rising edge and one on the falling edge of the clock signal).

Summary​

AMD Infinity Fabric efficiently handles the transfer of data from CPU cores to RAM by utilizing high-speed interconnects, memory controllers, and multiple memory channels. The 256-bit data is segmented, transmitted over the data bus, and stored in DDR memory, leveraging parallelism and the double data rate mechanism to ensure fast and reliable data writes.

 

[JWNoctis]

I asked chatgpt

how does amd infinity fabric writes 256 bit of data to ram?

short answer: data segmentation / the data is splitted

-snip-

Right now that's only Exhibit A of Why Generally Available AI is Still Not Useful for Factual Info.

It mostly appears reasonable, but falls to pieces near the end if one would take a closer look.

 

[teck]

I don't have another answer, I was just intrigued by this, as the zen core uses 256 bits data paths internally, while the RAM has only 64 bit or 128 bit (dual channel) data path.