# NVIDIA GeForce RTX 4090 PCI-Express Scaling



## W1zzard (Oct 14, 2022)

The new NVIDIA GeForce RTX 4090 is a graphics card powerhouse, but what happens when you run it on a PCI-Express 4.0 x8 bus? In our mini-review we've also tested various PCI-Express 3.0, 2.0 and 1.1 configs to get a feel for how FPS scales with bandwidth.

*Show full review*


----------



## Selaya (Oct 14, 2022)

looks like there's finally a point to 4.0x16 for high refresh.
interesting.


----------



## ZetZet (Oct 14, 2022)

Yeah, so about those PCIe Gen5x16 slots on higher end boards...


----------



## Upgrayedd (Oct 14, 2022)

Why not use Vulkan on RDR2?


----------



## CapNemo72 (Oct 14, 2022)

So, I am only checking 4K results as that is where you see what is GPU made off.

This article will be important for those deciding to go with AMD x670 or x670E or B650 or b650E. From your foundings, you should just be sure that m2 slot has that PCie Gen5 possibility as the GPU will not be able to use it. Saving maybe 50 bucks on that could be invested in a bit faster GPU.

Funny question: Is Nvidia SLI completely dead and if not, would it work with two 4090?


----------



## P4-630 (Oct 14, 2022)

So basically if you still got PCIe 3.0 you're still ok with that and a 4090....

But as @ir_cow mentioned you'd probably have a huge CPU bottleneck if you were still on PCIe 3.0 platform with a 4090....


----------



## Tek-Check (Oct 14, 2022)

W1zzard said:


> The new NVIDIA GeForce RTX 4090 is a graphics card powerhouse, but what happens when you run it on a PCI-Express 4.0 x8 bus? In our mini-review we've also tested various PCI-Express 3.0, 2.0 and 1.1 configs to get a feel for how FPS scales with bandwidth.


Would performance difference stay the same as shown in games (2-3%), if *graphics productivity workloads* are tested? We know that most games are not able to saturate x8 Gen4 link, but what happens if graphic designers, architects, engineers use their powerful software on x8 Gen4 link with 4090?


----------



## ModEl4 (Oct 14, 2022)

That settles the Pci-express conserns if you use Gen5 SSD.
It should be the same for the other upcoming 4080 models also:






NVIDIA cancels GeForce RTX 4080 12GB 

Hi @W1zzard in your upcoming article featuring RTX 4090 performance with 13900K, are you going to use the newer 522.25 drivers (11/10) that improve DX12 performance?

NVIDIA’s new driver enables “substantial” DirectX12 performance improvements for GeForce RTX GPUs 

"_Our DirectX 12 optimizations apply to GeForce RTX graphics cards and laptops, though improvements will vary based on your specific system setup, and the game settings used. In our testing, performance increases were found in a wide variety of DirectX 12 games, across all resolutions:
_

_Assassin’s Creed Valhalla: up to 24% (1080p)_
_Battlefield 2042: up to 7% (1080p)_
_Borderlands 3: Up to 8% (1080p)_
_Call of Duty: Vanguard: up to 12% (4K)_
_Control: up to 6% (4K)_
_Cyberpunk 2077: up to 20% (1080p)_
_F1Ⓡ 22: up to 17% (4K)_
_Far Cry 6: up to 5% (1440p)_
_Forza Horizon 5: up to 8% (1080P)_
_Horizon Zero Dawn: Complete Edition: up to 8% (4k)_
_Red Dead Redemption 2: up to 7% (1080p)_
_Shadow of the Tomb Raider: up to 5% (1080p)_
_Tom Clancy’s The Division 2: up to 5% (1080p)_
_Watch Dogs: Legion: up to 9% (1440p)"_


----------



## Wolfkin (Oct 14, 2022)

Overall it's about the difference I expected.
What surprised me though is the difference between individual games and at different resolution both within the same game and compared to other games, did not expect that big of a variance.


----------



## Bcannon2000 (Oct 14, 2022)

Why are you benchmarking GPUs with a 5800X? Shouldn't you be using the best CPUs for benchmarking GPUs? You have three other options for the best gaming CPUs, Alder Lake, Zen 4, and the 5800X3D


----------



## W1zzard (Oct 14, 2022)

Bcannon2000 said:


> Why are you benchmarking using a 5800X? Shouldn't you be using the best CPUs for benchmarking GPUs? You have three other options for the best gaming CPUs, Alder Lake, Zen 4, and the 5800X3D
> 
> Why are you benchmarking GPUs with a 5800X? Shouldn't you be using the best CPUs for benchmarking GPUs? You have three other options for the best gaming CPUs, Alder Lake, Zen 4, and the 5800X3D


I'm using 13900K

Edit: soon


----------



## wheresmycar (Oct 14, 2022)

W1zzard said:


> I'm using 13900K



You might need to amend the test setup page... its showing 5800X

actually i would have been happier if it was 5800X hehe... looks like I might end up with an AM4 platform if Zen4/RPL can't be had at a preferred budget (no thanks to NVIDIA for contaminating budget allocation)


----------



## Chrispy_ (Oct 14, 2022)

Hah! PCIe 2.0 still being fine almost 16 years later 

Sure, if you want that last 5% then you'll need PCIe 4.0 but these articles always prove that the PCIe race really isn't that necessary unless you're already chasing diminishing returns.


----------



## defaultluser (Oct 14, 2022)

Chrispy_ said:


> Hah! PCIe 2.0 still being fine almost 16 years later
> 
> Sure, if you want that last 5% then you'll need PCIe 4.0 but these articles always prove that the PCIe race really isn't that necessary unless you're already chasing diminishing returns.




its hard to make more of an impact when you're already massively CPU -bound at 1080p - *this s why the first time ever, the gap INCREASES  with resolution bump to 1440p*


----------



## birdie (Oct 14, 2022)

And I guess mostly AMD fans have trashed NVIDIA for not supporting PCIe 5.0.

Considering there's on average a 2% difference between PCIe 3.0 and 4.0 there would be a 0% performance inrovement from using PCIe 5.0.

DP 2.0 for RTX 4090 - that's relevant for _some_. PCIe 5.0? The time has yet to come.


----------



## Chrispy_ (Oct 14, 2022)

defaultluser said:


> its hard to make more of an impact when you're already massively CPU -bound at 1080p



Looks like many titles are massively CPU-bound even at 4K too.

I think it's fair to say that the 4090 is so much more GPU than almost any CPU, display, or game can fully use right now - that the entire _raison d'etre_ for the 4090 isn't here yet.

Maybe 240Hz 4K displays will give it a purpose beyond bragging rights.


----------



## defaultluser (Oct 14, 2022)

Chrispy_ said:


> Looks like many titles are massively CPU-bound even at 4K too.
> 
> I think it's fair to say that the 4090 is so much more GPU than almost any CPU, display, or game can fully use right now - that the entire _raison d'etre_ for the 4090 isn't here yet.
> 
> Maybe 240Hz 4K displays will give it a purpose beyond bragging rights.




disagree man, look at the gap at 1440p versus the 3080:






New much more GPU-limited gap once you double the res (now 80% versus 88  at pcie 1.11)






it would be nice if we could fix thew CPU limits at 1080p, but at least we can switch to analysis at 1440p.


----------



## Style68 (Oct 14, 2022)

If I were to use a two Gen 4 NVMe SSDs in RAID 0 on the M.2 slots that are wired to the chipset of a high end z790 board like the ASUS ROG Maximus z790 Extreme, would I get the full PCIe x16 for the graphics card and get speeds matching a single Gen 5 NVMe SSD that would have been installed on the Gen 5 slot?


----------



## BorisDG (Oct 14, 2022)

Still > PCI 3.0 is pointless I see.


----------



## GC_PaNzerFIN (Oct 14, 2022)

The relative performance, noise, perf/w and perf/money graphs are always really awsome at TPU and this is no different. thank you @W1zzard 

I would really like to see PCIe 5.0 graphics cards so I know for sure in the future whatever plans I have with the second PCIe x8 slot will definitely not affect graphics card performance. PCIe 5.0 x8 plenty for graphics and x8 for other stuff on platform.


----------



## defaultluser (Oct 14, 2022)

BorisDG said:


> Still > PCI 3.0 is pointless I see.




only for top hardware *- when they castrate lower-end cards, the drop from 3.0 to 2.0 is almost 20%




*


----------



## W1zzard (Oct 14, 2022)

ModEl4 said:


> Hi @W1zzard in your upcoming article featuring RTX 4090 performance with 13900K, are you going to use the newer 522.25 drivers (11/10) that improve DX12 performance?


Of course.. the press drivers that I used also have these improvements



Upgrayedd said:


> Why not use Vulkan on RDR2?


On cards with small VRAM it will just crash, with DirectX 12 it will run slower but stable



Tek-Check said:


> if graphic designers, architects, engineers use their powerful software


What software is that? The people you listed don't need such a fast GPU as far as I understand, they only need some basic viewport acceleration



Style68 said:


> If I were to use a two Gen 4 NVMe SSDs in RAID 0 on the M.2 slots that are wired to the chipset of a high end z790 board like the ASUS ROG Maximus z790 Extreme, would I get the full PCIe x16 for the graphics card and get speeds matching a single Gen 5 NVMe SSD that would have been installed on the Gen 5 slot?


Good question, hard to say without data for Gen 5 drives, "10 GB/s" is just sequential which is of almost no consequence in real-life


----------



## Bcannon2000 (Oct 14, 2022)

W1zzard said:


> I'm using 13900K
> 
> Edit: soon


So you didn’t want to change the benchmark system right before having to change it again for Raptor Lake? that makes sense.

The issue is that testing with the 5800X is disingenuous, not that I would blame you for not wanting to change setups 2 times so quickly.

I do hope you revisit the PCIe scaling and GPU performance when you get the 13900K. It seems the average 4K numbers for this 4090 review are 20-30% below what others got.


----------



## Bcannon2000 (Oct 14, 2022)

birdie said:


> And I guess mostly AMD fans have trashed NVIDIA for not supporting PCIe 5.0.
> 
> Considering there's on average a 2% difference between PCIe 3.0 and 4.0 there would be a 0% performance inrovement from using PCIe 5.0.
> 
> DP 2.0 for RTX 4090 - that's relevant for _some_. PCIe 5.0? The time has yet to come.


To me, if I am paying $1,600 or more for a GPU, it better have all of the newest bells and whistles. It doesn’t have DP 2.0 and PCIe 5.0, even if it or I can’t use them, I still want them because of how much it cost


----------



## Denver (Oct 14, 2022)

@W1zzard Wouldn't the most correct test methodology be pairing the 4090 with a 7700X? 
The 5800X cannot extract the full potential of this GPU.


----------



## Dirt Chip (Oct 14, 2022)

W1zzard said:


> What software is that? The people you listed don't need such a fast GPU as far as I understand, they only need some basic viewport acceleration


Davinci resolve see big improvment with 4090. This video edit program can also use 2*4090 GPUs.
Also, some of adobe pro such as premiere, after effect utilise high end GPU`s.


----------



## mahirzukic2 (Oct 14, 2022)

Style68 said:


> If I were to use a two Gen 4 NVMe SSDs in RAID 0 on the M.2 slots that are wired to the chipset of a high end z790 board like the ASUS ROG Maximus z790 Extreme, would I get the full PCIe x16 for the graphics card and get speeds matching a single Gen 5 NVMe SSD that would have been installed on the Gen 5 slot?


For sequential R/W speeds and access times you would get the 2x improvement, for the 4k random you'd still be limited by 1 SSD.


----------



## W1zzard (Oct 14, 2022)

Bcannon2000 said:


> So you didn’t want to change the benchmark system right before having to change it again for Raptor Lake? that makes sense.


Retesting everything is a 2-3 weeks task, full-time, doing nothing but running benchmarks all day. I'm sure you are aware of all the launches that happened in the last weeks, no way to do this kind of retesting in that time.

Maybe other sites that have 4 benchmarks and 5 comparison cards can do it, and they probably still recycle results .. I have 25 games, 30+ cards, 3 resolutions



Denver said:


> @W1zzard Wouldn't the most correct test methodology be pairing the 4090 with a 7700X?
> The 5800X cannot extract the full potential of this GPU.











						AMD Ryzen 7 7700X Review - The Best Zen 4 for Gaming
					

With the Ryzen 7 7700X, AMD is introducing their fastest Zen 4 processor for gamers. In our review we found out that gaming on the 7700X runs better than the 7900X and 7950X, thanks to the single CCD design of the 7700X. Just like on other high-end AM5 CPUs, temperatures are a problem though.




					www.techpowerup.com
				



Only 12900K, better 13900K


----------



## Bcannon2000 (Oct 14, 2022)

Denver said:


> @W1zzard Wouldn't the most correct test methodology be pairing the 4090 with a 7700X?
> The 5800X cannot extract the full potential of this GPU.


I think there are three valid CPU options for GPU testing right now that work. Each of them has its pros and cons performance-wise. The 5800X3D, Alder Lake, and Zen 4. I would say Zen 4, as even if Raptor Lake is (let's say) 10% faster in gaming, Zen 4X3D will most likely crush Raptor Lake in gaming. If you use the base Zen 4, then you can just drop in a Zen 4X3D CPU while keeping the motherboard and RAM the same.


----------



## BorisDG (Oct 14, 2022)

defaultluser said:


> only for top hardware *- when they castrate lower-end cards, the drop from 3.0 to 2.0 is almost 20%*


I said you don't need PCI 4.0 or more even for 4090... 3.0 seems enough


----------



## Denver (Oct 14, 2022)

W1zzard said:


> Retesting everything is a 2-3 weeks task, full-time, doing nothing but running benchmarks all day. I'm sure you are aware of all the launches that happened in the last weeks, no way to do this kind of retesting in that time.
> 
> Maybe other sites that have 4 benchmarks and 5 comparison cards can do it, and they probably still recycle results .. I have 25 games, 30+ cards, 3 resolutions
> 
> ...


3% for me is margin of error. This end result can change entirely depending on the games used, whether it's in favor of the i9 or the Ryzen 7xxx, the point is that this is the only PCIe 5 platform available.

Yeah, the i9 would also bring better results than the current CPU.


----------



## Bcannon2000 (Oct 14, 2022)

BorisDG said:


> I said you don't need PCI 4.0 or more even for 4090... 3.0 seems enough


The issue is that we can't say for sure yet. There are other bottlenecks before PCIe bandwidth. Maybe even if we remove said bottlenecks, nothing will change, but maybe it will be a 10% difference.


----------



## masterkaj (Oct 14, 2022)

Curious if PCIE 4.0 vs 3.0 matters if you are recording using the AV1 encoder at the same time?  Cause it to saturate a bit more?


----------



## W1zzard (Oct 14, 2022)

masterkaj said:


> Curious if PCIE 4.0 vs 3.0 matters if you are recording using the AV1 encoder at the same time?  Cause it to saturate a bit more?


Shouldn't .. moving the input frames to the encoder is free, because it happens within the GPU. The encoded result that goes over the bus is tiny in filesize, just a few MB/s


----------



## Denver (Oct 14, 2022)

Bcannon2000 said:


> The issue is that we can't say for sure yet. There are other bottlenecks before PCIe bandwidth. Maybe even if we remove said bottlenecks, nothing will change, but maybe it will be a 10% difference.


In some cases there is a considerable difference


----------



## masterkaj (Oct 14, 2022)

W1zzard said:


> Shouldn't .. moving the input frames to the encoder is free, because it happens within the GPU. The encoded result that goes over the bus is tiny in filesize, just a few MB/s


I figured that transfer over the bus with lossless quality would be more than a few MB/s.  Something like CQP 10 or lower.


----------



## Razrback16 (Oct 14, 2022)

Very helpful writeup, thank you.


----------



## Bcannon2000 (Oct 14, 2022)

Denver said:


> In some cases there is a considerable difference


I agree, I should have said the average numbers. There are some situations where PCIe doesn't matter. However, as you showed, there are some situations where it matters a ton. Based on what we have from this review/benchmarks, it seems there is only a 2-3% difference between 4.0 and 3.0. I can see it changing to 10% on average or more with updated CPU testing that removes some of the CPU bottlenecks.

Then that begs the question, how realistic is testing PCIe scaling with the same CPU? There shouldn't be any situation where you have PCIe 2.0 or PCIe 1.1 on a modern system.


----------



## Chrispy_ (Oct 14, 2022)

defaultluser said:


> disagree man, look at the gap at 1440p versus the 3080:


I have absolutely no idea what you're talking about; There's no 3080 in this chart, a 4090 was used and PCIe 2.0 is only 4% slower than PCIe 4.0, which is the point I'm making.

If you mean that the 4090 is 50% faster than a 3080, then that's still a CPU bottleneck because the 4090 is _twice_ as fast as a 3080 at 4K - ergo the scaling at lower resolutions is CPU-limited.

You can see many games exceeding 200fps which is well into the realms of high-fps gaming that frequently expose CPU weaknesses:




Add that to the problem of 240Hz 4K displays being vanishingly rare and expensive, and what we have an average framerate that frequently exceeds the capabilities of most displays and/or CPUs.


----------



## Tek-Check (Oct 14, 2022)

W1zzard said:


> What software is that? The people you listed don't need such a fast GPU as far as I understand, they only need some basic viewport acceleration


Workflows such as 3D GPU render engines - Octane, Redshift, V-Ray, Arnold GPU, Cycles, etc. Do we know how much data goes over PCIe bus in those workflows? I am asking out of curiocity to get an idea of possible saturation point at x8 Gen4 outside of gaming workflows.



ZetZet said:


> Yeah, so about those PCIe Gen5x16 slots on higher end boards...


You should only buy those boards if you need enough of PCIe data bandwidth for specific workflows. You can bifurcate x16 Gen5 lanes into x8 x8, keep GPU in the first slot and attach another device, such as NVMe RAID array to second x8 Gen 5 slot. PCIe 5.0 opens up many possibilties for those who need it and if motherboard vendors wire slots properly.


----------



## W1zzard (Oct 14, 2022)

Tek-Check said:


> Workflows such as 3D GPU render engines - Octane, Redshift, V-Ray, Arnold GPU, Cycles, etc. Do we know how much data goes over PCIe bus in those workflows? I am asking out of curiocity to get an idea of possible saturation point at x8 Gen4 outside of gaming workflows.


Should be very low, otherwise the GPU cores can't get loaded fully. Any idea if anyone has ever done a PCIe scaling test for these apps?


----------



## Jism (Oct 14, 2022)

Well done test.

We can conclude that the PCI-E adoption to even 6 in a few years is not really because of the ever demanding powerfull GPU's but simply enterprise markets having the need for more bandwidth. NVME storage, NIC's and everything else.

When the switch was made from AGP X8 vs PCI-E X16 there was barely any difference. It took a few generations of cards even to start taking benefit from that extra bandwidth.


----------



## Tek-Check (Oct 14, 2022)

CapNemo72 said:


> This article will be important for those deciding to go with AMD x670 or x670E or B650 or b650E. From your foundings, you should just be sure that m2 slot has that PCie Gen5 possibility as the GPU will not be able to use it. Saving maybe 50 bucks on that could be invested in a bit faster GPU.


Yes and no, depending on how many PCIe lanes someone needs for their peripherals and workflows. 
Both extreme and vanilla boards with two x16 Gen4/5 slots can be bifurcated to run at x8 and x8 Gen4/5. You can keep your GPU, either Nvidia or AMD, in one x8 slot (today's finding show that x8 Gen4 with Nvidia will work fine, so AMD GPU will work well too in x8), and attach another device to second x8 Gen4/5, such as NVMe RAID array.



Bcannon2000 said:


> Why are you benchmarking GPUs with a 5800X? Shouldn't you be using the best CPUs for benchmarking GPUs? You have three other options for the best gaming CPUs, Alder Lake, Zen 4, and the 5800X3D


Does it matter? He was testing PCIe bus data exchange.



Chrispy_ said:


> Hah! PCIe 2.0 still being fine almost 16 years later
> 
> Sure, if you want that last 5% then you'll need PCIe 4.0 but these articles always prove that the PCIe race really isn't that necessary unless you're already chasing diminishing returns.


It's more about providing more bandwidth over PCIe bus to give more flexibility to attach various peripherals to less lanes as PCIe generation widens data bandwidth. You can't do too much with x16 PCIe 2.0, but you can with x16 PCIe 4.0 or 5.0. You can bifurcate and trifurcate those lanes to several slots and attach numerous peripherals, such as AIC network card, NVMe RAID array, etc.



birdie said:


> Considering there's on average a 2% difference between PCIe 3.0 and 4.0 there would be a 0% performance inrovement from using PCIe 5.0.


Widening bandwidth on PCIe bus is more about providing more flexibility in assigning lanes to several slots for different peripherals. You can do a lot of configurations with 16 Gen5 lanes from CPU, such as two slots x8 x8, three slots x8 x4 x4, etc. 

Nowadays with x16 Gen5 lanes you need much less space on motherboard to devide into several peripherals. With Gen3, you would need 64 lanes to provide the same data bandwidth.



Style68 said:


> If I were to use a two Gen 4 NVMe SSDs in RAID 0 on the M.2 slots that are wired to the chipset of a high end z790 board like the ASUS ROG Maximus z790 Extreme, would I get the full PCIe x16 for the graphics card and get speeds matching a single Gen 5 NVMe SSD that would have been installed on the Gen 5 slot?


Two NVMe Gen4 drives on the chipset in RAID0 mode cannot exceed the speed of M.2 x4 slot wiring, so you get 64 Gbps of traffic.
You would need to attach two NVMe Gen4 drives to AIC card in x8 Gen4 slot to get this unified pool to transmit over eight lanes, which gives 128 Gbps. This would match one NVMe Gen5 traffic.



Bcannon2000 said:


> The issue is that testing with the 5800X is disingenuous,


You can test with any recent CPU. The purpose was to see whether PCIe bus throughput was affected by cutting lanes or between generations. Several CPUs and GPUs should be tested for the consistency of PCIe bus throughput data. This was the first test, but HUB have tested this before in more diverse PCIe configurations. Have a watch on youtube.


----------



## Bcannon2000 (Oct 14, 2022)

Tek-Check said:


> Does it matter? He was testing PCIe bus data exchange.


It absolutely does. He was testing GPU performance scaling, not bandwidth. Would you say the same if he tested PCIe 3.0 to PCIe 1.1 using an Athlon X4 950 Bristol Ridge? That is a quad-core AM4 Bulldozer CPU. Could you see the same PCIe scaling as the Athlon if he used the 7700X or 12900KS PCIe 3.0 to PCIe 1.1? No. 

If you have a limit with the CPU performance, you won't be able to see any difference in PCIe scaling or any other changes. If there is a bottleneck, it doesn't matter how fast or how wide the flow is after the bottleneck. It still will be limited by the bottleneck


----------



## Tek-Check (Oct 14, 2022)

W1zzard said:


> Maybe other sites that have 4 benchmarks and 5 comparison cards can do it, and they probably still recycle results .. I have 25 games, 30+ cards, 3 resolutions


Would you consider breaking down the three resolutions into SRD and HDR gaming performance? I could not find information on the testing page whether games are tested for SDR/HDR output. I can see "highest quality settings". Does that include HDR being on in each game?

As shown by Tim from HUB, some gaming engines deal with HDR differently and need more processing when enabled, which lowers frame rates on HDR monitor. This would mean that games need to be tested tested in SDR and HDR, to show differences in GPU performance in those scenarios. A game achieving say 70 fps on 4K SDR monitor could achieve 63 fps on 4K HDR monitor. Many games perform similarly in both modes, but it would be useful to identify the outliers, so that published charts are even more accurate.











Denver said:


> Yeah, the i9 would also bring better results than the current CPU.


This is irrelevant. He was testing PCIe bus throughput, not which CPU is better performing.



masterkaj said:


> Curious if PCIE 4.0 vs 3.0 matters if you are recording using the AV1 encoder at the same time?  Cause it to saturate a bit more?


Someone would need to test several workflows at the same time to tell us where saturation point is. This somebody would need to have a lot of time to do that.



Denver said:


> In some cases there is a considerable difference


Please make pictures smaller before posting, as they appear gigantic in the comment section! Try to drag picture angle diagnonally towards its centre.



Chrispy_ said:


> Add that to the problem of 240Hz 4K displays being vanishingly rare and expensive, and what we have an average framerate that frequently exceeds the capabilities of most displays and/or CPUs.


It depends on what game people play on what kind of gear. Here, average performance is a meaningless measure. If I play Flight Simulator and Cyberpunk in ultra settings only, which is very demanding on GPU, I will not need more than 4K/90hz HDR display. Even mighty 4090 cannot give more than 73 fps in Flight Simulator in this case. Do we need 73 fps in Flight Simulator? Not necessarily. 60 is plenty. It's a slow pace game mostly, unless someone enjoys fighter jet action. It's all about use scenario.

4090 is a halo product that unlocks higher frame rates on 4K display and could be purchased specifically for that purpose by those who need it to do that. Anyone playing 1080p or 1440p should never consider this card, unless needing insane frame rates. A total overkill. Anyone who needs future-proof DisplayPort 2.0 connectivity should never buy 4000 series cards.


----------



## mouacyk (Oct 14, 2022)

Upgrayedd said:


> Why not use Vulkan on RDR2?


I only got 70-80% GPU usage vs 99% with DX12 on my RTX3080.  It's probably not optimized.


----------



## Tek-Check (Oct 14, 2022)

W1zzard said:


> Should be very low, otherwise the GPU cores can't get loaded fully. Any idea if anyone has ever done a PCIe scaling test for these apps?


Not to my knowledge, that's why I am asking. I am curious about saturation point and data rates for different workflows. How much bandwidth different games use and how much those pro workflows send over the bus? Cannot find any chart or table with those GB/s. 

If we had such data, we would roughly be able to see saturation points for all generations of PCIe with simulataneous workflows, e.g. gaming while other app performs render and third one other graphics tasks.



Bcannon2000 said:


> It absolutely does. He was testing GPU performance scaling, not bandwidth. Would you say the same if he tested PCIe 3.0 to PCIe 1.1 using an Athlon X4 950 Bristol Ridge? That is a quad-core AM4 Bulldozer CPU. Could you see the same PCIe scaling as the Athlon if he used the 7700X or 12900KS PCIe 3.0 to PCIe 1.1? No.
> 
> If you have a limit with the CPU performance, you won't be able to see any difference in PCIe scaling or any other changes. If there is a bottleneck, it doesn't matter how fast or how wide the flow is after the bottleneck. It still will be limited by the bottleneck


True that. I stand corrected.
With DirectStorage 1.1, will CPU bottleneck become less relevant in testing PCIe scaling?


----------



## mahirzukic2 (Oct 15, 2022)

Tek-Check said:


> True that. I stand corrected.
> With DirectStorage 1.1, will CPU bottleneck become less relevant in testing PCIe scaling?


No, it will not. DirectStorage increases speed with which textures are loaded from SSD, and afterwards instead of decompressing the compressed textures and assets on the CPU they will be decompressed on the GPU for even faster loading of data which will decrease loading times any time new assets are being loaded by the game, e.g. in cutscenes, changing the map or zone, loading new assets in case of open world game, etc.
Other operations being done by the CPU, calculating game logic and issuing GPU draw calls will still be there and will still tax the CPU.
So no, CPU won't become less relevant in bottlenecking the PCIe scaling. With DirectStorage 1.1 you will see faster loading time and shorter cut scenes but PCIe scaling will still be the same as before in terms of raw FPS values.


----------



## The Von Matrices (Oct 15, 2022)

I always have wanted PCIe to not require powers of 2 for the number of lanes enabled .  This would be a perfect situation where an x16 slot with 12 enabled lanes would be great.


----------



## ir_cow (Oct 15, 2022)

P4-630 said:


> So basically if you still got PCIe 3.0 you're still ok with that and a 4090....


Yes, but any MB that is Gen3 probably has a CPU that will be holding it back.


----------



## mechtech (Oct 15, 2022)

Nice.   Just gotta hacksaw out the ends of my 1x slots so I can put a 16x card in there


----------



## defaultluser (Oct 15, 2022)

BorisDG said:


> I said you don't need PCI 4.0 or more even for 4090... 3.0 seems enough


agreed

you have to castrate the ever-loving-shit out of my ivy bridge h77 system by using the south-bridge-only second video slot at2.0 x4 \

or install a 2500k in place of my 3570lk


----------



## BigMack70 (Oct 15, 2022)

Good to know I don't need to upgrade from my 9900ks yet... Was hoping that would be the case


----------



## Hxx (Oct 15, 2022)

thank you for posting such great work!! 
As a 10th gen intel owner i am relieved i dont need to upgrade my platform lol... for at least two more years.
this is excellent news for all those still on pcie 3.0.



Selaya said:


> looks like there's finally a point to 4.0x16 for high refresh.
> interesting.


yes 2% lmao. massive improvement


----------



## Sora (Oct 15, 2022)

this testing is missing horizon zero dawn, where massive differences were noted.


----------



## W1zzard (Oct 15, 2022)

Sora said:


> this testing is missing horizon zero dawn, where massive differences were noted.


Just bad programming


----------



## Mussels (Oct 15, 2022)

Does halo infinite have issues with the 40 series? whats up with the 1080p results getting worse than 1440p?








ir_cow said:


> Yes, but any MB that is Gen3 probably has a CPU that will be holding it back.


*Slaps 5800x3D on x370*

Yeaaaaah booooooi.
To be fair that's about the only possible example it could work, Zen3/3D on 300 series boards - otherwise i'm totally agreeing with you.





I think people are missing that these results are showing that even an 8x 3.0 or 4.0 link will be fine for 99% of users, which is great when a lot of motherboards have issues with bandwidth being yoinked from the GPU slot for NVME and other PCI-E slots.


----------



## W1zzard (Oct 15, 2022)

Mussels said:


> Does halo infinite have issues with the 40 series? whats up with the 1080p results getting worse than 1440p?


CPU limited, and the higher CPU overhead of Ada causes it to fall behind older gen while CPU limited, because the driver eats a few extra CPU cycles that could otherwise go to the game to eke out of a few more frames


----------



## Calmmo (Oct 15, 2022)

W1zzard said:


> CPU limited, and the higher CPU overhead of Ada causes it to fall behind older gen while CPU limited, because the driver eats a few extra CPU cycles that could otherwise go to the game to eke out of a few more frames



Here's hoping  AMD releases the 3D variants soon after the 13900k...
even if it only manages to roughly match the 13900k id rather buy into a platform that will be getting at least one more launch


----------



## Bjorn_Of_Iceland (Oct 15, 2022)

I just wish they would add an Unreal Engine 5 based app to benchmark, as this would also indicate recent upcoming title performance.


----------



## mahirzukic2 (Oct 15, 2022)

mechtech said:


> Nice.   Just gotta hacksaw out the ends of my 1x slots so I can put a 16x card in there


You don't need to hacksaw anything. Just get a PCIe riser.


----------



## AnarchoPrimitiv (Oct 15, 2022)

It's a shame that USB4 was just released, because a Thunderbolt 5/USB5 based on a PCIe 5.0x4 link would have been awesome since in terms of bandwidth it equates to 3.0x16, which, with the proper chips, could mean an eGPU enclosure could offer 3.0x16 and basically make it so an external GPU would be just as performant as a GPU slotted into the motherboard.



birdie said:


> And I guess mostly AMD fans have trashed NVIDIA for not supporting PCIe 5.0.
> 
> Considering there's on average a 2% difference between PCIe 3.0 and 4.0 there would be a 0% performance inrovement from using PCIe 5.0.
> 
> DP 2.0 for RTX 4090 - that's relevant for _some_. PCIe 5.0? The time has yet to come.


Although I'm not a fan of AMD, I do cheer for them as any gains for AMD at Intel's expense equates to a better situation for consumers with the idea being that if they reach a 50/50 marketshare, they will reach a balance of power and be forced into the most competetive situation possible and therefore result in the best possible market for consumers.....but anyway, I think anyone who knows anything about PCIe knew that 5.0 wouldn't really provide any tangible benefits over 4.0 in the consumer space with respect to GPUs.  And thus far I uavent heard or read anyone, regardless of their allegiance,  making a big deal about it.  Regardless, there'll always be defenders on either side trying to pick on the perceived shortcomings of others....

Just like people trying to negatively point out that AM5 is a brand new platform and will therefore have associated costs with adopting a brand new platform.....and the sky is blue ...that's always going to be true with any new platform or any consumer building a computer from scratch and requiring all new hardware even if they're using an older platform....it's a reality that can never be avoided so I don't understand the reason behind pointing it out, especially in the summary/conclusion of a review on Ryzen 7000 when it's listed under "negatives", it implies that a company can do something to avoid such a situation, and besides artificially lowering prices and cutting into profit, something a publicly traded company can be sued for by their shareholders, there's nothing really that can be done....it's just the inherent nature of the situation, but by hypothetically listing with things such as, "DDR5 only, no choice of DDR4", it makes it seem like it was a design choice by AMD like the example I just gave of DDR5.


----------



## W1zzard (Oct 15, 2022)

Bjorn_Of_Iceland said:


> I just wish they would add an Unreal Engine 5 based app to benchmark, as this would also indicate recent upcoming title performance.


I've been thinking about make an UE5-based workload for various testing, but I feel it's too early for a rendering workloads and it might misrepresent what UE5 can do eventually. I added an UE5 developer workflow in my CPU benchmarks recently


----------



## Quicks (Oct 15, 2022)

Selaya said:


> looks like there's finally a point to 4.0x16 for high refresh.
> interesting.



are we reading the same charts?


----------



## mechtech (Oct 15, 2022)

mahirzukic2 said:


> You don't need to hacksaw anything. Just get a PCIe riser.













but that would be a better idea though


----------



## lightning70 (Oct 15, 2022)

As a result, PCI Express 5.0 is not yet required for graphics cards as there is almost no difference between 3.0 and 4.0.


----------



## Denver (Oct 16, 2022)

Tek-Check said:


> Would you consider breaking down the three resolutions into SRD and HDR gaming performance? I could not find information on the testing page whether games are tested for SDR/HDR output. I can see "highest quality settings". Does that include HDR being on in each game?
> 
> As shown by Tim from HUB, some gaming engines deal with HDR differently and need more processing when enabled, which lowers frame rates on HDR monitor. This would mean that games need to be tested tested in SDR and HDR, to show differences in GPU performance in those scenarios. A game achieving say 70 fps on 4K SDR monitor could achieve 63 fps on 4K HDR monitor. Many games perform similarly in both modes, but it would be useful to identify the outliers, so that published charts are even more accurate.
> 
> ...


This is very relevant, how do you want to measure a possible PCIe bottleneck when there is already a CPU bottleneck?


----------



## The Von Matrices (Oct 16, 2022)

lightning70 said:


> As a result, PCI Express 5.0 is not yet required for graphics cards as there is almost no difference between 3.0 and 4.0.


It isn't needed for performance but PCIe 5.0 would offer extra flexibility.  If the card supported PCIe 5 then you could run it in an x8 slot with a PCIe 5 SSD on Intel platforms without losing GPU performance.


----------



## Tek-Check (Oct 16, 2022)

lightning70 said:


> As a result, PCI Express 5.0 is not yet required for graphics cards as there is almost no difference between 3.0 and 4.0.


PCIe 5.0 is not required, but it is desirable for future-proofing PC systems. Nvidia was anti-consumer by not offering PCIe 5.0 and DisplayPort 2.0 on 4000 GPUs.

PCIe 5.0 is desirable for the entire PC ecosystem, including GPU. Although is makes motherboard and peripherals design a bit more expensive initially, it will greaty improve connectivity and expansion slot provision for peripherals, like never before. You can run GPUs in x8 or even x4 mode on 5.0 and free up another x8 or x12 lanes for other devices with direct traffic to CPU.

Having 24 Gen5 usable lanes on 7000 Ryzen CPUs, is like having 48 Gen4 lanes, practically entry workstation platform. First generation of AM5 motherboards is a safe, mainstream design, but I do expect some vendors to be more adventurous in new edition of boards next year. We can expect some developments:

- PCIe switch chip Gen5-to-Gen4 x4 (previosuly known as PLX), that will double downstream lanes. Current Promo21 chip uses PCIe 4.0 x4 link to CPU at 64 Gbps, so half of 128 Gbps that CPU can provide. A new double chipset wil be able to connect to CPU via PCIe switch chip independently, gaining x4 lanes for another NVMe drive or other peripheral. Current daisy-chaining consumes those x4 lanes.

- or new edition of AM5 chipset Promo21 from AsMedia will be PCIe 5.0 capable itself, taking all 128 Gbps from CPU chipset link

- several boards currently have two x16 Gen5 slots with ability to bifurcate lanes into x8 x8 configuration. This allows new GPUs to operate in x8 Gen5 slot without any performance loss and frees another x8 Gen5 lanes for AIC peripheral, such as NVMe RAID array or other solution. 

- three x16 slots can trifurcate 16 Gen5 lanes too, x8 or x4 electrically for GPU, and another x8 and x12 respectively for other peripherals.

There are many options, depending on how many PCIe slots vendors want to provide, how to wire electrically those slots and where to install PCIe switch chip. PCIe switch can also link to those bifurcarted lanes from GPU, say x8 Gen5, and provide another x16 Gen4 lanes for other non-AIC peripherals. 

Therefore, PCIe 5.0 brings a lot of flexibility to motherboard design and connectivity without compromising on performance or need to bi-wire two devices on one set of lanes, which has often been the case. For example, many boards have mutually exclusive option to either connect M.2 drive or PCIe x4 device. Both cannot work in the same time. With PCIe 5.0, this is not necessary anymore.



Denver said:


> This is very relevant, how do you want to measure a possible PCIe bottleneck when there is already a CPU bottleneck?


I corrected myself on this one already in another post.


----------



## AlwaysHope (Oct 16, 2022)

Chrispy_ said:


> Hah! PCIe 2.0 still being fine almost 16 years later
> 
> Sure, if you want that last 5% then you'll need PCIe 4.0 but these articles always prove that the PCIe race really isn't that necessary unless you're already chasing diminishing returns.


Yes, that is interesting. 4.0 will set you up for rest of this decade, at least on a single monitor.


----------



## wheresmycar (Oct 16, 2022)

Tek-Check said:


> PCIe 5.0 is not required, but it is desirable for future-proofing PC systems. Nvidia was anti-consumer by not offering PCIe 5.0 and DisplayPort 2.0 on 4000 GPUs.
> 
> PCIe 5.0 is desirable for the entire PC ecosystem, including GPU. Although is makes motherboard and peripherals design a bit more expensive initially, it will greaty improve connectivity and expansion slot provision for peripherals, like never before. You can run GPUs in x8 or even x4 mode on 5.0 and free up another x8 or x12 lanes for other devices with direct traffic to CPU.
> 
> ...



no doubt there will be a string of buyers who will benefit from the uncompromising 5.0 flex, more-so specific workstation-class builds but for the wider majority (gamers/general productivety/office work/etc) i can't see most of us needing anything above 3.0 and yet 4.0 is a probable long term blessing in itself. 

For most by the time 5.0 becomes an essential piece of the puzzle current platforms will be, from an overall performance enthusiast perspective, ANCIENT! Saying that, as a gamer i'm still keeping my options open and might end up pulling the trigger on Zen 4... not necessarily for PCIe 5.0 but socket longevity.


----------



## Wasteland (Oct 16, 2022)

wheresmycar said:


> no doubt there will be a string of buyers who will benefit from the uncompromising 5.0 flex, more-so specific workstation-class builds but for the wider majority (gamers/general productivety/office work/etc) i can't see most of us needing anything above 3.0 and yet 4.0 is a probable long term blessing in itself.
> 
> For most by the time 5.0 becomes an essential piece of the puzzle current platforms will be, from an overall performance enthusiast perspective, ANCIENT! Saying that, as a gamer i'm still keeping my options open and might end up pulling the trigger on Zen 4... not necessarily for PCIe 5.0 but socket longevity.



Agreed.  PCIe 5.0 isn't worth the apparent extra cost (on motherboards) to the average user.  Heck, to most users, the only practical difference between PCIe 3.0 and PCIE 4.0 SSDs is that the latter run hotter.  No lie, I actually bought a secondary PCIe 4.0 SSD by accident recently, because the price was low enough that I assumed it was a 3.0 model.  Learned otherwise when I saw the temperature after installing it, though, lol.  

Not to say that there isn't merit in pushing these newer technologies to the consumer space, but I just don't have any compelling reason to care about what they offer.  W1zzard's excellent testing here reinforces the point.


----------



## dicobalt (Oct 16, 2022)

This makes me more annoyed at PCIe 5 and how expensive it is. Just put the damn m.2 in a 8x slot instead. You don't need to explode the platform cost for the sake of a form factor.


----------



## samih22 (Oct 16, 2022)

so I pre-ordered Raptor-lake , and already gotten myself a 4090. 

my question is, should I leave the GEN 5 m.2 slot empty ? or is it fine if I use a GEN4 m.2 on it ? if I do , I would also lose half the lanes?

I do not want to lose 1% or 2%. I will just use the other m.2 slots , I just wish if I could use all the slots on my board. Thanks


----------



## Godrilla (Oct 16, 2022)

BigMack70 said:


> Good to know I don't need to upgrade from my 9900ks yet... Was hoping that would be the case


I have the same setup against all odds, 9900ks, itx, 750 watt sfx platinum psu by corsair, Pcie 3.0 and 4090 Suprim liquid to replace the 3090 xc3 ultra hybrid. I will keep you posted on the outcome. ( probably upgrade to zen 4 3d to improve frame variance and better 0.1 % lows)
I was hoping they test which cpu generation shows the significant cpu bottlenecks at a resolution that scales best for the 4090 like 4k also will using DSR to 8k on 4k display while mitigating the cpu bottlenecks and still improving image quality. This is where I see the next trend in upscaling technology going upscaling to a higher resolution than your monitor to mitigate bottlenecks when there is performance on the table and improve image quality. Aka Dldsr.


----------



## gQx (Oct 16, 2022)

I wonder since you have this fps chart what about the temperature and power consumption difference between those cards?


----------



## arni-gx (Oct 16, 2022)

NVIDIA GeForce RTX 4090 PCI-Express Scaling
					

The new NVIDIA GeForce RTX 4090 is a graphics card powerhouse, but what happens when you run it on a PCI-Express 4.0 x8 bus? In our mini-review we've also tested various PCI-Express 3.0, 2.0 and 1.1 configs to get a feel for how FPS scales with bandwidth.




					www.techpowerup.com
				



OMG, thank u so much, dude..... finally, my old PCIE 3.0 16x still can handle those rtx 4090 24gb...... the different is only 2-3%..... cool......


----------



## Tek-Check (Oct 16, 2022)

wheresmycar said:


> no doubt there will be a string of buyers who will benefit from the uncompromising 5.0 flex, more-so specific workstation-class builds but for the wider majority (gamers/general productivety/office work/etc) i can't see most of us needing anything above 3.0 and yet 4.0 is a probable long term blessing in itself.
> 
> For most by the time 5.0 becomes an essential piece of the puzzle current platforms will be, from an overall performance enthusiast perspective, ANCIENT! Saying that, as a gamer i'm still keeping my options open and might end up pulling the trigger on Zen 4... not necessarily for PCIe 5.0 but socket longevity.


Be mindful that several PCIe standards co-exist on same boards. You still have Gen3, Gen4 and Gen5 lanes an AM5 and 700 chipsets in different proportions.

The proportion of devices wired with different speeds will decide which platform appeals to your needs and budget. Diversity is the key here. We need most advanced, current and older PCIe lanes for variety of needs and peripherals. For example, it would be a waste to wire vast majority of LAN ports with Gen4 lanes, unless it's 10 GbE. It would also be a waste to wire entire motherboard with Gen5 lanes only, as noone really needs that.



Wasteland said:


> Agreed.  PCIe 5.0 isn't worth the apparent extra cost (on motherboards) to the average user.  Heck, to most users, the only practical difference between PCIe 3.0 and PCIE 4.0 SSDs is that the latter run hotter.  No lie, I actually bought a secondary PCIe 4.0 SSD by accident recently, because the price was low enough that I assumed it was a 3.0 model.  Learned otherwise when I saw the temperature after installing it, though, lol.
> 
> Not to say that there isn't merit in pushing these newer technologies to the consumer space, but I just don't have any compelling reason to care about what they offer.  W1zzard's excellent testing here reinforces the point.


There is a merit, as explained above. New technologies are needed. User base is very diverse. If you do not have a reason to care personally, that's fine. But, you should at least be open-minded to new technologies coming to consumer space for those who want to use it for their needs. You and other users coexist in the same consumer space.


----------



## dicobalt (Oct 16, 2022)

samih22 said:


> so I pre-ordered Raptor-lake , and already gotten myself a 4090.
> 
> my question is, should I leave the GEN 5 m.2 slot empty ? or is it fine if I use a GEN4 m.2 on it ? if I do , I would also lose half the lanes?
> 
> I do not want to lose 1% or 2%. I will just use the other m.2 slots , I just wish if I could use all the slots on my board. Thanks


Look at the block diagram in the motherboard manual, usually it's in the front pages of the manual. See if the m.2 connects direct to the CPU or direct to the chipset. If the diagram's line leading to the m.2 slot doesn't merge with any PCIe slots or m.2 slots then it is dedicated to that m.2 slot. If it connects to the chipset check the PCIe speed between the chipset and the CPU in the block diagram also. Also be prepared to purchase a m.2 heatpipe cooler if you get a 13GB/s PCIe 5 SSD because they get extremely hot. Make sure the location of that m.2 heatpipe does not interfere with the GPU heatsink or with a CPU heatsink if you decide on CPU air cooling. Also keep in mind that PCIe 5.0 m.2 are longer wide 25mm x 110mm instead of common 22mm x 80mm used today.


----------



## Tek-Check (Oct 16, 2022)

samih22 said:


> so I pre-ordered Raptor-lake , and already gotten myself a 4090.
> my question is, should I leave the GEN 5 m.2 slot empty ? or is it fine if I use a GEN4 m.2 on it ? if I do , I would also lose half the lanes?
> I do not want to lose 1% or 2%. I will just use the other m.2 slots , I just wish if I could use all the slots on my board. Thanks


If you use* any dedicated* M.2 Gen5 slot with *any* NVMe drive on *any* Z790 board supporting Gen5 NVMe drives, your GPU slot will operate at *x8 Gen5* with upcoming AMD GPUs and *x8 Gen4 *with Nvidia and Intel cards. This is because CPU provides only x16 Gen5 lanes for GPU and no additional dedicated Gen5 lanes for M.2 drives.

Motherboard vendors workaround is to "steal" x8 lanes from GPU in order to provide M.2 Gen5 capability. This means whichever NVMe drive you put into dedicated M.2 Gen5 slots, it will cut GPU slot lanes in half. You will need to check in manual or in BIOS whether M.2 Gen5 slots operate by default or the option needs to be toggled and enabled in BIOS. Different vendors may enable it by default or not in BIOS.

If you do not want to loose 1-2% on GPU, then run bandwidth and scaling tests, such as PCIe bandwidth speed in 3D Mark and a few games for scaling.
1. Test GPU in x16 Gen4 mode (Nvidia 4000 cards do not support Gen5) and without NVMe drive in Gen5 slot
2. Test GPU in x8 Gen4 mode with any NVMe drive in M.2 Gen 5 dedicated slot.
See if there is any difference in data throughput and fps, and let us know. I am quite curious to know.

If there is no difference, you are ok to put any NVMe drive into M.2 Gen5 dedicated slot. If there is 1-2% difference, it's negligible, practically zero. You will never notice this in using 4090. You can still leave your NVMe drive in that slot. If difference is 5% or more, then leave all M.2 Gen5 dedicated slots empty.

Below is a diagram of Z790 CPU and HSIO lanes. I hope this answers your questions.








dicobalt said:


> Look at the block diagram in the motherboard manual, usually it's in the front pages of the manual. See if the m.2 connects direct to the CPU or direct to the chipset. If the diagram's line leading to the m.2 slot doesn't merge with any PCIe slots or m.2 slots then it is dedicated to that m.2 slot. If it connects to the chipset check the PCIe speed between the chipset and the CPU in the block diagram also. Also be prepared to purchase a m.2 heatpipe cooler if you get a 13GB/s PCIe 5 SSD because they get extremely hot. Make sure the location of that m.2 heatpipe does not interfere with the GPU heatsink or with a CPU heatsink if you decide on CPU air cooling. Also keep in mind that PCIe 5.0 m.2 are longer wide 25mm x 110mm instead of common 22mm x 80mm used today.


You did not answer his question. There is no M.2 Gen5 slot connected to chipset. CPU supports only x16 Gen5 lanes for GPU, which could be bifurcated into x8 Gen5 for GPU and x8 Gen5 for NVMe drives. Intel's chipset operates at Gen4 speeds for all peripherals.


----------



## dicobalt (Oct 16, 2022)

Tek-Check said:


> You did not answer his question. There is no M.2 Gen5 slot connected to chipset. CPU supports only x16 Gen5 lanes for GPU, which could be bifurcated into x8 Gen5 for GPU and x8 Gen5 for NVMe drives. Intel's chipset operates at Gen4 speeds for all peripherals.


Wow I didn't realize PCIe 5.0 support was so limited. I figured even on the budget boards it would support at least one PCIe 5 m.2 without stealing from the GPU.


----------



## AusWolf (Oct 16, 2022)

So when limited to pci-e gen 3.0 or 4.0 x8, the $1600 4090 is slower in Metro: Exodus than the $900 6950 XT. Interesting. Still over 200 FPS at 1080p, but still interesting.


----------



## Tek-Check (Oct 16, 2022)

dicobalt said:


> Wow I didn't realize PCIe 5.0 support was so limited. I figured even on the budget boards it would support at least one PCIe 5 m.2 without stealing from the GPU.


It's the hard limit on Raptor Lake CPUs. Vendors need to work with x16 Gen5 lanes and make a compromise if they want to enable M.2 Gen5 drives.
For more Gen5 lanes, either from CPU or chipset, consumers who need it will need to wait for Meteor Lake. Or consider Zen4 AM4 platform, where CPUs have x16 Gen5 for GPU and another x8 for Gen5 drives. Below is a diagram of all lanes for Ryzen X670E platform.






AusWolf said:


> So when limited to pci-e gen 3.0 or 4.0 x8, the $1600 4090 is slower in Metro: Exodus than the $900 6950 XT. Interesting. Still over 200 FPS at 1080p, but still interesting.


True. People will need to look into specific scenarios. On average, difference is small, but in some games situation is different.


----------



## HTC (Oct 16, 2022)

Tek-Check said:


> It's the hard limit on Raptor Lake CPUs. Vendors need to work with x16 Gen5 lanes and make a compromise if they want to enable M.2 Gen5 drives.
> For more Gen5 lanes, either from CPU or chipset, consumers who need it will need to wait for Meteor Lake. Or consider Zen4 AM4 platform, where CPUs have x16 Gen5 for GPU and another x8 for Gen5 drives. Below is a diagram of all lanes for Ryzen X670E platform.
> 
> View attachment 265735
> ...



Do you happen to have the diagrams for X670 non-E and 650(E)?


----------



## Viruzz (Oct 16, 2022)

Bcannon2000 said:


> To me, if I am paying $1,600 or more for a GPU, it better have all of the newest bells and whistles. It doesn’t have DP 2.0 and PCIe 5.0, even if it or I can’t use them, I still want them because of how much it cost


Ill be happy if they remove displayport  from high end card
IMO useless, give me more HDMI 2.1
Its time to abandon this idiotic dual format and go with single universal display connection, HDMI 2.1 is great and lets anyone with whatever device they have use pc monitor.
DP is limited to just PC monitors.


----------



## Tek-Check (Oct 16, 2022)

HTC said:


> Do you happen to have the diagrams for X670 non-E and 650(E)?


It's the same for X670, apart from GPU, which is x16 Gen4

Below is B650E. For B650, GPU and NVMe are Gen4.







Viruzz said:


> Ill be happy if they remove displayport  from high end card
> IMO useless, give me more HDMI 2.1
> Its time to abandon this idiotic dual format and go with single universal display connection, HDMI 2.1 is great and lets anyone with whatever device they have use pc monitor.
> DP is limited to just PC monitors.


DisplayPort *IS* monitor standard. Nothing will change that. It is in several ways better than HDMI. You can tunnel DP through USB4 and have it in Alt Mode. DP is more flexible than HDMI. Over one single USB-C cable you can have DP, USB, PCIe data and power. It's brilliant. Plus, multi-stream transport. You can daisy-chain monitors over DP. HDMI does not offer that level of flexibility. And DP 2.0 will offer up to 80 Gbps ports. It's better.

I do agree that GPU should have one more HDMI port though. Some vendors offer it. Look up some Gigabyte and Asus cards. Also, HDMI has royalties. There are adapters too.


----------



## Viruzz (Oct 16, 2022)

AusWolf said:


> So when limited to pci-e gen 3.0 or 4.0 x8, the $1600 4090 is slower in Metro: Exodus than the $900 6950 XT. Interesting. Still over 200 FPS at 1080p, but still interesting.



Why are you looking at 1080p for 1600$ card? Any resolution besides 4K is irrelevant
In 4K, even at PCIe Gen 2.0 x16 [*Gen 4.0 x2*] mode the 4090 is 20fps faster than 6950

In gen 3.0 x16/Gen 4.0 x8, its 50fps faster.


----------



## AusWolf (Oct 16, 2022)

Viruzz said:


> Why are you looking at 1080p for 1600$ card? Any resolution besides 4K is irrelevant
> In 4K, even at PCIe Gen 2.0 x16 [*Gen 4.0 x2*] mode the 4090 is 20fps faster than 6950
> 
> In gen 3.0 x16/Gen 4.0 x8, its 50fps faster.


Because that's the resolution that I'm using. I know the 4090 isn't meant for that, just like it isn't meant for me, but that's my point of reference anyway.


----------



## Viruzz (Oct 16, 2022)

Tek-Check said:


> DisplayPort *IS* monitor standard. Nothing will change that. It is in several ways better than HDMI. You can tunnel DP through USB4 and have it in Alt Mode. DP is more flexible than HDMI. Over one single USB-C cable you can have DP, USB, PCIe data and power. It's brilliant. Plus, multi-stream transport. You can daisy-chain monitors over DP. HDMI does not offer that level of flexibility. And DP 2.0 will offer up to 80 Gbps ports. It's better.
> 
> I do agree that GPU should have one more HDMI port though. Some vendors offer it. Look up some Gigabyte and Asus cards. Also, HDMI has royalties. There are adapters too.



Royalties is the main reason why DP is used.
HDMI also has alt mode and can run over USB-C with power
with monitors, very few large models come out so all the speed of DP is wasted.

I been using OLEDs as my monitor, so at least i use the 4K/120 mode and its necessary, 
for PC monitor users, and lots of people for some reason by TINY useless monitors like 27 inch [and some people even smaller model there is no need for 4K at that size.
Even 21:9 "35" inch models not yet 4K and they don't need it

What im saying is that monitor users, the majority of them, like over 95% use tiny sizes and they dont go above 1440p, so 80gbps, 8K and so on is not for them.

What happens now is that people that cant afford a TV, cant buy any PC monitor and use it, it will have royalty free DP, or like garbing a cheap PC monitor and connecting to photography equipment which uses HDMI.
If we move to single format, it will solve lots of issue


----------



## Tek-Check (Oct 17, 2022)

Viruzz said:


> What happens now is that people that cant afford a TV, cant buy any PC monitor and use it, it will have royalty free DP, or like garbing a cheap PC monitor and connecting to photography equipment which uses HDMI.
> If we move to single format, it will solve lots of issue


True for HDMI Alt Mode. I forgot about it. I have checked. And the reason I forgot is because it uses HDMI 1.4b standard, not implemented on many devices in PC industry. I also have OLED TV, but do not use it as monitor. It has its own quirks. Firstly, it's too big, then text, etc. I do use it for 4K/120 gaming sometimes.

I do not think DP speed is wasted. For monitors that need up to 21 Gbps of bandwidth, vendors install DP 1.2 ports. Above that, you get DP 1.4, for 4K and some HDR monitors. It's quite rational decision. 

There are billions of monitors at home, at work, at public places, of very different sizes, from small laptop displays to professional 32/40 inch, to gigantic public displays over 100 inch. It's fine. 80 Gbps ports will find their use. There is no doubt about it. Remember, with one 80 Gbps cable you can daisy chain and simplify cabling in workplace, for example. Super convenient.

There are simple adaptors for all connectors. It's not a big deal. Increasing number of audio, video and storage devices are moving to USB-C. This is the one to rule them all, as you can flexibly choose which data protocols you want to offer over it, e.g. Thunderbolt, DP, PCIe, USB and power.


----------



## jigar2speed (Oct 17, 2022)

Control feels like a very bad coded game, graphic looks like they are from 2012 but performance requirement are way to high.


----------



## W1zzard (Oct 17, 2022)

jigar2speed said:


> Control feels like a very bad coded game, graphic looks like they are from 2012 but performance requirement are way to high.


The world disagrees with you: https://store.steampowered.com/app/870780/Control_Ultimate_Edition/#app_reviews_hash

Have you actually played it for a couple of hours? I finished it and have to say I enjoyed it


----------



## AusWolf (Oct 17, 2022)

W1zzard said:


> The world disagrees with you: https://store.steampowered.com/app/870780/Control_Ultimate_Edition/#app_reviews_hash
> 
> Have you actually played it for a couple of hours? I finished it and have to say I enjoyed it


I agree. Control is one of the few rare gems in our modern-day triple-A franchise-based sequel after sequel yawnfest.


----------



## jigar2speed (Oct 17, 2022)

W1zzard said:


> The world disagrees with you: https://store.steampowered.com/app/870780/Control_Ultimate_Edition/#app_reviews_hash
> 
> Have you actually played it for a couple of hours? I finished it and have to say I enjoyed it


I am glad i am wrong here. I have not played that game yet but now that you and a lot of folks are recommending it, i will give it a try.


----------



## Mussels (Oct 18, 2022)

W1zzard said:


> CPU limited, and the higher CPU overhead of Ada causes it to fall behind older gen while CPU limited, because the driver eats a few extra CPU cycles that could otherwise go to the game to eke out of a few more frames


That's something i've never seen mentioned before that the 40 series *need* a faster CPU


----------



## AusWolf (Oct 18, 2022)

Mussels said:


> That's something i've never seen mentioned before that the 40 series *need* a faster CPU


It's not new.


----------



## Bjorn_Of_Iceland (Oct 19, 2022)

W1zzard said:


> I've been thinking about make an UE5-based workload for various testing, but I feel it's too early for a rendering workloads and it might misrepresent what UE5 can do eventually...


Ahh yes.. true. Nevertheless, good review as always!


----------



## Hello_im_snowman (Oct 19, 2022)

NVIDIA GeForce RTX 4090 PCI-Express Scaling
					

The new NVIDIA GeForce RTX 4090 is a graphics card powerhouse, but what happens when you run it on a PCI-Express 4.0 x8 bus? In our mini-review we've also tested various PCI-Express 3.0, 2.0 and 1.1 configs to get a feel for how FPS scales with bandwidth.




					www.techpowerup.com
				




Absolutely brilliant article that has answered every question and curiosity I had about the 4090's PCIe scaling. I made an account just to upvote the content, well done!


----------



## sizzling (Oct 21, 2022)

I could not see it confirmed, did the pcie 4.0 results have resizable bar enabled? Seeing as it’s only supported on 4.0 it’s a key factor for understanding the difference between 3.0 and 4.0.


----------



## The Von Matrices (Oct 22, 2022)

Tek-Check said:


> Below is a diagram of Z790 CPU and HSIO lanes. I hope this answers your questions.
> 
> View attachment 265727


I'm not sure whether you generated that image or you got it from somewhere else, but according to Intel's ARK and other Intel documentation the Alder Lake/Raptor Lake CPU only supports bifurcating the PCIe x16 into two x8 slots. There is no evidence of x8/x4/x4 like was supported on Z590 and before. So there can only be one PCIe 5.0 M.2 slot on an Intel CPU. The other 4 lanes would go unused in that case.

Also, Intel says "Up to 20 PCIe 4.0" and "Up to 8 PCIe 3.0" for Z790, but your diagram would be 18 and 10, respectively.  Two of the PCIE 3.0 lanes in your diagram should be PCIe 4.0.


----------



## Tek-Check (Oct 22, 2022)

The Von Matrices said:


> I'm not sure whether you generated that image or you got it from somewhere else, but according to Intel's ARK and other Intel documentation the Alder Lake/Raptor Lake CPU only supports bifurcating the PCIe x16 into two x8 slots. There is no evidence of x8/x4/x4 like was supported on Z590 and before. So there can only be one PCIe 5.0 M.2 slot on an Intel CPU. The other 4 lanes would go unused in that case.
> 
> Also, Intel says "Up to 20 PCIe 4.0" and "Up to 8 PCIe 3.0" for Z790, but your diagram would be 18 and 10, respectively.  Two of the PCIE 3.0 lanes in your diagram should be PCIe 4.0.


All lanes are there, in total 28. There are two PCIe 4.0 lanes under SATA. Forgot to change the colour and text for those two. Thanks for the pedantic reminder. Will do once back home.

As for PCIe birurcation, several options are possible, depending on what vendors want to do. There could be AIC x8 for two M.2 drives. There could be two separate x4 M.2 slots if vendors install PLX switch chip on bifurcated lanes.

I marked it as two x4 for the sake of how many Gen5 drives could use full bandwidth from CPU, rather than as implemented solution on motherboard.


----------



## The Von Matrices (Oct 22, 2022)

Tek-Check said:


> All lanes are there, in total 28. There are two PCIe 4.0 lanes under SATA. Forgot to change the colour and text for those two. Thanks for the pedantic reminder. Will do once back home.
> 
> As for PCIe birurcation, several options are possible, depending on what vendors want to do. There could be AIC x8 for two M.2 drives. There could be two separate x4 M.2 slots if vendors installs PLX switch chip on vifurcated lanes.
> 
> I marked it as two x4 for the sake of how many Gen5 drives could use full bandwidth from CPU, rather than as implemented solution on motherboard.


Where are these diagrams from, by the way?  They're very helpful.


----------



## Tek-Check (Oct 23, 2022)

The Von Matrices said:


> Where are these diagrams from, by the way?  They're very helpful.


I made them, based on the same documents you referred to. Thanks for nice words.


----------



## grilli4nt (Oct 26, 2022)

This is a very interesting topic, as I building my gen 13 build with a 4090, and want to be future proof for m2 gen5, but dont really want to jeopardize any performance from my 4090, not even a few percentages if possible, than I'd rather go with a cheaper MB (such as the strix-f) and go PCIe nvme gen5 next upgrade in a few years when this is more mainstream.

I am particiarly looking at buying the z790 maximus hero. It has a PCIe gen 5 x16 but if I connect a gen5 m2 ssd in the gen2 slot, the GPU PCIe slot halves. But is this really a problem, or am I missing something here? I know the GPU dont run PCIe 5, but the bandwidth from the motherboard is still based on gen 5?

So halving a gen 5 PCIe x 16 into two gen 5 x 8 slots, that is still the same as x16 for each slot if translated to gen 4? I.e, there should be plenty room and absolutely no loss in performance running an m2 gen5?

Is it really accurate to say that when the PCIe gen 5 x 16 gets halved (as a result of inserting m2 gen5), it results in 2 x gen4 x 8 slots? Shouldn't it rather be 2x gen5 x8 slots, which essentially is the same as 2x gen4 x 16 slots?

This is a snippet from the Maximus Hero z790 specs:





Forgive me if im completely wrong here, just wanted to see if this makes any sense at all?


----------



## Nopa (Oct 26, 2022)

grilli4nt said:


> This is a very interesting topic, as I building my gen 13 build with a 4090, and want to be future proof for m2 gen5, but dont really want to jeopardize any performance from my 4090, not even a few percentages if possible, than I'd rather go with a cheaper MB (such as the strix-f) and go PCIe nvme gen5 next upgrade in a few years when this is more mainstream.
> 
> I am particiarly looking at buying the z790 maximus hero. It has a PCIe gen 5 x16 but if I connect a gen5 m2 ssd in the gen2 slot, the GPU PCIe slot halves. But is this really a problem, or am I missing something here? I know the GPU dont run PCIe 5, but the bandwidth from the motherboard is still based on gen 5?
> 
> ...


For Alder Lake & Raptor Lake, PCIe 5.0 GPU slot share lanes with PCIe 5.0 M.2 NVMe slots.

For Zen 4's X670E & B650E, you can have both PCIe 5.0 GPU (16 lanes) and PCIe 5.0 M.2 NVMe (8 lanes) separately.


----------



## grilli4nt (Oct 26, 2022)

Nopa said:


> For Alder Lake & Raptor Lake, PCIe 5.0 GPU slot share lanes with PCIe 5.0 M.2 NVMe slots.
> 
> For Zen 4's X670E & B650E, you can have both PCIe 5.0 GPU (16 lanes) and PCIe 5.0 M.2 NVMe (8 lanes) separately.


I get that, but if the PCIe 5.0 GPU slot is shared with the PCIe 5.0 M.2 NVMe slot, doesnt that mean that each slot gets 8 lanes? Since Gen 5 should be 2x the bandwidth from gen 4, doesnt this mean that Gen5 slot x 8 equals gen 4 x 16? Maybe thats not how it works, but shouldn't above mean that when the PCIe 4.0 RTX 4090 is connected into the "halved" Gen 5.0 PCIe slot, the slot itself should be able to produce bandwidth similar to a Gen 4.0 x16 slot?


----------



## Nopa (Oct 26, 2022)

grilli4nt said:


> I get that, but if the PCIe 5.0 GPU slot is shared with the PCIe 5.0 M.2 NVMe slot, doesnt that mean that each slot gets 8 lanes? Since Gen 5 should be 2x the bandwidth from gen 4, doesnt this mean that Gen5 slot x 8 equals gen 4 x 16? Maybe thats not how it works, but shouldn't above mean that when the PCIe 4.0 RTX 4090 is connected into the "halved" Gen 5.0 PCIe slot, the slot itself should be able to produce bandwidth similar to a Gen 4.0 x16 slot?


Yes, x8 of Gen 5 is equal x16 of Gen 4 in Bandwidth. But 4090 doesn't support Gen 5 and once running it on Intel's Z790 MB alongside an M.2 NVMe (Gen 4/5), it will divide Bandwidth with GPU slot which means it'll maximize at x8 of Gen4.

The GPU slot can fit one x4, one x8, one x16 of PCIe 3.0/4.0/5.0 (PCIe backward compatibility).

The M.2 slot can fit two Gen 5 x4 (Zen 4) (can run M.2 & GPU at maximum Bandwidth) and one Gen 5 x4 (Z690/Z790) (share two x4 to M.2 NVMe slot once running it alongside Gen 4/5 GPU).


----------



## AusWolf (Oct 26, 2022)

grilli4nt said:


> I get that, but if the PCIe 5.0 GPU slot is shared with the PCIe 5.0 M.2 NVMe slot, doesnt that mean that each slot gets 8 lanes? Since Gen 5 should be 2x the bandwidth from gen 4, doesnt this mean that Gen5 slot x 8 equals gen 4 x 16? Maybe thats not how it works, but shouldn't above mean that when the PCIe 4.0 RTX 4090 is connected into the "halved" Gen 5.0 PCIe slot, the slot itself should be able to produce bandwidth similar to a Gen 4.0 x16 slot?


That's not how it works. If you have x8 lanes available, it will be x8 lanes of whatever gen the installed card supports. The 4090 is a gen 4 x16 card, so it will connect via gen 4 x8 (which is the equivalent of gen 3 x16).


----------



## grilli4nt (Oct 27, 2022)

Nopa said:


> Yes, x8 of Gen 5 is equal x16 of Gen 4 in Bandwidth. But 4090 doesn't support Gen 5 and once running it on Intel's Z790 MB alongside an M.2 NVMe (Gen 4/5), it will divide Bandwidth with GPU slot which means it'll maximize at x8 of Gen4.
> 
> The GPU slot can fit one x4, one x8, one x16 of PCIe 3.0/4.0/5.0 (PCIe backward compatibility).
> 
> The M.2 slot can fit two Gen 5 x4 (Zen 4) (can run M.2 & GPU at maximum Bandwidth) and one Gen 5 x4 (Z690/Z790) (share two x4 to M.2 NVMe slot once running it alongside Gen 4/5 GPU).





AusWolf said:


> That's not how it works. If you have x8 lanes available, it will be x8 lanes of whatever gen the installed card supports. The 4090 is a gen 4 x16 card, so it will connect via gen 4 x8 (which is the equivalent of gen 3 x16).



Thanks both for clarifying.


----------



## Tek-Check (Oct 31, 2022)

AusWolf said:


> That's not how it works. If you have x8 lanes available, it will be x8 lanes of whatever gen the installed card supports. The 4090 is a gen 4 x16 card, so it will connect via gen 4 x8 (which is the equivalent of gen 3 x16).


Yes. In this case, 4090 might lose some performance, 2-3%, if you run it in Gen4 x8 mode, with bifurcated M.2 Gen5 slots. Have a look into PCIe scaling tests.








						NVIDIA GeForce RTX 4090 PCI-Express Scaling
					

The new NVIDIA GeForce RTX 4090 is a graphics card powerhouse, but what happens when you run it on a PCI-Express 4.0 x8 bus? In our mini-review we've also tested various PCI-Express 3.0, 2.0 and 1.1 configs to get a feel for how FPS scales with bandwidth.




					www.techpowerup.com
				





grilli4nt said:


> Thanks both for clarifying.


This is what you could get installed on Z790 platform. Motherboard vendors can play with available lanes.






The Von Matrices said:


> Where are these diagrams from, by the way?  They're very helpful.


I updated the Z790 diagram above.


----------



## Disco Dog (Dec 1, 2022)

Tek-Check said:


> Yes. In this case, 4090 might lose some performance, 2-3%, if you run it in Gen4 x8 mode, with bifurcated M.2 Gen5 slots. Have a look into PCIe scaling tests.
> 
> 
> 
> ...


I made an account to ask you a question. I realize this a month old now, but you have explained this better than anyone I've seen online so far (in my opinion).

In the diagram above, the pcie 4.0 M.2 NVM X 4 that shows four lanes in green on the CPU.
Are you saying that I could run (1) gen 4 nvme X 4 on the processor without knocking down to X 8 on the gpu? Are those the 4 lanes for the M2_2 position (which is also on the processor) I feel like this is it, I just wanted to make sure.

On my motherboard (Asus z790e) the M2_1 Gen 5 and M2_2 Gen 4 are on the processor. Other 3 are on the chipset.

I realize earlier you said "If you use* any dedicated* M.2 Gen5 slot with *any* NVMe drive on *any* Z790 board supporting Gen5 NVMe drives, your GPU slot will operate at *x8 Gen5* with upcoming AMD GPUs and *x8 Gen4 *with Nvidia and Intel cards."  The Gen 4 should just need 4 lanes, which show available, so I just wanted to be absolutely sure I couldn't run one Gen 4 in the M2_1 slot and another off the chipset. Heatsink just looks like it would be better on the M2_1 slot. Thanks for all your info.


----------



## The Von Matrices (Dec 1, 2022)

Disco Dog said:


> I made an account to ask you a question. I realize this a month old now, but you have explained this better than anyone I've seen online so far (in my opinion).
> 
> In the diagram above, the pcie 4.0 M.2 NVM X 4 that shows four lanes in green on the CPU.
> Are you saying that I could run (1) gen 4 nvme X 4 on the processor without knocking down to X 8 on the gpu? Are those the 4 lanes for the M2_2 position (which is also on the processor) I feel like this is it, I just wanted to make sure.
> ...


The Z790 platform has a dedicated M.2 Gen4 slot, but if you want a Gen5 slot you have to steal lanes from the PCIe x16 slot since those are the only Gen5 lanes on the platform.

On that ASUS Z790-E board, if you use the M2_1 slot, it steals lanes from the PCIe slot, and the PCIe slot runs at X8. You should only use the M2_1 slot if you have a Gen5 M.2 drive or if every other slot is full.

If you have a Gen4 M.2 drive, then you should use it in the M2_2 slot so that it uses the dedicated CPU lanes and the PCIe slot runs at x16.  If you have multiple Gen4 M.2 drives, then put the other drives in M2_3, M2_4, and M2_5.  They are connected to the chipset and have a PCIe 4.0 x8 uplink, which will not present a bandwidth limitation and will also not steal lanes from the GPU.


----------



## Disco Dog (Dec 2, 2022)

The Von Matrices said:


> The Z790 platform has a dedicated M.2 Gen4 slot, but if you want a Gen5 slot you have to steal lanes from the PCIe x16 slot since those are the only Gen5 lanes on the platform.
> 
> On that ASUS Z790-E board, if you use the M2_1 slot, it steals lanes from the PCIe slot, and the PCIe slot runs at X8. You should only use the M2_1 slot if you have a Gen5 M.2 drive or if every other slot is full.
> 
> If you have a Gen4 M.2 drive, then you should use it in the M2_2 slot so that it uses the dedicated CPU lanes and the PCIe slot runs at x16.  If you have multiple Gen4 M.2 drives, then put the other drives in M2_3, M2_4, and M2_5.  They are connected to the chipset and have a PCIe 4.0 x8 uplink, which will not present a bandwidth limitation and will also not steal lanes from the GPU.


Awesome. Thank you!


----------



## Frag_Maniac (Dec 2, 2022)

@ W1zzard,

in my PM to you, my question as to why a Gen 5 NVMe drive steals 8 Pci-Ex lanes on an Intel based MB, but not on an AMD based MB, with this response...

_"It's simply due to the reason Intel configured the lane availability in their chipset, so they steal them from the GPU to offer Gen 5 NVMe, which otherwise wouldn't be avilable on the platform

If you switch the lanes to x16 3.0, you're still using 16 of 16 lanes available"_

... but said nothing about why this is not the case with AMD based MBs? Do you know how AMD is getting that bandwidth for Gen 5 NVMe drives, if not from the Pci-Ex lanes?

And, yes, as I said, I know your test (and Steve Burke's at GamersNexus), showed negligible frame loss on Pci-Ex, but I'm concerned about how that will play out going forward, especially on the few games where the frame loss ISN'T as minuscule. After all, when one gets a top of the line GPU, he expects it to perform at it's best.

And yes, as I also said, I'm fully aware you can still use 16 lanes if you switch to Gen 3 Pci-Ex, but you did not answer my question as to whether Steve Burke at GamersNexus was correct in saying in 8x mode you don't necessarily get the same quality of lanes as in 16x mode.


----------



## W1zzard (Dec 2, 2022)

Frag_Maniac said:


> ... but said nothing about why this is not the case with AMD based MBs? Do you know how AMD is getting that bandwidth for Gen 5 NVMe drives, if not from the Pci-Ex lanes?


They simply have more Gen 5 lanes coming out of their processor


----------



## Tek-Check (Dec 2, 2022)

Frag_Maniac said:


> whether Steve Burke at GamersNexus was correct in saying in 8x mode you don't necessarily get the same quality of lanes as in 16x mode


Average loss is 2-3% on Gen4 x8 vs. Gen4 x16 mode, in some games a bit more, in others nothing. Practically, negligible. You won't see any difference unless you are unlucky to play a specific game with 5-10% loss. Even then, this 5-10% could be in context of 300 fps vs 280 fps, which you wouldn't notice anyway. 

Analyse the games tested and if there arenot perceivable losses in percentages converted into fps, you have nothing to worry about.

More performance is expected on Gen3 x8 vs. Gen3 x16 due to less available bandwidth.


----------



## Frag_Maniac (Dec 2, 2022)

Thanks for the responses guys. While it's impressive the Ryzen 7000 series CPUs have that many Gen 5 lanes, especially given the high transfer rate on Gen 5, all the comparison reviews I've read on i7-13700K vs R7 7700X show the Intel chip winning on performance, so it doesn't appear to be worth a slight difference in gaming FPS that is only noticeable in benchmark stats.

Truth be told, I'm not even sure I will use the Gen 5 m.2 slot on the MB I'm considering, which are said to be able to produce read and write speeds as high as 12000 to 13000 Mb/s. Most are saying they will likely run a lot hotter, and elaborate heatpipe equipped heatsink/fan assemblies are already available to address that issue, many with so so results. My current NVMe isn't even running at full speed in the rig I have now, because I didn't want to give up the SATA 3 port it would require. The ASRock Z790 PG Riptide I'm considering has 8 vs my current 6 SATA 3 ports, but my NVMe drive even at full speed is only about 1/3 the speed of what a Gen 5 drive will be, and I've been sufficing with less than 2500 MB/s.

I mean I like my OS and programs to run fast, yeah, but I don't think I need them moving faster than I can blink.


----------



## The Von Matrices (Dec 3, 2022)

Tek-Check said:


> Average loss is 2-3% on Gen4 x8 vs. Gen4 x16 mode, in some games a bit more, in others nothing. Practically, negligible. You won't see any difference unless you are unlucky to play a specific game with 5-10% loss. Even then, this 5-10% could be in context of 300 fps vs 280 fps, which you wouldn't notice anyway.
> 
> Analyse the games tested and if there arenot perceivable losses in percentages converted into fps, you have nothing to worry about.
> 
> More performance is expected on Gen3 x8 vs. Gen3 x16 due to less available bandwidth.


What confuses me is that I have tried these supposedly bandwidth-limited games on my own system with the same settings (I realize it's not the exact same configuration), and with GPU-Z open I never see bus utilization go over a few percent. If they are bandwidth limited then I would think that there would be at least one point where I would see high if not max bus utilization. Is that counter just not updated frequently enough to show these differences or am I looking at the wrong counter?


----------



## Mussels (Dec 3, 2022)

The Von Matrices said:


> What confuses me is that I have tried these supposedly bandwidth-limited games on my own system with the same settings (I realize it's not the exact same configuration), and with GPU-Z open I never see bus utilization go over a few percent. If they are bandwidth limited then I would think that there would be at least one point where I would see high if not max bus utilization. Is that counter just not updated frequently enough to show these differences or am I looking at the wrong counter?


It's only going to be an issue at very specific times, you'd need logging to have any record of it spiking up
HWinfo shows my recent gaming session as using 9% of the bus bandwidth and ~6GB of VRAM

I'd hazard a guess that something else may be part of the limiting factor here, like RAM speed and storage speed and that the extra lane bandwidth just helps timings match up (Like waiting for a frame update, a 60Hz display *must* wait 16.6ms but a 120Hz display at 60FPS might wait 16.6 or 8.3, if that content is ready at the exact right moment

TL;DR: might be a latency thing, not a bandwidth thing?


----------



## armlegx (Dec 3, 2022)

The Von Matrices said:


> The Z790 platform has a dedicated M.2 Gen4 slot, but if you want a Gen5 slot you have to steal lanes from the PCIe x16 slot since those are the only Gen5 lanes on the platform.
> 
> On that ASUS Z790-E board, if you use the M2_1 slot, it steals lanes from the PCIe slot, and the PCIe slot runs at X8. You should only use the M2_1 slot if you have a Gen5 M.2 drive or if every other slot is full.
> 
> If you have a Gen4 M.2 drive, then you should use it in the M2_2 slot so that it uses the dedicated CPU lanes and the PCIe slot runs at x16.  If you have multiple Gen4 M.2 drives, then put the other drives in M2_3, M2_4, and M2_5.  They are connected to the chipset and have a PCIe 4.0 x8 uplink, which will not present a bandwidth limitation and will also not steal lanes from the GPU.


I did this in my build because I misunderstood how the M2_1 slot stole lanes. I thought it would drop to PCIe 5.0x8 and not 4.0x8. I have the M2_1 and 2 slots populated with gen 4 ssds in raid 0. 

I need to get back into the case because additional storage arrived along with a new psu. This seems like a good time to move the M2_1 slot drive to a different slot. Does  anyone know if this would break the array? The raid is Intel rst through the bios.


----------



## Upgrayedd (Dec 3, 2022)

W1zzard said:


> They simply have more Gen 5 lanes coming out of their processor


So are they made with 16 lanes of PCIe5 and 20 lanes of PCIe4?


Frag_Maniac said:


> Thanks for the responses guys. While it's impressive the Ryzen 7000 series CPUs have that many Gen 5 lanes, especially given the high transfer rate on Gen 5, all the comparison reviews I've read on i7-13700K vs R7 7700X show the Intel chip winning on performance, so it doesn't appear to be worth a slight difference in gaming FPS that is only noticeable in benchmark stats.
> 
> Truth be told, I'm not even sure I will use the Gen 5 m.2 slot on the MB I'm considering, which are said to be able to produce read and write speeds as high as 12000 to 13000 Mb/s. Most are saying they will likely run a lot hotter, and elaborate heatpipe equipped heatsink/fan assemblies are already available to address that issue, many with so so results. My current NVMe isn't even running at full speed in the rig I have now, because I didn't want to give up the SATA 3 port it would require. The ASRock Z790 PG Riptide I'm considering has 8 vs my current 6 SATA 3 ports, but my NVMe drive even at full speed is only about 1/3 the speed of what a Gen 5 drive will be, and I've been sufficing with less than 2500 MB/s.
> 
> I mean I like my OS and programs to run fast, yeah, but I don't think I need them moving faster than I can blink.


So when you populate the M2(Gen5) slot, the main PCIe (G5) slot where your GPU goes would be reduced to x8 Gen5 lanes. Which is equivalent to x16 lanes of Gen4. Nvidia cards max at Gen4 x16 lanes. So by using a Gen5 M2 you should not be choking your GPU of any bandwidth unless it uses Gen5 lanes already, which nvidia cards don't. 
8x Gen5 lanes provide the same bandwidth to a card as 16x Gen4 lanes. 


This next part I'm just assuming because I couldn't find anything concrete on it. It seems like Intel CPUs have 16lanes of G5 and 20lanes of G4? Because usually when populating the CPU M2 for Intel the main PCIe slot isn't reduced because there is still 16lanes left. On the Intel website it just says 20 lanes and doesn't state if Gen5 and Gen4 have the same amount lanes.


----------



## dinvlad (Dec 4, 2022)

Upgrayedd said:


> So are they made with 16 lanes of PCIe5 and 20 lanes of PCIe4?
> 
> So when you populate the M2(Gen5) slot, the main PCIe (G5) slot where your GPU goes would be reduced to x8 Gen5 lanes. Which is equivalent to x16 lanes of Gen4. Nvidia cards max at Gen4 x16 lanes. So by using a Gen5 M2 you should not be choking your GPU of any bandwidth unless it uses Gen5 lanes already, which nvidia cards don't.
> 8x Gen5 lanes provide the same bandwidth to a card as 16x Gen4 lanes.


I thought so too but I don't think that's correct, unfortunately. Someone else on YT explained it better. But even the evidence in this article doesn't support this - otherwise, we wouldn't see any difference when the PCIE 5.0 port was reduced to x8..

Here's how that explanation goes. When NVidia (or any other 4.0 card) gets plugged into a PCIE 5.0 x16 slot, it "sees" this slot as if it was just 4.0 x16 _physically_. And so when an M.2 slot steals x8 lanes from this slot, it's as if the slot was physically cut in half. But the card still sees the _full_ port as if it was 4.0 x16 - and when the lanes are stolen those lanes are cut in half in the same exact way - i.e. the card is not able to "convert" 5.0 x8 lanes into 4.0 x16 lanes - it now sees them as 4.0 x8, because it actually does physically communicate with it in 4.0 format from the start, it never sees it as 5.0 and is not able to utilize that extra 5.0 bandwidth at any point, even though the mobo would very much understand that. Which is why when those lanes are lost, they're lost :/


----------



## Mussels (Dec 4, 2022)

Lanes are physically wired and only come in 16/8/4/1 - theres no "12x" lane option for GPU's to be wired to support

The GPU cant convert 5.0 x8 into 4.0 x16, it's the same bandwidth but you'd need dedicated hardware for that specific purpose, and that hasn't been done
For a 4.0 x16 GPU to understand a 5.0 x8 signal, it would have to be a 5.0 GPU in the first place - for anything else you'd literally need a physical device that was a PCI-E 5.0 card accepting signal from the first 8x connectors and passing it to the remaining 8x, and that translation delay would likely offset any gains from the extra bandwidth.


What i'm pretty confident on but need to investigate is how these split lanes work with different gen devices - would a PCI-E 2.0 NVME drive lock your primary slot to 8x 2.0?
I get the feeling that's a yes, but i'm also not confident on it

I'll go test this and get a definitive answer, at least for AM4

3090 @ 4.0 16

Add a 3.0 riser card with an x4 3.0 NVME






and receive...

(to be concluded, stay tuned for the thrilling confusion of "why cant i stop investigating all these things and just play video games")

Okay there we go:

Lanes can be 16/8/4, as determined by using the extra slots or not *at boot*

PCI-E gen can be different between the devices on the split lanes, and can change in real-time as my 3090 is running 8x 4.0, the NVME is 4x 3.0, and 4x just vanishes to never be seen again
(3090 can lower down to 1.1 in power saving and 4.0 when in use)


----------



## W1zzard (Dec 4, 2022)

Upgrayedd said:


> So are they made with 16 lanes of PCIe5 and 20 lanes of PCIe4?
> 
> So when you populate the M2(Gen5) slot, the main PCIe (G5) slot where your GPU goes would be reduced to x8 Gen5 lanes. Which is equivalent to x16 lanes of Gen4. Nvidia cards max at Gen4 x16 lanes. So by using a Gen5 M2 you should not be choking your GPU of any bandwidth unless it uses Gen5 lanes already, which nvidia cards don't.
> 8x Gen5 lanes provide the same bandwidth to a card as 16x Gen4 lanes.
> ...


The zen 4 cpu has 28 lanes gen 5. 4 of those go to the chipset. Since the chipset supports only gen4, they operate in gen 4 mode. The other lanes are typically used for x16 graphics and x4 nvme, but other configs are certainly possible, too

on lower cost boards the mobo vendor can lock the lanes to gen 4 if they can’t guarantee signal integrity for gen 5 (to save cost)


----------



## Tek-Check (Dec 4, 2022)

Mussels said:


> I'll go test this and get a definitive answer, at least for AM4


AM4 platform doesn't support NVMe 2.0 standard, only up to 1.4, I believe.



dinvlad said:


> Which is why when those lanes are lost, they're lost


It is that simple.
Bifurcation of x16 to x8x8 leaves eight lanes of whatever generation for any device to try to use it.

As no commercial device supports Gen5 speeds, slotting Gen4 device into Gen5 M.2 or PCIe slot will work as Gen4 device.

Eight Gen5 lanes left for GPU in x16 slot after bifurcation will work as Gen4 x8.


----------



## Mussels (Dec 5, 2022)

Tek-Check said:


> AM4 platform doesn't support NVMe 2.0 standard, only up to 1.4, I believe.
> 
> 
> It is that simple.
> ...


PCI-E 2.0, not NVME 2.0
PCI-E 5.0 slots can run at 1,1/2/3/4/5 at x1/x2/x4/8/x16

I wanted to check that a PCI-E 2.0 NVME drive in that 5.0 slot wouldnt lock a GPU to 2.0 x8


----------



## Tek-Check (Dec 5, 2022)

Mussels said:


> PCI-E 2.0, not NVME 2.0
> PCI-E 5.0 slots can run at 1,1/2/3/4/5 at x1/x2/x4/8/x16
> 
> I wanted to check that a PCI-E 2.0 NVME drive in that 5.0 slot wouldnt lock a GPU to 2.0 x8



There is no locking. Bifurcation works via PCIe switch chip, which allows all devices on split slots to work independently up to PCIe standard supported by the switch chip in the first place, then slot, then device.


----------



## Frag_Maniac (Dec 5, 2022)

It would be a real shocker if AMD came out with some Gen 5 GPUs that made good use of Pci-Ex Gen 5 bandwidth, and smeared egg on Nvidia's face by taking the performance crown, but I just don't see that happening. Still Blue and Green, just like my Seahawks.


----------



## Mussels (Dec 6, 2022)

Tek-Check said:


> There is no locking. Bifurcation works via PCIe switch chip, which allows all devices on split slots to work independently up to PCIe standard supported by the switch chip in the first place, then slot, then device.


It's done at boot, and cant change til a reboot. It's "locked in"


----------



## Godrilla (Dec 6, 2022)

Frag_Maniac said:


> It would be a real shocker if AMD came out with some Gen 5 GPUs that made good use of Pci-Ex Gen 5 bandwidth, and smeared egg on Nvidia's face by taking the performance crown, but I just don't see that happening. Still Blue and Green, just like my Seahawks.


Because the 4090 is chocked by even pcie 3.0 with 1 outlier lol? Tell us more!


----------



## Frag_Maniac (Dec 6, 2022)

Godrilla said:


> Because the 4090 is chocked by even pcie 3.0 with 1 outlier lol? Tell us more!



Trust me, I'd be completely happy to see AMD thoroughly trounce Nvidia on performance. If nothing else it might bring prices down a smidge. The fact is though, at best usually AMD can only come close to matching them, and they're no saints when it comes to GPU pricing either lately on their high end models at launch. Of course they always come down to more reasonable levels when people realize they don't perform as well though. So yeah, LOL all you want bud, in denial much? 

And only a fool would buy a 4090 and put it on a Pci-Ex 3 board, talk about LOLs. I wouldn't even trust Nvidia's cheesy 16 pin adapter myself.


----------



## Godrilla (Dec 6, 2022)

Frag_Maniac said:


> Trust me, I'd be completely happy to see AMD thoroughly trounce Nvidia on performance. If nothing else it might bring prices down a smidge. The fact is though, at best usually AMD can only come close to matching them, and they're no saints when it comes to GPU pricing either lately on their high end models at launch. Of course they always come down to more reasonable levels when people realize they don't perform as well though. So yeah, LOL all you want bud, in denial much?
> 
> And only a fool would buy a 4090 and put it on a Pci-Ex 3 board, talk about LOLs. I wouldn't even trust Nvidia's cheesy 16 pin adapter myself.


But as techpowerup proved that there is no statistical significant delta when using pcie 3.0 like if using a z790 board and a next gen pcie 5.0 nvme as well as a 4090 except for the one outlier. Questions?


----------



## Frag_Maniac (Dec 6, 2022)

Godrilla said:


> But as techpowerup proved that there is no statistical significant delta when using pcie 3.0 like if using a z790 board and a next gen pcie 5.0 nvme as well as a 4090 except for the one outlier. Questions?



You're only pointing out the obvious there. Every time a new Pci-Ex standard comes out, the new GPUs releasing can't make anywhere near use of it's bandwidth. So what if some of us keep checking if that remains to be true. You want a cookie for being overly pedantic or something? JEEEZ!


----------



## Godrilla (Dec 6, 2022)

Frag_Maniac said:


> You're only pointing out the obvious there. Every time a new Pci-Ex standard comes out, the new GPUs releasing can't make anywhere near use of it's bandwidth. So what if some of us keep checking if that remains to be true. You want a cookie for being overly pedantic or something? JEEEZ!


Well the blindside is that AM5 doesn't have that problem so at least its something plus they have zen4 3d chips coming out. So while yes I love for AMD to stick it to Nvidia as much as the next person in reality they are coming close. If you can extrapolate the Zen multi chiplet design from launch to actually becoming a competitive contender it took about 2 generations for everyone to want it  besides a few loyalists. I predict we will have something similar with next generation MCM AMD gpus when I say Next generation I mean 7000 rdna3 successor.


----------



## Tek-Check (Dec 6, 2022)

Frag_Maniac said:


> And only a fool would buy a 4090 and put it on a Pci-Ex 3 board, talk about LOLs.


This is not correct. The loss in gaming of 4090 is 2% on average in x16 PCIe 3.0 slot. it's negligible.
As for productivity apps, we have no measurements at the moment. It's still ok to put Gen4 x16 device into Gen3 xq16 slot.
This might change in 2024, but it's still ok.


----------



## Upgrayedd (Dec 6, 2022)

@W1zzard I was meaning the RPL CPUs and their PCIe lanes.


----------



## W1zzard (Dec 6, 2022)

Upgrayedd said:


> @W1zzard I was meaning the RPL CPUs and their PCIe lanes.


Intel has 16 lanes of Gen 5, the rest are Gen 4


----------



## Frag_Maniac (Dec 6, 2022)

Tek-Check said:


> This is not correct. The loss in gaming of 4090 is 2% on average in x16 PCIe 3.0 slot. it's negligible.
> As for productivity apps, we have no measurements at the moment. It's still ok to put Gen4 x16 device into Gen3 xq16 slot.
> This might change in 2024, but it's still ok.



Yeah it's "OK" so to speak, but anyone going for a 4090 is probably going to want the tech in the Gen 4 MBs, and be able to afford it. That's all I'm saying, no need to get marmy.


----------



## Godrilla (Dec 6, 2022)

Frag_Maniac said:


> Yeah it's "OK" so to speak, but anyone going for a 4090 is probably going to want the tech in the Gen 4 MBs, and be able to afford it. That's all I'm saying, no need to get marmy.


Like calling people stupid oh wait!


----------



## Tek-Check (Dec 6, 2022)

Upgrayedd said:


> @W1zzard I was meaning the RPL CPUs and their PCIe lanes.


Here is entire diagram for Z790 and B760.


----------



## Mussels (Dec 7, 2022)

You're good at making diagrams. You should make a thread and post them all in, adding for socket/chipset as you have the time.

Guarantee it'll be popular.
Oh and watermark them, you'll find em appearing all over the web soon enough


----------



## W1zzard (Dec 7, 2022)

Mussels said:


> You're good at making diagrams. You should make a thread and post them all in, adding for socket/chipset as you have the time.
> 
> Guarantee it'll be popular.
> Oh and watermark them, you'll find em appearing all over the web soon enough


Indeed. I'd love to add such diagrams to our CPU database. @Tek-Check?


----------



## Tek-Check (Dec 7, 2022)

Mussels said:


> Oh and watermark them,


Thank you for your kind words. 


W1zzard said:


> Indeed. I'd love to add such diagrams to our CPU database. @Tek-Check?


Thank you. Could you PM me and explain how this would work?


----------



## W1zzard (Dec 7, 2022)

Tek-Check said:


> Thank you. Could you PM me and explain how this would work?


Draw them for all the major CPUs in our database, send to me, I'll add


----------



## armlegx (Dec 8, 2022)

armlegx said:


> I did this in my build because I misunderstood how the M2_1 slot stole lanes. I thought it would drop to PCIe 5.0x8 and not 4.0x8. I have the M2_1 and 2 slots populated with gen 4 ssds in raid 0.
> 
> I need to get back into the case because additional storage arrived along with a new psu. This seems like a good time to move the M2_1 slot drive to a different slot. Does  anyone know if this would break the array? The raid is Intel rst through the bios.


I just went and did it.  Everything works perfectly and I have my 4x16 back.  Have not noticed a difference  Modern motherboards are something else lol.


----------



## JimGrongle (Dec 13, 2022)

Can anyone help me understand the lane splitting for the Intel vs AMD Asus ProArt Creator boards?

Intel: https://www.asus.com/motherboards-components/motherboards/proart/proart-z790-creator-wifi/

AMD: https://www.asus.com/motherboards-components/motherboards/proart/proart-x670e-creator-wifi/

The AMD says it can run 3 gpus in x8, x8, then x2 for the third slot
Whereas the Intel says it will run x8, x8, x4?

I would NOT be using an NVME in a PCIE5 slot. Either a) an NVME in a gen4 slot, or b) no nvme at all.

If AMD has more lanes, why is the Intel running the third slot at a better speed?
And while a lot of games have been tested, I would use this for 3D rendering, where render power/speed scales linearly with each GPU added. 3 GPUs is 3x faster than 1.

I am just confused about how these two systems are allocating lanes to the slots vs other components.

Thanks!!


----------



## Mussels (Dec 13, 2022)

JimGrongle said:


> Can anyone help me understand the lane splitting for the Intel vs AMD Asus ProArt Creator boards?
> 
> Intel: https://www.asus.com/motherboards-components/motherboards/proart/proart-z790-creator-wifi/
> 
> ...


Because the AMD has an entire extra 5.0 M.2 slot?

You wouldnt use GPUs for rendering in any of these 2x or 4x slots they'd be crippled. If you're truly running multiple high end GPU's you're going to want a HEDT board, or at least boards with better slot layouts


----------



## JimGrongle (Dec 13, 2022)

Mussels said:


> Because the AMD has an entire extra 5.0 M.2 slot?
> 
> You wouldnt use GPUs for rendering in any of these 2x or 4x slots they'd be crippled. If you're truly running multiple high end GPU's you're going to want a HEDT board, or at least boards with better slot layouts



Appreciate the insight. I didn't even realise myself until they were next to each other. 

You _can_ use GPUs for rendering on these boards... why do you say not? PCIE4x4 is the same as 3x8 as seen in the article parent of these comments, and as GPUs (even 4090s) don't saturate PCIE3, it's been functionally shown that high end GPUs don't take a render hit when going from PCIE3x16 down to 3x8 (which is the same as 4x4). It's not frame counts for games I'm talking about it, it's just sending data to VRAM then waiting 10mins for the CUDA cores to chew over it. Once the data is transferred, the PCIE speed doesn't materially affect the calculation of the image - that's down to the CUDA cores and other such things.

Anyway maybe 4x2 would see a hit more so than 4x4, but the question is, is it economically worth it to put the third 4090 on this board and lose 5%, or build an entire second machine with all associated costs (or build a threadripper machine, well that's certainly not economical at all, I'd take the 5% hit rather than spend another $3k on parts).
Even then it's only the 3rd slot taking the performance hit. The other 2 will render 3D as fast as anything else out there.

What I was more interested in was* if the AMD board would see a 3rd slot speed of 4x2 with zero NVMEs installed*, or how you tell which NVME slot to use that won't affect slot3's speed. But maybe it's better to use the intel board anyway, even if I won't be able to upgrade the CPU in 2yrs.

edit: It seems the AMD will run at x2 no matter what? Even if no NVME installed in that slot. This is what confuses me as I thought AMD had more lanes or at least the same as Intel, the way they "count" and market them is confusing. Someone else said Intel doesn't "count" chipset lanes which is why AMD is listed at 24 but Intel lists 20. But even so why the difference on that 3rd slot... I was simply hoping for "one simple trick" to run that 3rd slot at x4 not x2.   Ah well.


----------



## Mussels (Dec 14, 2022)

JimGrongle said:


> Appreciate the insight. I didn't even realise myself until they were next to each other.
> 
> You _can_ use GPUs for rendering on these boards... why do you say not? PCIE4x4 is the same as 3x8 as seen in the article parent of these comments, and as GPUs (even 4090s) don't saturate PCIE3, it's been functionally shown that high end GPUs don't take a render hit when going from PCIE3x16 down to 3x8 (which is the same as 4x4). It's not frame counts for games I'm talking about it, it's just sending data to VRAM then waiting 10mins for the CUDA cores to chew over it. Once the data is transferred, the PCIE speed doesn't materially affect the calculation of the image - that's down to the CUDA cores and other such things.
> 
> ...


You cant just add 4+2 and make 6, as 6x slots dont exist

And intel boards only do the lane swapping because they don't have enough lanes and they're all from the chipset - when the 5.0 NVME slots are from the AMD CPU, they cant mix with 4.0 ones from the chipset


----------



## JimGrongle (Dec 15, 2022)

Mussels said:


> You cant just add 4+2 and make 6, as 6x slots dont exist


Uh I didn't say that? 6?? Not sure what you read here. I said it's either going to run at PCIE4x2 or PCIE4x4, the manual makes it look like the slot will run at PCIE4x2 no matter what, even if the NVME slot is empty, which is dumb, but unsure if a BIOS update in the future could just see the slot run 4x4 when no NVME installed - depends on how it's wired? And the manual doesn't tell me that or if it does I didn't see it.


----------



## Mussels (Dec 15, 2022)

JimGrongle said:


> Uh I didn't say that? 6?? Not sure what you read here. I said it's either going to run at PCIE4x2 or PCIE4x4, the manual makes it look like the slot will run at PCIE4x2 no matter what, even if the NVME slot is empty, which is dumb, but unsure if a BIOS update in the future could just see the slot run 4x4 when no NVME installed - depends on how it's wired? And the manual doesn't tell me that or if it does I didn't see it.


You're missing the point that these are physically wired. You cant use the BIOS to change that.
You want more slots and less NVME - go buy a board that has that internal wiring set up to do so.


----------



## pavle (Dec 15, 2022)

Depending on amount of data that's being transferred via PCI-Express, but if you got 2.0 x16 you're good to go, one could easily live with 1.1 though.


----------



## JimGrongle (Dec 15, 2022)

Mussels said:


> You're missing the point that these are physically wired. You cant use the BIOS to change that.
> You want more slots and less NVME - go buy a board that has that internal wiring set up to do so.



Ah cool well that's what I didn't know. I couldn't tell if it was physically limited to only ever being at 4x2. I thought if the PCIE and M2 were connected they'd be sharing lanes, but my assumption is wrong I see. Thanks for pointing that out!
The additional of a billion NVMEs does get me down. I'm coming from a board with 7 slots (on broadwell) 4 of which will run at PCIE3x16. And now the most a board has is 3, and the third is often crippled. It's terrible!



pavle said:


> Depending on amount of data that's being transferred via PCI-Express, but if you got 2.0 x16 you're good to go, one could easily live with 1.1 though.


For some things indeed, for games or heaven forbid 3D workflows, not so much.


----------



## Mussels (Dec 16, 2022)

JimGrongle said:


> Ah cool well that's what I didn't know. I couldn't tell if it was physically limited to only ever being at 4x2. I thought if the PCIE and M2 were connected they'd be sharing lanes, but my assumption is wrong I see. Thanks for pointing that out!
> The additional of a billion NVMEs does get me down. I'm coming from a board with 7 slots (on broadwell) 4 of which will run at PCIE3x16. And now the most a board has is 3, and the third is often crippled. It's terrible!
> 
> 
> For some things indeed, for games or heaven forbid 3D workflows, not so much.




Intel shared lanes from the chipset because they had such a shortage of them it's all they could do
It wasn't ever done to be user friendly or helpful, but as a way to get the features users wanted and hoping they didn't ever use them all at once.

AM5 vs intel at present, AMD simply has 4x more lanes on the CPU, 28 5.0 on AM5 vs 16 5.0 + 4 3.0 on 13th gen intel.
Where those x4 lanes go is upto the board maker, but they cant be mixed with lanes from the chipset 

I'm having trouble finding the exact wording to explain why this is, it seems obvious to me but i also can't word it well - it's like getting a USB hub and plugging it back into itself, hoping to get more wattage and speed.


----------



## JimGrongle (Dec 17, 2022)

All good, I am understanding this more, as I read more and more board reviews and GPU reviews.
Probably a lot of people's confusion comes from the update cycle - for myself I was on X99 with 7 slots and 48 lanes (40? I forget) and everything was great. Boards cost half as much as they do now, and still came with dual 10gbe and 4 of the 7/8 PCIE slots could all run at 3x16. Now we get two slots, 5 NVMEs and 2.5gb LAN for 2x the price. My confusion and frustration feed off each other.  And around we go. 
But assuming others are like me and coming from the 10xx or 20xx generation of GPUs (and matching boards), well things have obviously changed a lot and not so much for the better, for those of us looking to use these machines for work and not games. In 3D where CUDA cores are king, Quadros just don't make any sense any more unless you work in CAD or maybe Architecture visualisation. For anyone creating 3D content, all we want is the most GPUs in a single box for the lowest price.

This article has been an absolute lifesaver though (now that I have a better understanding of the current state of affairs) in learning how I can evaluate different systems and generations, even down to how my X99 board will handle some 4090s (and I plan to throw some on it!) compared to a new rig.


----------



## Mussels (Dec 17, 2022)

Lanes back then were all from the chipset, back before everything was integrated into CPUs
It wasnt as fast as it seemed because you might have had four 16x PCI-E slots but up-stream they'd all have a singular link back to the CPU choking them all anyway


x58 had "36 Lanes, 4x8, and 1x4" from the chipset with 25.6GB/s bandwidth on the QPI link from chipset to CPU
12.8GB/s each direction on the QPI, while the PCI-E lanes totaled 18GB/s - there was never enough bandwidth for everything at the same time

Eventually i guess they made the decision to just cut devices off entirely, with a method to choose one or the other instead of competing devices (The good old days of your network card causing your audio to crackle, etc)


----------



## JimGrongle (Dec 18, 2022)

Ah well that explains some of this, I didn't realise they were all from the chipset on those older generations! Thanks for the explanation, the detail you've provided me in these replies has really shown me a lot, and it's much appreciated.

I will have to do a bandwidth test of some kind on my older x99 board to see how they're running. Going to replace 4x1080s with 2x4090s on that board, parts are ordered (and pushing a new system build to the new year).

On the plus side, going from 4 GPUs down to 2, sounds like it will be better for any potential bottlenecks I may have had with 4 GPUs fighting for data. Will run some tests to find out, before and after upgrade to compare.


----------



## Mussels (Dec 18, 2022)

To be fair i had to google it to get accurate information myself, i just had a heads up remembering some of the keywords and acronyms

You are aware there is no such thing as SLI any more?


----------



## JimGrongle (Dec 18, 2022)

Mussels said:


> To be fair i had to google it to get accurate information myself, i just had a heads up remembering some of the keywords and acronyms
> 
> You are aware there is no such thing as SLI any more?



Yes, for 3D rendering I don't need SLI, it's really down to how many CUDA cores there are. More cores, faster render.  I mean SLI would be nice for machine learning maybe, and that's becoming a bigger thing, but my own day to day work is not in that field - I just need renders out the fastest possible, and right now that means 4090s, whatever I can get hold of!


----------



## Tek-Check (Dec 21, 2022)

JimGrongle said:


> *if the AMD board would see a 3rd slot speed of 4x2 with zero NVMEs installed*,


Dude, open .pdf manuals of both boards on Asus website, and under each board you will find literally all answers to your questions, with simple diagrams showing which slots share lanes and which do not.



Mussels said:


> AM5 vs intel at present, AMD simply has 4x more lanes on the CPU, 28 5.0 on AM5 vs 16 5.0 + 4 3.0 on 13th gen intel.
> Where those x4 lanes go is upto the board maker, but they cant be mixed with lanes from the chipset


Check the chipset link speeds in diagrams in CPU data base. 
AM5 - 24+4 Gen5, x4 lanes reserved for the chipset link
Z790 - 16 Gen5 + x4 Gen4 and x8 Gen4 for DMI chipset link.

Both CPUs have the same amount of PCIe lanes (28), but differently distributed and of different generation. 
Although AMD's AM5 platform has more usable Gen5 lanes, Intel's chipset link to CPU is twice as fast comparing to AM5 (15.7 GB/s vs 7.6 GB/s).
This means that if you copy data from two NVMe drives on Intel's chipset somewhere to CPU, you will not have speed bottleneck. With AM5, you will, so it's important to be aware of bottlenecks.


----------



## Mussels (Dec 22, 2022)

Tek-Check said:


> This means that if you copy data from two NVMe drives on Intel's chipset somewhere to CPU, you will not have speed bottleneck. With AM5, you will, so it's important to be aware of bottlenecks.


Yes, chipset connected ones would be slower - but AMD have two 5.0 NVME slots direct from the CPU to avoid that

What does stand out is that AM4 had SATA on the CPU while AM5 is PCI-E only, so SATA devices could be slower on AM5 while NVME shouldn't

Board makers could slap that 4x somewhere else if wanted, but i'd be surprised if they did
The next gen chipsets using a 5.0 link would catch up on chipset connected devices


----------



## Tek-Check (Dec 22, 2022)

Mussels said:


> AMD have two 5.0 NVME slots direct from the CPU to avoid that


True that, but only if two drives communicate with each other. Besides, I have seen just a few double NVMe Gen5 drive implementations. 


Mussels said:


> next gen chipsets using a 5.0 link would catch up on chipset connected devices


Hopefully. Promontory 21 chipset does need to evolve for this to happen.


----------

