Don't insult my intelligence with this "pragmatism" bulls**t. A single PCIe x16 GPU and single PCIe x16 NVMe add-in card is not HEDT in any way shape or form, yet HEDT is literally the only option if I want to use those components together - no, I can't run GPU at x8 and NVMe at x16 because of the 24 CPU lanes available
No one insults your intelligence and it's not a topic here. You are just barking on a wrong target. AIC producers can easily design Gen5 x8 AIC for NVMe array with four Gen4 drives at full speed or two Gen5 drives. It's on them to provide such product, not AMD. Such solution would allow for two decent Gen5 x8 connections to primary and secondary PCIe slots, for GPU and NVMe AIC. Perfectly enough for a desktop platform at the moment.
PLX chips are not as cheap anymore as they used to be due to industry and ownership shifts in recent years, so we are unlikely to see dekstop motherboards with two or more x16 connections on two PCIe slots. There is HEDT for such needs.
ow this could be overcome if the 8 lanes of PCIe 4.0 put out by Prom21 were assigned to a single PCIe slot, which I could then use for the GPU and put the NVMe card in the PCIe 5.0 x16 slot - except there are zero motherboards that do this, because AMD doesn't allow it. x4 is the most you get.
I have not seen any document where Gen4 x8 connection from the Promo21 chipset is explicitly disallowed. x4/x2/x1 links are most likely hardwired limitations of the chipset logic and PHY. Even then, additional PCIe chip could combine two Gen4 x4 links into x8 link on the second Promo21 chipset, but this would not be practical due to inter-chipset link being x4. Chipsets were not designed for x8 solutions as the main peripherals being attached are NVMe drives, network, SATA and USB devices.
You are trying to invent a problem that does not exist on desktop by redefining what HEDT-style connectivity means. Sure, you can try to make a case for it. If you need plus x32 lanes to two PCIe slots and other peripherals, you know what you need to buy, either from AMD or Intel. It's not that choice does not exist. Pay for it. Or wait for new GB300 AI desktop system and pay Jensen for it $10,000.
Given that EPYC CPUs have upwards of 96 lanes of PCIe, it's absolutely pathetic that consumer CPU are artificially limited to a quarter of that (or only 20 lanes if you are using X850/X870 with its mandatory USB4). It's also absolutely pathetic that AMD thought Prom21 was in any way shape or form fit for purpose, with its abysmal number of PCIe lanes and ridiculous daisy-chained layout.
You may not like or prefer solutions provided on AM5 platform and that's fine, but saying it's "pathetic" sounds a little bit desperate and emotive to my ear. You can always resort to alternative solutions from Intel's 800 boards if those would meet your connectivity preferences. EPYC is a completely different category of products and it's not helpful to make such comparison with desktop platform. It's like saying that elephant baby is "pathetic" because it's much smaller than its adult member of herd. Nonsense.
The fix for this is to dump Prom21 in the bin where it belongs, and open up all the CPU lanes to be fully addressable in any configuration (i.e. not limited to blocks of x4, no mandatory NVMe) and instead allow board manufacturers to add SATA and USB controllers to segment their products - in the exact same way that any server board does. Those controllers then connect directly to the CPU, without the unnecessary chipset in the middle limiting everything to x4. Increase the CPU lane count to 40 to accommodate for the lost lanes from Prom21 and the mandatory USB4, and off you go. This is a far more sensible way to accomplish market segmentation and allow board manufacturers to produce products that are more than just the same recipe with slightly different aesthetics: if a manufacturer wants to build a board that has nothing but two x16 5.0 slots and a NIC on the remaining 4, they now can.
I like this idea, to an extent. It's much cheaper to produce chipset chips for usual peripherals than increase IOD logic size and cost by integrating everything in it. CPUs are already expensive enough, so we can't have it all both ways. IOD could certainly evolve on desktop, and it will. Their IOD cadence is usually two generations, so they are preparing some changes for Zen6 products. More sizeable changes will be available on AM6 platform.
Ask Asus and others to provide Gen5 x8 AIC with four Gen4 NVMe drives and most of your needs are met.
Yes, the CPU has 28 lanes. But on B650/X670 you lose 4 of those lanes to the link to the chipset, and on B850/X870 you lose 4 more lanes to the USB4 add-in chipset. So you have 24 available in the former platform, and 20 in the latter.
Some motherboards allow USB4 to be switched off in BIOS, so another x4 connection is available. This solution is available to all motherboard vendors. It's on them to enable it.
That means on X670 the total available lanes are 24 + 12, and on X870 it's 20 + 12 - yes, the newer X870 has less available connectivity than its predecessor. Now in theory either of these configurations would allow for a PCIe 5.0 x16 slot and a PCIe 4.0 x8 slot, except that the chipset is artificially limited to only allow up to 4 lanes to be aggregated. So really what you have is (24 or 20) + 4 + 4 + 4.
We know this since 2022. Not sure what the fuss is about now.
To crown this s**t sandwich, the platform massively wastes the bandwidth of the lanes it reserves because of AMD's arbitrary penny-pinching. The 4 lanes that are used to connect the CPU and chipset(s) are PCIe 5.0 on the CPU side, but 4.0 on the chipset side; similarly on X870, the USB4 chip is also a PCIe 4.0 x4 part connected to the CPU via PCIe 5.0 x4. So there's an entire 4 lanes of PCIe 5.0 bandwidth that sits unused.
This is due to disparity in development cycles of different products in the industry. And prices. It was far cheaper to produce Promo21 as Gen4 than Gen5 chip. If it was Gen5 chip, motherboards would have been way more expensive, with more layers needed for routing Gen5 traces and keeping integrity. Had this solution been adopted, you would complained that motherboards are insanely expensive. AsMedia prepared USB4 chip according to USB IF specification, so Gen4 PCIe transport. They are now preparing v2 chip that might house Gen5 x4 link. Both chips were delayed due to Covid.
At the end of the day, it's never possible to make everyone happy with available solutions. There are solutions for you out there. You just need to pay for it.
Meanwhile on the Intel side of the fence, Arrow Lake has "only" 20 PCIe 5.0 lanes but USB4 is integrated into the CPU so you don't lose any. And the link between CPU and chipset is a dedicated PCIe 4.0 x8 one, so not only are no lanes or bandwidth wasted, it's also twice as fast as when AMD offers. In short Arrow Lake CPUs, despite having 8 fewer PCIe 5.0 lanes than Zen 4 or 5, have exactly the same number of available 5.0 lanes compared to their competition. Unfortunately the Z890 chipset, despite having 24 PCIe 4.0 lanes, also restricts them to maximum groups of 4...
Intel's current chipset is a bit better, I agree, but it's not fundamentally better. It's always a nitpick for nerds. They have integrated Thunderbolt 4, not native USB4, so their PCIe traffic over TB4 port is ~2.8 GB/s, at best, due to PCIe 3.0 x4 link on the PHY and the link overhead, whereas native ASM4242 AMS2464 controllers allow ~3.8 GB/s due to Gen4 x4 support. Those are speeds I get on my USB4 external drive from Akasa.
I agree that dekstop IO can improve, and it will, but there's nothing fundamentally wrong with it for 99% of users.
