I Have Three Identical NVMes, One is Significantly Slower Than the Others

[Downinla]

I recently installed three Samsung 980 Pro 2TB into my system, and I ran CrystalDiskMark on all three drives, and one is SIGNIFICANTLY slower than the other two, and I'm stumped as to the reason why.
2 of the NVMes have read/write MB/s of around 3500, but one only as read/write of 1780 MB/s. I ran the test multiple times, and nothing was running in the background, and the slow drive isn't my OS drive either. Any ideas or clues is appreciated.

My Specs:
AMD Threadripper 2950X
Gigabyte Designare X399
Cosair 4x16Gb 3200
3x Samsung 980 Pro 2TB
Western Digital 4TB
EVGA RTX 3090

 

[TheoneandonlyMrK]

Are they all totally empty?

Or is one attached via an nvme slot attached to your chipset.

 

[cadaveca]

Can you explain what slots you are using for each drive?

Are they all totally empty?

CDM shows that each drive varies in its usage. My first thought was that maybe one drive is full, but it turns out that the fuller drive works fine.

 

[eidairaman1]

Can you explain what slots you are using for each drive?

This, not all m. 2 or pcie are equal

 

[AnotherReader]

What else do you have installed in your system? According to Guru3D, in some circumstances, the m.2 slots can drop to 2 lanes instead of 4. This is what they say:

With multiple cards or full SATA port occupation, you will run out of PCIe lanes.

 

[Downinla]

This, not all m. 2 or pcie are equal

My board has three M.2 slots, and I am not using a PCIe card. According to Gigabyte's description of the M.2 slots, each slot uses 4 PCIe lanes directly to the CPU, so it seems all three are equal.

What else do you have installed in your system? According to Guru3D, in some circumstances, the m.2 slots can drop to 2 lanes instead of 4.

Ironically, the slow drive is the game installation drive, since I wasn't obviously playing games, when I started the tests, I figure that would be one of the fastest, and that the OS drive would be the slow one.

Are they all totally empty?

Or is one attached via an nvme slot attached to your chipset.

No, they are not emply. One (my OS drive) has 1.54 TB free, the other one has 630GB free, and the slow one has 567 GB free.

What else do you have installed in your system? According to Guru3D, in some circumstances, the m.2 slots can drop to 2 lanes instead of 4. This is what they say:

Interesting article. My system has 60 PCIe lanes, so running out of lanes was not something I considered. I check the BIOS to see if there is a setting to force 4 lanes.

Thank you

 

[ColinB123]

Gigabyte describe the Storage specs of this board as:
Storage Interface

1. 2 x M.2 connectors (Socket 3, M key, type 2260/2280/22110 SATA and PCIe x4/x2 SSD support) (M2M_32G)(M2Q_32G)
2. 1 x M.2 connector (Socket 3, M key, type 2242/2260/2280 SATA and PCIe x4/x2 SSD support) (M2P_32G)
3. 8 x SATA 6Gb/s connectors
4. Support for RAID 0, RAID 1, and RAID 10

They don't provide any further detail on "PCIe x4/x2", but I suspect you are discovering the limitations on PCIe lanes.
Amazing mobo, though!

 

[TheoneandonlyMrK]

My board has three M.2 slots, and I am not using a PCIe card. According to Gigabyte's description of the M.2 slots, each slot uses 4 PCIe lanes directly to the CPU, so it seems all three are equal.


Ironically, the slow drive is the game installation drive, since I wasn't obviously playing games, when I started the tests, I figure that would be one of the fastest, and that the OS drive would be the slow one.


No, they are not emply. One (my OS drive) has 1.54 TB free, the other one has 630GB free, and the slow one has 567 GB free.


Interesting article. My system has 60 PCIe lanes, so running out of lanes was not something I considered. I check the BIOS to see if there is a setting to force 4 lanes.

Thank you

Pciex to nvme x4 drive adapter might do it, worked for me.

 

[Aquinus]

The slow drive almost appears to be running with only 2 PCIe lanes. It makes me wonder if reseating the NVMe card would make a difference.

 

[docnorth]

Like arleady mentioned, probably different slots or line limitations.

 

[Aquinus]

Like arleady mentioned, probably different slots or line limitations.

The user guide does not mention anything about that which is why I'm inclined to not think that's the problem because most of the time, that kind of behavior is called out in the user manual and tends to be more common with CPUs with fewer PCIe lanes, which Threadripper has no short supply of.

Edit: The only thing that might make me think otherwise is this statement on Guru3D's review of the board:

Be advised though.With multiple cards or full SATA port occupation, you will run out of PCIe lanes. If that happens you will revert to x2 PCIe Gen 3.0 configuration. Most motherboards will allow you to switch to x4 however as a BIOS setting and at the cost of disabling a SATA3 port or PCIe-Slot.

Gigabyte X399 DESIGNARE EX review (Page 22)

Let's perform some mega-threading with a Ryzen Threadripper processors In this review we take the mightly Gigabyte X399 DESIGNARE EX for a spin....

www.guru3d.com

 

[Assimilator]

I'll guarantee you that the M.2 slot in question is being limited to PCIe x2 either because of chipset IO limitations or because Gigabyte is incompetent - not because the CPU doesn't have enough PCIe lanes.

Remember, these are essentially server parts, and servers don't have a whole bunch of IO onboard, maybe a couple of SATA ports and one M.2 slot - almost all the CPU PCIe lanes are directed towards PCIe slots, to serve the bandwidth-hungry expansion cards expected to be installed there.

 

[Downinla]

Based on all the analysis from you (thank you all very much), I'm inclined to think that the slow drive is only running on two PCIe lanes. I suppose the question is why is this happening? I only have one PCIe device which is my GPU, and only using two of the six SATA ports. I don't see an option to disable a PCI port in the BIOS. On the BIOS, while I can't disable a PCIe port, it's giving me the option of turning x16 to 8x8 or 4x4x4x4 which I don't understand at all.

 

[Nike_486DX]

Based on all the analysis from you (thank you all very much), I'm inclined to think that the slow drive is only running on two PCIe lanes. I suppose the question is why is this happening? I only have one PCIe device which is my GPU, and only using two of the six SATA ports. I don't see an option to disable a PCI port in the BIOS. On the BIOS, while I can't disable a PCIe port, it's giving me the option of turning x16 to 8x8 or 4x4x4x4 which I don't understand at all.

If its not a X299 or threadripper platform then there is nothing you can do, you can only use the few pcie lanes the platform is actually able to provide. also on your mobo there could be a specific layout that doesnt let you assign full x4 speed to all three m2 slots at once. Btw keep in mind that internal devices such as ethernet or usb controllers are attached via pcie, thus eating up the lanes.

Btw the first 2 ssds perform according to spec (primarily looking at RND4K, which reflects the real performance), the third one (drive G) is kinda slower maybe cuz its filled up. Here is a result of my 32gb optane standalone drive benchmarked (os drive). just look at these rnd4k results

 

[erocker]

I check the BIOS to see if there is a setting to force 4 lanes.

There should be an option to select x2/x4. Not familiar with newer Gigabyte boards but on my Asus boards one m.2 slot seems to default at x2 and I have to change it.

 

[eidairaman1]

Based on all the analysis from you (thank you all very much), I'm inclined to think that the slow drive is only running on two PCIe lanes. I suppose the question is why is this happening? I only have one PCIe device which is my GPU, and only using two of the six SATA ports. I don't see an option to disable a PCI port in the BIOS. On the BIOS, while I can't disable a PCIe port, it's giving me the option of turning x16 to 8x8 or 4x4x4x4 which I don't understand at all.

Contact gigabyte

 

[Dr. Dro]

Are you using the bottom 2 SATA ports? Even if the others are empty, at least on my B550-E (admittedly no Threadripper board as far as I/O goes), the NVMe and SATA resource overlap occurs if the bottom 2 sata ports are used. If you only have two SATA devices, make sure they are using ports 0 and 1 only. Good luck, TR should be capable of running three x4 NVMe drives plus a GPU at full speed and have plenty of lanes left to use

 

[brummyfan]

The slower one could be using only 2 lanes of PCIe.

 

[Downinla]

I think It's the drive itself. I round-ribboned the drives in the three NVMe slots, and it remains slower than the others. If it was due to a PCIe lane issue, the drive in the "slow 2-lane" slot would always be slow, but that's not the case.

It's disappointing that a brand new Samsung drive would be defective.

Thank you all for your help.

 

[looniam]

I think It's the drive itself. I round-ribboned the drives in the three NVMe slots, and it remains slower than the others. If it was due to a PCIe lane issue, the drive in the "slow 2-lane" slot would always be slow, but that's not the case.

It's disappointing that a brand new Samsung drive would be defective.

Thank you all for your help.

by any chance you look at the storage config in the bios, not pci-e for graphics but showing your populated nvme drives.
may be an option to select x2/x4.

 

[Assimilator]

I think It's the drive itself. I round-ribboned the drives in the three NVMe slots, and it remains slower than the others. If it was due to a PCIe lane issue, the drive in the "slow 2-lane" slot would always be slow, but that's not the case.

It's disappointing that a brand new Samsung drive would be defective.

Thank you all for your help.

Are all drives on the same firmware?

 

[Wirko]

Have you checked SMART data with a program like CrystalDiskInfo? Also to confirm that the interface is actually PCIe 3.0 x4.

For some reason (probably hardware error on the SSD), it could fall back to PCIe 3.0 x2 as others have speculated, but there's another possibility, namely PCIe 2.0 x4.

 

[chrcoluk]

You might get more bandwidth using a PCIE m.2 card for that 3rd drive.

On my Z370, I have 2 m.2, one is 4 lanes, the second is 2 lanes. I put a PCIE m.2 card in my bottom PCIE slot which is only fed by chipset, but still is a 4 lane slot. It gets full performance and even slightly outperforms the first m.2 slot.

 

[TheoneandonlyMrK]

You might get more bandwidth using a PCIE m.2 card for that 3rd drive.

On my Z370, I have 2 m.2, one is 4 lanes, the second is 2 lanes. I put a PCIE m.2 card in my bottom PCIE slot which is only fed by chipset, but still is a 4 lane slot. It gets full performance and even slightly outperforms the first m.2 slot.

And he is using only one pciex for a GPU afaik , I would.
Some manufacturers have been called out for substitution of parts, spec recently, I don't recall Samsung though.

 

[qubit]

Assuming you have ruled out all other variables, it's possible that you've fallen victim to the bait and switch scam from hardware manufacturers. It's literally impossible to tell the better product from the inferior one by looking at the packaging or product labelling, yet they're not the same. LTT in the video below specifically gives examples of m.2 drives that have fallen prey to this.

I know you've rotated the connections and the slowness followed that SSD, but if practical, you could try one more thing to rule out all confounding variables:

• Backup the data from all three SSDs to a big HDD.
• Format them all.
• Connect each one in turn to the same port on the mobo, the one you know for sure is fast and benchmark them.

This will rule out any other factor. If the slowness still follows it, then it's either faulty, or maybe you've been scammed. Let us know how you get on.

SSD manufacturers bait and switch inferior components while keeping the same model numbers

This isn't a new practice either. I know that monitor manufacturers do this with their display panels and have been doing so for years. All this leads to inferior performance and reliability for a later manufactured version of the same product, with the same model number and it's not even...

nerdzone.uk