# I Have Three Identical NVMes, One is Significantly Slower Than the Others



## Downinla (Mar 29, 2022)

I recently installed three Samsung 980 Pro 2TB into my system, and I ran CrystalDiskMark on all three drives, and one is SIGNIFICANTLY slower than the other two, and I'm stumped as to the reason why.
2 of the NVMes have read/write MB/s of around 3500, but one only as read/write of 1780 MB/s. I ran the test multiple times, and nothing was running in the background, and the slow drive isn't my OS drive either. Any ideas or clues is appreciated.












My Specs:
AMD Threadripper 2950X
Gigabyte Designare X399
Cosair 4x16Gb 3200
3x Samsung 980 Pro 2TB 
Western Digital 4TB
EVGA RTX 3090


----------



## TheoneandonlyMrK (Mar 29, 2022)

Are they all totally empty?

Or is one attached via an nvme slot attached to your chipset.


----------



## cadaveca (Mar 29, 2022)

Can you explain what slots you are using for each drive? 



TheoneandonlyMrK said:


> Are they all totally empty?



CDM shows that each drive varies in its usage. My first thought was that maybe one drive is full, but it turns out that the fuller drive works fine.


----------



## eidairaman1 (Mar 29, 2022)

cadaveca said:


> Can you explain what slots you are using for each drive?


This, not all m. 2 or pcie are equal


----------



## AnotherReader (Mar 29, 2022)

What else do you have installed in your system? According to Guru3D, in some circumstances, the m.2 slots can drop to 2 lanes instead of 4. This is what they say: 


> With multiple cards or full SATA port occupation, you will run out of PCIe lanes.


----------



## Downinla (Mar 29, 2022)

eidairaman1 said:


> This, not all m. 2 or pcie are equal


My board has three M.2 slots, and I am not using a PCIe card. According to Gigabyte's description of the M.2 slots, each slot uses 4 PCIe lanes directly to the CPU, so it seems all three are equal.



AnotherReader said:


> What else do you have installed in your system? According to Guru3D, in some circumstances, the m.2 slots can drop to 2 lanes instead of 4.


Ironically, the slow drive is the game installation drive, since I wasn't obviously playing games, when I started the tests, I figure that would be one of the fastest, and that the OS drive would be the slow one.



TheoneandonlyMrK said:


> Are they all totally empty?
> 
> Or is one attached via an nvme slot attached to your chipset.


No, they are not emply. One (my OS drive)  has 1.54 TB free, the other one has 630GB free, and the slow one has 567 GB free.



AnotherReader said:


> What else do you have installed in your system? According to Guru3D, in some circumstances, the m.2 slots can drop to 2 lanes instead of 4. This is what they say:


Interesting article. My system has 60 PCIe lanes, so running out of lanes was not something I considered. I check the BIOS to see if there is a setting to force 4 lanes.

Thank you


----------



## ColinB123 (Mar 29, 2022)

Gigabyte describe the Storage specs of this board as:
Storage Interface

2 x M.2 connectors (Socket 3, M key, type 2260/2280/22110 SATA and PCIe x4/x2 SSD support) (M2M_32G)(M2Q_32G)
1 x M.2 connector (Socket 3, M key, type 2242/2260/2280 SATA and PCIe x4/x2 SSD support) (M2P_32G)
8 x SATA 6Gb/s connectors
Support for RAID 0, RAID 1, and RAID 10
They don't provide any further detail on "PCIe x4/x2", but I suspect you are discovering the limitations on PCIe lanes. 
Amazing mobo, though!


----------



## TheoneandonlyMrK (Mar 29, 2022)

Downinla said:


> My board has three M.2 slots, and I am not using a PCIe card. According to Gigabyte's description of the M.2 slots, each slot uses 4 PCIe lanes directly to the CPU, so it seems all three are equal.
> 
> 
> Ironically, the slow drive is the game installation drive, since I wasn't obviously playing games, when I started the tests, I figure that would be one of the fastest, and that the OS drive would be the slow one.
> ...


Pciex to nvme x4 drive adapter might do it, worked for me.


----------



## Aquinus (Mar 29, 2022)

The slow drive almost appears to be running with only 2 PCIe lanes. It makes me wonder if reseating the NVMe card would make a difference.


----------



## docnorth (Mar 29, 2022)

Like arleady mentioned, probably different slots or line limitations.


----------



## Aquinus (Mar 29, 2022)

docnorth said:


> Like arleady mentioned, probably different slots or line limitations.


The user guide does not mention anything about that which is why I'm inclined to not think that's the problem because most of the time, that kind of behavior is called out in the user manual and tends to be more common with CPUs with fewer PCIe lanes, which Threadripper has no short supply of.

Edit: The only thing that might make me think otherwise is this statement on Guru3D's review of the board:


> Be advised though.With multiple cards or full SATA port occupation, you will run out of PCIe lanes. If that happens you will revert to x2 PCIe Gen 3.0 configuration. Most motherboards will allow you to switch to x4 however as a BIOS setting and at the cost of disabling a SATA3 port or PCIe-Slot.











						Gigabyte X399 DESIGNARE EX review
					

Let's perform some mega-threading with a Ryzen Threadripper processors In this review we take the mightly Gigabyte X399 DESIGNARE EX for a spin.... Performance - M.2 PCIe NVMe SSD Storage Performance




					www.guru3d.com


----------



## Assimilator (Mar 29, 2022)

I'll guarantee you that the M.2 slot in question is being limited to PCIe x2 either because of chipset IO limitations or because Gigabyte is incompetent - not because the CPU doesn't have enough PCIe lanes.

Remember, these are essentially server parts, and servers don't have a whole bunch of IO onboard, maybe a couple of SATA ports and one M.2 slot - almost all the CPU PCIe lanes are directed towards PCIe slots, to serve the bandwidth-hungry expansion cards expected to be installed there.


----------



## Downinla (Mar 29, 2022)

Based on all the analysis from you (thank you all very much), I'm inclined to think that the slow drive is only running on two PCIe lanes. I suppose the question is why is this happening? I only have one PCIe device which is my GPU, and only using two of the six SATA ports. I don't see an option to disable a PCI port in the BIOS. On the BIOS, while I can't disable a PCIe port, it's giving me the option of turning x16 to 8x8 or 4x4x4x4 which I don't understand at all.


----------



## Nike_486DX (Mar 29, 2022)

Downinla said:


> Based on all the analysis from you (thank you all very much), I'm inclined to think that the slow drive is only running on two PCIe lanes. I suppose the question is why is this happening? I only have one PCIe device which is my GPU, and only using two of the six SATA ports. I don't see an option to disable a PCI port in the BIOS. On the BIOS, while I can't disable a PCIe port, it's giving me the option of turning x16 to 8x8 or 4x4x4x4 which I don't understand at all.


If its not a X299 or threadripper platform then there is nothing you can do, you can only use the few pcie lanes the platform is actually able to provide. also on your mobo there could be a specific layout that doesnt let you assign full x4 speed to all three m2 slots at once. Btw keep in mind that internal devices such as ethernet or usb controllers are attached via pcie, thus eating up the lanes. 

Btw the first 2 ssds perform according to spec (primarily looking at RND4K, which reflects the real performance), the third one (drive G) is kinda slower maybe cuz its filled up. Here is a result of my 32gb optane standalone drive benchmarked (os drive). just look at these rnd4k results


----------



## erocker (Mar 29, 2022)

Downinla said:


> I check the BIOS to see if there is a setting to force 4 lanes.


There should be an option to select x2/x4. Not familiar with newer Gigabyte boards but on my Asus boards one m.2 slot seems to default at x2 and I have to change it.


----------



## eidairaman1 (Mar 29, 2022)

Downinla said:


> Based on all the analysis from you (thank you all very much), I'm inclined to think that the slow drive is only running on two PCIe lanes. I suppose the question is why is this happening? I only have one PCIe device which is my GPU, and only using two of the six SATA ports. I don't see an option to disable a PCI port in the BIOS. On the BIOS, while I can't disable a PCIe port, it's giving me the option of turning x16 to 8x8 or 4x4x4x4 which I don't understand at all.


Contact gigabyte


----------



## Dr. Dro (Mar 30, 2022)

Are you using the bottom 2 SATA ports? Even if the others are empty, at least on my B550-E (admittedly no Threadripper board as far as I/O goes), the NVMe and SATA resource overlap occurs if the bottom 2 sata ports are used. If you only have two SATA devices, make sure they are using ports 0 and 1 only. Good luck, TR should be capable of running three x4 NVMe drives plus a GPU at full speed and have plenty of lanes left to use


----------



## brummyfan (Mar 30, 2022)

The slower one could be using only 2 lanes of PCIe.


----------



## Downinla (Mar 31, 2022)

I think It's the drive itself. I round-ribboned the drives in the three NVMe slots, and it remains slower than the others. If it was due to a PCIe lane issue, the drive in the "slow 2-lane" slot would always be slow, but that's not the case. 

It's disappointing that a brand new Samsung drive would be defective.

Thank you all for your help.


----------



## looniam (Mar 31, 2022)

Downinla said:


> I think It's the drive itself. I round-ribboned the drives in the three NVMe slots, and it remains slower than the others. If it was due to a PCIe lane issue, the drive in the "slow 2-lane" slot would always be slow, but that's not the case.
> 
> It's disappointing that a brand new Samsung drive would be defective.
> 
> Thank you all for your help.


by any chance you look at the storage config in the bios, not pci-e for graphics but showing your populated nvme drives. 
may be an option to select x2/x4.


----------



## Assimilator (Mar 31, 2022)

Downinla said:


> I think It's the drive itself. I round-ribboned the drives in the three NVMe slots, and it remains slower than the others. If it was due to a PCIe lane issue, the drive in the "slow 2-lane" slot would always be slow, but that's not the case.
> 
> It's disappointing that a brand new Samsung drive would be defective.
> 
> Thank you all for your help.


Are all drives on the same firmware?


----------



## Wirko (Mar 31, 2022)

Have you checked SMART data with a program like CrystalDiskInfo? Also to confirm that the interface is actually PCIe 3.0 x4.

For some reason (probably hardware error on the SSD), it could fall back to PCIe 3.0 x2 as others have speculated, but there's another possibility, namely PCIe 2.0 x4.


----------



## chrcoluk (Mar 31, 2022)

You might get more bandwidth using a PCIE m.2 card for that 3rd drive.

On my Z370, I have 2 m.2, one is 4 lanes, the second is 2 lanes.  I put a PCIE m.2 card in my bottom PCIE slot which is only fed by chipset, but still is a 4 lane slot.  It gets full performance and even slightly outperforms the first m.2 slot.


----------



## TheoneandonlyMrK (Mar 31, 2022)

chrcoluk said:


> You might get more bandwidth using a PCIE m.2 card for that 3rd drive.
> 
> On my Z370, I have 2 m.2, one is 4 lanes, the second is 2 lanes.  I put a PCIE m.2 card in my bottom PCIE slot which is only fed by chipset, but still is a 4 lane slot.  It gets full performance and even slightly outperforms the first m.2 slot.


And he is using only one pciex for a GPU afaik , I would.
Some manufacturers have been called out for substitution of parts, spec recently, I don't recall Samsung though.


----------



## qubit (Apr 1, 2022)

Assuming you have ruled out all other variables, it's possible that you've fallen victim to the bait and switch scam from hardware manufacturers. It's literally impossible to tell the better product from the inferior one by looking at the packaging or product labelling, yet they're not the same. LTT in the video below specifically gives examples of m.2 drives that have fallen prey to this.

I know you've rotated the connections and the slowness followed that SSD, but if practical, you could try one more thing to rule out all confounding variables:

Backup the data from all three SSDs to a big HDD.
Format them all.
Connect each one in turn to the same port on the mobo, the one you know for sure is fast and benchmark them.
This will rule out any other factor. If the slowness still follows it, then it's either faulty, or maybe you've been scammed. Let us know how you get on.






						SSD manufacturers bait and switch inferior components while keeping the same model numbers
					

This isn't a new practice either. I know that monitor manufacturers do this with their display panels and have been doing so for years. All this leads to inferior performance and reliability for a later manufactured version of the same product, with the same model number and it's not even...




					nerdzone.uk


----------



## RUSTA (Apr 1, 2022)

A few things to check.

Check for firmware updares using Samsung Magician if you haven't already
Magician & SSD Tools & Software Download | Samsung Semiconductor Global

According to the chipset block diagram the 3 SSD's should be all running off the CPU. Unless Gigabyte made the third m.2 run off the chipset?

One thing to try is to clean the connector with some isopropyl. I often get video cards not connecting at x16 and PCIe 1.1 or 2.0 instead of 3.0.  I clean the PCIe slot and the video cards connector and
then bingo, x16 and PCIe 3.0.


----------



## Wirko (Apr 1, 2022)

Does any software exist that could measure the actual speed of the PCIe interface? For example, by writing and reading small amounts of data so everything stays in the drive's DRAM cache?


----------



## Valantar (Apr 1, 2022)

RUSTA said:


> A few things to check.
> 
> Check for firmware updares using Samsung Magician if you haven't already
> Magician & SSD Tools & Software Download | Samsung Semiconductor Global
> ...


Yeah, cleaning the connector is a logical next step. It would only require a small speck of dirt or corrosion on one pin to make it fall back to x2 operation, after all.


----------



## Downinla (Apr 12, 2022)

I just wanted to update all of you as a thank you for all your help

I messed up in my analysis where I blame the drive. When I did a round-ribbon of the 3 NVMes into the different M.2 slots, I said the drive was still slow, regardless of what M.2 slot it was installed in. However, I was a bit clumsy when physcially organzing the drives, and didn't do the round ribbon of the NVMes correctly.

I found this program called HWInfo64, and it indeed confirmed that the M.2 slot that the slow NVME drive was on was only using 2 lanes. I tried to work with Gigabyte support, and asking if there is a BIOS setting (I'm using the latest bios ver), but Gigabtye support was absoutely worthless. All they stated is something to the effect, that all the "built-in M.2 slots support x4 lanes". I also checked the Gigabyte forums, and apparently, the newer boards do have a setting to change the number of PCIe lans in the BIOS, but my BIOS simply does not have a setting for this.

I ended up buying a cheap NVMe PCIe card and installed it into a PCIe 8x slot, and now I get 4 PCIe lanes, but I am supremely disappointed/pissed off on Gigabyte. This board is geared towards the workstation market, to take advantage of a CPU that supports 64 PCIe lanes, and they can't setup a "*BUILT-IN*" M.2 slot properly to reach its fully potential! There no settings in the BIOS to properly set this! This board was an expensive investment, and the fact I had to buy addtional hardware to get something that should have worked in the first place is insulting. On a slightly lighter note, I'm also angry that the NVMe card that I had buy and install impacts the "clean-look" aesthetics of my computer. I know many of you take pride and joy in the appearance of your computers, so I am sure many of you can relate.

In conclusion, thank you all for your help and reading my post if you got this far. This was my first post in Tech Power, and the fact so many of you offered your help shows there great community here.


----------



## eidairaman1 (Apr 12, 2022)

If there was a problem, should of pushed for an RMA.


----------



## looniam (Apr 12, 2022)

Downinla said:


> In conclusion, thank you all for your help and reading my post if you got this far. This was my first post in Tech Power, and the fact so many of you offered your help shows there great community here.


yw. its a shame on gigabyte.
explody psus
another member's' experience w/aorus itx board - bad xmp -went asus.
now this - not getting "updated" bios support just sux.

still even w/workaround i might (on bad days) keep screaming at gigabyte to fix it.


----------



## qubit (Apr 12, 2022)

Thanks for the update. Good to know you weren't cheated on the M.2 card.

That Gigabyte support does suck and I've heard several stories like this about them over the years. Indeed, one of the members here started a thread a few years ago saying he'll have never buy Gigabyte again due to bad customer service from them.

@newtekie1 was that you? Apologies if wrong.

Personally, my current PC has an Asus mobo which has worked like a champ over the last 11 years now and is still going strong. Asus and EVGA seem to be the best brands as far as I can tell, but don't hold me to it.


----------



## newtekie1 (Apr 13, 2022)

qubit said:


> @newtekie1 was that you? Apologies if wrong.


Yes, it was me.


----------



## Mac the Geek (Apr 16, 2022)

qubit said:


> That Gigabyte support does suck and I've heard several stories like this about them over the years. Indeed, one of the members here started a thread a few years ago saying he'll have never buy Gigabyte again due to bad customer service from them.



It pains me to hear that Gigabyte has spiraled down the toilet.  I have two Gigabyte boards (one for an AMD build, one for Intel) and they've both been rock-solid for a decade.  But I went with MSI this time around.


----------



## dogwitch (Apr 16, 2022)

ltt mention about ssd manf. 
idk if mention or not.


----------

