Very low random w/r speeds on NVME

[Giovanni]

Hello everyone

I just build a new PC but I have a problem with the NvME so I can't get over it

The PC is MB ASUS proart Z690
I9 12900K
64 Mb RAM DDR5 5600 kingston
graphics card 1660Ti
1 NVME 2 TB samsung 980 Pro with OS WIN10
1 NVME 2 TB samsung 980 Pro with some dat
1 NVME 2 TB samsung 980 Pro with no data on it just formatted 500MB

The problem is that I have these disappointing performances from NVME, all three are more or less on these values (even the empty one just formatted)
Tests performed with crystal disk 7

------------- -------------------------------------------------- ---------------
CrystalDiskMark 7.0.0 x64 (C) 2007-2019 hiyohiyo
Crystal Dew World: https://crystalmark.info/
----------- -------------------------------------------------- -----------------
* MB / s = 1,000,000 bytes / s [SATA / 600 = 600,000,000 bytes / s]
* KB = 1000 bytes, KiB = 1024 bytes

[Read]
Sequential 1MiB (Q = 8, T = 1): 6793.875 MB / s [6479.1 IOPS] <1208.98 us>
Sequential 1MiB (Q= 1, T= 1): 2964.959 MB/s [ 2827.6 IOPS] < 345.60 us>
Random 4KiB (Q= 32, T=16): 75.915 MB/s [ 18533.9 IOPS] < 26273.34 us> <----------------- HERE THE PROBLEM
Random 4KiB (Q= 1, T= 1): 28.914 MB/s [ 7059.1 IOPS] < 133.97 us> <----------------- HERE THE PROBLEM

[Write]
Sequential 1MiB (Q= 8, T= 1): 4922.388 MB/s [ 4694.4 IOPS] < 1690.48 us>
Sequential 1MiB (Q= 1, T= 1): 3078.317 MB/s [ 2935.7 IOPS] < 332.76 us>
Random 4KiB (Q= 32, T=16): 56.373 MB/s [ 13762.9 IOPS] < 35562.51 us> <----------------- HERE THE PROBLEM
Random 4KiB (Q= 1, T= 1): 35.573 MB/s [ 8684.8 IOPS] < 107.19 us> <----------------- HERE THE PROBLEM

Profile: Default
Test: 1 GiB (x5) [Interval: 5 sec] <DefaultAffinity=DISABLED>
Date: 2022/09/09 13:50:31
OS: Windows 10 Enterprise [10.0 Build 19044] (x64)


------------------

as you can see the random write and read values are very different from those shown here while those sequentials are similar:

https://www.guru3d.com/articles-page...review,14.html

I presume given the date of the review the reposted values were taken from the same version of crystal disk, anyway I tried also with version 8.0 (current) and these values are the same

I tried to change brand by putting a NVME WD black 850 but the values are very similar, slightly lower.

NVMEs work via the microsoft driver
I tried to make them work with the Intel RST driver but the values get a little worse

I tried to unplug all the mechanical hard disks connected on the sata ports and also to disable them via BIOS but nothing changes.

I tried to run the crystal disk test in W10 safe mode to rule out interference from drivers or applications but nothing changes.

I tried to force the PCIe 4x mode from bios for the NVME slots (which otherwise by default are on auto) but nothing changes.

Using samsung magician i put all the nvme on high performance but nothing changes.

Magician also confirms low performance values.

Sequential read 1685 MB / s
Sequential write 1708 MB / s
random read 17,570 IOPS
random write 10,986 IOPS

Much lower than those reported as typical on the samsung website.
In particular, the random values are 1/10 of the expected.

Magician does not report errors with NVME drivers

The NVME firmware is updated to the latest release.

W10 is up to date.

I honestly don't know what ot check / do now.

 

[ERazer]

Not enough pcie-lanes? remove the third NVME and check performance.

 

[Giovanni]

Not enough pcie-lanes? remove the third NVME and check performance.

Thank you for answering

I tried to remove 2 NVME leaving only the one where is installed Win10
The results are the same

I checked also if the NVMEs are misaligned with AOEMEI but it report that all NVME have aligned optimized partitions.

I checked alo attaching a crucial MX500 to a sata port and crystal disk give these results

It seems that problem is not isolated to NVME

------------------------------------------------------------------------------
CrystalDiskMark 7.0.0 x64 (C) 2007-2019 hiyohiyo
Crystal Dew World: https://crystalmark.info/
------------------------------------------------------------------------------
* MB/s = 1,000,000 bytes/s [SATA/600 = 600,000,000 bytes/s]
* KB = 1000 bytes, KiB = 1024 bytes

[Read]
Sequential 1MiB (Q= 8, T= 1): 561.049 MB/s [ 535.1 IOPS] < 14890.25 us>
Sequential 1MiB (Q= 1, T= 1): 446.177 MB/s [ 425.5 IOPS] < 2341.50 us>
Random 4KiB (Q= 32, T=16): 75.641 MB/s [ 18467.0 IOPS] < 26522.76 us> <-----------------------------------
Random 4KiB (Q= 1, T= 1): 22.433 MB/s [ 5476.8 IOPS] < 175.32 us> <--------------------------------------

[Write]
Sequential 1MiB (Q= 8, T= 1): 512.087 MB/s [ 488.4 IOPS] < 16307.82 us>
Sequential 1MiB (Q= 1, T= 1): 471.898 MB/s [ 450.0 IOPS] < 2211.70 us>
Random 4KiB (Q= 32, T=16): 56.654 MB/s [ 13831.5 IOPS] < 35492.42 us> <-----------------------------
Random 4KiB (Q= 1, T= 1): 32.870 MB/s [ 8024.9 IOPS] < 117.06 us> <--------------------------------

Profile: Default
Test: 1 GiB (x5) [Interval: 5 sec] <DefaultAffinity=DISABLED>
Date: 2022/09/10 11:38:17
OS: Windows 10 Enterprise [10.0 Build 19044] (x64)

More info

The same CX500 SSD attached to a 10 years old system with 3Gb sata ports give these results that for R/W in 4K are neraly 2 times higher!

------------------------------------------------------------------------------

[Read]
SEQ 1MiB (Q= 8, T= 1): 284.693 MB/s [ 271.5 IOPS] < 29357.17 us>
SEQ 1MiB (Q= 1, T= 1): 253.530 MB/s [ 241.8 IOPS] < 4129.51 us>
RND 4KiB (Q= 32, T= 1): 138.235 MB/s [ 33748.8 IOPS] < 914.58 us>
RND 4KiB (Q= 1, T= 1): 30.089 MB/s [ 7345.9 IOPS] < 132.76 us>

[Write]
SEQ 1MiB (Q= 8, T= 1): 267.353 MB/s [ 255.0 IOPS] < 31089.64 us>
SEQ 1MiB (Q= 1, T= 1): 256.710 MB/s [ 244.8 IOPS] < 4076.87 us>
RND 4KiB (Q= 32, T= 1): 118.180 MB/s [ 28852.5 IOPS] < 1076.62 us>
RND 4KiB (Q= 1, T= 1): 48.744 MB/s [ 11900.4 IOPS] < 78.98 us>

Profile: Default
Test: 1 GiB (x5) [D: 23% (425/1863GiB)]
Mode: [Admin]
Time: Measure 5 sec / Interval 5 sec
Date: 2022/09/10 11:59:27
OS: Windows 10 Professional [10.0 Build 19043] (x64)

 

[b1k3rdude]

Thank you for answering

I tried to remove 2 NVME leaving only the one where is installed Win10
The results are the same

I checked also if the NVMEs are misaligned with AOEMEI but it report that all NVME have aligned optimized partitions.

I checked alo attaching a crucial MX500 to a sata port and crystal disk give these results

It seems that problem is not isolated to NVME

Did you ever get to the bottom of this..?

 

[brave]

Try this on your Bios:
Secure boot > Disabled
VMD - Disabled
HDD Smart Test > Disabled
CSM - Disabled If Using Uefi Booting, if no is need to be Enable

 

[evernessince]

This appears to be a dead thread but your answer could lie in your platform. The review is using an AMD platform whereas you are using Intel.

Aside from that you can use HWInfo in summary mode to ensure the SSDs are getting full bandwidth. Make sure you have the driver installed for both the chipset and the SSD installed (if a non-generic one exists for this particular SSD).

 

[chrcoluk]

Check your CPU utilisation during the 4k i/o tests, bear in mind the number of threads vs number of logical CPU's in your system and take that into account, please report the CPU usage here.

4k random i/o is effectively a CPU i/o bench once the underlying nand is fast enough. Which is usually the case on modern NVME drives.

You have reported you running Windows 10, but are using a hybrid CPU.

I suspect things might be going via the e-cores on your 12900k.

So you can try downloading power settings explorer, and look for this setting, set it to prefer performant cores.

Will attach my 980 pro results from my 13700k, also on win 10, and 13700k has same core counts as 12900k.

 

[csendesmark]

Try this on your Bios:
Secure boot > Disabled
VMD - Disabled
HDD Smart Test > Disabled
CSM - Disabled If Using Uefi Booting, if no is need to be Enable

I would not recommend to turn the S.M.A.R.T. test out,
That will setting will not impact performance, but on booting will inform you if one of your drives are close to dying.

On the PCIe lane limitation I think there is no limitation for your system:
Only found limitation is with the M.2_4 and SATA ports
Your mainboard looks quite good actually