Would you guys be ok with 70C idle temp on NVME storage.

[chrcoluk]

So I have a N100 NUC on firewall duty, and the thing runs much hotter than when running Windows.

At 19C ambient temp (today) with its default passive cooling setup, case as heat sink, CPU is at 57C which is hot for idle but not close to TJMAX, however the NVME is at 68C. This room will get over 30C in the summer.

Blowing a small USB fan over the top of the case fins makes everything similar to desktop temps so CPU in 30s and NVME down to 40s. But trying to avoid having to use active cooling, so curious if you guys would be ok with your drive at those sort of temps 24/7.

 

[evernessince]

70c idle is high, have to wonder if it's properly entering sleep states or if it's just poor circulation. You could always try a middle-ground approach and get a metal heatsink / shim to reduce temps.

Whether it's fine long term depends on the NAND chips, some are fine with those temps while others will start throttling.

 

[xtreemchaos]

i would not be happy with that temp, wonder what temp it starts to throttle it can not be far at 68c.

 

[Marcus L]

I swapped 500GB back and forth today between 2 NVME drives, the hottest one was 47c max during transfer, the other barely got over 40c, 70c is an insane temp for an NVME drive, you need to get a H/S on it or just deal with the USB fan blowing over it

 

[bonehead123]

Hopefully you do realize that 70c is 154F, which is way, way too hot for almost any electronic device, drives or otherwise....

As someone here said along time ago (can't remember who): "heat is the bane of all electronics" and "chill it or kill it"...which is true in all respects, unless of course, you just enjoy keeping a fire extinguisher next to your pc and/or frying/replacing your mobo's every month or so

Hell, even in my mini-me box with it's semi-active cooling system, in normal everyday uses, it rarely exceeds 52c..

 

[TheLostSwede]

No, that's insane I would say. The NAND isn't likely to take any damage, but other components on the SSD might degrade quicker than intended at that kind of temperature.
My hottest drive is 45 degrees idle, with the next hottest 40 and the coolest at a mere 29 degrees. Admittedly, the coolest one has a massive heatsink on it, whereas the other two drives and under a piece of shared metal that sits under the GPU.

 

[LabRat 891]

Something is likely wrong w/ the SSD. AFAIK, It should be throttling itself down about that temperature.
Alternatively, a particularly high-current NVME might be stressing the 3.3V rail to the NVME, drooping voltage, increasing current consumed, and getting noisy/dirty power, making it run exceedingly hot.
(I have an early USB 3.2 gen2 2xM.2 NVME 'dock' that runs drives crazy hot, too. So, apparently 'that's a thing')

 

[chrcoluk]

Ok thanks, so right now fan is on.

In the mean time I discovered this.

Other - NVMe Autonomous Power State Transition (APST)

So I'm playing around with APST (I had a slow day thankfully). I noticed when I'm using Debian, the power states will shift around depending on what the drive (1TB EX900 Plus M.2) needs to perform. The NVMe has 5 power states (0-4) and in Debian it will typically remain in power state 4...

forums.freebsd.org

Power States Supported: 5

#   Max pwr  Enter Lat  Exit Lat RT RL WT WL Idle Pwr  Act Pwr Workloadd
--  --------  --------- --------- -- -- -- -- -------- -------- --
0:  3.5000W    0.000ms   0.000ms  0  0  0  0  0.6300W  0.0240W 2
1:  2.4000W    0.000ms   0.000ms  0  0  0  0  0.6300W  0.0180W 2
2:  1.9000W    0.000ms   0.000ms  0  0  0  0  0.6300W  0.0150W 2
3:  0.0250W*   3.900ms  11.000ms  3  3  3  3  0.0250W  0.0000W 0
4:  0.0050W*   5.000ms  44.000ms  4  4  4  4  0.0050W  0.0000W 0

# nvmecontrol power nvme0   
Current Power State is 0
Current Workload Hint is 0

Looks like a FreeBSD problem with NVME power management, if I take the NVME out the CPU temp goes down about 10C as well, so the NVME is a mini heater in the case. Or maybe the drive is the issue, as when I was testing Windows it was on a different SSD, the drive is a WD SN570.

I can change it 2 manually and it will stay there, but 3 or 4 seem out of bounds.

 

[LabRat 891]

Ok thanks, so right now fan is on.

In the mean time I discovered this.

Other - NVMe Autonomous Power State Transition (APST)

So I'm playing around with APST (I had a slow day thankfully). I noticed when I'm using Debian, the power states will shift around depending on what the drive (1TB EX900 Plus M.2) needs to perform. The NVMe has 5 power states (0-4) and in Debian it will typically remain in power state 4...

forums.freebsd.org

Power States Supported: 5

#   Max pwr  Enter Lat  Exit Lat RT RL WT WL Idle Pwr  Act Pwr Workloadd
--  --------  --------- --------- -- -- -- -- -------- -------- --
0:  3.5000W    0.000ms   0.000ms  0  0  0  0  0.6300W  0.0240W 2
1:  2.4000W    0.000ms   0.000ms  0  0  0  0  0.6300W  0.0180W 2
2:  1.9000W    0.000ms   0.000ms  0  0  0  0  0.6300W  0.0150W 2
3:  0.0250W*   3.900ms  11.000ms  3  3  3  3  0.0250W  0.0000W 0
4:  0.0050W*   5.000ms  44.000ms  4  4  4  4  0.0050W  0.0000W 0

# nvmecontrol power nvme0   
Current Power State is 0
Current Workload Hint is 0

Looks like a FreeBSD problem with NVME power management, if I take the NVME out the CPU temp goes down about 10C as well, so the NVME is a mini heater in the case. Or maybe the drive is the issue, as when I was testing Windows it was on a different SSD, the drive is a WD SN570.

That makes sense. Also, the NVME staying in a high-power state will keep the CPU 'more awake' for the x4 PCIe link, too (AFAIK).
Any chance you could try out Windows temporarily, to see if the power states are switching correctly in a more expected/normal operating environment?

 

[chrcoluk]

It would be a pain on this actual SSD as I would have to wipe the OS and install windows, is only one NVME slot.

Do you know how to check the active power state in windows? If you do what I could do I suppose is move the drive into this system, and just watch its behaviour.

I do also have a boxed new sabrent rocket, but to put in there I need to buy a 2230 to 2280 extender, as it has no 2230 standoff.

The original SSD I used was a chinese branded NVME that came with it, and that idled in the high 40s in windows, but it had repeated incidents of corruption so I replaced it.

 

[docnorth]

Personally I wouldn't accept such temps at 19°C ambient not even under load. With the motherboards basic heatsink my Samsung 980 pro reached 55°C under load (in a relatively large case of course, with it's stock fans) and I'm wearing a summer shirt in my office...
Since you already use some passive cooling, the small fan you tried is the next step (I guess there's no space inside the case?)

 

[chrcoluk]

Personally I wouldn't accept such temps at 19°C ambient not even under load. With the motherboards basic heatsink my Samsung 980 pro reached 55°C under load (in a relatively large case of course, with it's stock fans) and I'm wearing a summer shirt in my office...
Since you already use some passive cooling, the small fan you tried is the next step (I guess there's no space inside the case?)

Its a NUC, case is like a little bit larger HDD. NVME is massive overkill for the use case, but the board has no M.SATA, although it does have a normal SATA port and I think some holes for securing that on the underside, given it looks like FreeBSD doesnt support NVME power management, I may end up putting a normal SATA drive in there. It might potentially be possible to secure a really thin small profile fan in there, but I think wont be needed if I use a normal SATA.

--

TrueNAS guys did a bit more digging here.

NVMe Autonomous Power State Transition (APST) - Not Working in CORE, Works in SCALE

So I'm playing around with APST (I had a slow day thankfully). I noticed when I'm using SCALE, the power states will shift around depending on what the drive (1TB EX900 Plus M.2) needs to perform. Do not get distracted by the fact that I have only one M.2 drive, it is purely for testing. My...

www.truenas.com

 

[TheLostSwede]

Ok thanks, so right now fan is on.

In the mean time I discovered this.

Other - NVMe Autonomous Power State Transition (APST)

So I'm playing around with APST (I had a slow day thankfully). I noticed when I'm using Debian, the power states will shift around depending on what the drive (1TB EX900 Plus M.2) needs to perform. The NVMe has 5 power states (0-4) and in Debian it will typically remain in power state 4...

forums.freebsd.org

Power States Supported: 5

#   Max pwr  Enter Lat  Exit Lat RT RL WT WL Idle Pwr  Act Pwr Workloadd
--  --------  --------- --------- -- -- -- -- -------- -------- --
0:  3.5000W    0.000ms   0.000ms  0  0  0  0  0.6300W  0.0240W 2
1:  2.4000W    0.000ms   0.000ms  0  0  0  0  0.6300W  0.0180W 2
2:  1.9000W    0.000ms   0.000ms  0  0  0  0  0.6300W  0.0150W 2
3:  0.0250W*   3.900ms  11.000ms  3  3  3  3  0.0250W  0.0000W 0
4:  0.0050W*   5.000ms  44.000ms  4  4  4  4  0.0050W  0.0000W 0

# nvmecontrol power nvme0 
Current Power State is 0
Current Workload Hint is 0

Looks like a FreeBSD problem with NVME power management, if I take the NVME out the CPU temp goes down about 10C as well, so the NVME is a mini heater in the case. Or maybe the drive is the issue, as when I was testing Windows it was on a different SSD, the drive is a WD SN570.

I can change it 2 manually and it will stay there, but 3 or 4 seem out of bounds.

3 & 4 are power saving modes.

Technology Power Features - NVM Express

NVMe™ Technology Power Management Features NVM Express (NVMe) architecture contains various features to manage the power of SSDs. In the data center, this can be used to throttle an SSD to a specific TDP (thermal design power) to manage platform thermals and total power draw. Max power is...

nvmexpress.org

 

[chrcoluk]

3 & 4 are power saving modes.

Technology Power Features - NVM Express

NVMe™ Technology Power Management Features NVM Express (NVMe) architecture contains various features to manage the power of SSDs. In the data center, this can be used to throttle an SSD to a specific TDP (thermal design power) to manage platform thermals and total power draw. Max power is...

nvmexpress.org

Left the script running that the TrueNAS user made, and its not changed to 3 or 4 once.

Will move the OS to SATA then do some testing with the drive in Windows, I do have another NUC sitting here, when I get RAM for it, I will test on that.

 

[Wirko]

the drive is a WD SN570

Take a look at TPU's SSD reviews. WD drives, as a rule, are unable to go into low power active states (APST), at least in W1zzard's Windows desktop system. That doesn't mean they behave the same in a notebook but if they do then they're more or less unusable in notebooks - and in PCs such as your passively cooled NUC.

 

[A Computer Guy]

So I have a N100 NUC on firewall duty, and the thing runs much hotter than when running Windows.

At 19C ambient temp (today) with its default passive cooling setup, case as heat sink, CPU is at 57C which is hot for idle but not close to TJMAX, however the NVME is at 68C. This room will get over 30C in the summer.

Blowing a small USB fan over the top of the case fins makes everything similar to desktop temps so CPU in 30s and NVME down to 40s. But trying to avoid having to use active cooling, so curious if you guys would be ok with your drive at those sort of temps 24/7.

That seems fairly high. What NVMe is it and what is the manufacturer tolerance spec? Also do you know if that's the nand temp or the controller temp?

 

[chrcoluk]

Take a look at TPU's SSD reviews. WD drives, as a rule, are unable to go into low power active states (APST), at least in W1zzard's Windows desktop system. That doesn't mean they behave the same in a notebook but if they do then they're more or less unusable in notebooks - and in PCs such as your passively cooled NUC.

Yeah now you mention it I do remember it from his WD reviews, although interestingly I dont think he has power graphs on his SN570 review and I found this quote.

Thermals, on the other hand, were impressive. Thanks to a highly energy-efficient controller design, there is no thermal throttling, not even in our worst-case thermal stress test, which hammers the drive with incoming writes non-stop. This is good news for users of mini PCs or laptops with limited cooling capability—no heatsink required.

That seems fairly high. What NVMe is it and what is the manufacturer tolerance spec? Also do you know if that's the nand temp or the controller temp?

I got no idea where the temp sensor is placed, its a 250 gig SN570. Smart says the warning threshold is 80C.

 

[A Computer Guy]

Yeah now you mention it I do remember it from his WD reviews, although interestingly I dont think he has power graphs on his SN570 review and I found this quote.
I got no idea where the temp sensor is placed, its a 250 gig SN570. Smart says the warning threshold is 80C.

For reference: https://www.westerndigital.com/products/internal-drives/wd-blue-sn570-nvme-ssd?sku=WDS250G3B0C

 

[chrcoluk]

Ok about to take this offline, but I did give it some running time on the older FreeBSD NVME driver (seems is two), and also on the power mode 2 instead of 0, to see what impact it would have, the driver wasnt allowing the idle modes to work, however temps stopped rising about 5-6C lower on both CPU and SMART so drive is now 62C and CPU is in 40s, ambient temp is the same. I will test windows on it, changed my mind as I am too curious as well as the firewall on a SATA. Multiple sources of FreeBSD not supporting NVME power management so I think I will end up with it on SATA anyway.

 

[Wirko]

Yeah now you mention it I do remember it from his WD reviews, although interestingly I dont think he has power graphs on his SN570 review and I found this quote.

He did measure the SN570, and many other older SSDs, again in his newer reviews, for example that of the SN580. 0.9W when it should be close to zero... that's a considerable amount in a fanless NUC. Granted, it was a larger capacity variant.

 

[chrcoluk]

I did do some testing, but not much as I have had to order some stuff, as an example the SATA power cable from an older NUC is a 2 pin cable but this board needs a 4 pin cable. So SATA is a no go for 2 weeks.

However I did test the SN570 in windows, here is some numbers.

Windows 10 Idle - Ambient 19.1C

Chinese NVME 48C CPU 42C
WD SN570 NVME 59C (55C in notes, but might be error, as screenshot is 59C) CPU 48C

pfSense based on FreeBSD 14 CURRENT - Ambient 19.2C - Stock NVME power state 0.

WD SN570 NVME 68C CPU 55C

pfSense based on FreeBSD 14 CURRENT - Ambient 19.2C - Configured NVME power state 2.

WD SN570 NVME 61C CPU 49C

I will be able to try Sabrent Rocket SB-2130-256 hopefully tomorrow in the unit.

 

[Wirko]

I did do some testing, but not much as I have had to order some stuff, as an example the SATA power cable from an older NUC is a 2 pin cable but this board needs a 4 pin cable. So SATA is a no go for 2 weeks.

However I did test the SN570 in windows, here is some numbers.

Windows 10 Idle - Ambient 19.1C

Chinese NVME 48C CPU 42C
WD SN570 NVME 59C (55C in notes, but might be error, as screenshot is 59C) CPU 48C

pfSense based on FreeBSD 14 CURRENT - Ambient 19.2C - Stock NVME power state 0.

WD SN570 NVME 68C CPU 55C

pfSense based on FreeBSD 14 CURRENT - Ambient 19.2C - Configured NVME power state 2.

WD SN570 NVME 61C CPU 49C

I will be able to try Sabrent Rocket SB-2130-256 hopefully tomorrow in the unit.

Hmm, CPU temp goes up a lot too. That can't be just because it's in the same small box as the SSD. My guess is that the SN570 is unable to put the PCIe bus into a lower power state, and as a consequence, the CPU also keeps its PCIe interface in a higher-power active state.

Can you at least mount a small heatsink on the SSD?

 

[docnorth]

Its a NUC, case is like a little bit larger HDD. NVME is massive overkill for the use case, but the board has no M.SATA, although it does have a normal SATA port and I think some holes for securing that on the underside, given it looks like FreeBSD doesnt support NVME power management, I may end up putting a normal SATA drive in there. It might potentially be possible to secure a really thin small profile fan in there, but I think wont be needed if I use a normal SATA.

--

TrueNAS guys did a bit more digging here.

NVMe Autonomous Power State Transition (APST) - Not Working in CORE, Works in SCALE

So I'm playing around with APST (I had a slow day thankfully). I noticed when I'm using SCALE, the power states will shift around depending on what the drive (1TB EX900 Plus M.2) needs to perform. Do not get distracted by the fact that I have only one M.2 drive, it is purely for testing. My...

www.truenas.com

Sorry for the late response, only now found some time.
1)Yeah I know it usually has the width of my hand, but some are bigger and can even take a compact GPU, I guess it was worth asking.
2)About power states a NUC should behave like a laptop and allow ASPM, so that the SSD can enter low power state. @W1zzard tests for some time now with ASPM off and on. By fixing that (if needed) your temps will improve, but still may be problematic under load, especially during summer.

 

[ty_ger]

I would be okay with it, yes. Idle isn't critical. In fact, a warm idle is good for NAND longevity. The thing I wonder is the load temperature. If it idles at 70 and heats up to 80, that would be fantastic. I would be ecstatic.

 

[freeagent]

Mine run in the 20s-40s so I would say 70 is a bit warm compared to what I am used to