Tuesday, April 21st 2020
Western Digital Defends DM-SMR on WD Red HDDs, Points Users to WD Red Pro or WD Gold
Western Digital gave its first response to allegations of the company implementing SMR (shingled magnetic recording) on its WD Red internal hard drives without properly documenting it. The WD Red series is extensively marketed as being "NAS optimized," which caused many NAS and RAID DAS enthusiasts to pick it up for home-office use, only to discover that the company's implementation of drive-managed SMR (DM-SMR) makes them effectively unfit for RAID use, as DM-SMR is vital for some of the higher-capacity WD Red models to achieve their nameplate capacity, while coming at a heavy cost of random write performance.
"SMR is tested and proven technology that enables us to keep up with the growing volume of data for personal and business use. We are continuously innovating to advance it. SMR technology is implemented in different ways - drive-managed SMR (DMSMR), on the device itself, as in the case of our lower capacity (2 TB - 6 TB) WD Red HDDs, and host-managed SMR, which is used in high-capacity data center applications. Each implementation serves a different use case, ranging from personal computing to some of the largest data centers in the world.," Western Digital writes.
Western Digital explains that the WD Red family of HDDs were designed for smaller-scale home-office NAS applications - "one to eight bays" in scale, with workload rate of 180 TB/year. The company states that the drives have been tested and validated by major NAS manufacturers - a response to the SMR controversy blowing up on NAS manufactrurers' support forums.
Western Digital was vague about how it plans to make up to aggrieved WD Red users. It points them to their support site, stating "We know you entrust your data to our products, and we don't take that lightly. If you have purchased a WD Red drive, please call our customer care if you are experiencing performance or any other technical issues. We will have options for you."
The company pointed serious NAS customers (applications of a scale higher than mentioned), to consider WD Red Pro, WD Gold, or even Ultrastar enterprise HDDs.
Source:
Western Digital Blog
"SMR is tested and proven technology that enables us to keep up with the growing volume of data for personal and business use. We are continuously innovating to advance it. SMR technology is implemented in different ways - drive-managed SMR (DMSMR), on the device itself, as in the case of our lower capacity (2 TB - 6 TB) WD Red HDDs, and host-managed SMR, which is used in high-capacity data center applications. Each implementation serves a different use case, ranging from personal computing to some of the largest data centers in the world.," Western Digital writes.
Western Digital was vague about how it plans to make up to aggrieved WD Red users. It points them to their support site, stating "We know you entrust your data to our products, and we don't take that lightly. If you have purchased a WD Red drive, please call our customer care if you are experiencing performance or any other technical issues. We will have options for you."
The company pointed serious NAS customers (applications of a scale higher than mentioned), to consider WD Red Pro, WD Gold, or even Ultrastar enterprise HDDs.
42 Comments on Western Digital Defends DM-SMR on WD Red HDDs, Points Users to WD Red Pro or WD Gold
2. The originally issue, the reason all of this was discovered, was because the drives were so slow the RAID controllers assumed they were bad drives.
3. If I break the seal on the drives I have I can't return them, and I'm not keeping these piles of garbage.
DM-SMR drives aren't just "a little" slower - and to top it off DM-SMR REDs have a serious firmware bug which makes them throw hard IO errors at the host system
How much slower?
Well if you put one in an existing parity RAIDset (RAID5/6 or RAIDZ1/2/3) to replace a failed or aging drive, you can see 90-98% degradation in rebuild times
RAID rebuilds which used to take a day or less can easily take 8-19 DAYS
RAID scrub/verification passes can take 8-12 DAYS
In the case of WD REDs, they won't get that far - because during the raid rebuild process they will generate drive internal errors:
smartctl -x will show drive internal error logs like this:
" 10 -- 51 00 00 00 00 1f 3b c0 b0 40 00 Error: IDNF at LBA = 0x1f3bc0b0 = 524009648 "
Linux reports them like this (samples):
"blk_update_request: I/O error, dev sdd, sector 734925768 op 0x1:(WRITE) flags 0x700 phys_seg 1 prio class 0
blk_update_request: critical target error, dev sdd, sector 734925408 op 0x1:(WRITE) flags 0x700 phys_seg 1 prio class "
And the results above ARE ON AN IDLE ARRAY - the only thing going on is the rebuild or the scrub
It goes well beyond DM-SMR in NAS drives.
Here's why DM-SMR in "desktop drives" is a spectacularly bad idea:
If you know what you're doing and treat them "just right", then DM-SMR drives will have a performance hit of between 40-60% - which isn't minor.
The problem is this: Virtually every filesystem (except FAT) writes a metadata update back to the filesystem every time a file is read, updating the last read time of that file.
ie: Every file read, causes a write
THIS is the WORST POSSIBLE CASE for DM-SMR drives - and why operators MUST know what they have, as you need to be able to turn off "last accessed" times.
On linux, that's easy: "noatime" mount options - but doing this on the OS mounts and in areas where atime monitoring is critical WILL cause secondary issues
On Windows, it appears to be an "all or nothing" proposition across the entire system, which destroys the functionality of a lot of system automation utilty builtins.
Someone in WD messed up bad. SMR shouldn't be in NAS (Red) or Performance (Black) lines.
It was also pointed out that their combative stance will probably result in exemplary damages awards and the only hope they have of migitating judgements now is to come utterly clean on everything including their entire DM-SMR range and future plans
They had a radical change of tune in public shortly afterwards - but still dodging full responsibility on this.
Seagate and Toshiba aren't immune to legal action either. Seagate in particular have been gaslighting complaints about the exact same issues for the last 18 months, whilst Toshiba have simply been "stealthy", but as the affected items are no longer "business products" they have fewer legal roadblocks they can throw up to small claims actions.
Consumer and market regulators need to swoop in and deal with this.
I'm calling BS on the numbers you are stating as they do not jive with the numbers I/we are seeing. I think this problem very much depends on array configurations on case by case basis.
Kindly stump up and state the EXACT MODELS of the "drives in question" in your work enterprise arrays
Then let us know if you have atime enabled or disabled on your arrays at work, or if the admins have performed other optimizations to take SMR into account such as copious write caching
In any case:
Try knocking out a drive from an array of CMR drives and rebuilding your array with a clean DM-SMR one - bearing in mind that WD have been going out of their way to HIDE that these drives are DM-SMR up until now, so this is a valid scenario (replacing a failed older "red drive" with a new "red drive")
Report back on what you see.
DM-SMR drives will work "OK" for very small quantities of writes, but as soon as given sustained writes (such as when rebuilding parity RAID5/6 arrays), sustained random seeks and writes (rebuilding ZFS RAIDZ arrays) or lots of read/writes (normal filesystem access gives a metadata update write for every file read) the performance goes down the toilet
The last case is why they're giving such awful performance as desktop drives.
Use as a NAS drive under _normal_ conditions (write occasionally/read mostly) can be mitigated by turning off access time metadata updates completely (relatime is not good enough), BUT you have to know what you are dealing with in order to do that.
In the case of REDs, raid rebuilds of the kinds described ALSO tickles a firmware bug which will throw critical write IO errors back to the host system - which is invariably fatal for the rebuild
Instead of buying a second 4TB Seagate drive for mirroring, bought two 1TB SATA SSDs.
Haven't used one in many many years since I switched to ssd's initially then to nvme, not goin back no wayz no howz, that wayz I gotz no worries about all this here nonsense, hahaha :)
And speakin of WD, all 6 of my Black SN750's are screaming along at 3500MB/s this very second, have been since day 1.... so in case you didn't catch it the 1st time....
Die, spinners, die :D