Which 20 TB drive to get?

[Klemc]

Imagine you accidently replace a file without knowing and then copy it to your RAID (RAID 0 is not a RAID), now you have lost the file as there is no backup.

All this file managing in Windows is stupid, can't be worst:

why the file with the same name instead being put in the recycle-bin (like it's logic) would totally nuke the replaced file, there is NO reason, or a lie...

Since it's a well known problem causing human (mostly) errors, why did they keep this behavior, is therte a license to the first guy that got the idea and implemented it in the OS then it's like a proprietary thing that would cost MS money to change it ?... or what !?

I pity them all, ingeniors/devs look to be strange people (no insult).

 

[AusWolf]

Imagine you accidently replace a file without knowing and then copy it to your RAID (RAID 0 is not a RAID), now you have lost the file as there is no backup.

Sure, it doesn't protect against user error, but it does against hardware error. In this sense, just having two drives, and manually synchronising their contents is safer.

Backup is an extra copy of your data. A RAID1 solution would be one extra copy, albeit with hardware redundancy, thus a bit more reliable than a bare drive. (See: https://unix.stackexchange.com/ques...-personal-desktop-backup-system-pros-and-cons)

A proper solution looks like this (though few can afford it): https://www.backblaze.com/blog/the-3-2-1-backup-strategy/

I don't get no.3, the off site part. Why would I want to keep my data miles away from where I can access it?

Also, just throwing this for completeness' sake: assuming we're not looking at 20TB of sensitive data, have you considered could storage? I know it's not ideal, but for the cost of one 20TB drive, you can rent that amount for years more than the physical drive would serve you.

I have, but it's too expensive on the long run. The most sensitive part of my data is roughly 300 GB (family photos). This is the only thing I keep copied over several different drives at the moment.

 

[Klemc]

Sure, it doesn't protect against user error, but it does against hardware error. In this sense, just having two drives, and manually synchronising their contents is safer.


I don't get no.3, the off site part. Why would I want to keep my data miles away from where I can access it?


I have, but it's too expensive on the long run. The most sensitive part of my data is roughly 300 GB (family photos). This is the only thing I keep copied over several different drives at the moment.

You can also print your family photos, protect with plastic (photo book), yes ?.. instead buying more and more HDD to duplicate those, it's even less pricy & more secure.

 

[AusWolf]

You can also print your family photos, protect with plastic (photo book), yes ?.. instead buying more and more HDD to duplicate those, it's even less pricy & more secure.

Print circa 25,000 photos? How big of a book would I need to store them?

Also, printed media isn't duplicable. If it's lost, it's lost forever. I don't see it as more secure than HDDs.

 

[JoeTheDestroyer]

I don't get no.3, the off site part. Why would I want to keep my data miles away from where I can access it?

If your backups are all on-site, and your house burns down, all your data is gone.

 

[Klemc]

Print circa 25,000 photos? How big of a book would I need to store them?

Also, printed media isn't duplicable. If it's lost, it's lost forever. I don't see it as more secure than HDDs.

Digital data doesn't even exists physically, it's less strong.

If you have 25000 photos, i'm sure there is a company capable of making them for you, the whole

edit: OKay, just don't smoke too near the album, and all will be fine

 

[AusWolf]

Digital data doesn't even exists physically, it's less strong.

True, but as long as it exists in multiple copies and locations...

If you have 25000 photos, i'm sure there is a company capable of making them for you, the whole

edit: OKay, just don't smoke too near the album, and all will be fine

Yeah, and I might as well rent another house just for storage.

 

[Klemc]

True, but as long as it exists in multiple copies and locations...


Yeah, and I might as well rent another house just for storage.

Or just a caravan that you can drive and share your photos with everybody on the road

 

[agent_x007]

@AusWolf Q : What about WD's HC 560/HC 570 20TB ?

 

[unwind-protect]

What's ZFS? Do you have some resource I could read?

OpenZFS - Data Security vs. Integrity - Klara Systems

Understand the difference between data security and integrity. Learn how ZFS ensures both in this article on protecting and validating your data.

klarasystems.com

https://en.wikipedia.org/wiki/ZFS skip to "Data integrity".

 

[Harlow]

You can also print your family photos, protect with plastic (photo book), yes ?

I print out 1,500 photos each year on A4 glossy paper, laminate them in plastic, trim them with a guillotine, then hand them out to the people I've photographed. It takes me around 4 weeks and uses over 30 'after market' ink cartridges (must get a printer with large refillable ink tanks).

I find home printing cheaper than an on-line service, but still quite expensive. With digital, I sometimes shoot over 1,000 photos each day at festivals. No way would I print them all out. Just some of the best ones.

Also, printed media isn't duplicable. If it's lost, it's lost forever. I don't see it as more secure than HDDs.

Prints are not necessarily more secure than HDDs, but they might still be around, long after the hard disk has stopped working (if you don't copy the files to a new drive every 5 years). Although quality drops off, you can scan old prints at 3000dpi+, fix them in Photoshop, upscale them in GigaPixel, then print them out again. Works for old B&W family photos from times gone by, where the negs are lost.

Quote from Epson:
"Photographs can last from just a few months to over one hundred years. High humidity, higher temperatures, and UV light from bright light sources will cause photos to fade faster. Prints made from its high end Stylus photo R1800 printer using pigments on archival paper will last 250 years according to Epson."
https://www.better-digital-photo-tips.com/photograph-last.html.

Although I won't be around in 100 years time to see if this is true, neither will any of my hard disks. As for the prints, who knows. They'll probably end up in the skip.

 

[Wirko]

If your backups are all on-site, and your house burns down, all your data is gone.

Said Joe the Destroyer...

But seriously, you only need to access that far removed, last-in-the-line backup disk if something goes very wrong at your home. Fire, flood, tornado, theft. IT-era burglars can take everything that looks like a computer or a disk (or a piece of paper with passwords on it).

Also, backup media should be kept at some physical distance from your computer(s), for similar reasons. Next room, or whatever is reasonable in your house or apartment.

 

[A Computer Guy]

Voted for Exos but if Toshiba is more affordable then that is an option I suppose.

I see the RAID is not a Backup subject has been flogged to death so I won't comment on that any further other than to say you should always TEST your backup from time to time. This might be via running some validation if you are using backup software or manually by using a file comparison utility that will compare file content at the byte level.

Regardless of drive model here are my thoughts...

• If your intent is to only get 2 drives then I would forgo the RAID1 in favor of 1 for normal use and 1 as a backup copy.
• If you can afford 3 drives then I would forgo the RAID1 in favor of 1 for normal use and 2 as a backup copy.

alternatively ...if affordability allows introduce redundancy...

• If you can afford 3 drives then I would setup 2 in a RAID-1 (for hardware redundancy/availability) for normal use and 1 as a backup.
• If you can afford 4 drives then I would setup 2 in a RAID-1 (for hardware redundancy/availability) for normal use and 2 as a backup.

( expanded below... in a 4 disk RAID: If setup time and read/write speed does not matter to you RAID-6 to rotate or replace any 2 disks at any time which includes disk failure or upgrading capacity. Includes scrubbing to help protect against bit rot. Replacing high capacity disks will take a long time when rebuilding the array. )

• If you can afford 6 drives then I would setup 4 in a RAID-6 (for maximum hardware redundancy/availability) 1 for (cold or hot) spare, and 1 as a backup.
• If you can afford 7 drives then I would setup 4 in a RAID-6 (for maximum hardware redundancy/availability) 1 for (cold or hot) spare, and 2 as a backup.
• If you can afford 8 drives then I would setup 4 in a RAID-6 (for maximum hardware redundancy/availability) 2 for (cold or hot) spare, and 2 as a backup.

...as time moves on...

• At the first affordable opportunity get one replacement disk to store on the shelf when needed.
• At least about 1/2 way though the warranty of your first disk in your initial setup get another disk and incrementally improve your initial setup based on the scaling above to add as a backup or cold/hot spare. Over time if you accumulate enough disks you can improve redundancy.

...other thoughts...

• Typically enterprise drives will be nosier and hotter so if those drives will be in your PC or room where you have to listen to them constantly or adding to your PC case temperature take that into consideration.
• Enterprise drives need to be tougher because of more strenuous workloads they are used for. If you are NOT going to punish your drive without mercy then going for more affordable consumer drives might also allow you to get an overall greater quantity of drives for backup and/or redundancy.

 

[ir_cow]

Well, the Toshiba is £310, the Seagates are around £350 each, but the WD Gold is £460 - this is why I didn't list it in the poll.

You pay for what you get. Toshiba is the bottom of the barrel. Like eveyone else said, a Raid-1 isn't a true data backup, but you'll going to need a copy when those Toshiba drives fail

 

[Harlow]

What's ZFS? Do you have some resource I could read?

This Youtube video shows a cheap home built ZFS system running TrueNAS Scale. This is not a system for the purists, because it doesn't use ECC RAM.

I use TrueNAS Core (formerly FreeNAS) running on 6 and 8-disk arrays in RAID-Z2 (two disk redundancy, similar to RAID 6).
https://www.truenas.com/truenas-core/

Typically enterprise drives will be nosier and hotter

True. I've got a few WD Golds and Seagate Exos, but I don't find them particularly annoying.
The five 140mm cooling fans in my Fractal R4 and similar cases drown out any hard disk noise with the side panels on, even with the fans set to low RPM.

My loudest systems are HP servers with four Delta fans screaming their little heads off at start up. Not suitable for use in a bedroom.

This video clip gives an idea of start up noise on a machine similar to mine - skip 35 seconds into the video.

The noise dies down to an "acceptable" level if you fast forward to 3 minutes 30 seconds, after the system boots up.

Having worked in large server halls containing hundreds of machines, I got used to wearing ear defenders.

You pay for what you get. Toshiba is the bottom of the barrel.

Agreed. One of my 6TB Toshiba N300 NAS drives in RAID-Z2 failed early with three bad blocks. The chassis of the faulty drive vibrates significantly louder than the other five drives. It's only a matter of time before the next Toshiba in the array kicks the proverbial.

Instead of paying £460 for one 20TB WD Gold, you could buy a bunch of "Consumer" level drives and cross your fingers. Data integrity is important, but so is paying the rent or mortgage, gas, electric and water bills, car repayments and insurance, supermarket bills, etc. It's a question of priorities.

 

[DaemonForce]

thinking about replacing my internal hard drives with an external RAID 1 array of 20 TB enterprise drives for some extra data retention in case of failure

Nope, don't do it. Get an internal HDD, plug it into your most reliable 24/7 accessible server and let it go BRRRRRRRR.
Nearline/offline backups are important but the moment you start climbing into the xxTB of content, you're not using exclusively "important" non-replaceable data.
As data hoarders, we know better. I will shill Toshiba all decade long, love these drives...But as hobbyist or pro you've got a LOT of thinking to do before any migration.

I'm leaning towards the Toshiba, as that's the cheapest of the bunch, but I don't want to regret my decision later.

Get one, verify it CLEAN, performance test then consider how to maximize your experience.
I keep a 16TB in the eMachines. Yes there's a sata2 speed cap and yes they do get warm from extended writes.
Make sure you get a fan and don't pair these up with hot disks or any on the way out:

Incredible drives.

Performance isn't extremely important, but reliability is (two of such drives aren't cheap).

The performance isn't important until you need it.
You're likely going to see similar speeds so be prepared for it.
An HDD is not just a magic box to drop files. Let it thrash.

What do you think?

Okay, time to get a little autistic. It makes far more sense to strategize your data provisioning than going into this like it's some extension of your current storage.
A new disk isn't a tumor but a new lease on life. One of the reasons we're talking about HDDs in the first place is their long standing reputation in storage and how they behave.
The disks that currently hold your data have TIME on them, which translates to WEAR. We don't know how much but if you're the type to hold onto disks for a while, it adds up.
There are performance characteristics of new drives that obsolete your older ones. Not by a lot but just enough for you to consider them and then some.
Maybe you double up on storage every other year or wait it out to quadruple. Maybe Toshiba's half gigabyte cache appeals to you too. You've got options and they're great.

If your storage strategy is good, you're going to shuffle a bit of data around.
Avoid extreme jumps like I did: 300GB...750GB...2TB...4.9TB...16TB.
Roughly 2.2x your current largest is often safe and cost effective.
If something happens and you need to suddenly recover a ton of data, the time and money involved is relative to these capacity jumps.

Write a chart.
A chart of your disks. Their sizes, speeds, features you care about, their behaviors and how you want to arrange them for use.
Try something like most frequent access (online) to less frequent to cold spare (nearline/offline).
If you're already setup good, this will change a little bit. If it's a really big deal, expect a very big change.

Write a chart of your data.
I'm not sure how to put this into human language but consider serious uses like game libraries, music, movies, various video backups, photos, memes, whatever.
You'll start to notice other things that stand out as you plot these details.
Some data takes up more space than others. Some data takes WAAAAY more space.
Some kinds of data fit way better on a volume arranged in 16K clusters than 64K. Some data is just slow no matter what. Be wary of the behaviors.
You may need to be aware of variations in volume types like NTFS and ReFS if you ever messed with that, they behave a little differently.
More importantly, be aware of the data you want to keep and the data that can be dumped. I'll get into that in a bit.

Be aware of bad provisions where you're likely to experience a serious balloon in library sizes if you weren't able to do it before but want/need it.
My Steam library went from 544GB (561GB on disk) to 2619GB of a 3TB iSCSI chunk sitting at the start of the data partition of my 16TB Toshiba.
I had a lot of stuff to grab. I also know my own purchase habits and it will take ~2 years to fill another 500GB.
By then I will either chunk it out again or have another strategy for the volume.
I then made a 2.75TB iSCSI provision for Epic and various other game libraries that finally get a drive letter.
Having both volumes on one disk might be a bad idea if there's lots of updates from both stores at the same time so if you do something like this, be warned.
At this point, nearly half the disk is decided and good.

About dumping data...You have data that can be sorted into categories like constantly used, frequently accessed, rarely used and they can be in nets like LAN only or Internet distribution.
The importance rank is entirely up to you but you need to make a choice when determining the value of those different pieces of data and how they all fit together.
They're all types of data that behave differently from one another, with varying priorities, securities and growth patterns.
Most of us are just single users of each device but if you're some kind of NetAdmin, have quotas or just a really restrictive bandwidth policy, it's something to think about.

Figure out which data sets are:
nonreplaceable personal/isolate
nonreplaceable historic/archive
nonreplaceable work
replaceable non-burden
replaceable fair burden
replaceable extreme burden

A noteworthy example: A Steam Library can always be replaced, a Steam Archive probably not.
It's also troublesome to replace and validate tons of files or lose too much space to cluster.

A million files with ~9GB lost to clusters.

It also matters way more to certain people. If you're just an ordinary gamer but fickle, you tend to game hop a lot. The biggest 8 titles in my Steam add up to 60% of my monthly bandwidth quota and that's without updates, which thankfully have calmed down lately. If you're some kind of retro gamer, of course you're going to have your own repo for playing and streaming but if you're a variety streamer that bounces from title to title every day or every hour, that's a lot of data to retain in your library to just be on call like that. It can all be replaced but it's a pretty heavy burden to do so.

Emulation kits, ISOs, YouTube archives, Gallery-DL archives, 3D Models, Virtual Machine stores, Unreal and Unity backups, system driver packages, keys, functional IIS website backups, language material archives, other study kits, patches for various kinds of games/software, phone backups, photo archives, photostock, entire user profile backups, various open toolkits, torrent backups, archives of antique websites that no longer exist and can't be wayback'd, music storage pools and their archives...

All of these fit somewhere in the chart and I'm sure you're going to figure out more than I can put here.
If you really want to keep a RAID for everything, I'm sure it's fine.
I avoid that setup until using M.2 but for HDDs, manual mirroring has worked fine.

 

[Wirko]

Figure out which data sets are:
nonreplaceable personal/isolate
nonreplaceable historic/archive
nonreplaceable work
replaceable non-burden
replaceable fair burden
replaceable extreme burden

Good thing you pointed that out. It's an important thing to consider.

 

[bug]

I don't get no.3, the off site part. Why would I want to keep my data miles away from where I can access it?

That's the easiest to explain: that's to protect your data against disasters. Doesn't need to be miles away, unless you want your data to survive a nuclear strike or smth. It can be something that you keep at a friend or relative's house.

 

[A Computer Guy]

That's the easiest to explain: that's to protect your data against disasters. Doesn't need to be miles away, unless you want your data to survive a nuclear strike or smth. It can be something that you keep at a friend or relative's house.

Or bury in a time capsule or maybe even lost in a "boat accident" with any guns you may or may not have.

 

[chrcoluk]

Out of the three iron wolf pro, looks like it has the best acoustics.

They all helium and same warranty.

I highly recommend ZFS or a similar file system, it has features that concentrate on data integrity such as managing bit rot (helps avoid the issue someone mentioned on their old fake raid setup), and also supports snapshots, so file roll back would also be possible.

TrueNAS can run on pretty much any hardware (documented minimum ram to use is 8 gigs), or you can buy NAS systems that either use ZFS out of the box or something like btrfs. Also setting up windows file sharing on TrueNAS core, there is a very useful page for it on the official documentation, following the guide its probably about 15 minutes to do it.

The subject of backups is a bit controversial as storage is expensive, and the problem as has been pointed out already is that raid isnt a backup, the raid mode thats closest to a backup is raid 1, as you can pull a disk and that has a copy of the data usable in single disk mode, but of course if you do that then you have lost your redundancy and is out of date as soon as the data changes on the remaining drive in the pool. My own backups are going to onedrive, I have a family account so 5TB of cloud storage on there and I have configured a automated sync in TrueNAS. That was probably the most cost efficient remote backup solution I could find at the time I set it up, of course you can do a local backup solution but more capital expenditure in disks, it also isnt quite covering all data on my NAS, some data is duplicated still on PC, otherwise I prioritised it, I consider my game recordings of up most importance so they all get sent to the cloud as a backup. Those refurbished drives may be a good bet for local backups if not using cloud.

 

[lexluthermiester]

Hi guys,

I'm thinking about replacing my internal hard drives with an external RAID 1 array of 20 TB enterprise drives for some extra data retention in case of failure. I have 3 options available at my no.1 go-to store, but having no experience with enterprise/NAS drives, I don't know which one is the best. I'm leaning towards the Toshiba, as that's the cheapest of the bunch, but I don't want to regret my decision later.

Performance isn't extremely important, but reliability is (two of such drives aren't cheap).

What do you think?

While Seagate drives have been very good in recent years, I still prefer Toshiba and that got my vote. It sucks you don't have the choice of Western Digital as the Gold drives are easily the best.

I don't get no.3, the off site part. Why would I want to keep my data miles away from where I can access it?

In case your home is robbed or burn down.

You pay for what you get. Toshiba is the bottom of the barrel.

Not according to BackBlaze.

Backblaze Drive Stats for 2023

Read the 2023 Drive Stats Report and the latest insights on drive failure from Andy Klein. Read the 2023 Drive Stats Report and the latest insights on drive failure from Andy Klein.

www.backblaze.com

By that chart Seagate is very much the worse company for drives with WD being best.

And this remains true going back several years;

Backblaze Drive Stats for 2022

Check out the 2022 year-end Drive Stats data, and let us know your thoughts in the comments.

www.backblaze.com

Backblaze Drive Stats for 2021

Understanding our hard drives is key to understanding our company. Read the Backblaze drive stats for 2021 to learn more about our expansions.

www.backblaze.com

2020 Hard Drive Reliability Report by Make and Model

The AFR for 2020 dropped below 1% down to 0.93%. In 2019, it stood at 1.89%. That’s over a 50% drop year over year.

www.backblaze.com

2019 Hard Drive Reliability: Failure Rates Continue to Rise

In 2019 we monitored the performance of 122,507 hard drives. In this article we look at the failure rates by make and model.

www.backblaze.com

According to these stats, Toshiba drives are solidly reliable and have been for many years.

@AusWolf
If you do choose Toshiba drives, buy with confidence because they really are solid drives. Especially if you can get them for a good price.

 

[ir_cow]

It's important to get the exact model as listed in backblaze. Easy to get the wrong model for some of these.

 

[lexluthermiester]

It's important to get the exact model as listed in backblaze. Easy to get the wrong model for some of these.

Context is important. The overall performance of a company lineup is was the data from BackBlaze shows. Individual model numbers are not as important. The overall numbers for the last 5 years clearly show that WD is consistently #1. HGST & Toshiba trade places for #2 & #3 and Seagate is consistently the last, and by a considerable percentage margin. That trend continues as you go further back in time. So to call Toshiba "bottom of the barrel" is disingenuous at best and misleading at worst.

 

[ir_cow]

So to call Toshiba "bottom of the barrel" is disingenuous at best and misleading at worst.

Eh. If your like me and always come across dead Toshiba drives in servers and personal computer, you tend to avoid them. I just stick to WD Golds now. Don't waste my money on the others.

 

[DaemonForce]

dead Toshiba drives in servers and personal computer

What era is this? I get that I'm the odd one out for slumping anywhere from 20-40TBs of HDD into an eMachines and it's hilarious but if you're encountering a dead Tosh or more in SERVERS...They were either DOA or have 100K hours on them. Personal computer? What are you doing? How desperate do you have to be to use HDDs in a primary workstation? I went all SSD with mine in 2018 and dumped all spinners to servers.

We are also fast approaching the still very much radical idea I have about sata SSDs as well.
Basically goes like "cable connected disks are for servers." Now that's a line that goes hard.