Which 20 TB drive to get?

[AusWolf]

Hi guys,

I'm thinking about replacing my internal hard drives with an external RAID 1 array of 20 TB enterprise drives for some extra data retention in case of failure. I have 3 options available at my no.1 go-to store, but having no experience with enterprise/NAS drives, I don't know which one is the best. I'm leaning towards the Toshiba, as that's the cheapest of the bunch, but I don't want to regret my decision later.

Performance isn't extremely important, but reliability is (two of such drives aren't cheap).

What do you think?

 

[Harlow]

I'm thinking about replacing my internal hard drives with an external RAID 1 array of 20 TB enterprise drives

The word "replacing" is worrying, if it means the new RAID 1 array will store the only copy of your treasured files.

To be safe, you need at least three copies of important files, stored on different devices/media, with one copy held off-site.

Can you afford to lose up to 20TB of data, if the RAID array goes up in smoke or is attacked by Ransomware?

Disadvantages of RAID 1​

• RAID 1 is not the best provider of redundancy for very large servers that are storing terabytes of important data.
• RAID 1 is commonly used as a backup solution, but it’s not reliable as the only backup method for data. It doesn’t protect data from power surges or other physical damage (since it’s not in the cloud or on another server). And as previously mentioned, RAID 1 doesn’t protect data from corruption; if part of a file is corrupted, the same corruption will be exactly mirrored on the other drive.
• It offers half the capacity of true data storage because half is used for copies.

https://www.enterprisestorageforum.com/backup/raid-1/

RAID 0 should be avoided like the plague. Lose one drive and all your data's gone.

Back in the days before SSDs, I built a RAID 1 array with two 1TB hard disks. Two years later I discovered both drives had developed bad blocks and some files were corrupted. From then on I abandoned simple motherboard RAID and saved multiple copies on separate drives in different machines.

Instead of installing two 20TB drives in a RAID 1 enclosure, put the drives in two smaller enclosures, then use file synchronisation software to keep the files backed up. That way if one drive gets clobbered, you might still have a second copy.

20TB drives are still quite expensive and hard disks can fail at any time. To reduce the risk of losing huge amounts of data on one drive, it might be better to buy four 10TB drives for roughly the same cost as two 20TB drives and divide the risk.

I have adopted an alternative approach and store my data on a number of home-built TrueNAS Core servers, using multi-disk arrays. By running RAID-Z2 (equivalent to RAID-6) I can lose up to 2 drives and my data remains "safe". This does of course mean you need at least 4 identical hard disks for RAID-Z2 and I use 6 or 8 drives per array.

I would not recommend TrueNAS Core unless you're prepared to master the complicated set up procedure to configure and share the array (it took me ages), but you can build a TrueNAS system for little more than the cost of the drives, if you have an old computer gathering dust. Any dual core CPU with at least 8GB RAM in a big case will do.
https://computingforgeeks.com/install-and-configure-truenas-core/

I also back up files to hard disks in various computers and to 800GB LTO-4 tapes for very little cost ($2 per tape). These archive tapes are far less susceptible to virus attack.

If you like a easier life, consider buying a Q-Nap or Synology box and fill it with drives. The downside of these systems is the expense of the empty NAS box. That's why I prefer TrueNAS.

Whatever you do, keep more than one copy.

 

[unwind-protect]

I have good luck with (smaller) Toshibas, so that is what I would pick.

 

[ir_cow]

WD Gold.

 

[Solaris17]

but it’s not reliable as the only backup method

That’s because it isn’t a backup and OP never let anyone tell you raid is a backup. RAID is hardware redundancy not data redundancy.

anyway out of the choices given exos

 

[Steevo]

A separate unplugged drive is a backup, RAID is fault tolerance for a single drive but as mentioned doesn't prevent data loss if you have a electrical issue or any other reason than a single drive failed. I wouldn't consider a USB drive attached 24/7 to be backup either.

 

[AusWolf]

The word "replacing" is worrying, if it means the new RAID 1 array will store the only copy of your treasured files.

To be safe, you need at least three copies of important files, stored on different devices/media, with one copy held off-site.

Can you afford to lose up to 20TB of data, if the RAID array goes up in smoke or is attacked by Ransomware?

Disadvantages of RAID 1​

• RAID 1 is not the best provider of redundancy for very large servers that are storing terabytes of important data.
• RAID 1 is commonly used as a backup solution, but it’s not reliable as the only backup method for data. It doesn’t protect data from power surges or other physical damage (since it’s not in the cloud or on another server). And as previously mentioned, RAID 1 doesn’t protect data from corruption; if part of a file is corrupted, the same corruption will be exactly mirrored on the other drive.
• It offers half the capacity of true data storage because half is used for copies.

https://www.enterprisestorageforum.com/backup/raid-1/

RAID 0 should be avoided like the plague. Lose one drive and all your data's gone.

Back in the days before SSDs, I built a RAID 1 array with two 1TB hard disks. Two years later I discovered both drives had developed bad blocks and some files were corrupted. From then on I abandoned simple motherboard RAID and saved multiple copies on separate drives in different machines.

Instead of installing two 20TB drives in a RAID 1 enclosure, put the drives in two smaller enclosures, then use file synchronisation software to keep the files backed up. That way if one drive gets clobbered, you might still have a second copy.

20TB drives are still quite expensive and hard disks can fail at any time. To reduce the risk of losing huge amounts of data on one drive, it might be better to buy four 10TB drives for roughly the same cost as two 20TB drives and divide the risk.

I have adopted an alternative approach and store my data on a number of home-built TrueNAS Core servers, using multi-disk arrays. By running RAID-Z2 (equivalent to RAID-6) I can lose up to 2 drives and my data remains "safe". This does of course mean you need at least 4 identical hard disks for RAID-Z2 and I use 6 or 8 drives per array.

I would not recommend TrueNAS Core unless you're prepared to master the complicated set up procedure to configure and share the array (it took me ages), but you can build a TrueNAS system for little more than the cost of the drives, if you have an old computer gathering dust. Any dual core CPU with at least 8GB RAM in a big case will do.
https://computingforgeeks.com/install-and-configure-truenas-core/

I also back up files to hard disks in various computers and to 800GB LTO-4 tapes for very little cost ($2 per tape). These archive tapes are far less susceptible to virus attack.

If you like a easier life, consider buying a Q-Nap or Synology box and fill it with drives. The downside of these systems is the expense of the empty NAS box. That's why I prefer TrueNAS.

Whatever you do, keep more than one copy.

The problem is that my data currently sits on several hard drives of different capacities as single copies. If one of the drives fails, part of my data is lost. That's why I thought a RAID 1 array would at least protect my data in case of drive failure. I know it doesn't account for any other failure, but I don't have the money, nor would I ever want to pay for 15-20 TB worth of cloud storage. And keeping it as an external array, it wouldn't have to be connected to my PC and a power source all the time, either. My other option is mirroring the disks manually.

WD Gold.

Well, the Toshiba is £310, the Seagates are around £350 each, but the WD Gold is £460 - this is why I didn't list it in the poll.

A separate unplugged drive is a backup, RAID is fault tolerance for a single drive but as mentioned doesn't prevent data loss if you have a electrical issue or any other reason than a single drive failed. I wouldn't consider a USB drive attached 24/7 to be backup either.

I'm planning a RAID array of unplugged drives. I'd only plug the enclosure in when I'm doing a backup or reading files. It wouldn't be plugged in 24/7.

That’s because it isn’t a backup and OP never let anyone tell you raid is a backup. RAID is hardware redundancy not data redundancy.

Sure, but with RAID 1, if a drive fails, you still have your data intact on the second one, right?

 

[Harlow]

The problem is that my data currently sits on several hard drives of different capacities as single copies.

I have many hard disks scattered across multiple machines. This means I have at least 3 copies of all my holiday photos (typically 600GB per trip). Then there are backups on 8TB USB drives, plus backups on 800TB tape, plus backups on 4 TrueNAS core servers, plus copies at a relatives house. It sounds like complete overkill, but some of them should survive a Ransomware attack or another lightning strike near my house.

I know it doesn't account for any other failure, but I don't have the money, nor would I ever want to pay for 15-20 TB worth of cloud storage.

I quite understand about expense, especially the bit about Cloud storage. The problem is, if you place any value on keeping your files safe, you need more than one copy. If you can't afford enough brand new drives to keep three copies, consider second hand "pulls" from servers. It's a thorny question as to whether or not you should trust a hard disk that's been running continuously for 3 to 5 years, but they're often Enterprise Class drives and cheap enough to buy in quantity. If one or two medium capacity drives stop working, you've got plenty more to provide redundancy.

I'm planning a RAID array of unplugged drives. I'd only plug the enclosure in when I'm doing a backup or reading files. It wouldn't be plugged in 24/7.

Same here. I only switch on my four TrueNAS Core arrays when copying new data, or running data scrubs. In addition, these servers are not connected to the Internet at any time. When copying files from a computer to a RAID-Z2 array, I use a separate 10GbE network isolated from the Internet. This wouldn't stop a Ransomware virus on a computer from locking the files on an isolated server, but it does stop webots finding my arrays.

Although it's labouring the point, RAID is for redundancy, NOT backups. If I lose one of my RAID-Z2 arrays in a catastrophic failure, I have three more to fall back on, plus the files on other computers, external drives, BD-R and LTO tapes.

As an example, I recently switched on a RAID-Z2 array and one of the six 6TB drives started generating SMART errors. The drives were purchased brand new and had only 6 days accumulated run time. Despite this, a drive had started to go bad. I replaced the drive and "resilvered" the array. I'm now waiting for the next drive failure in any machine. I have stacks of dead disks, accumulated over decades. All disks die eventually. Some earlier than others.

Sure, but with RAID 1, if a drive fails, you still have your data intact on the second one, right?

Oh, but how I wish this was always true. There are countless stories of people who trusted their data to RAID1, only to discover their files had gone. There are many reasons why RAID1 fails to keep data safe. The crucial thing to remember is you need multiple copies stored on different physical devices. If a RAID1 NAS power supply goes bang, it could destroy both drives. If Ransomware gets into the NAS, it will encrypt both copies. If one drive starts to go bad, corrupted files will be mirrored to the good drive.

 

[AusWolf]

Oh, but how I wish this was always true. There are countless stories of people who trusted their data to RAID1, only to discover their files had gone. There are many reasons why RAID1 fails to keep data safe. The crucial thing to remember is you need multiple copies stored on different physical devices. If a RAID1 NAS power supply goes bang, it could destroy both drives. If Ransomware gets into the NAS, it will encrypt both copies. If one drive starts to go bad, corrupted files will be mirrored to the good drive.

I see... So do you think it would be better to keep the data copied over to 2 drives with no RAID? Unfortunately, having such an extensive setup as yours is off the table due to financial reasons. Two 20 TB drives are expensive enough as they are.

 

[Solaris17]

Sure, but with RAID 1, if a drive fails, you still have your data intact on the second one, right?

People miss and confuse this all the time. Iv straight not hired people for this.

It’s not really a game or a philosophical debate. Hardware redundancy is just that. Data redundancy is a backup to another medium.

Raid in any form, does not give a shit, if your data is corrupted, or if you have a virus, or if it’s encrypted, if you accidentally delete it.

Raid is NOT a backup.

 

[Shou Miko]

If you want to RMA a Seagate drive it might take up to 6months just for the agents to actually get the RMA case started this is what happened to my previous X16 drive before I brought the X18 I have now.

Raid is NOT a backup.

I use this line earlier this week because a customer was talking about new server for NIS2 and I kept saying RAID ain't a backup just so they didn't forget.

The better RMA service for me has been Western Digital with their Gold series it haven't been as long as Seagate, but if you can live with that Seagate takes an age to get your RMA started because you cannot do it yourself go ahead

 

[AusWolf]

Raid is NOT a backup.

I kept saying RAID ain't a backup just so they didn't forget.

Okay, fine, RAID 1 isn't a backup solution. So what is it, then?

Please keep in mind that I'm not an IT specialist applying for a job, not even a client looking for company storage solutions. I'm just a random dude trying to store data safely, protected from hardware failure.

 

[Solaris17]

Okay, fine, RAID 1 isn't a backup solution. So what is it, then?

Please keep in mind that I'm not an IT specialist applying for a job, not even a client looking for company storage solutions. I'm just a random dude trying to store data safely, protected from hardware failure.

It’s just hardware redundancy. It doesn’t protect the data itself.

And it’s ok, this happens all the time to consumers or those that don’t work with data.
but for those that do and especially if you backup offsite the people storing this data very much make this distinction.

At a previous job one of my domains was our data systems. Just our client backup system was mmmm math 5.4Pb (peta bytes) per site x13 sites. I built it myself after the previous failed when its maintainer left. That’s a whole story.

But if I had those on raid 1,5,6,10,60 w/e doesn’t matter and I lost that array there data would straight up be gone.

Anyway got way off topic sorry OP

 

[Ferrum Master]

The golden rule. If you don't have a backup of your backup, you don't have a backup.

 

[AusWolf]

It’s just hardware redundancy. It doesn’t protect the data itself.

And it’s ok, this happens all the time to consumers or those that don’t work with data.
but for those that do and especially if you backup offsite the people storing this data very much make this distinction.

At a previous job one of my domains was our data systems. Just our client backup system was mmmm math 5.4Pb (peta bytes) per site x13 sites. I built it myself after the previous failed when its maintainer left. That’s a whole story.

But if I had those on raid 1,5,6,10,60 w/e doesn’t matter and I lost that array there data would straight up be gone.

Anyway got way off topic sorry OP

It's not off topic, as all this info has an influence on my buying decisions.

So basically, do you think storing data on two drives separately is better than RAID 1?

 

[Solaris17]

It's not off topic, as all this info has an influence on my buying decisions.

So basically, do you think storing data on two drives separately is better than RAID 1?

Storing data in two places is what you should do if you care about it regardless of if you use raid or not.

 

[Harlow]

Remember, you already have a single copy of your data spread over a number of hard disks. If you keep these original drives safe, you'll "only" have to copy it to two new devices, to stand a reasonable chance of keeping your precious files safe.

Well, the Toshiba is £310,

If you were considering buying two disks at £310 each and spending another £60+ on a RAID enclosure, you've got a pretty good budget of around £680. Even if each individual backup amounts to 20TB, that's still a reasonable budget.

How you divide up your £680 budget it up to you.
You could buy a couple of 20TB Toshiba MG10ACA20TE drives on Amazon for £309.67 each and bung them in good quality USB3 powered housings.
Some people have recommended Seagate Exos (£317.52 for 20TB on Amazon) or WD Gold (starting at £472 for 20TB, way outside your budget).

Alternatively, if you're prepared to buy hard disks refurbished by the manufacturer, Amazon are selling 10TB Seagate Enterprise Capacity v6 ST10000NM0046 drives for £98.71 each. At that price I might buy a couple myself. You could buy 40TB for less than £400, plus the USB housings, saving up to 200 quid.
https://www.amazon.co.uk/Seagate-En...eywords=10tb+hard+drive&qid=1728139424&sr=8-4

If you're put off by the thought of buying refurbished drives or used "pulls" from servers, remember one of my six 6TB Toshiba N300 NAS drives (purchased new) developed bad blocks in my HP ML350P server, after only 6 days total elapsed running time. Buying brand new doesn't guarantee long life. All drives die eventually, some sooner than others. If you want to see how well various drives survive, check out the Backblaze stats:
https://www.backblaze.com/blog/backblaze-drive-stats-for-q1-2024/
As an example, Backblaze had 37,676 16TB Toshiba MG08ACA16TA drives running for an aggregate total of 3,285,527 hours and 81 drives failed, average fail rate 0.91%.

Storage Solutions

In addition to TrueNAS servers, my data is copied to hard drives in a number of desktop PCs (more accurately, full towers). Do you own more than one desktop computer, with room inside for additional 3.5in hard disks? If so, you could dispense with the external USB3 housing(s) or RAID enclosure and spread your backup drives around at least three machines. You could repurpose an old machine or buy a cheap second hand Dell office PC, then use them as backup devices. There are lots of possibilities. Of course you'd need more room to keep backup computers, instead of a RAID or USB housing.

I'm thinking about replacing my internal hard drives with an external RAID 1 array of 20 TB enterprise drives for some extra data retention in case of failure.

The word "replacing" makes it sound as if you're going to copy the data off the old drives, then wipe them and use the disks for something else, but the term "extra data retention" implies you'll keep the old disks and data intact.

I'm just a random dude trying to store data safely, protected from hardware failure.

Likewise. My backups at home are a hodge-podge of disparate storage solutions, accumulated over the years. I started off saving programmes to cassette tape, next it was 5.25" and 3.5" floppy discs, then hard disks and finally back to (LTO) tape for some archives.
https://en.wikipedia.org/wiki/Linear_Tape-Open
Various LTO manufacturers claim you can store files on tape for up to 30 years, but you'd still need a working drive to read the tape back on.
https://www.stockroomlondon.com/2024/09/16/how-long-does-lto-storage-last/

Please keep in mind that I'm not an IT specialist applying for a job, not even a client looking for company storage solutions.

The reason forum members with more experience are trying to steer you clear of RAID as your only copy, is because no single device constitutes a backup. You need multiple copies on different devices/media.

RAID stands for Redundant Array of Independent (or Inexpensive) Disks. Nowhere in the title does it mention the word "Backup".
https://en.wikipedia.org/wiki/RAID

You can use RAID as a backup, provided it's not your only copy.

We're simply trying to help you avoid pitfalls other folk have fallen into, losing their irreplaceable files.

Good luck.

 

[unwind-protect]

If you are concerned about data corruption run ZFS. It has internal checksumming. Works on a plain drive or RAID-whateveryoulike.

Some NAS products run ZFS.

 

[AusWolf]

Remember, you already have a single copy of your data spread over a number of hard disks. If you keep these original drives safe, you'll "only" have to copy it to two new devices, to stand a reasonable chance of keeping your precious files safe.

I guess I worded it wrong, sorry.

The only thing I have a copy on several drives is my family photos. The rest of my data exists only as a single copy, some of it on drive A, some on drive B, and so on. It's not backed up anywhere else. So if drive A fails, then part of my data is lost, as I currently do not have any duplicates. This is what I'm planning to change.

Storage Solutions

In addition to TrueNAS servers, my data is copied to hard drives in a number of desktop PCs (more accurately, full towers). Do you own more than one desktop computer, with room inside for additional 3.5in hard disks? If so, you could dispense with the external USB3 housing(s) or RAID enclosure and spread your backup drives around at least three machines. You could repurpose an old machine or buy a cheap second hand Dell office PC, then use them as backup devices. There are lots of possibilities. Of course you'd need more room to keep backup computers, instead of a RAID or USB housing.

That's actually a good idea. I guess my HTPCs wouldn't mind some of their free space taken by some stuff. Still not enough for all of my data, though.

Likewise. My backups at home are a hodge-podge of disparate storage solutions, accumulated over the years. I started off saving programmes to cassette tape, next it was 5.25" and 3.5" floppy discs, then hard disks and finally back to (LTO) tape for some archives.
https://en.wikipedia.org/wiki/Linear_Tape-Open
Various LTO manufacturers claim you can store files on tape for up to 30 years, but you'd still need a working drive to read the tape back on.
https://www.stockroomlondon.com/2024/09/16/how-long-does-lto-storage-last/

I just looked at the technology... the tape seems cheap as heck, but man, those drives are expensive!

The reason forum members with more experience are trying to steer you clear of RAID as your only copy, is because no single device constitutes a backup. You need multiple copies on different devices/media.

RAID stands for Redundant Array of Independent (or Inexpensive) Disks. Nowhere in the title does it mention the word "Backup".
https://en.wikipedia.org/wiki/RAID

You can use RAID as a backup, provided it's not your only copy.

We're simply trying to help you avoid pitfalls other folk have fallen into, losing their irreplaceable files.

Good luck.

And I'm happy for all the advice here.

You guys have more experience in storage than I do, so if you say RAID, even RAID 1 isn't any better for data retention than several copies of the same data on several different drives, then I'll believe you.

 

[Harlow]

Some NAS products run ZFS.

Many thanks. That prompted me to run quick search for commercial NAS using ZFS.
https://www.qnap.com/en-us/operating-system/quts-hero

For the unititiated, ZFS is the underlying file system behind TrueNAS and Proxmox, in the same way NTFS lurks under Windows.
https://en.wikipedia.org/wiki/ZFS
https://en.wikipedia.org/wiki/NTFS

 

[AusWolf]

If you are concerned about data corruption run ZFS. It has internal checksumming. Works on a plain drive or RAID-whateveryoulike.

Some NAS products run ZFS.

What's ZFS? Do you have some resource I could read?

 

[Solaris17]

For a home consumer. That wants to try and explore all this I would propose:

1:
Raid 1 so data is still available if a drive dies
Backup that raid to an external drive that can hold the data.

2:
Raid 1 so data is still available if a drive dies
Backup that raid to a cloud service so you can recover.

every professional managing data will use that recipe to some degree. Usually following the 3-2-1 strategy.

data is sacred. Everything else can be replaced. That’s why it’s important for those to dispel that kinda thought is all.

For consumers though no need to go into crazy raid 10 multi disk systems. With a copy at your friends and a cloud backup. Though you can.

the two scenarios above are really what anyone with data they care about should already be doing. Since raid isn’t special . You can replace the work raid with “my hard drive” and it still applies.

What's ZFS? Do you have some resource I could read?

Not a replacement for any of the advice so far. Just another layer to the onion.

 

[Harlow]

I just looked at the technology... the tape seems cheap as heck, but man, those drives are expensive!

You'd need to sell your car to afford an LTO-9 tape drive, with prices starting around £4,000 and 18TB tapes from £90 upwards.
Older (second hand) LTO drives are much cheaper.

I moved into LTO-4 (800GB per tape native) back in 2018, when I bought a couple of Quantum external SAS tape drives for £85 each.
https://en.wikipedia.org/wiki/Serial_Attached_SCSI

Initially I purchased several boxes of brand new LTO-4 tapes (5 per box) for £12 to £15 per tape.
Then I got a really good deal on barely used tapes for £1.50 each and bought loads.

Finally I found a clunky full height clunky 5.25in LTO-4 SAS drive on eBay for £25.
Another £10 to £15 for a cheap LSI (manufacturer) HBA (Host Bus Adapter) PCIe SAS card on eBay and you're up and running.
https://docs.broadcom.com/doc/12352187

These days I'd opt for LTO-5 or later, because you can write files straight to tape from Windows File Manager using LTFS.
https://en.wikipedia.org/wiki/Linear_Tape_File_System

With older LTO-4, pre-LTFS, you need special backup software with an option to encode data on tape. Most non-Enterprise software does not support tape.

Your computer needs to be able to keep up with the demands of the tape drive, as you stream data on or off tape continuously.
My tape drive writes/reads at 80MB/s. It's OK with large video files, 50MB RAW and 15MB JPGs, but anything smaller and the tape "shoe shines".
https://community.spiceworks.com/t/...ausing-shoe-shining-should-i-zip-first/565423

LTO-4 is way, way behind the times and off the bottom of this chart, with a mere 800GB native, 1.6TB (theoretical) compressed capacity.
1.6TB 2-to-1 compression doesn't work with JPGs and RAW files, which are already compressed, so I use the native 800GB capacity.
Still, a single LTO-4 tape copes with a typical 600 to 700GB of snaps taken on holiday.
For LTO-5 and above, you'll need to stream data considerably faster to keep up with the tape. Zipping millions of small files into one huge file is a solution.

 

[Shrek]

Okay, fine, RAID 1 isn't a backup solution. So what is it, then?

Imagine you accidently replace a file without knowing and then copy it to your RAID (RAID 0 is not a RAID), now you have lost the file as there is no backup.

 

[bug]

Okay, fine, RAID 1 isn't a backup solution. So what is it, then?

Please keep in mind that I'm not an IT specialist applying for a job, not even a client looking for company storage solutions. I'm just a random dude trying to store data safely, protected from hardware failure.

Backup is an extra copy of your data. A RAID1 solution would be one extra copy, albeit with hardware redundancy, thus a bit more reliable than a bare drive. (See: https://unix.stackexchange.com/ques...-personal-desktop-backup-system-pros-and-cons)

A proper solution looks like this (though few can afford it): https://www.backblaze.com/blog/the-3-2-1-backup-strategy/

Also, just throwing this for completeness' sake: assuming we're not looking at 20TB of sensitive data, have you considered could storage? I know it's not ideal, but for the cost of one 20TB drive, you can rent that amount for years more than the physical drive would serve you.