Which 20 TB drive to get?

[DaemonForce]

If I have concerns about bit rot, I do CRC checks. I have codes baked into half the stuff that needs to stay 100%.
Anything that can be replaced, usually gets auto-checked and verified through some other system designed to do exactly that.
If it passes, everything is good. If it fails, it looks for the problem chunk and replaces it from a 3rd party.
This system and the way it saves all of this unfixable media from disappearing is a huge part of why I believe in distribution.

 

[lexluthermiester]

Bit rot shouldn't be an issue if I have at least 2 copies of all data, and regularly rewrite them from one drive to another, right?

It shouldn't be an issue at all if your drives have been made within the last 12years and have been used within the last 5.

Are there any programs that can detect bit rot on NTFS to know which drive one should use to rewrite the other?

Most partitioning utilities have a surface scan function that will automagically detect and correct any data corruption, including "bit rot"(bit degradation). My fav is of AOMEI.

If I have concerns about bit rot, I do CRC checks. I have codes baked into half the stuff that needs to stay 100%.
Anything that can be replaced, usually gets auto-checked and verified through some other system designed to do exactly that.
If it passes, everything is good. If it fails, it looks for the problem chunk and replaces it from a 3rd party.
This system and the way it saves all of this unfixable media from disappearing is a huge part of why I believe in distribution.

You will not be affected in any way by bit-rot. It is extremely rare.

 

[AusWolf]

I shouldn't be an issue at all if your drives have been made within the last 12years and have been used within the last 5.


Most partitioning utilities have a surface scan function that will automagically defect and correct any data correction, including "bit rot"(bit degradation). My fav is of AOMEI.


You will not be affected in any way by bit-rot. It is extremely rare.

That's all good to know. I have EaseUS Partition manager which I use when cloning drives for OS migration. I'll have a look if it has surface scan functions.

How about HDD utilities like HD Tune Pro, or just a simple "chkdsk /f /r" command?

 

[lexluthermiester]

I have EaseUS Partition manager which I use when cloning drives for OS migration. I'll have a look if it has surface scan functions.

I believe it does..

How about HDD utilities like HD Tune Pro

No idea. Haven't used it in a long time.

or just a simple "chkdsk /f /r" command?

That's a good question. The /R function scans for bad sectors which would find & correct any bit degradation by either refreshing the bit marking it as bad. So yes, that would do the job.

 

[Nhonho]

No vote for this??

https://www.newegg.com/p/2U3-0005-000M7

or 2 of these?

https://www.newegg.com/p/N82E16820363144

The IronWolf is made to be a NAS/RAID drive though.

I would buy the one from the link below. The one you mentioned is SAS and the person would have to buy a SAS-PCIe controller to use it.

Since the SSD in the link below is U.3 (PCIe-NVMe), I think it's only necessary to buy an M.2 adapter and a cable (like this one: http://www.iocrest.com/index.php?id=2404) and it would work on a home PC.

https://www.newegg.com/micron-30-72-tb-9400/p/N82E16820363150

That's all good to know. I have EaseUS Partition manager which I use when cloning drives for OS migration. I'll have a look if it has surface scan functions.

How about HDD utilities like HD Tune Pro, or just a simple "chkdsk /f /r" command?

Have you already purchased the HDs?

I only buy HDs that have at least a 5-year manufacturer's warranty.

I also make a table in Excel and see which models have the lowest cost per GB and buy the one with the lowest cost per GB (from a reliable seller)...

 

[lexluthermiester]

No vote for this??

https://www.newegg.com/p/2U3-0005-000M7

or 2 of these?

https://www.newegg.com/p/N82E16820363144

The IronWolf is made to be a NAS/RAID drive though.

I would buy the one from the link below. The one you mentioned is SAS and the person would have to buy a SAS-PCIe controller to use it.

Since the SSD in the link below is U.3 (PCIe-NVMe), I think it's only necessary to buy an M.2 adapter and a cable (like this: http://www.iocrest.com/index.php?id=2404) and it would work on a home PC.
https://www.newegg.com/micron-30-72-tb-9400/p/N82E16820363150

Um, gentlemen.. The op has already bought both the enclosure and drives they're using. They're NOT interesting in drives that have $2500 & $5000+ price tags. Just throwing it out there..

Have you already purchased the HDs?

Yeah, welcome to the party!

I also make a table in Excel and see which models have the lowest cost per GB and buy the one with the lowest cost per GB (from a reliable seller)...

Seriously? Who does that for personal purchasing?

 

[AusWolf]

Have you already purchased the HDs?

I have bought an external enclosure, but I haven't bought any drive that the vote is about, although I'm leaning towards the Toshiba for the reason you mentioned (cost per GB). I'm probably gonna get one eventually, and use it as an internal drive, while the current external ones (8 TB and 8+4 TB in the enclosure) will serve as backup. This seems to be the most cost-efficient solution without sacrificing data redundancy.

As @lexluthermiester mentioned, I'm not interested in a several thousand USD/GBP solution. I only want to safely store my personal data with adequate data redundancy in case of drive failure.

 

[unwind-protect]

Bit rot shouldn't be an issue if I have at least 2 copies of all data, and regularly rewrite them from one drive to another, right?

Are there any programs that can detect bit rot on NTFS to know which drive one should use to rewrite the other?

You point out the problem right there. Potentially overwriting a good backup with a now-corrupted master version, whatever the reason for the master (workstation) copy being bad might be.

One way is to do checksumming yourself. You keep a database of sha checksums for files somewhere and execute a new backup only after you re-check those checksums.

fsck and friends (filesystem checkers) won't detect problems within files that didn't affect metadata.

 

[Wasteland]

Bit rot shouldn't be an issue if I have at least 2 copies of all data, and regularly rewrite them from one drive to another, right?

Are there any programs that can detect bit rot on NTFS to know which drive one should use to rewrite the other?

Backup software will often handle checksumming on the destination automatically. I mentioned Borg/Vorta in the other thread, for example. If you're using something like that, then as @unwind-protect suggested, your only concern becomes the source storage, on which you can save a checksum for periodic checking. There are any number of GUI tools that you can use for this purpose, 7zip for example, TeraCopy on Windows, Double Commander. But even this step is probably gratuitous for most people, these days.

Even as an avowed ZFS cultist, I wouldn't worry about bit rot on any data that isn't absolutely critical, and by critical I mean the sort of thing the loss of which might threaten your livelihood. You probably don't have any data that fits that description. I certainly don't. If you did, then (I hope) we wouldn't be having this conversation.

As I said earlier, any system you put in place is only as good as your ability to maintain it. This is true in all areas of life. @Solaris17 nailed it earlier: if you're making regular back ups to an external device, you're already better off than overwhelming majority of the population. If you want to go beyond that, then great; game on, but you should do it because you want to, not because you're tying yourself in knots over what are manifestly trivial concerns for 99% of the global userbase.

 

[AusWolf]

Backup software will often handle checksumming on the destination automatically. I mentioned Borg/Vorta in the other thread, for example. If you're using something like that, then as @unwind-protect suggested, your only concern becomes the source storage, on which you can save a checksum for periodic checking. There are any number of GUI tools that you can use for this purpose, 7zip for example, TeraCopy on Windows, Double Commander. But even this step is probably gratuitous for most people, these days.

It sounds interesting, I'll look into it. TeraCopy looks nice at first glance. Things like Borg are a no-go. I really can't be asked with CLI on any OS.

How does a checksum work?

Even as an avowed ZFS cultist, I wouldn't worry about bit rot on any data that isn't absolutely critical, and by critical I mean the sort of thing the loss of which might threaten your livelihood. You probably don't have any data that fits that description. I certainly don't. If you did, then (I hope) we wouldn't be having this conversation.

I'd say family photos are pretty critical. Beyond that, I agree with you.

As I said earlier, any system you put in place is only as good as your ability to maintain it. This is true in all areas of life. @Solaris17 nailed it earlier: if you're making regular back ups to an external device, you're already better off than overwhelming majority of the population. If you want to go beyond that, then great; game on, but you should do it because you want to, not because you're tying yourself in knots over what are manifestly trivial concerns for 99% of the global userbase.

Oh definitely.

 

[Harlow]

I'd say family photos are pretty critical.

Have you considered printing out a few of the most important photos for your descendants?
A box full of prints might outlive your digital images, if the hardware required to view them becomes obsolete.

https://www.bhphotovideo.com/explor...uide-to-archiving-and-storing-film-and-prints

 

[AusWolf]

Have you considered printing out a few of the most important photos for your descendants?
A box full of prints might outlive your digital images, if the hardware required to view them becomes obsolete.

https://www.bhphotovideo.com/explor...uide-to-archiving-and-storing-film-and-prints

I have printed a few, yes. But the thing is, printed photos fade after a while, just like digital ones go through bit rot. Printing doesn't keep your memories for life, either. If I keep the digital copies on multiple drives, though, I'll always have at least one good copy of each. And if the drive becomes obsolete (let's say, SATA connectors aren't a thing anymore), I can just move it all to a different drive.

 

[Harlow]

printed photos fade after a while

This article claims that prints made with pigment inks (on appropriate paper) can last up to 200 years, when kept under ideal conditions, e.g. in a light proof box (see below):

Pigment inks, on the other hand, use fine particles suspended in liquid and are much less susceptible to the same issues. This gives some formulations ratings of more than 200 years.
https://www.bhphotovideo.com/explora/photography/tips-and-solutions/what-is-an-archival-print

Occasionally, I print out photographs on roughly 400 sheets of A4 glossy paper using inexpensive third-party (non-Canon) ink tanks. After trimming, I laminate the prints and hand them over to the people I've photographed. A protective layer reduces exposure to ultra violet light. My laminated prints seem to last for years, but most of the time they're kept out of direct or indirect sunlight. For individual prints, mounting under glass in a frame is probably the best idea, if you want to keep the image out on display.

I can just move it all to a different drive.

That's certainly the best idea.