Which 20 TB drive to get?

[chrcoluk]

Not really, NTFS has similar features built into it's journaling system. Lot's of people have been pushing ZFS as the new hotness, but it's really not. It's good and solid, but no more or less than NTFS. ZFS's main advantage is volume size and file size limits, which far exceeds NTFS and most other file systems. Only Ext4 is comparable in size limits. Sun envisioned a future where truly ginormous data arrays would be a thing and designed ZFS accordingly. That's it. All it's other features are very much on-par with other file systems. For Windows based systems, NTFS has the native support advantage and that fact can not be understated.


It's not new. It was first used on Sun Solaris workstations in 2005. That's nearly 2 decades ago. https://en.wikipedia.org/wiki/ZFS

REFS is the one thats more comparable to ZFS, NTFS has no bit rot detection.

Resilient File System (ReFS) overview

Learn more about: Resilient File System (ReFS) overview

learn.microsoft.com

Thats another option for you @AusWolf since Windows seems to be a requirement.

 

[lexluthermiester]

NTFS has no bit rot detection.

Not really a problem. Very niche feature..

ReFS is not officially supported for Windows 11 as a bootable partition.

 

[chrcoluk]

Not really a problem. Very niche feature..

ReFS is not officially supported for Windows 11 as a bootable partition.

Doesnt matter if its just a data partition, bit rot detection is one of the biggest talking points of modern file systems.

 

[AusWolf]

I'd say that's about right. I have 320GB/500GB laptop drives (Seagate and WD) from many moons ago and 30 MB/s is about what they do after the cache runs out during a sustained file write so by mixing very different drives your getting the poor performance from the least capable drive in the array.

Yeah, but that's not what the drive can do on its own. It can do 90 MB/s maintained all day and night, just not in the array.

Have you decided how you're going to arrange your storage yet?

I'm thinking about leaving the externals as they are. I've got my 8 and 4 TB drives in the enclosure as separate units, and I've got an 8 TB external drive next to them. That's 20 TB in total.
Then, if I get a single 20 TB internal drive, I can have 2 copies of my data - one internally on the enterprise drive, and one spread across the external ones.
Later, I could be looking at getting another 20 TB drive either for RAID 1, or just manually mirroring the two.

I don't think I have seen a laptop HDD go from 90 down to 30 MB/sec.

Are you sure this is not the fault of Windows software raid, or a general windows problem?

It could very well be, I'm not sure.

REFS is the one thats more comparable to ZFS, NTFS has no bit rot detection.

Resilient File System (ReFS) overview

Learn more about: Resilient File System (ReFS) overview

learn.microsoft.com

Thats another option for you @AusWolf since Windows seems to be a requirement.

Interesting. Am I gathering it right that server editions of Windows support it natively?

Windows is a requirement for now. It might not be in the future.

 

[lexluthermiester]

bit rot detection is one of the biggest talking points of modern file systems.

Ok, but how useful is it? Bitrot is not something the vast majority of users, even us tech junkies, are going to cross paths with but once in a blue-moon. I've only seen it twice in the last decade(with hard drives) and the fix was simple, long format with surface scan and done. It's something that should be built into a utility, not a file system.

Interesting. Am I gathering it right that server editions of Windows support it natively?

Sort of. ReFS is not currently bootable, unless I'm missing a recent development. Not that you need that with your DAS.

 

[AusWolf]

Sort of. ReFS is not currently bootable, unless I'm missing a recent development. Not that you need that with your DAS.

Exactly, I only need it for storage. I'd be daft to boot from HDD in late 2024 anyway.

 

[lexluthermiester]

I'd be daft to boot from HDD in late 2024 anyway.

I actually still do for one of my systems. It's my Lenovo Windows XP system. Granted it's a nice performing HDD so it's not an issue.

 

[bug]

I don't think I have seen a laptop HDD go from 90 down to 30 MB/sec.

They can do better than that.
I tested this once, copied two large files to the same HDD. Transfer speed plummeted 10x after it started to copy the second file. That is what HDD heads going back and forth will do. It is also the greatest advantage SSDs have over HDDs, but that's another story.

 

[Ruru]

I actually still do for one of my systems. It's my Lenovo Windows XP system. Granted it's a nice performing HDD so it's not an issue.

I also have a HDD on my retro gaming XP laptop. Actually pretty snappy as it's a 7200rpm one.

edit: Also on a C2D Win10 laptop which is next to my sofa. I use it to play media, when it has booted and loaded all stuff, it feels okay. 7200rpm drive as well.

 

[lexluthermiester]

I also have a HDD on my retro gaming XP laptop. Actually pretty snappy as it's a 7200rpm one.

edit: Also on a C2D Win10 laptop which is next to my sofa. I use it to play media, when it has booted and loaded all stuff, it feels okay. 7200rpm drive as well.

Exactly. Good performance HDD's are still perfectly acceptable as boot drives.

 

[azrael]

Must admit I haven't read all the posts (currently at work ) but for something something RAID I would always prefer RAID5 over RAID1. Does require a minimum of three drives though. Problem is, with large drives the possibility of bit rot also increases, meaning that there's a risk that another drive fails during the rebuild of the array, and while you can lose one, you can't lose two.

I can see someone mentioned ZFS. Doesn't that require ECC memory? Or have they dropped that requirement?

 

[Wasteland]

I can see someone mentioned ZFS. Doesn't that require ECC memory? Or have they dropped that requirement?

It never required ECC memory.

 

[azrael]

It never required ECC memory.

I see. You learn something new every day. I was pretty sure I've always heard that ECC was a requirement for using ZFS. Not that I would use anything else myself, but nice to know that you don't need to.

 

[unwind-protect]

Ok, but how useful is it? Bitrot is not something the vast majority of users, even us tech junkies, are going to cross paths with but once in a blue-moon. I've only seen it twice in the last decade(with hard drives) and the fix was simple, long format with surface scan and done. It's something that should be built into a utility, not a file system.

That's a strange stance. You already had bitrot twice and you want to risk having it again?

Keep in mind that a majority of users, even if they do backup at all, do one-shot backups to locally connected harddrives. Which are not only subject to bitrot themselves, but that scheme would overwrite good backups with rotted data before you notice the bitrot.

 

[chrcoluk]

Yeah, but that's not what the drive can do on its own. It can do 90 MB/s maintained all day and night, just not in the array.


I'm thinking about leaving the externals as they are. I've got my 8 and 4 TB drives in the enclosure as separate units, and I've got an 8 TB external drive next to them. That's 20 TB in total.
Then, if I get a single 20 TB internal drive, I can have 2 copies of my data - one internally on the enterprise drive, and one spread across the external ones.
Later, I could be looking at getting another 20 TB drive either for RAID 1, or just manually mirroring the two.


It could very well be, I'm not sure.


Interesting. Am I gathering it right that server editions of Windows support it natively?

Windows is a requirement for now. It might not be in the future.

If you think you might port over to a new OS later then ZFS might be better, but if you either dont mind rebuilding the raid later or see yourself using it on Windows for a while I would consider storage spaces with ReFS, of course play round with it first.

ReFS isnt as mature as NTFS though so I suggest checking compatibility with any software you might intend to use. Seems good on paper but I have no experience with using it.

ZFS is a much more known quantity but ReFS is natively in Windows.

Sorry for overloading you with ideas, I know there has been a ton of suggestions in the thread overall.

 

[bug]

Ok, but how useful is it? Bitrot is not something the vast majority of users, even us tech junkies, are going to cross paths with but once in a blue-moon. I've only seen it twice in the last decade(with hard drives) and the fix was simple, long format with surface scan and done. It's something that should be built into a utility, not a file system.

Every time you hear an audio file crackle or pop, every time you open an image and see square(ish) artifacts, that's bit rot. Happens a lot more than a couple of times per decade (though SSDs may be less susceptible since they move data around behind the scenes). And while the next write will get the cell back into working shape, that doesn't recover the files already corrupted.

 

[A Computer Guy]

It never required ECC memory.

Oh no pandora's box is open.

Yeah, but that's not what the drive can do on its own. It can do 90 MB/s maintained all day and night, just not in the array.

Your saying the 5400 rpm laptop drive will do 90MB/s sustained writes? If so then my laptop drives must be a lot older than I remember.

 

[lexluthermiester]

That's a strange stance. You already had bitrot twice and you want to risk having it again?

You missed some context. Twice in a decade. Here's something you might not know, I run a computer shop and handle thousands of drives a year. So for two older PC's to be brought into my shop needing service/upgrades having drives that need a fresh long-form formatting is not unexpected. It happens. Both of the systems in question had been sitting un-used for years. It's rare, but not unheard of. Such things do not justify the need for mitigation on a file-system level.

Every time you hear an audio file crackle or pop, every time you open an image and see square(ish) artifacts, that's bit rot.

No, those are bit errors. The notion that it's "bitrot" is an assumption and a very bad one..

Happens a lot more than a couple of times per decade

Moose muffins.

(though SSDs may be less susceptible since they move data around behind the scenes).

Please read more at the link below.

Understanding Bit Rot: Causes, Prevention & Protection | DataCore

This comprehensive guide explores bit rot, its causes, and most importantly, how to combat this common enemy of digital preservation.

www.datacore.com

 

[kapone32]

RAID 0 over software is the easiest way to create a RAID array. Just picked Stripped or Spanned. You cannot stripe a disk that already exists but you can span one in Windows. The absolute fastest drive for all applications is a card with a RAID 0 controller on the PCB. The WD AN 1500 is a RAID 0 based product. They don't seem to sell them anymore though.

 

[unwind-protect]

Interestingly I just logged into one of my rarely used laptops and ZFS warns me that 93 errors in files occurred. It also lists the files affected.

 

[mechtech]

Hi guys,

I'm thinking about replacing my internal hard drives with an external RAID 1 array of 20 TB enterprise drives for some extra data retention in case of failure. I have 3 options available at my no.1 go-to store, but having no experience with enterprise/NAS drives, I don't know which one is the best. I'm leaning towards the Toshiba, as that's the cheapest of the bunch, but I don't want to regret my decision later.

Performance isn't extremely important, but reliability is (two of such drives aren't cheap).

What do you think?

No vote for this??

https://www.newegg.com/p/2U3-0005-000M7

or 2 of these?

https://www.newegg.com/p/N82E16820363144

The IronWolf is made to be a NAS/RAID drive though.

 

[Bagerklestyne]

No vote for this??

https://www.newegg.com/p/2U3-0005-000M7

or 2 of these?

https://www.newegg.com/p/N82E16820363144

The IronWolf is made to be a NAS/RAID drive though.

I'll take 6 of the first one (when they're about 1/10th of that price)

True about the Ironwolf. Not mentioned though is the the Ironwolf is designed for smaller arrays whereas the pro's are designed for 8+ disk arrays with better balance so less 'harmonic' rotational vibration. Not that's it's a huge issue, but more of a mitigation of a possible array issue/crash.

 

[bug]

Interestingly I just logged into one of my rarely used laptops and ZFS warns me that 93 errors in files occurred. It also lists the files affected.

Lesson learned: don't log into rarely used laptops using ZFS. It corrupts your files

 

[unwind-protect]

Lesson learned: don't log into rarely used laptops using ZFS. It corrupts your files

Well, the laptop was hanging and needed a good powercycle. I think the hardware might be a bit tired

 

[AusWolf]

You missed some context. Twice in a decade. Here's something you might not know, I run a computer shop and handle thousands of drives a year. So for two older PC's to be brought into my shop needing service/upgrades having drives that need a fresh long-form formatting is not unexpected. It happens. Both of the systems in question had been sitting un-used for years. It's rare, but not unheard of. Such things do not justify the need for mitigation on a file-system level.


No, those are bit errors. The notion that it's "bitrot" is an assumption and a very bad one..

Moose muffins.

Please read more at the link below.

Understanding Bit Rot: Causes, Prevention & Protection | DataCore

This comprehensive guide explores bit rot, its causes, and most importantly, how to combat this common enemy of digital preservation.

www.datacore.com

Bit rot shouldn't be an issue if I have at least 2 copies of all data, and regularly rewrite them from one drive to another, right?

Are there any programs that can detect bit rot on NTFS to know which drive one should use to rewrite the other?