# Critical Flaw in Windows 10 Could Corrupt Your Hard Drive



## AleksandarK (Jan 18, 2021)

Windows OS security is taken seriously, as the OS is wide-spread across millions of PCs around the world, however, there may be issues where OS has some security flaw that is found by external researchers. Due to the sheer code base of the new OS like Windows 10, there are a plethora of bugs and security flaws waiting to get discovered by someone. And today, thanks to the team of cybersecurity researchers, we have found out that in Windows 10 file-system called NTFS, there is a bug that corrupts your hard drive by simply triggering a specific variable name in a file.

If the end-user inside Windows 10 tries to access the NTFS attribute called "$i30" in a specific way, the flaw is exploited. The NTFS search index attribute, specifically the string "$i30", is containing a list of files and subfolders in a directory, and there is even a log of deleted files and folders. After running a specific command inside the command line (CMD) or inside the browser, Windows will start to display warnings of "File or directory is corrupted and cannot be read". After that, the OS will prompt a user to restart the machine and repair the damaged drive, so the Windows disk check utility will start. Once corrupted, Windows 10 will start displaying a notification indicating that the main file table (MFT) on the particular disk is corrupted and thus can not operate. Starting from the build Windows 10 Build 1803 the OS is vulnerable until the current version and a possible fix is expected to be released soon.


 



*View at TechPowerUp Main Site*


----------



## ThrashZone (Jan 18, 2021)

Hi,
Yeah I miss 1709.


----------



## FreedomEclipse (Jan 18, 2021)

Windows 10 - the gift that keeps on giving.


----------



## N3utro (Jan 18, 2021)

Instead of giving specific details of a 0day problem that could lead to some people exploiting it, you could just be more vague and wait for a fix.

You're article is basically an invitation for hackers to have fun with this. Shame on you.


----------



## TheoneandonlyMrK (Jan 18, 2021)

N3utro said:


> Instead of giving specific details of a 0day problem that could lead to some people exploiting it, you could just be more vague and wait for a fix.
> 
> You're article is basically an invitation for hackers to have fun with this. Shame on you.


It's widely reported the same?!


----------



## Vayra86 (Jan 18, 2021)

N3utro said:


> Instead of giving specific details of a 0day problem that could lead to some people exploiting it, you could just be more vague and wait for a fix.
> 
> You're article is basically an invitation for hackers to have fun with this. Shame on you.


Your.

Shame on you


----------



## Fouquin (Jan 18, 2021)

N3utro said:


> Instead of giving specific details of a 0day problem that could lead to some people exploiting it, you could just be more vague and wait for a fix.
> 
> You're article is basically an invitation for hackers to have fun with this. Shame on you.



This has been reported on since August 2020. This is not even remotely news. This 'vulnerability' is so incredibly dangerous that chkdsk can just fix it on boot...


----------



## wurschti (Jan 18, 2021)

ThrashZone said:


> Hi,
> Yeah I miss 1709.


same... I hate it that MS force updates the OS even if most of what the new shit we get is useless features.


----------



## windwhirl (Jan 18, 2021)

3rold said:


> I hate it that MS force updates the OS even if most of what the new shit we get is useless features.


I don't mind the forced updates much, but I wish they'd invest more money in QA and bug fixing.


----------



## londiste (Jan 18, 2021)

This quite definitely does not damage a hard drive.
It corrupts NTFS file system. The degree of that corruption is not clear. It appears that in many or most cases, running chkdsk as prompted will fix the corruption.

There are also some reports saying this can corrupt file system so that chkdsk will fail to fix it and result in bluescreen at boot. At the same time I have seen links to an existing chkdsk issue that reportedly causes these boot failures so it is possible that failure to boot is a combination of the two.


----------



## lemonadesoda (Jan 18, 2021)

3rold said:


> same... I hate it that MS force updates the OS even if most of what the new shit we get is useless features.


Yep. I have another unwanted forced update last night.

And YES I have done all the registry policy and O&O shut ups to stop forced update. But somehow, MS, automagically does it anyway.


----------



## ThrashZone (Jan 18, 2021)

3rold said:


> same... I hate it that MS force updates the OS even if most of what the new shit we get is useless features.


Hi,
1709 was the best performer benchmark wise.



lemonadesoda said:


> Yep. I have another unwanted forced update last night.
> 
> And YES I have done all the registry policy and O&O shut ups to stop forced update. But somehow, MS, automagically does it anyway.


I use update mini tool it hasn't failed me yet.


----------



## windwhirl (Jan 18, 2021)

lemonadesoda said:


> Yep. I have another unwanted forced update last night.
> 
> And YES I have done all the registry policy and O&O shut ups to stop forced update. But somehow, MS, automagically does it anyway.


Microsoft: "You can not stop what is to come"

Seriously, though, maybe there's some hidden subroutine that verifies that Windows Update is up and running? 

Heck, at this point I'm surprised Microsoft hasn't really locked down WU.


----------



## TheUn4seen (Jan 18, 2021)

N3utro said:


> Instead of giving specific details of a 0day problem that could lead to some people exploiting it, you could just be more vague and wait for a fix.
> 
> You're article is basically an invitation for hackers to have fun with this. Shame on you.


You get really worked up over nothing. First, this was reported many months ago, second, it does nothing. Windows is, at it's core, layers of hastily written and slapped together code, this is just part of NTFS index which gets exposed to everything, including the browser. Laziness on Microsoft's part, yes. Dangerous in itself, not really. Unless you start digging and checking what else is exposed in a similar way...


----------



## 1d10t (Jan 18, 2021)

I already disable indexing option along with file history. I just don't like 'em.


----------



## DeathtoGnomes (Jan 18, 2021)

FreedomEclipse said:


> Windows 10 - the gift that keeps on giving.


I'm sorry I just cannot tell if you are being sarcastic or not, but here let me fix that for you..



> Windows 10Microsoft - the gift that keeps on giving.


  

--
It a good thing to delay updates, althought windows finally added a timed delay, nothing beats an indefinite delay like a good firewall can provide.


----------



## qubit (Jan 18, 2021)

Damn, this is bad, probably one of the worst, as the potential cost in terms of money and other ways of data loss is huge. Thankfully, this kind of problem is rare.


----------



## bigdogge (Jan 18, 2021)

ThrashZone said:


> Hi,
> Yeah I miss 1709.


Wasn't 1709 the one with that bug in where file explorer windows would snap to horizontal right when selecting multiple files? Or was it 1709 that fixed that bug whilst simultaneously resetting your default browser to Edge?


----------



## ThrashZone (Jan 18, 2021)

bigdogge said:


> Wasn't 1709 the one with that bug in where file explorer windows would snap to horizontal right when selecting multiple files? Or was it 1709 that fixed that bug whilst simultaneously resetting your default browser to Edge?


Hi,
Don't remember those bugs
I just benchmark with 10 and 1709 did it best.


----------



## laszlo (Jan 18, 2021)

nice "feature" ... for sure those scam support services will use this...


----------



## AltCapwn (Jan 18, 2021)

Can I post the command line? 




Don't try it on your PC.


----------



## Vayra86 (Jan 18, 2021)

Oh this is going to be a LOT of fun at work tomorrow.

No I won't.
Probably


----------



## AltCapwn (Jan 18, 2021)

Vayra86 said:


> Oh this is going to be a LOT of fun at work tomorrow.
> 
> No I won't.
> Probably


The new gag if someone left its PC unlocked.


----------



## zlobby (Jan 18, 2021)

N3utro said:


> Instead of giving specific details of a 0day problem that could lead to some people exploiting it, you could just be more vague and wait for a fix.
> 
> You're article is basically an invitation for hackers to have fun with this. Shame on you.


Nah, that's not how progress is made. Besides, if one doesn't have a backup plan for a corrupted drive, one probably deserves a corrupted drive.


----------



## sam_86314 (Jan 18, 2021)

3rold said:


> same... I hate it that MS force updates the OS even if most of what the new shit we get is useless features.


I use WuMgr and a script I wrote to disable automatic updates.

By default, I have the update services disabled. The script enables them, launches WuMgr, and waits for it to be closed. Then it disables the services again.

Aside from some issues WuMgr has, this solution has been working fine for me.

If only I knew C# so I could take over development on WuMgr...


----------



## lexluthermiester (Jan 18, 2021)

N3utro said:


> Instead of giving specific details of a 0day problem that could lead to some people exploiting it, you could just be more vague and wait for a fix.
> 
> *Your* article is basically an invitation for hackers to have fun with this. Shame on you.


Really? So you think hackers who want to exploit this problem are going to look up TechPowerUp for hacking info? Hmmm? 

Think about that very carefully while you ponder your next condescending comment.


----------



## R-T-B (Jan 18, 2021)

N3utro said:


> Instead of giving specific details of a 0day problem that could lead to some people exploiting it, you could just be more vague and wait for a fix.
> 
> You're article is basically an invitation for hackers to have fun with this. Shame on you.


That's not how modern security reporting works.  We must operate on the principal that knowledge is power, and the bad guys already surely have this knowledge.


----------



## efikkan (Jan 18, 2021)

AleksandarK said:


> Windows OS security is taken seriously, as the OS is wide-spread across millions of PCs around the world, however, there may be issues where OS has some security flaw that is found by external researchers. *Due to the sheer code base of the new OS like Windows 10*, there are a plethora of bugs and security flaws waiting to get discovered by someone. And today, thanks to the team of cybersecurity researchers, we have found out that in Windows 10 file-system called NTFS, there is a bug that corrupts your hard drive by simply triggering a specific variable name in a file.


When simply typing one file path in user space causes file system corruption, and typing another causes instant BSODs, then there are fundamental design flaws in the system.
The size code base is irrelevant, such problems are inexcusable. These problems has plagued Windows for decades, and will continue to do so until MS does a complete overhaul of the kernel. Patchwork can't solve this.



lemonadesoda said:


> Yep. I have another unwanted forced update last night.
> 
> And YES I have done all the registry policy and O&O shut ups to stop forced update. But somehow, MS, automagically does it anyway.


What about setting the network connection as metered and disable updates on a metered connection?


----------



## Xuper (Jan 18, 2021)

I am experienced them every one or two week.I thought It's from sleep mode.


----------



## lexluthermiester (Jan 18, 2021)

efikkan said:


> When simply typing one file path in user space causes file system corruption, and typing another causes instant BSODs, then there are fundamental design flaws in the system.
> The size code base is irrelevant, such problems are inexcusable.


Could not agree more here...


efikkan said:


> and will continue to do so until MS does a complete overhaul of the kernel.


...but not with this. What MS needs to do is slow things down development-wise and refine, refine, refine.


----------



## ThrashZone (Jan 18, 2021)

Hi,
Wonder if linux pukes too


----------



## lexluthermiester (Jan 18, 2021)

ThrashZone said:


> Hi,
> Wonder if linux pukes too


No, this is exclusively a Windows problem, AFAIK.


----------



## ixi (Jan 18, 2021)

FreedomEclipse said:


> Windows 10 - the gift that keeps on giving.


Make better OS.


----------



## efikkan (Jan 18, 2021)

lexluthermiester said:


> ...but not with this. What MS needs to do is slow things down development-wise and refine, refine, refine.


Fundamental design flaws in the kernel, driver model and the file system can't be solved with just refinements, a total overhaul of the NT kernel is required. It should not be possible to damage or comprimise a system like this from user space, yet Windows has a seemlingly endless stream of such bugs.


----------



## windwhirl (Jan 18, 2021)

efikkan said:


> Fundamental design flaws in the kernel, driver model and the file system can't be solved with just refinements, a total overhaul of the NT kernel is required.


Off-topic, but I kinda been wanting to ask Microsoft (hah, if only) why they never pushed for more than just two privilege rings after all the other platform compatibility plans (Alpha, PowerPC, MIPS, etc.) got scrapped, considering there were some opportunities here and there (probably Vista with its massive changes would have been the best time to just push all the pain at once and be done with it). I know ARMv7 has three levels, while x86 has four at least.


----------



## trparky (Jan 18, 2021)

Outside of academic or small open source projects, there's not been a new operating system kernel written in decades. Writing a new kernel from scratch is a monumental task and one that someone doesn't just wake up one morning and decide "Oh, I'm going to write a new operating system kernel today". That just doesn't happen. Even the Linux kernel is twenty years old (or more) and has been hacked on and hacked on for just as long. Sure, there's been some big changes inside the Linux kernel but largely it's the same stuff. I highly doubt that Linus Torvalds would say to rewrite the whole damn kernel and if you tried to tell him to do that, he'd tell you where to go and how to get there in a very swear word laden message.

Oh, and by the way, I watched a YouTube video where a malformed NTFS file system crashes even Linux, BSD, and MacOSX. Simply plugging in a USB drive with said malformed NTFS file system will crash even those operating systems.


----------



## Night (Jan 18, 2021)

Honestly this is useful info, if you run a simple batch script without knowing what this command does, you get corrupt NTFS. I used to write these scripts and to be honest this doesn't seem malicious at first, maybe just a bit suspicious. Useful if you like to check the scripts before you run them.


----------



## 80-watt Hamster (Jan 18, 2021)

Night said:


> ... I used to write these scripts and to be honest this doesn't seem malicious at first, maybe just a bit suspicious ...



*looks at Night's avatar* 

I am not filled with confidence.
.
.
.


----------



## AusWolf (Jan 18, 2021)

"If the end-user inside Windows 10 tries to access the NTFS attribute called "$i30" in a specific way..."

Why would I even want to do that?


----------



## lexluthermiester (Jan 19, 2021)

efikkan said:


> Fundamental design flaws in the kernel, driver model and the file system can't be solved with just refinements, a total overhaul of the NT kernel is required.


That is an opinion not everyone agrees with. For example Windows 7 does not suffer from the problem detailed in the article above. Most of the problems with Windows 10 are solvable without over-hauling the kernel. Microsoft just needs to take the time to actually do it.


----------



## R-T-B (Jan 19, 2021)

The NT kernel is actually an incredibly cool piece of tech that dates back to the Microsoft collaborative effort on OS/2 with IBM.  It's not something that needs a redesign, it's always being tweaked this way or that.  It's very adaptable.


----------



## windwhirl (Jan 19, 2021)

trparky said:


> Outside of academic or small open source projects, there's not been a new operating system kernel written in decades. Writing a new kernel from scratch is a monumental task and one that someone doesn't just wake up one morning and decide "Oh, I'm going to write a new operating system kernel today". That just doesn't happen. Even the Linux kernel is twenty years old (or more) and has been hacked on and hacked on for just as long. Sure, there's been some big changes inside the Linux kernel but largely it's the same stuff. I highly doubt that Linus Torvalds would say to rewrite the whole damn kernel and if you tried to tell him to do that, he'd tell you where to go and how to get there in a very swear word laden message.


I imagine the main reason why new kernels don't really take off is software compatibility. Not many want to deal with all the main kernels of this era, that is Linux, Windows and whatever macOS is (dare I mention BSD, too?). If they had to deal with a thousand others, with very different concepts each of how to handle things (plus those microkernels that decidedly do not handle some stuff at all), devs would go mad. Heck, how many care to go outside of whatever main platform they use/code for?

That is besides the fact that new kernels need a lot of work, which you clearly stated.



R-T-B said:


> The NT kernel is actually an incredibly cool piece of tech that dates back to the Microsoft collaborative effort on OS/2 with IBM.  It's not something that needs a redesign, it's always being tweaked this way or that.  It's very adaptable.


You know, if I had the programming/coding knowledge for it, I'd love to have a one-on-one with the people behind the NT 3.1 kernel and ask perhaps even the silliest questions about how it came to be. I honestly would feel like the child that it's being told a fantasy story with how I'd be so invested in that


----------



## sam_86314 (Jan 19, 2021)

Looks like there's another similar issue.


















						Windows 10 bug crashes your PC when you access this location
					

A bug in Windows 10 causes the operating system to crash with a Blue Screen of Death simply by opening a certain path in a browser's address bar or using other Windows commands.




					www.bleepingcomputer.com
				




Entering the following string into cmd or a web browser will cause a BSOD (Try this at your own risk, preferably in a VM).







Similar to the NTFS issue, this doesn't do anything in versions older than 1709. Even Windows 7 and XP are unaffected. Kinda makes me wonder what M$ changed that would cause this.

M$ needs to get their shit together.

EDIT: Apparently the NTFS flaw in the article isn't as severe as people think it is.










I guess it corrupts some metadata file somewhere, which causes the entire FS to be marked as dirty, which triggers the chkdisk scan. Accessing other $i30 attributes doesn't do anything.

Also apparently it works in XP, but not 7.

EDIT 2: Just tried both flaws in a Win10 1809 VM. The BSOD one caused the VM to lock up and restart, no BSOD.

The NTFS one caused chkdisk to run upon rebooting, and pretty much nothing else happened. The VM started right back up as if nothing happened. Interestingly I didn't get any notifications that anything happened after running the command. Cmd returned that the file was corrupted, and I also manually checked the disk for errors, and it said it needed to repair the disk. I also ran SFC after rebooting and it passed with no errors.

Pretty sure the NTFS one is mostly harmless (save for maybe some extra wear on your boot drive), and the BSOD one carries any risks associated with system crashes.


----------



## Frick (Jan 19, 2021)

3rold said:


> same... I hate it that MS force updates the OS even if most of what the new shit we get is useless features.





lemonadesoda said:


> Yep. I have another unwanted forced update last night.
> 
> And YES I have done all the registry policy and O&O shut ups to stop forced update. But somehow, MS, automagically does it anyway.



I'm all for MS forcing people like you to update. If people had the option they would just never update machines, not even for security.


----------



## 300BaudBob (Jan 19, 2021)

trparky said:


> Outside of academic or small open source projects, there's not been a new operating system kernel written in decades. Writing a new kernel from scratch is a monumental task and one that someone doesn't just wake up one morning and decide "Oh, I'm going to write a new operating system kernel today". That just doesn't happen. Even the Linux kernel is twenty years old (or more) and has been hacked on and hacked on for just as long. Sure, there's been some big changes inside the Linux kernel but largely it's the same stuff. I highly doubt that Linus Torvalds would say to rewrite the whole damn kernel and if you tried to tell him to do that, he'd tell you where to go and how to get there in a very swear word laden message.
> 
> Oh, and by the way, I watched a YouTube video where a malformed NTFS file system crashes even Linux, BSD, and MacOSX. Simply plugging in a USB drive with said malformed NTFS file system will crash even those operating systems.


Periodically I enjoy writing up specs for an all new OS.  Just for fun...a what if I could do this.  Of course I don't have a thousand years to write it or the money to hire a team to shorten the time.  Just a crazy old coder with delusions of grandeur.


----------



## rtwjunkie (Jan 19, 2021)

N3utro said:


> Instead of giving specific details of a 0day problem that could lead to some people exploiting it, you could just be more vague and wait for a fix.
> 
> You're article is basically an invitation for hackers to have fun with this. Shame on you.


You don’t realize this information has already been published? The sources are on the main page.


----------



## DeathtoGnomes (Jan 19, 2021)

lexluthermiester said:


> and refine, refine, refine.


not something m$ does easily, its not like they listen to their community much or read their own support forums.


----------



## Arc1t3ct (Jan 19, 2021)

I really like Windows 10. I hope they fix this soon.


----------



## Octopuss (Jan 19, 2021)

Over at Guru3D, they wrote this is been reported since 2018 or something. What the actual F if true?


----------



## Aquinus (Jan 19, 2021)

I feel like my decision to use just Linux and Mac OS is becoming more and more justified as time goes on.


sam_86314 said:


> Pretty sure the NTFS one is mostly harmless (save for maybe some extra wear on your boot drive), and the BSOD one carries any risks associated with system crashes.


Haha. No. When NTFS disks aren't unmounted cleanly, a bit doesn't get set on the disk which indicates that there wasn't a clean shutdown. If you BSOD and your install gets wrecked, that bit doesn't get set and if you try to go into Linux and mount the filesystem, it won't do it because that bit isn't set and fixing it without Windows is a freaking nightmare. Not to mention that it's scary because you're writing raw data to the disk with something like dd to fix it. This was literally the last nail in the coffin when I actively chose to stop using Windows.

NTFS is hot garbage when you compare it to options like APFS, ext4, btrfs, and f2fs.


----------



## lexluthermiester (Jan 19, 2021)

Frick said:


> I'm all for MS forcing people like you to update. If people had the option they would just never update machines, not even for security.


Gotta disagree with you there. It's not your place or even Microsoft's place to tell people what they have to do with their own PC's.



Octopuss said:


> Over at Guru3D, they wrote this is been reported since 2018 or something. What the actual F if true?


To be fair, how many people are A. Going to actually know enough to have the ability to type that command, and B. Will actually do it?



Aquinus said:


> NTFS is hot garbage when you compare it to options like APFS, ext4, btrfs, and f2fs.


That is both an unfair and inaccurate comparison. Let's stay within the realm of reality shall we..


----------



## HD64G (Jan 19, 2021)

So much so for a big company that stopped supporting their previous OS before the one forced on many users get out of the beta stage...


----------



## ThrashZone (Jan 19, 2021)

Frick said:


> I'm all for MS forcing people like you to update. If people had the option they would just never update machines, not even for security.


Hi,
Security for one of their own lame apps I don't use and likely would love to uninstall if I haven't already lol



HD64G said:


> So much so for a big company that stopped supporting their previous OS before the one forced on many users get out of the beta stage...


Hi,
10 is never out of beta stage is one beta after another.


----------



## Muck Muster (Jan 19, 2021)

3rold said:


> same... I hate it that MS force updates the OS even if most of what the new shit we get is useless features.


Yeah.. .  my temp files get piled with useless installers. Though my machine glides like a champ with them hangin' on, I muse at finding 1 or 2gb of useless installers loungin' around my computer.


----------



## lemonadesoda (Jan 19, 2021)

Frick said:


> I'm all for MS forcing people like you to update. If people had the option they would just never update machines, not even for security.


Enforce a machine to be updated to a more stable build. I agree with that concept.

MS externally Forcing an untimely reboot on a private machine, without owner agreeing to it, to a new OS build, causing loss of time and work and potentially farking a machine. Do not agree with that. It’s pretty shady ethically and legally. 

So, the question should be, what method could be designed to enforce the updating of machines without forcing reboots that have not been agreed to. There are a thousand different ways to make that work.


----------



## Aquinus (Jan 20, 2021)

lexluthermiester said:


> That is both an unfair and inaccurate comparison. Let's stay within the realm of reality shall we..


Then perhaps you should enlighten me. What's so great about NTFS where it excels beyond APFS, ext4, btrfs, and f2fs?


----------



## Frick (Jan 20, 2021)

ThrashZone said:


> Hi,
> Security for one of their own lame apps I don't use and likely would love to uninstall if I haven't already lol



Lame apps like the 0 day stuff they've found in Windows itself?



lexluthermiester said:


> Gotta disagree with you there. It's not your place or even Microsoft's place to tell people what they have to do with their own PC's.



It shouldn't be. Like how it shouldn't be up to manufacturers of anything to enforce any security for anything, but the thing is that attitude is actively bad for us. Botnets wouldn't be a problem if people was sensible about security, but they aren't, so someone has to be.


----------



## AusWolf (Jan 20, 2021)

Aquinus said:


> Then perhaps you should enlighten me. What's so great about NTFS where it excels beyond APFS, ext4, btrfs, and f2fs?


I don't think the question is what makes NTFS great, but what makes it so much worse like you described. I for one, have never had any issues with NTFS. The incorrect shutdown issue can be resolved by using the safe removal feature, or shutting your PC down before disconnecting your drives. Each of these takes 2 clicks maximum.



lemonadesoda said:


> Enforce a machine to be updated to a more stable build. I agree with that concept.
> 
> MS externally Forcing an untimely reboot on a private machine, without owner agreeing to it, to a new OS build, causing loss of time and work and potentially farking a machine. Do not agree with that. It’s pretty shady ethically and legally.
> 
> So, the question should be, what method could be designed to enforce the updating of machines without forcing reboots that have not been agreed to. There are a thousand different ways to make that work.


There are many options in Windows 10 already. Active hours, deferring updates, custom reboot time once your uptades have been installed...


----------



## ThrashZone (Jan 20, 2021)

Frick said:


> Lame apps like the 0 day stuff they've found in Windows itself?
> 
> It shouldn't be. Like how it shouldn't be up to manufacturers of anything to enforce any security for anything, but the thing is that attitude is actively bad for us. Botnets wouldn't be a problem if people was sensible about security, but they aren't, so someone has to be.


Hi,
Yet I'm still on win-7 and the world hasn't ended without that patch plus any spectre/ meltdown nonsense either


----------



## DeathtoGnomes (Jan 20, 2021)

Frick said:


> Botnets wouldn't be a problem if people was sensible about security, but they aren't,


Most people ignore any warnings about security, hell, most are not even taught about basic security, there are those that just  lack common sense. The question is, is it m$'s responsibility teach or just plain warn us?? 

 It usually winds up being too late to do anything  about the brick sitting next to the lava lamp you have sitting on your RGB embedded computer desk, guess they'll learn next time and be more  sensible, right?


----------



## Frick (Jan 20, 2021)

DeathtoGnomes said:


> Most people ignore any warnings about security, hell, most are not even taught about basic security, there are those that just  lack common sense. *The question is, is it m$'s responsibility teach or just plain warn us??*
> 
> It usually winds up being too late to do anything  about the brick sitting next to the lava lamp you have sitting on your RGB embedded computer desk, guess they'll learn next time and be more  sensible, right?



That's the question, and it's not an easy one. I mean yeah MS deserves flak for pushing bad updates (and drivers that bricked computers...) but I really think it's the lesser of evils, the other one being people just not updating. I'm sure you've seen unupdated XP computers riddled with nasty stuff and the user running everything as admin by default.


----------



## lexluthermiester (Jan 20, 2021)

Aquinus said:


> Then perhaps you should enlighten me. What's so great about NTFS where it excels beyond APFS, ext4, btrfs, and f2fs?


How about Windows won't run on them? Or perhaps that NTFS is very stable, reliable and nearly error proof? And before you mention that drive disconnection thing, that it a Windows problem, not an NTFS problem.



DeathtoGnomes said:


> The question is, is it m$'s responsibility teach or just plain warn us??


No, it is not. But it is also not their place to tell us what, when and how we may or may not use our own PC's.



Frick said:


> It shouldn't be. Like how it shouldn't be up to manufacturers of anything to enforce any security for anything, but the thing is that attitude is actively bad for us. Botnets wouldn't be a problem if people was sensible about security, but they aren't, so someone has to be.


That requires education of the public, not strong-arm enforcement of rules that don't always make sense are certainly do not respect our rights as users.


----------



## lemonadesoda (Jan 20, 2021)

AusWolf said:


> I don't think the question is what makes NTFS great, but what makes it so much worse like you described. I for one, have never had any issues with NTFS. The incorrect shutdown issue can be resolved by using the safe removal feature, or shutting your PC down before disconnecting your drives. Each of these takes 2 clicks maximum.
> 
> 
> There are many options in Windows 10 already. Active hours, deferring updates, custom reboot time once your uptades have been installed...
> ...


Come back and comment after you have tried deferring updates, and Windows reboots you overnight when you have work on the desktop, in MS office, etc. You are talking theory, and not practice. Just listen to the people in this thread that have had W10 reboot on them when they didnt want it.


----------



## DeathtoGnomes (Jan 20, 2021)

lexluthermiester said:


> No, it is not. But it is also not their place to tell us what, when and how we may or may not use our own PC's.


Yea and they still try to tell us things. Its the "I know what you want better than you do" syndrome


----------



## Frick (Jan 20, 2021)

lexluthermiester said:


> That requires education of the public, not strong-arm enforcement of rules that don't always make sense are certainly do not respect our rights as users.



Like how anyone connected to the internet has access to all this knowledge so there shouldn't be anymore ignorance, right? If the internet has taught us anything it is that people are aggressively uninterested in education and we as a spiecies cannot be trusted to do the right thing, especially when the right thing is slightly inconvenient. Like updating your OS, or picking non terrible passwords.



ThrashZone said:


> Hi,
> Yet I'm still on win-7 and the world hasn't ended without that patch plus any spectre/ meltdown nonsense either



And I've never been robbed, so logically robberies is a myth.


----------



## ThrashZone (Jan 20, 2021)

Frick said:


> Like how anyone connected to the internet has access to all this knowledge so there shouldn't be anymore ignorance, right? If the internet has taught us anything it is that people are aggressively uninterested in education and we as a spiecies cannot be trusted to do the right thing, especially when the right thing is slightly inconvenient. Like updating your OS, or picking non terrible passwords.
> 
> 
> 
> And I've never been robbed, so logically robberies is a myth.


Hi,
Nope just a bad example of yours really but I do have a gun carry permit = Check  

There is more to security than os patches and MS browser which have always been targets
Remote access..... disabled = Check
Mbam Pro license = Check
Ublock Origin browser security = Check 
Don't use MS apps = Check


----------



## Aquinus (Jan 20, 2021)

lexluthermiester said:


> How about Windows won't run on them? Or perhaps that NTFS is very stable, reliable and nearly error proof? And before you mention that drive disconnection thing, that it a Windows problem, not an NTFS problem.


A filesystem is pretty useless if it's not stable, reliable, or error proof so that's a pretty low bar. The benefits come from all the features that these file systems implement beyond doing what any FS should be capable of doing. Just because Windows can't run them doesn't mean NTFS is better. It means that Microsoft shoves it down your throat without giving you the option.


----------



## lexluthermiester (Jan 20, 2021)

Frick said:


> we as a spiecies cannot be trusted to do the right thing


That is an assumption you have no right or merit to make.


Frick said:


> And I've never been robbed, so logically robberies is a myth.


That statement lacks both logical context and sense.



Aquinus said:


> A filesystem is pretty useless if it's not stable, reliable, or error proof so that's a pretty low bar.


You made the statement that NTFS is lacking, so prove up, how is it so bad?



DeathtoGnomes said:


> Yea and they still try to tell us things. Its the "I know what you want better than you do" syndrome


Such is a deeply flawed "god-complex" kind of thinking that needs serious smack-down, not adherence.


----------



## AusWolf (Jan 20, 2021)

lemonadesoda said:


> Come back and comment after you have tried deferring updates, and Windows reboots you overnight when you have work on the desktop, in MS office, etc. You are talking theory, and not practice. Just listen to the people in this thread that have had W10 reboot on them when they didnt want it.


What makes you think I'm not talking from my own experience?

I have deferred updates countless times, set my active hours to when I'm actually using the PC, and never had a single automatic reboot. I occasionally have updates running in the background, but when they're done, I get a reminder that my PC will need a restart as soon as I'm ready. That's it.

I've been using Windows 10 basically since day one, so I know what I'm talking about. Those people that you refer to are either using an ancient version (pre-1809), or haven't changed the settings that they need to change. The picture I posted is an actual screenshot from the update settings. If you set it to off, you will no longer experience the random restart syndrome. Magic.

Old Windows OSes didn't give you anything, you had to install and configure everything yourself. Windows 10 gives you a basic set of apps and settings. If you don't like them, change them. Don't be lazy.



Frick said:


> Like how anyone connected to the internet has access to all this knowledge so there shouldn't be anymore ignorance, right? If the internet has taught us anything it is that *people are aggressively uninterested in education and we as a spiecies cannot be trusted to do the right thing*, especially when the right thing is slightly inconvenient. Like updating your OS, or picking non terrible passwords.
> 
> And I've never been robbed, so logically robberies is a myth.


1. Drawing any sort of conclusion about humanity as a whole ignores the infinite number of individual differences, and as such, is totally pointless and false.
2. You are describing 'attitude', which is an individual variable, and has nothing to do with the fact that using a PC and the internet requires some basic knowledge. Like I said above: just because you don't know about certain things, it doesn't mean that you shouldn't care to know about them.


----------



## Aquinus (Jan 20, 2021)

lexluthermiester said:


> You made the statement that NTFS is lacking, so prove up, how is it so bad?


Lack of compatibility with POSIX file permissions is a good start which limits interoperability with non-Windows operating systems since practically everything else is POSIX compliant so there is no effective way to handle permissions on these systems. Copying a file with btrfs doesn't actually copy all of the data until new data is written which is a nice perk which also lends itself to really fast sub-volume snapshotting due to being CoW (APFS does this too.) Btrfs also has support for more than one kind of file compression at the file system level depending on your needs. Btrfs also checksums files and supports more than just CRC (such as SHA256,) which is useful for ensuring data integrity instead of assuming that bits don't flip. Btrfs also basically has a transaction log of all the changes to the file system which allows for efficient streaming of data for backups. Btrfs also has the added benefit of being open source.

Like I said before:


Aquinus said:


> A filesystem is pretty useless if it's not stable, reliable, or error proof so that's a pretty low bar. *The benefits come from all the features that these file systems implement beyond doing what any FS should be capable of doing.* Just because Windows can't run them doesn't mean NTFS is better. It means that Microsoft shoves it down your throat without giving you the option.


----------



## windwhirl (Jan 21, 2021)

Aquinus said:


> Lack of compatibility with POSIX file permissions is a good start which limits interoperability with non-Windows operating systems since practically everything else is POSIX compliant so there is no effective way to handle permissions on these systems. Copying a file with btrfs doesn't actually copy all of the data until new data is written which is a nice perk which also lends itself to really fast sub-volume snapshotting due to being CoW (APFS does this too.) Btrfs also has support for more than one kind of file compression at the file system level depending on your needs. Btrfs also checksums files and supports more than just CRC (such as SHA256,) which is useful for ensuring data integrity instead of assuming that bits don't flip. Btrfs also basically has a transaction log of all the changes to the file system which allows for efficient streaming of data for backups. Btrfs also has the added benefit of being open source.
> 
> Like I said before:


On that matter, are CoW and file checksum automatically enabled on all cases or must be manually enabled by the user for each drive/volume/partition/etc. ?


----------



## Aquinus (Jan 21, 2021)

windwhirl said:


> On that matter, are CoW and file checksum automatically enabled on all cases or must be manually enabled by the user for each drive/volume/partition/etc. ?


Both btrfs and apfs are CoW by design. Checksumming is on by default on btrfs, but can be disabled (opt-out.) APFS does checksumming on certain files, not all.


----------



## lexluthermiester (Jan 21, 2021)

Aquinus said:


> Lack of compatibility with POSIX file permissions is a good start which limits interoperability with non-Windows operating systems since practically everything else is POSIX compliant so there is no effective way to handle permissions on these systems.


Interesting, but not a flaw so much as it's deliberate engineering choice.


Aquinus said:


> Copying a file with btrfs doesn't actually copy all of the data until new data is written which is a nice perk which also lends itself to really fast sub-volume snapshotting due to being CoW (APFS does this too.)


Now that is a flaw. Copying a file without actually copying the file? Flawed design indeed.


Aquinus said:


> Btrfs also has support for more than one kind of file compression at the file system level depending on your needs. Btrfs also checksums files and supports more than just CRC (such as SHA256,)


Those are nice features, however NTFS has similar features of it's own.


Aquinus said:


> Btrfs also has the added benefit of being open source.


Now THAT's a good point. No argument there. However that doesn't mean NTFS is not stable. 

You kinda failed to sell your argument there.

Microsoft has MORE than it's share of problems, but the NTFS file system isn't one of them.


----------



## Melvis (Jan 21, 2021)

Windows 7 users be like


----------



## ThrashZone (Jan 21, 2021)

Melvis said:


> Windows 7 users be like


Hi,
You can't handle the truth lol


----------



## windwhirl (Jan 21, 2021)

lexluthermiester said:


> Microsoft has MORE than it's share of problems, but the NTFS file system isn't one of them.


I think NTFS is due for some upgrades, honestly. Though, I agree that for the majority of users, specially in normal use-cases, it's reliable enough. 

A while back Ars Technica wrote an interesting article about that:









						Bitrot and atomic COWs: Inside “next-gen” filesystems
					

We look at the amazing features in ZFS and btrfs—and why you need them.




					arstechnica.com


----------



## lexluthermiester (Jan 21, 2021)

Melvis said:


> Windows 7 users be like


Yeah, pretty much.



windwhirl said:


> I think NTFS is due for some upgrades, honestly.


It could do with a few improvements, sure.


----------



## Aquinus (Jan 21, 2021)

lexluthermiester said:


> Now that is a flaw. Copying a file without actually copying the file? Flawed design indeed.


CoW is not a design flaw, it is literally a design decision. If you have a 10GB file and you copy it (not change it, copy it,) why do you need to store a second copy of the data if the data didn't change? It's a waste of space. The file reference and metadata is copied, but the actual data isn't unless one of those "files" changes. Only then will it become a different copy, but until that happens, there is absolutely no reason why you can't share that data under the hood. It's almost like you're advocating for wasting disk space.


lexluthermiester said:


> Those are nice features, however NTFS has similar features of it's own.


NTFS only supports LZ77, isn't configurable, and doesn't support checksumming files at the FS level.


lexluthermiester said:


> Interesting, but not a flaw so much as it's deliberate engineering choice.


Now this is what I would call a poor engineering choice, not CoW.


lexluthermiester said:


> You kinda failed to sell your argument there.
> 
> Microsoft has MORE than it's share of problems, but the NTFS file system isn't one of them.


Do you have trouble reading or something? Do I need to quote *myself* again?


Aquinus said:


> A filesystem is pretty useless if it's not stable, reliable, or error proof so that's a pretty low bar. The benefits come from all the features that these file systems implement beyond doing what any FS should be capable of doing. Just because Windows can't run them doesn't mean NTFS is better. It means that Microsoft shoves it down your throat without giving you the option.


----------



## lexluthermiester (Jan 21, 2021)

Aquinus said:


> CoW is not a design flaw, it is literally a design decision. If you have a 10GB file and you copy it (not change it, copy it,) why do you need to store a second copy of the data if the data didn't change? It's a waste of space.


That's an opinion. Stupid design idea. What happens when a change is made to one of the "copies"? Does the actual copy function happen then? One way or the other an actual copy function is going to happen. For it not to happen on command is a design flaw, not a design decision. Technology needs follow user commands. If I command a file copy, I expect the file to be copied, bit for bit, as commanded at the moment I command it. A file system should NEVER have the "option" to do something other than what a user has instructed.


Aquinus said:


> It's almost like you're advocating for wasting disk space.


"Wasting" is a subjective term. What I expect from my technology is that it follows my commands as I issue them, the way that I issue them and does NOT take liberties interpreting what I "might" mean or need in future or whatever. So no, I care not about "wasting disk space". It's my space to waste if I so choose.


Aquinus said:


> NTFS only supports LZ77, isn't configurable, and doesn't support checksumming files at the FS level.


Incorrect. NTFS has, since the days of NT 4.0, supported error correction which requires a form of check sum calculations. It might be MS's private brand of it, but it is there.


Aquinus said:


> Do you have trouble reading or something? Do I need to quote *myself* again?


No you don't. What you need to do is re-read your own statements and think very carefully about the context you are attempting, and failing, to convey.

You're still not selling your point. Would you like to continue trying?


----------



## Aquinus (Jan 21, 2021)

lexluthermiester said:


> That's an opinion. Stupid design idea. What happens when a change is made to one of the "copies"? Does the actual copy function happen then? One way or the other an actual copy function is going to happen. For it not to happen on command is a design flaw, not a design decision. Technology needs follow user commands. If I command a file copy, I expect the file to be copied, bit for bit, as commanded at the moment I command it. A file system should NEVER have the "option" to do something other than what a user has instructed.


It's a filesystem, not a block device. I would expect it to do what I ask it to do the most efficient way possible. There is zero reason to duplicate date on the filesystem. You're advocating for wasting space, it's really that simple.


lexluthermiester said:


> "Wasting" is a subjective term. What I expect from my technology is that it follows my commands as I issue them, the way that I issue them and does NOT take liberties interpreting what I "might" mean or need in future or whatever. So no, I care not about "wasting disk space". It's my space to waste if I so choose.


It's wasting space because there is absolutely no good reason to duplicate data on the same disk. As the consumer of that data, you shouldn't care where on the disk it comes from. You should only care that it's there and is valid. Once again, you're advocating for duplicating data, which is wasteful if it's on the same disk. There aren't any ifs about this, it simply is. If it were on another disk, that'd be a different story, but it's not. Structural sharing when data is immutable is never a bad thing.


lexluthermiester said:


> Incorrect. NTFS has, since the days of NT 4.0, supported error correction which requires a form of check sum calculations. It might be MS's private brand of it, but it is there.


Wat? CRC errors you hear about in Windows aren't from the filesystem but from the disk during transfers. *NTFS does not store checksums.*








						Does Windows calculate CRCs to check every file operation?
					

When Windows encounters a file corruption on a file system, it may return the following error message:   Data error (cyclic redundancy check). This suggests Windows checks every file operation (c...




					superuser.com
				





lexluthermiester said:


> No you don't. What you need to do is re-read your own statements and think very carefully about the context you are attempting, and failing, to convey.
> 
> You're still not selling your point. Would you like to continue trying?


Says the guy who doesn't have his facts straight and who has an ass backwards view of copy on write. It's almost like you've never used a sparse disk image before. Maybe we should just store every zero for empty space in a disk image because you asked for a copy.


----------



## Muck Muster (Jan 22, 2021)

Judging from everyone's replies, I'm not sure I wanna install Windows 10 on all my computers. I use Windows 10 regularly but my B-52 browser is still Windows 7. Clearly aside from a few hardware updates and security ( which I use to 3rd party security programs), I find gliding along with Windows 7 still pretty convenient. However, I do get perturbed about how often Windows keeps updating itself.  I know Microsoft works with the Govt. and sometimes feel Windows 10 ability to spy a little unnerving. Just what do they look for? I know all Windows copies of Windows have spy tools implanted inside since NT but to what degree, I have no clue.
I wonder if these updates to spy programs have a direct or indirect impact as Windows keeps updating.
These are the former spy programs I've been notified of and kept tabs on. I don't know what they do. The last I think was in 2016.
NSAKEY_key2
ADVAP132
Does anyone have any clues on how they conduct passing information when asked to do so? If I uninstall them, what harm could it do? I could just say, if asked, "Oh it looked like a form of malware so I uninstalled it." On the other hand, could it lock Windows if I do?


----------



## lexluthermiester (Jan 22, 2021)

Aquinus said:


> It's a filesystem, not a block device. I would expect it to do what I ask it to do the most efficient way possible. There is zero reason to duplicate date on the filesystem. You're advocating for wasting space, it's really that simple.


That is a series of opinions. And you're welcome to them.


Aquinus said:


> It's wasting space because there is absolutely no good reason to duplicate data on the same disk.


Also an opinion, especially if the copied data is going to be modified. Wasted space or not, I expect the copy function to actually make a copy of whatever I commanded it to copy. That is simple common sense and a functional requirement.


Aquinus said:


> Wat? CRC errors you hear about in Windows aren't from the filesystem but from the disk during transfers. *NTFS does not store checksums.*
> 
> 
> 
> ...


Your citation isn't one. SuperUser is a public forum and does not qualify as a citable source. But the following does. Do read..





						Data Integrity and Recoverability with NTFS - NTFS.com
					

NTFS is a recoverable file system that guarantees the consistency of the volume by using standard transaction logging and recovery techniques.




					ntfs.com
				





Aquinus said:


> Says the guy who doesn't have his facts straight and who has an ass backwards view of copy on write. It's almost like you've never used a sparse disk image before. Maybe we should just store every zero for empty space in a disk image because you asked for a copy.


Typical straw-man argument. You need a mirror.


----------



## Aquinus (Jan 22, 2021)

lexluthermiester said:


> That is a series of opinions. And you're welcome to them.





lexluthermiester said:


> Also an opinion, especially if the copied data is going to be modified. Wasted space or not, I expect the copy function to actually make a copy of whatever I commanded it to copy. That is simple common sense and a functional requirement.


It's not an opinion. It's a waste of space because there is no good reason to copy data multiple times. You're free to give me good examples of why you want to store the same data on disk multiple times though. Your view is misguided and you've yet to prove to me why it's not. Until it's modified, why would you proactively waste space? It's dumb because your assumption rests on the premise that it will be eventually modified which is quite the expectation.


lexluthermiester said:


> Your citation isn't one. SuperUser is a public forum and does not qualify as a citable source. But the following does. Do read..


That link says nothing about NTFS having checksums.  


lexluthermiester said:


> Typical straw-man argument. You need a mirror.


That's because you're not giving any good reason and you're just saying I'm wrong. I'm not misrepresenting anything, your take is just bad. Take your own advice and stop being a tool.


----------



## DeathtoGnomes (Jan 22, 2021)

Aquinus said:


> You're free to give me good examples of why you want to store the same data on disk multiple times though


main frame data redundancy was a huge over-protective thing at one time because of the failure rates of hard drives and how easily tape drive date could  be lost from a simple stupid mistake. Imagine a whole raid array lost without "a copy". Some are still concerned  about losing data so the practice of storing duplicate data continues.


----------



## Aquinus (Jan 22, 2021)

DeathtoGnomes said:


> main frame data redundancy was a huge over-protective thing at one time because of the failure rates of hard drives and how easily tape drive date could  be lost from a simple stupid mistake. Imagine a whole raid array lost without "a copy". Some are still concerned  about losing data so the practice of storing duplicate data continues.


We're talking about file systems, not block devices. I'm not advocating for people to not backup their data to different disks or to use technologies like RAID. I'm saying that copying the same data on the same logical disk is wasteful and copy on write helps prevent redundancy. It literally lets you get more out of the storage you have.


----------



## 80-watt Hamster (Jan 22, 2021)

Aquinus said:


> We're talking about file systems, not block devices. I'm not advocating for people to not backup their data to different disks or to use technologies like RAID. I'm saying that copying the same data on the same logical disk is wasteful and copy on write helps prevent redundancy. It literally lets you get more out of the storage you have.


Are CoW systems by necessity journalled?  Or is there another mechanism that directs a read op on the copy to the original, and forces a write on the copy if the original is deleted or altered? I don't know much about filesystems.


----------



## Aquinus (Jan 22, 2021)

80-watt Hamster said:


> Are CoW systems by necessity journalled?  Or is there another mechanism that directs a read op on the copy to the original, and forces a write on the copy if the original is deleted or altered? I don't know much about filesystems.


Most implementations of structural sharing under the hood has references to the spot in memory or on disk and much like the garbage collector in an application or in a VM like the JVM, those areas are cleaned up once all references are gone. There are a number of strategies for accomplishing this, but the most relevant similar technology I can think of are persistent data structures. Copy on write essentially means that data becomes immutable because it's not updated in place, which is a requirement for something like this because a new version requires copying of the non-shared parts, depending on how the underlying technology handles it. Much like MVCC, the "changed" data doesn't get overwritten, it gets stored someone else then the old area gets cleaned up later. A big advantage here is that since the old data is never written over, a crash during a transfer does not corrupt the old data. The new data just doesn't get saved. NTFS actually does something like this via its transaction system, but it's still not checksummed so you can't validate down the line to make sure that no bits flipped after the write is complete.


----------



## windwhirl (Jan 22, 2021)

80-watt Hamster said:


> Are CoW systems by necessity journalled?  Or is there another mechanism that directs a read op on the copy to the original, and forces a write on the copy if the original is deleted or altered? I don't know much about filesystems.


They are, not so much by necessity for the CoW feature itself, but because there is hardly ever a reason to disable journaling. In general, I think all modern filesystems (ext4, NTFS, btrfs, ZFS, etc.) are journaled.

Regarding how the duplicate copies are handled depends on the filesystem, but btrfs for example simply makes a pointer that says that DuplicateFile uses the same blocks (clusters, disk sectors, etc.) as OriginalFile, which makes creating a duplicate an instant operation, regardless of the file size. As long as at least one of that set of files (Original and Duplicate/s) uses those blocks, they're not marked for deletion.

Let's say you change one of those files. The change is written down in new filesystem blocks and then the filesystem adds and/or changes pointers that for a human would read like "DuplicateFile uses the same blocks as Original File except it replaces block 5 for block 12879821 and discards blocks 50 and 51". So the filesystem shows the original file as it was and the duplicate that changed shows up with the changes you may have made to it.

CoW is also important for file system snapshots, as it basically saves the entire state of the filesystem in an instant. Say, you took a snapshot of the drive at 5:30:01 AM, July 4th 2019. The snapshot is instantaneous, as all changes after the snapshot are written down in new blocks, without changing the original ones. And this changes can be "atomic", that is that they don't have to save the whole file in new blocks, the file system writes down only the exact changes made and nothing more. So, if you "flip a bit" in a 400 GB file, the operation won't require writing down the full 400 GB file, it will simply save the change and add a pointer in the filesystem index so that it knows to look for the specific block of that file that was changed when reading said file.

EDIT: Ninja-ed by Aquinus, lol


----------



## Aquinus (Jan 22, 2021)

windwhirl said:


> EDIT: Ninja-ed by Aquinus, lol


No, I like your response better. Snapshotting is such a nice feature of filesystems like btrfs. It's such a powerful tool. I'd actually argue that ext4 is inferior to btrfs because it lacks this.


----------



## lexluthermiester (Jan 23, 2021)

Aquinus said:


> That link says nothing about NTFS having checksums.


You're missing some context, read a little closer.


----------



## Aquinus (Jan 23, 2021)

lexluthermiester said:


> You're missing some context, read a little closer.


I did and there are no references to checksums or hashes. It only retries and recovers from a bad write. If something happens to the data after it has already been written, you're SoL. Your source even plainly spells this out. The checksum isn't to ensure that data is correct just after it has been written. It's to ensure that it's always correct after being written after any arbitrary amount of time. There is a big difference.


----------



## lexluthermiester (Jan 24, 2021)

Aquinus said:


> I did and there are no references to checksums or hashes.


What is the purpose of check-suming and hashing? Hmm? Data integrity, is it not? Let's review...


> Data Integrity and Recoverability with NTFS


That's the TITLE of the information page. It's even in the web address;
http://ntfs.com/data-integrity.htm

You were saying?


----------



## Aquinus (Jan 24, 2021)

lexluthermiester said:


> What is the purpose of check-suming and hashing? Hmm? Data integrity, is it not? Let's review...
> 
> That's the TITLE of the information page. It's even in the web address;
> http://ntfs.com/data-integrity.htm


Just because it commits writes in transactions doesn't mean that it ensures the integrity of your data in the long term, only at the time of writing. This does not ensure that the data written 5 years ago is still the way it should be. That's the point of the checksum, not to make sure that the write was successful like with NTFS' transaction system. What NTFS does is to ensure that write failures due to something like a crash or power loss event won't corrupt data already on disk which you get out of the box with a CoW solution. It does not ensure the correctness of data after it has been written and committed.


lexluthermiester said:


> You were saying?


Apparently things you've been ignoring.


Aquinus said:


> NTFS actually does something like this via its transaction system, but it's still not checksummed so you can't validate down the line to make sure that no bits flipped after the write is complete.


----------



## lexluthermiester (Jan 25, 2021)

Aquinus said:


> Just because it commits writes in transactions doesn't mean that it ensures the integrity of your data in the long term, only at the time of writing. This does not ensure that the data written 5 years ago is still the way it should be. That's the point of the checksum, not to make sure that the write was successful like with NTFS' transaction system. What NTFS does is to ensure that write failures due to something like a crash or power loss event won't corrupt data already on disk which you get out of the box with a CoW solution. It does not ensure the correctness of data after it has been written and committed.
> 
> Apparently things you've been ignoring.


You're dismissing the entire context of the point by marginalizing the proven data integrity functionality built-in to NTFS in a vain attempt to support your argument. This debate is over. You have failed to prove your original claim about NTFS. Move along.


----------



## Aquinus (Jan 25, 2021)

lexluthermiester said:


> You're dismissing the entire context of the point by marginalizing the proven data integrity functionality built-in to NTFS in a vain attempt to support your argument. This debate is over. You have failed to prove your original claim about NTFS. Move along.


I've posted several times about when it ensures that data integrity; which at write time. There is no checksum and nothing to enable you to validate data integrity after the write transaction has be committed.


Aquinus said:


> Just because it commits writes in transactions doesn't mean that it ensures the integrity of your data in the long term, only at the time of writing. This does not ensure that the data written 5 years ago is still the way it should be. That's the point of the checksum, not to make sure that the write was successful like with NTFS' transaction system. What NTFS does is to ensure that write failures due to something like a crash or power loss event won't corrupt data already on disk which you get out of the box with a CoW solution. It does not ensure the correctness of data after it has been written and committed.





Aquinus said:


> I did and there are no references to checksums or hashes. It only retries and recovers from a bad write. If something happens to the data after it has already been written, you're SoL. Your source even plainly spells this out. The checksum isn't to ensure that data is correct just after it has been written. It's to ensure that it's always correct after being written after any arbitrary amount of time. There is a big difference.





Aquinus said:


> Wat? CRC errors you hear about in Windows aren't from the filesystem but from the disk during transfers. *NTFS does not store checksums.*


Also:


lexluthermiester said:


> This debate is over. You have failed to prove your original claim about NTFS. Move along.





lexluthermiester said:


> You were saying?





lexluthermiester said:


> Typical straw-man argument. You need a mirror.


^: I've gotten responses like this every time I've attempted to provide more context. You've yet to prove anything yourself, yet your attitude remains bad and you keep digging your heels in despite not even being able to address any of the things I brought up. You're right, the conversation is over, but it's not because I failed to prove my point. It's because you failed listen. Since you seem to suck at listening, I'll re-quote yet another thing I said, because I'm getting tired of being a broken record for some schmuck who can't read, is ignorant, or both.


Aquinus said:


> That's because you're not giving any good reason and you're just saying I'm wrong. I'm not misrepresenting anything, your take is just bad. Take your own advice and stop being a tool.


----------



## trparky (Jan 25, 2021)

Aquinus said:


> There is no checksum and nothing to enable you to validate data integrity after the write transaction has be committed.


My argument now is... OK, the data is corrupt, the data isn't what it's supposed to be. What are you going to do about it? It's not like you can do anything once the data has been corrupted except restore it from another source of the same data.

As for corrupted data happening, I guess that the reasoning behind NTFS not having built-in checksums is because Microsoft doesn't believe that data corruption is as big of a deal as you may think it is on a consumer level. Perhaps at the data center level like in Microsoft Azure they have that kind of support but that's at the corporate level and that's the kind of stuff that one would expect at that level. Whereas at the home level, data corruption has happened to me twice in ten years and that's only because the drive itself was failing which at that point I assumed all the data was corrupt. But again, that's why you have data backups and why you should always have multiple copies of the same data that you can't lose. Don't trust one repository for your most important of your data, have it in multiple places.

I, myself, have the most important of data saved in the cloud in Microsoft OneDrive. Vacation photos mostly, the kinds of memories that I absolutely cannot lose. There's other data that I keep saved up in the cloud but that's a story for another day. Suffice it to say, if the data is that important to you, keep it in multiple places and most importantly, keep it off site so if a disaster happens like God forbid a house fire, robbery, flood, electrical damage, etc. that data is safely stored offsite where you can recover said data.

As for CoW, I really don't see a problem considering that drives are reaching absolutely ungodly high capacities so at this point, CoW is a bit of a moot point. When you have a ten TB drive, who gives a damn? Are you really going to care? Nope, not I.


----------



## windwhirl (Jan 25, 2021)

trparky said:


> As for corrupted data happening, I guess that the reasoning behind NTFS not having built-in checksums is because Microsoft doesn't believe that data corruption is as big of a deal as you may think it is on a consumer level. Perhaps at the data center level like in Microsoft Azure they have that kind of support but that's at the corporate level and that's the kind of stuff that one would expect at that level


There is checksum support on ReFS, introduced with Windows Server 2012. By default, it only checksums metadata, but file data checksums can be turned on. However, if the checksum doesn't match, the file is deleted. There is no repair there. Btrfs at the very least can detect a bit flip and repair the affected file (though I'd like to know how far the repair can go before it hits a dead-end)


trparky said:


> Whereas at the home level, data corruption has happened to me twice in ten years and that's only because the drive itself was failing which at that point I assumed all the data was corrupt. But again, that's why you have data backups and why you should always have multiple copies of the same data that you can't lose. Don't trust one repository for your most important of your data, have it in multiple places.


Agreed. Home users and most prosumers will probably not care (and dare I say should not care?) at all as long as they are using at least some sort of backup.


trparky said:


> As for CoW, I really don't see a problem considering that drives are reaching absolutely ungodly high capacities so at this point, CoW is a bit of a moot point


To be fair, it's not so much for the capacity, but rather that it's useful for snapshotting the drive in a quick and painless fashion. And without the snapshots consuming too much space, specially if you want to do something like a snapshot per hour or so, which is useful for rolling back changes or keeping multiple versions of a file. 

Also, someone must be asking for CoW support, seeing as it is supported not just on ZFS and BTRFS, but also on APFS.


----------



## trparky (Jan 25, 2021)

windwhirl said:


> Btrfs at the very least can detect a bit flip and repair the affected file (though I'd like to know how far the repair can go before it hits a dead-end)


I still wouldn't trust it. If I knew that a file was corrupt, I'd wouldn't trust any repair of it. Replace it from a known good source.


windwhirl said:


> To be fair, it's not so much for the capacity, but rather that it's useful for snapshotting the drive in a quick and painless fashion. And without the snapshots consuming too much space, specially if you want to do something like a snapshot per hour or so, which is useful for rolling back changes or keeping multiple versions of a file.


Windows does have this capability, it's called Shadow Copies.


----------



## R-T-B (Jan 26, 2021)

ThrashZone said:


> Hi,
> Nope just a bad example of yours really but I do have a gun carry permit = Check
> 
> There is more to security than os patches and MS browser which have always been targets
> ...


Using ms os that is obsolete with a built in netstack that will never get patched again, check?



lexluthermiester said:


> Now that is a flaw. Copying a file without actually copying the file? Flawed design indeed.


You just described write buffering.

ReFS is the future for Windows anyways.


----------



## lexluthermiester (Jan 26, 2021)

R-T-B said:


> Using ms os that is obsolete with a built in netstack that will never get patched again, check?


As long as you don't use the built-in Windows Firewall, that's not really a problem.


R-T-B said:


> You just described write buffering.


Write-buffering actually completes the writes as some point.


----------



## R-T-B (Jan 26, 2021)

lexluthermiester said:


> As long as you don't use the built-in Windows Firewall, that's not really a problem.


Yet.  But someday it will be.  Hopefully by the time that comes around though most users will have migrated.



lexluthermiester said:


> Write-buffering actually completes the writes as some point.


True.  At least, usually true.  There are some exceptions but not generally worth mentioning.  JFS on linux for example, has the amazing property of a write buffer that won't flush until it's full:  that's right, no timeout.

No one really uses that anymore though.


----------



## Aquinus (Jan 26, 2021)

trparky said:


> I still wouldn't trust it. If I knew that a file was corrupt, I'd wouldn't trust any repair of it. Replace it from a known good source.


Now there's the rub. Without a checksum, how are you going to know that data become corrupted in the first place if you don't have a signature to check it against? You won't until you try to use it and if (and only if,) the application using it throws an error due to the corruption as opposed to still continuing to operate with the bad data.


lexluthermiester said:


> Write-buffering actually completes the writes as some point.


So does CoW. Metadata gets copied, but why copy the data when it doesn't change? The data is already there and it's not going to change. If you as the consumer of data get your data when you ask for it, why do you care if under the hood there is structural sharing to save on space along with all the other benefits of going that route? You shouldn't. You should however care about things like snapshotting and disk space optimization because it enables you to have more control over your data without having to do all of this yourself and things like optimizing disk usage gets you more out of your drive. That's definitely not a bad thing.


----------



## trparky (Jan 26, 2021)

R-T-B said:


> has the amazing property of a write buffer that won't flush until it's full: that's right, no timeout.


I'm not sure I like that concept. We've already had discussions in other parts of these forums where people have complained about how SSDs (except for those expensive enterprise drives) don't have some form of onboard power device to complete the write transactions in case of a power failure. Or am I barking up the wrong tree here?


Aquinus said:


> Now there's the rub. Without a checksum, how are you going to know that data become corrupted in the first place if you don't have a signature to check it against? You won't until you try to use it and if (and only if,) the application using it throws an error due to the corruption as opposed to still continuing to operate with the bad data.


For my most important data, I make my own checksums. Do I wish I didn't have to do that? Yes. But I only do that for about 1 to 2% of the data that I store, mostly ripped DVD ISO files (yes, I own them). As for the rest of the six or seven terabytes of data that I have, oh well... it's not a big deal.

If you guys must know, I have a TV series set that is really old and the DVD set is experiencing bit-rot. Some of the disks are physically delaminating. Needless to say, I have all the disks ripped to ISO files on my drives and store them in multiple locations with manually created SHA512 checksums.


----------



## R-T-B (Jan 26, 2021)

trparky said:


> I'm not sure I like that concept. We've already had discussions in other parts of these forums where people have complained about how SSDs (except for those expensive enterprise drives) don't have some form of onboard power device to complete the write transactions in case of a power failure. Or am I barking up the wrong tree here?


No, you are completely correct.


----------



## trparky (Jan 26, 2021)

I thought so. Any write buffer that doesn't start spooling the data to the disk as fast as possible opens the door for data corruption the likes of which... yeah, I really don't want to think about it.


----------



## lexluthermiester (Jan 26, 2021)

R-T-B said:


> Yet. But someday it will be.


If you really think that, you need to brush up on how firewalls work. If a person using a third party firewall that is currently being maintained and continuing to support Win7, if properly configured, that user will not be exposed to anything that will put them at risk.


R-T-B said:


> Hopefully by the time that comes around though most users will have migrated.


Time will tell.


R-T-B said:


> No one really uses that anymore though.


Likely because it's not very useful.


trparky said:


> I thought so. Any write buffer that doesn't start spooling the data to the disk as fast as possible opens the door for data corruption the likes of which... yeah, I really don't want to think about it.


Exactly.


----------



## R-T-B (Jan 26, 2021)

lexluthermiester said:


> If you really think that, you need to brush up on how firewalls work.


No, I don't.  a firewall can't supersede the TCP/IP stack that accepts the incoming connection in the first place.  Find a hole in that and you've got the keys to the kingdom.

It'll take time.  It took A REAL LONG TIME for 2000 to get there.  I don't think XP is even there yet.  But it'll happen, because software isn't perfect.  As I said, I doubt it'll be a real issue because by then, the install base will be gone.  We hope anyways.



lexluthermiester said:


> Likely because it's not very useful.


It's fast and has low cpu cycles, but it's largerly from IBM datacenter land and doesn't care if you lose power, screw you in that situation lol.  You also should be using ECC according to them.  Yeah.  Old school IBM exec logic... lol.

NTFS as far as a filesystem has it's lineage in HPFS from OS/2.  It's a very old design.  That doesn't mean it's bad though.  It's decently vetted and proven.  But Microsoft is aware it is aging, and is working on a replacement already.  That's why I mentioned ReFS.  It's already available in Server, but it doesn't support extended attributes yet, so you can't install to it.


----------



## lexluthermiester (Jan 26, 2021)

R-T-B said:


> No, I don't. a firewall can't supersede the TCP/IP stack that accepts the incoming connection in the first place. Find a hole in that and you've got the keys to the kingdom.


Oh yes you do. Most good firewalls use kernel level implementations to regulate network traffic coming into and out of a system being firewalled. I'm aware that IT security is your line of work, however that doesn't mean you have nothing to learn.


R-T-B said:


> But Microsoft is aware it is aging, and is working on a replacement already.


NTFS doesn't need replacement, it needs refinement. Nothing more. You don't fix something that isn't broken.


----------



## R-T-B (Jan 26, 2021)

lexluthermiester said:


> Oh yes you do. Most good firewalls use kernel level implementations to regulate network traffic coming into and out of a system being firewalled. I'm aware that IT security is your line of work, however that doesn't mean you have nothing to learn.


Your saying they use their own TCP stack?  I really doubt that.  Mind you stack level exploits are really rare.  Like I said the last I'm aware of is in Windows 2000...  or maybe it's even NT, I forget.  They aren't common, is the point.
Regardless, I basically agree with you.


lexluthermiester said:


> NTFS doesn't need replacement, it needs refinement. Nothing more. You don't fix something that isn't broken.


In the same sense that FAT32 doesn't need replacement, yes (I'm aware they are not in the same league but they both fill roles, so bear with me).  See, it's good at what it does but having a newer flashier "heir to the throne" would not hurt the PR people.  That's what MS is after, honestly.  ReFS is slowly being retrofitted for that, I feel.


----------



## trparky (Jan 26, 2021)

R-T-B said:


> Your saying they use their own TCP stack? I really doubt that.


Exactly. The firewall still needs to insert itself into the networking stack. If at any stage of the networking stack there is a vulnerability below that of where the firewall is loaded, as @R-T-B has said before... you have the keys to the kingdom.


----------



## R-T-B (Jan 26, 2021)

trparky said:


> Exactly. The firewall still needs to insert itself into the networking stack. If at any stage of the networking stack there is a vulnerability, as @R-T-B has said before... you have the keys to the kingdom.


But again, these kind of exploits are rare, because the best coders are hired for that sort of stuff.  Because they know that's entry point #1.

Not impossible, but pretty far out.

So we both have points here.


----------



## trparky (Jan 26, 2021)

R-T-B said:


> But again, these kind of exploits are rare, because the best coders are hired for that sort of stuff. Because they know that's entry point #1.


Ah but I still remember the days of using punters back in the old Windows 9x days. Oh those were the days. Losing an online game? Load the punter and sit back as your opponent dropped out of the game. I believe it used a malformed ICMP packet that crashed the old Windows 9x TCP/IP stack.


----------



## R-T-B (Jan 26, 2021)

trparky said:


> Ah but I still remember the days of using punters back in the old Windows 9x days. Oh those were the days. Losing an online game? Load the punter and sit back as your opponent dropped out of the game. I believe it used a malformed ICMP packet that crashed the old Windows 9x TCP/IP stack.


9x was kind of an abomination in that regard, lol.

I think it's the last one Bill Gates actually had lines of code in too.  Makes me wonder how competent he really was...


----------



## trparky (Jan 26, 2021)

R-T-B said:


> 9x was kind of an abomination in that regard, lol.
> 
> I think it's the last one Bill Gates actually had lines of code in too.  Makes me wonder how competent he really was...


Oh yeah, it was only when Windows XP came out that the consumer world graduated to using a real OS. Everything before Windows XP was an absolute dumpster fire.


----------



## windwhirl (Jan 26, 2021)

trparky said:


> Everything before Windows XP was an absolute dumpster fire.


Hey, my Windows 2000 nostalgia is insulted by that comment! 

I do agree, though. I would go even further and say XP before SP2 was garbage.


----------



## trparky (Jan 26, 2021)

windwhirl said:


> Hey, my Windows 2000 nostalgia is insulted by that comment!


Windows XP was essentially Windows 2000 with a candy shell.


windwhirl said:


> I do agree, though. I would go even further and say XP before SP2 was garbage.


I agree.


----------



## danwat1234 (Jan 28, 2021)

lemonadesoda said:


> Yep. I have another unwanted forced update last night.
> 
> And YES I have done all the registry policy and O&O shut ups to stop forced update. But somehow, MS, automagically does it anyway.



You can COMPLETELY disable Windows updates on Windows 10. Tested on 1903,

First go to Services and shut off Windows Update and disable it. Refresh Services to make sure it isn't running.

Now navigate to \windows\system32\ and find files wuaueng.dll and wuauclt.exe. If you can find the former, then look for wuauserv.dll.

For each, go to properties, security, advanced.
Click change owner and type in your user name, Click Check Names to select your user name and Click ok. Then Click Apply or Ok on the main window and close it and reopen it.
Now, you can change permissions for all users.
Delete/Remove permissions from all users and Click Ok.

If that doesn't work, then change owner to Administrator, close the window and try again.

That's it.
To re-enable, add "Read/Execute" permissions to System on wuaueng.dll or whichever dll you have. Doesn't need it on wuauclt.exe for some reason.


----------



## lemonadesoda (Feb 6, 2021)

...we need an update script, to allow permissions, force manual update, then deny permissions.


----------



## R-T-B (Feb 7, 2021)

lemonadesoda said:


> ...we need an update script, to allow permissions, force manual update, then deny permissions.


I mean you basically described WuMgr.


----------



## lemonadesoda (Feb 7, 2021)

RTB, thanks for the tip. Installed. Looks nifty. But look at this screenshot:






Please tell me what i am doing wrong. Look, the screenshot shows that "Disable Automatic Update" was ALREADY set. But guess what, Windows autoupdates, auto reboots. Can you lot even begin to understand my frustration!


----------



## R-T-B (Feb 7, 2021)

lemonadesoda said:


> RTB, thanks for the tip. Installed. Looks nifty. But look at this screenshot:
> 
> View attachment 187436
> 
> Please tell me what i am doing wrong. Look, the screenshot shows that "Disable Automatic Update" was ALREADY set. But guess what, Windows autoupdates, auto reboots. Can you lot even begin to understand my frustration!


Try running as Administrator?  Maybe it'll unlock other checkboxes there, I usually use "Disable Update Facilities."


----------



## lexluthermiester (Feb 9, 2021)

lemonadesoda said:


> Can you lot even begin to understand my frustration!


We really can.


R-T-B said:


> Try running as Administrator?


This and....


R-T-B said:


> "Disable Update Facilitators"


...this. Also check the "Disable Store Auto Update"


----------



## InVasMani (Feb 17, 2021)

Is this specific to the C:/ drive only or any drive!?!? Curious if you can use this to corrupt then reformat a bitlocker encrypted drive.


----------



## bsvols (Jun 7, 2021)

ThrashZone said:


> Hi,
> 1709 was the best performer benchmark wise.
> 
> 
> I use update mini tool it hasn't failed me yet.


"I use update mini tool it hasn't failed me yet." ***** This!


----------



## lexluthermiester (Jun 8, 2021)

bsvols said:


> "I use update mini tool it hasn't failed me yet." ***** This!


Why? Have you had problems with it?


----------

