SSD Defragging: The safe way

[goms]

If you manually run optimise it will only show re-trim in the log. But the log also show that defrag is completed at the same second as the re-trim. My conclusion is that the log just show the start and instant end of a scheduled defrag run, since it was not necessary because the drive in question was a ssd. This is just another round of confirmation bias.

No. "Optimise" doesn't normally defrag drives. As an example, today I manually run optimise on my drive and only retrim was performed:

Windows is supposed to retrim drives every week and defrag once a month. Your SSD was defragged because it was probably close to the normal schedule.

 

[A Computer Guy]

Yes, we all know defragmenting an SSD as a whole is a terrible idea. Don't do it.
If you don't know, it's because flash memory has near infinite reads, but limited writes - and cheaper drives can have very very low write values.

The exception to this is modern defragging tools that can defragment individual *files* - I use defraggler for this task every 6 months to hunt down just the worst files.

Is it worth it? Yes. You get much faster read speeds on those files in throughput and latency, as well as lower CPU usage while it has to process the thousands of scattered locations as it loads them into RAM.
Heres an older thread on another forum on the topic, on a plain old SATA SSD. The difference could be from that 150MB/s (or worse) to the 15GB/s a modern NVME drive can do.
Yes, file system fragmentation DOES affect SSD read speed | Overclock.net
View attachment 311225

Another user in the thread tested a 250MB file, but with ever greater amounts of fragments.
View attachment 311227

The bigger a file the less of it can sit in your RAM - if you cant fit the entire file in RAM (large game files) it'll be far worse than smaller files that loaded once, can stay in memory.

The files that get fragmented tend to be ones updated regularly on drives that are mostly full - windows or game files that have been patched or updated but have no empty space next to the existing parts of the file, so new fragments get added to the next free space over and over again.
This is normally not an issue or builds up very slowly but it's worth defragmenting the worst of the worst.
A video or logfile? skip it! The content for a game you run every day? absolutely worth it.


This is my C: drive with a 15-month-old install, sorted by number of fragments.

View attachment 310431

Somewhere around 8,000 fragments there, that would run at 4K random performance of the SSD instead of sequential.
Yet the actual size of those files is just a hair over a single gigabyte - so defragging them won't use much of that drive lifespan at all.

I'll defrag every file with 100 fragments or more now:

Before and after:
View attachment 310432View attachment 310434

Spending 1GB of the limited writes on the drive (My 970 Pro 2TB has 1,200 Terabytes writable as it's lifespan (known as TBW) to remove 7,000 fragments is worthwhile since that's barely a drop of water in the ocean - especially if it's in programs or games you run regularly.

I ran a disk cleanup as well and emptied the recycle bin, and suddenly the value plummeted. A good example of why it's not worth defragmenting the entire drive - I'd have defragged files that should have been deleted instead.
View attachment 310437


I sarcastically wonder if 7 days to die would load faster, if it was a single contiguous file instead 4,482 pieces?
Considering the second worst file on the disk has only 230 fragments, it stands out as a single file worth fixing.
View attachment 310436

Ok this thread finally peaked my curiosity so I'd thought I'd share some observations with defraggler.

On the left below is an example of fragmentation that happens when you clone a 277GB VMWare virtual machine (multi-file virtual disk) to an otherwise mostly clean partition of a Samsung 980 Pro NVMe. That little red dot on the left contains 29 fragments of my virtual machine files for some reason.

On the right below is an example of what happens after ripping an PS3 disk into a Linux virtual machine (single-file virtual disk) on a Samsung 970 Evo Plus.
(to clarify the screenshots are the host file system not the guest file system)

On the 970 Evo here below is an example of what happens when you take a VMWare virtual machine snapshot looking at 1 file in a multi-file virtual disk snapshot.
I find it interesting how the snapshot files are not even remotely contiguous but just spread all over the place for some reason.
I can say that after like 30 snapshots there becomes a noticeable lag in the loading of the virtual machine but also the act of taking a snapshot can actually freeze VMWare Workstation. (it doesn't recover, end task needed)
When this starts to happen I clone the VM (about once a month) to consolidate the snapshots and start the snapshot cycle over again so VMWare won't freeze anymore when taking snapshots.
This makes me think the VMWare freezing might have something to do with the fragmentation that happens with snapshots.

 

[chrcoluk]

I changed my VM now, previously system restore was completely off and its had no defrags, I now added a second SSD, but only enabled system restore for C: not the second SSD, so will see if both get defragged.

 

[Wirko]

Interesting observation, Window's will defrag SSD's, but what you have observed from the timestamps of the logs makes sense as it should be only defragging heavily fragmented drives that utilise volume shadow copies, not every single SSD.

Yes, probably. The existence of shadow copies may matter or not, but unless there's severe fragmentation, the defragmenter just makes a log entry. A few thousand fragments (of all files combined) does not count as a severe fragmentation on an SSD.

It might be a similar thing with TRIM: the retrim procedure runs, and either has very little work to do, or skips any work until the next time. If actual TRIM commands are sent, they must be spread out in time, to make (mostly) sure that the SSD processes (most of) them. Accepting and processing TRIM commands seems to be a low-priority work for the SSD controller.

@A Computer Guy
Fragmentation within the guest OS is of course also possible, and defragmentation of .vmdk files on the host OS can't solve that. Does VMWare have any guidelines about when and how you should, or shouldn't, defragment guests?

 

[A Computer Guy]

Fragmentation within the guest OS is of course also possible, and defragmentation of .vmdk files on the host OS can't solve that. Does VMWare have any guidelines about when and how you should, or shouldn't, defragment guests?

It's been forever since I even considered that. I never defragment guests since the guest file system is virtualized into the vdmk format onto the host filesystem. In a way it's like an SSD decides how to store the data internally without the OS knowledge of how the data is physically stored. I'll have to look at their online help and see what they say about defragging guests.

The screenshots I did were of the host, out of curiosity I was going to look at the guest next.

(edit guest screenshot)

The guest at the moment doesn't look too bad. I dare NOT defrag it since I'm using snapshots and dynamic disk allocation. Here is what VMWare has to say about it. Unfortunately the swap file is fragmented but I might be able to correct that by disabling it and re-enabling it in the guest OS. And yes that looks like a 27GB page file for 16GB of RAM. Looks like I'm gonna have to talk to someone about their web app.

This reminds me in many many days past I think I used to use Norton Ghost to rebuild heavily fragmented virtual disks.

Why defragment an SSD?

I think the point for defragging files on SSD's ( if needed ) is really consolidating the number of pointers or lookups needed to retrieve a file. If there are a very high number of fragments the OS and possibly the storage device must do more work for the read operation. With SSD's generally speaking random I/O is much better compared to HDD's so you could get away with a certain amount of fragmentation without ever feeling it, maybe even never. Given the wear and tear nature of SSD's thought of defragging a file should be considered carefully in cost vs benefit. For example if you have a database on an SSD keeping a defragmented index could be really helpful for performance however enter the dragon of wear leveling and other magic the SSD might be doing under the hood. This built into SSD's enters into question a reality that at best you may be improving logical fragmentation for software (OS or otherwise) however on the physical device level it may have minimal to no benefit, in fact you bear at least the cost of wear and tear. So the answer is really "It Depends".

To stay on topic, @Mussels I had a thought once upon a time, that installing a game on a HDD, defragging it, and then moving it on the SSD would net a lot less fragments than installing it straight on an SSD. Any thoughts?

Yes that does work. In fact you don't even need to defrag it first. I just did this the other day between my two SSD's after analyzing them. Just a simple file copy and enough contiguous free space. I used to do this with HDD's too.

 

[Mussels]

I once found this same article, unfortunately it's too old to be still relevant.

It still happens, It's harder to find the information in google but it's still a thing

Interesting observation, Window's will defrag SSD's, but what you have observed from the timestamps of the logs makes sense as it should be only defragging heavily fragmented drives that utilise volume shadow copies, not every single SSD.

It should only defragment specific files and volumes 'as needed'


Hence the point of this thread, once I realized people thought that SSD's did not fragment - and they most definitely do.

A few thousand fragments (of all files combined) does not count as a severe fragmentation on an SSD.

Absolutely - it's more like tens of thousands, possibly even in a single file. A thousand will slow down a read/write for sure, but if that's a 10GB file a thousand fragments still has each piece fairly large so the impact may be small - theres a ratio of fragments per MB/GB that we don't know yet, where it's worth defragging

Unfortunately the swap file is fragmented but I might be able to correct that by disabling it and re-enabling it in the guest OS

Yes, you disable the page file (or shrink it) then free up contiguous space on teh drive and recreate it - by deleting files or defragging whatever is in that place. Defraggler is good with that since you can see what's in the space and send it to the end of the disk, freeing up a big large chunk before re-enabling

To stay on topic, @Mussels I had a thought once upon a time, that installing a game on a HDD, defragging it, and then moving it on the SSD would net a lot less fragments than installing it straight on an SSD. Any thoughts?

It will make itself with as few fragments as possible either way, if the drives close to full it can help.

You download an 80GB file from the internet via steam, which then gets unpacked to 120GB - deleting each compressed file as it goes and then trying to fit a larger uncompressed file in the smaller space, fragmenting at least once per file
Vs
120GB of files from another drive, going into whatever existing spaces are on the drive - could be empty clear space, could already have every second 4KB cluster filled.

 

[A Computer Guy]

Yes, you disable the page file (or shrink it) then free up contiguous space on teh drive and recreate it - by deleting files or defragging whatever is in that place. Defraggler is good with that since you can see what's in the space and send it to the end of the disk, freeing up a big large chunk before re-enabling

Sending to the end of the virtual disk is probably has a nice advantage to avoid rewriting blocks. I am running a test now actually of 10,000-ish fragments on 65GB of data in the VM. It seems normal defrag in the guest OS visually looks like the old school defrag where it's collecting fragments and writing new blocks but may also be rewriting (logical) blocks that haven't been filled yet . So writing to the end of the virtual disk I think should simply write new blocks and since the disk is virtual the actual placement of the data doesn't matter as long as the fragments are reduced. What I am getting at is I am going to see if the performance improves in the guest OS after defragmenting and how this causes fragmentation in the host OS - for science!

Some feed back on VMWare Workstation behaviors while my tests are running

• When cloning a virtual machine in the destination I observed the vmdk (virtual disk files) were written contagiously however they were written in a few fragments from 2 to 4 fragments (most of my vdmk files were 11GB each) This doesn't seem to be too bad really.
• When cloning a virtual machine (and consolidating snapshots) it does nothing to defrag the guest - which I expected but it's kinda a shame it doesn't.
• The Defragment button in Virtual Machine Settings does nothing to defrag the guest. I think this just defragments the vmdk on the host so that's nice.
• In the VM defragging a single file that had 156 fragments resulted in read speeds increasing from 97MB /s to 118 MB/s inside the VM
• Moving files to the end of the drive seems pretty quick

 

[Mussels]

Sending to the end of the virtual disk is probably has a nice advantage to avoid rewriting blocks. I am running a test now actually of 10,000-ish fragments on 65GB of data in the VM. It seems normal defrag in the guest OS visually looks like the old school defrag where it's collecting fragments and writing new blocks but may also be rewriting (logical) blocks that haven't been filled yet . So writing to the end of the virtual disk I think should simply write new blocks and since the disk is virtual the actual placement of the data doesn't matter as long as the fragments are reduced. What I am getting at is I am going to see if the performance improves in the guest OS after defragmenting and how this causes fragmentation in the host OS - for science!

Some feed back on VMWare Workstation behaviors while my tests are running

• When cloning a virtual machine in the destination I observed the vmdk (virtual disk files) were written contagiously however they were written in a few fragments from 2 to 4 fragments (most of my vdmk files were 11GB each) This doesn't seem to be too bad really.
• When cloning a virtual machine (and consolidating snapshots) it does nothing to defrag the guest - which I expected but it's kinda a shame it doesn't.
• The Defragment button in Virtual Machine Settings does nothing to defrag the guest. I think this just defragments the vmdk on the host so that's nice.
• In the VM defragging a single file that had 156 fragments resulted in read speeds increasing from 97MB /s to 118 MB/s inside the VM
• Moving files to the end of the drive seems pretty quick

Depending on the defrag method used, some use less writes than others.
Ultimate Defrag was my favourite for it's detailed options, but it's been so long without major updates and the price is absurd now.

It allowed you to set high/low performance folders and it would put them at the start/end of the drive, and the rest it could either just shrink the number of fragments or try to maximise free space - which is the method that re-writes *everything* and isn't great for SSDs. You can use defraggler if its windows based and focus on just the most fragmented files, rather than the entire thing.

 

[A Computer Guy]

Found something interesting. A14GB windows.edb file after defragging with the instructions provided in the following link reduced the file to 6GB but smashed it into 390 fragments!

Windows.edb becomes larger than expected - Windows Client

Address an issue in which Windows.edb becomes larger than expected when PST files are indexed in Windows 10, 8.1, or 8.

learn.microsoft.com

 

[Mussels]

Found something interesting. A14GB windows.edb file after defragging with the instructions provided in the following link reduced the file to 6GB but smashed it into 390 fragments!

Windows.edb becomes larger than expected - Windows Client

Address an issue in which Windows.edb becomes larger than expected when PST files are indexed in Windows 10, 8.1, or 8.

learn.microsoft.com

How many fragments was it originally?

If it was broken into lots of 4KB chunks on a fairly full drive, that can still be a big reduction

 

[A Computer Guy]

How many fragments was it originally?

If it was broken into lots of 4KB chunks on a fairly full drive, that can still be a big reduction

It was only 45 fragments before. Well I guess it's something funky with Outlook and Windows Indexing. I've been scratching my head trying to find wtf I was losing disk space at an unusual rate.

Ok so here are my testing results, its a bit weird and perhaps calls into question relying on defraggler's read benchmark to gauge performance.

Started with 10,679 fragments in total across 65GB of data. I realized after a started to defrag the screenshot didn't include the benchmark run so I stopped and took a bench at 7,357 fragments.

Ok so it looks promising. In the VM guest we went from 88 MB/s to 118 MB/s random reads.
In the host system vmdks are now more fragmented from the guest defrag but we are getting 130 MB/s random reads.

After shutting down the vm and using the file copy technique between drives I essentially defragmented the vdmk files in just a few minutes as opposed to the hours it took in the guest VM but random reads drop to 62 MB/s on the host drive! Starting up the VM and retesting the random reads are up 122 MB/s but you can see how Windows goes right to work borking your nicely defragged drive with seemingly illogical nonsense of a file writing pattern after a shutdown and reboot and can't seem to keep files together.

Shutting down the VM again and retesting the host drive we are back up to 133 MB/s random reads on the host drive! WTF is going on here?

So after seeing this it seems my testing was not just poor it was junk and a better method of testing and validation is needed. I probably should have run some different benchmark maybe crystal disk info instead. On the surface it hints that a VM guest file performance might benefit from defragging including if the vmdk files are also in good order. After reviewing @Mussels original post the level of fragmentation I had for individual files across the board to test with likely was not significant enough to matter so there is that.

So does defragging the VM guest worth it? As a whole (on an SSD) I don't think so but my quick test isn't very conclusive. Keep in mind that although the guest might think a file isn't fragmented the vmdk file could very well be thus making the effort self defeating and requiring yet more wear and tear to defrag the vmdk file for top performance.

Anyway I hope this was entertaining.

 

[claes]

I’m not claiming to have any answers here or trying to be critical, as this is something I’m curious about myself, but some food for thought

CCleaner Support Community

support.ccleaner.com

CCleaner Support Community

support.ccleaner.com

This made me curious if the differences you’re seeing in performance are due to trim or actual defragmentation. Does trim defragment? When defraggler says “optimize,” do they mean defragment? Suppose we may not know until Microsoft tells us.

They then link to this article which didn’t really add much to previous conversations, but made some bold claims

Should You Defrag an SSD?

“Defrag your hard drive” has been the cure-all advice for PC performance woes since the command line was the standard interface for home computers. It’s a technique meant to improve the performance of traditional mechanical hard drives, but do you need to defrag an SSD? These newer solid-state...

helpdeskgeek.com

Also I was reading through that old OCN thread and noticed that the app they used to test is for windows? Perhaps you could test for us as I imagine things are different with nvme?

Read speeds dropping dramatically on older files...

www.overclock.net

 

[A Computer Guy]

This made me curious if the differences you’re seeing in performance are due to trim or actual defragmentation. Does trim defragment? When defraggler says “optimize,” do they mean defragment? Suppose we may not know until Microsoft tells us.

So in the case of copying the fragmented vmdk around I did trim the drive when the destination partition was empty before copying them back. So I think by that we know for sure at least the file copy essentially defragmented the vmdk files naturally from that function. Now why defraggler random read dropped so much afterward is an interesting observation. I happened to do a Samsung bench before and after the test was complete and I think the performance on the drive itself was not significantly different but a bit of a drop in random reads for some reason perhaps within a margin of run to run error. I don't fancy running it too many times for obvious reasons.

One last thing for the night. I noticed Cinebench 2024 was fragmented on my drive so I defraggled it and my multi-core score went up! I ran it twice to be sure.

 

[claes]

I’m going to say that cinebench score is within the margin of error but neat

 

[Wirko]

Here is what VMWare has to say about it.

Great. That's next to nothing.

Anyway I hope this was entertaining.

Sure, and very instructive, thanks a lot. Some results are realy impossible to explain, though, like this one: "In the VM defragging a single file that had 156 fragments resulted in read speeds increasing from 97MB /s to 118 MB/s inside the VM". It's alarming if a very small (for an SSD) number of fragments has such a great effect on read speed.

***

I think I should patent "Continuous Defragmentation". Method and apparatus and all. But, as always, too many people have thought of useful things before I did. Such as these guys.

What I mean is: the file system driver should be gathering some statistics about the files being read. It would detect and count situations where a sequential read has to jump all over the disk because of fragmentation, and temporarily store (in RAM) some metadata about these problematic parts of files. That operation would consume very little time. Some time later, perhaps when the disk were idle, a process akin to a garbage collector would defragment only those specific parts of specific files, putting priority on fragments that were read most often, and also trying to reduce the fragmentation of free space along the way.

I would of course throw in some advanced (but still dumb) statistics, to make the whole process somewhat adaptive depending on what's going on on the disk. Violà, artificial intelligence!

 

[A Computer Guy]

Great. That's next to nothing.

Sure, and very instructive, thanks a lot. Some results are realy impossible to explain, though, like this one: "In the VM defragging a single file that had 156 fragments resulted in read speeds increasing from 97MB /s to 118 MB/s inside the VM". It's alarming if a very small (for an SSD) number of fragments has such a great effect on read speed.

***

Yea I thought it was kind of interesting. I wonder how they are actually calculating that number.

I think I should patent "Continuous Defragmentation". Method and apparatus and all. But, as always, too many people have thought of useful things before I did. Such as these guys.

What I mean is: the file system driver should be gathering some statistics about the files being read. It would detect and count situations where a sequential read has to jump all over the disk because of fragmentation, and temporarily store (in RAM) some metadata about these problematic parts of files. That operation would consume very little time. Some time later, perhaps when the disk were idle, a process akin to a garbage collector would defragment only those specific parts of specific files, putting priority on fragments that were read most often, and also trying to reduce the fragmentation of free space along the way.

I would of course throw in some advanced (but still dumb) statistics, to make the whole process somewhat adaptive depending on what's going on on the disk. Violà, artificial intelligence!

Sounds like a great idea.

 

[Calenhad]

Since we're still in conspiracy theory country, I dug into the Windows task scheduler to find out why the log says re-trim and defrag at the same time stamp.

The scheduled defrag task is set to run the "optimize" job on every drive when it is triggered. Do defrag.exe -? in a command prompt and you can see for yourself that the optimize operation will do the appropriate defrag OR re-trim operation depending what kind of drive it is run on. Meaning that unless your ssd is wrongfully flagged as a hdd it will never be automatically defragmented by Windows.

So can we please put this nonsense to rest? A ssd should never, ever be defragged because it is physically impossible to do so. The nand controller is in charge of this by itself. There is nothing Windows (or any other os) can do about the actual placement of bits in the nand chips. We can issue a re-trim command which incidentally is what Windows is doing with the scheduled optimize task. But that is the end of it. The entire disk topology information that a ssd present in a defrag program is a lie to emulate how hdds are constructed. In reality the entire thing is just a translation layer presented by the nand controller. When you write anything to a ssd the bits, even if you bitstream an entire file in sequence, will be placed into nand cells according to whatever internal statistics the nand controller have. Think of wear-levelling as one example.

Messing around with defrag software is a fools errand. Best case you dig yourself a nice, big hole of confirmation bias. Worst case you kill your drive prematurely by triggering a ton of pointless writes.

 

[A Computer Guy]

Since we're still in conspiracy theory country, I dug into the Windows task scheduler to find out why the log says re-trim and defrag at the same time stamp.

The scheduled defrag task is set to run the "optimize" job on every drive when it is triggered. Do defrag.exe -? in a command prompt and you can see for yourself that the optimize operation will do the appropriate defrag OR re-trim operation depending what kind of drive it is run on. Meaning that unless your ssd is wrongfully flagged as a hdd it will never be automatically defragmented by Windows.

So can we please put this nonsense to rest? A ssd should never, ever be defragged because it is physically impossible to do so. The nand controller is in charge of this by itself. There is nothing Windows (or any other os) can do about the actual placement of bits in the nand chips. We can issue a re-trim command which incidentally is what Windows is doing with the scheduled optimize task. But that is the end of it. The entire disk topology information that a ssd present in a defrag program is a lie to emulate how hdds are constructed. In reality the entire thing is just a translation layer presented by the nand controller. When you write anything to a ssd the bits, even if you bitstream an entire file in sequence, will be placed into nand cells according to whatever internal statistics the nand controller have. Think of wear-levelling as one example.

Messing around with defrag software is a fools errand. Best case you dig yourself a nice, big hole of confirmation bias. Worst case you kill your drive prematurely by triggering a ton of pointless writes.

While I don't disagree with everything you have said here I think perhaps there is also very niche corner case involving the software aspect of dealing with fragmentation. For example taking additional compute time managing file system overhead but this likely only pertains (or might matter) in extreme cases of fragmentation.

 

[claes]

In addition, I think part of what’s being overlooked here is that fragmentation and problems like compute time are filesystem issues, not hardware issues (with SSDs).

 

[Waldorf]

havent used defraggler for years, just remember something "annoyed" me so that i stopped using it.
never seen any numbers (perf) wise that made me think a drive is "slow", then again i have been swapping/adding drives every 6-10 month in the past 3y,
and copied most stuff from one to another, probably limiting fragmentation.
but i dont mess with VMs, and havent used imaging stuff for almost a year, as the MSI replacing my Gb board, is rock stable (clocks/volts).

there is a (free) version of Auslogics that allows you to optimize OS etc for ssd drives/use, as well as "changing" how it works on those (vs HDD),
and allows to set offline defrag (prior OS booting), adjust for VSS etc, but i havent messed with telling it to (only) defrag certain files.

make sure to go thru settings, as well as "view"->"ssd optimization", and enable.
of course do not update when it shows in the taskbar, as that removes the ssd options (in the new version its paid feature).
auslogic defrag

 

[chrcoluk]

Since we're still in conspiracy theory country, I dug into the Windows task scheduler to find out why the log says re-trim and defrag at the same time stamp.

The scheduled defrag task is set to run the "optimize" job on every drive when it is triggered. Do defrag.exe -? in a command prompt and you can see for yourself that the optimize operation will do the appropriate defrag OR re-trim operation depending what kind of drive it is run on. Meaning that unless your ssd is wrongfully flagged as a hdd it will never be automatically defragmented by Windows.

So can we please put this nonsense to rest? A ssd should never, ever be defragged because it is physically impossible to do so. The nand controller is in charge of this by itself. There is nothing Windows (or any other os) can do about the actual placement of bits in the nand chips. We can issue a re-trim command which incidentally is what Windows is doing with the scheduled optimize task. But that is the end of it. The entire disk topology information that a ssd present in a defrag program is a lie to emulate how hdds are constructed. In reality the entire thing is just a translation layer presented by the nand controller. When you write anything to a ssd the bits, even if you bitstream an entire file in sequence, will be placed into nand cells according to whatever internal statistics the nand controller have. Think of wear-levelling as one example.

Messing around with defrag software is a fools errand. Best case you dig yourself a nice, big hole of confirmation bias. Worst case you kill your drive prematurely by triggering a ton of pointless writes.

I take it you didnt see my screenshots then, you will notice the SSD was identified as an SSD, and there is Microsoft developers as well as a Microsoft support article (linked in this thread) stating SSD's will be auto defragged from time to time.

Fragmentation affects actual mapping in the file system, more fragments means more mapping data.

Now you did confirm the theory that the defrag log does not necessarily mean a defrag was run, that was good, but its not confirming what you just typed.

There is a flag of some sort, under normal conditions if you click optimise it will just do a retrim, but if the SSD is identified as having a partition thats too fragmented it will initiate a defrag, in the thread I made, if I interrupted the auto defrag, it would run the defrag when I clicked optimsie, but once I let the defrag finish, clicking optimise only then did a retrim.

If it was misidentifying my drive as an HDD, it would be doing a defrag every time I click optimise and no retrim's at all.

 

[Waldorf]

@chrcoluk
+1
a nice "concentrate" (if i had to post it be much longer ) of how i (already) expected win to handle drives.

still not sure tho, if an actual defrag is always making a difference (short of "not every avg user" like VMs or shadow copy),
at least with newer drives from the last few years.

 

[R-T-B]

I honestly think even if this is possibly an edge case benefit of a thing, it's really not worth the extra write cycles beyond what windows does. I personally fail to see any revelations here, this whole thread is just kinda looking at SSD controller logic and screaming "holy crap the black blox is black"

 

[A Computer Guy]

I honestly think even if this is possibly an edge case benefit of a thing, it's really not worth the extra write cycles beyond what windows does. I personally fail to see any revelations here, this whole thread is just kinda looking at SSD controller logic and screaming "holy crap the black blox is black"

I agree but also I wonder if SSD's hadn't come along would NTFS have been replaced with something more file system fragmentation overhead friendly?
Also how does Linux/EXT4 handle fragmentation overhead when the disk gets nearly full? What about BTRFS, ZFS, and ESXI in general?
Questions I haven't thought about in a long time since fast storage became available in the consumer space.

In particular I wonder about ESXI and how much filesystem overhead is incurred within guest fragmentation as I am about to put back together my test bench with my Asus Hyper M2 x16 card after recently liberating my older SSD's in testing for x8x4x4 bifurcation that will at least allow me to use 3 SSD's (1 dedicated, maybe 2 in raid 0). I think I've demonstrated trying to resolve VM guest fragmentation is a bit of a doubous matter especially if you are not using a passthrough disk but virtual disks instead.

Admittedly I haven't had time yet to do a deep dive with the info chrocoluk provided.

 

[chrcoluk]

I agree but also I wonder if SSD's hadn't come along would NTFS have been replaced with something more file system fragmentation overhead friendly?
Also how does Linux/EXT4 handle fragmentation overhead when the disk gets nearly full? What about BTRFS, ZFS, and ESXI in general?
Questions I haven't thought about in a long time since fast storage became available in the consumer space.

In particular I wonder about ESXI and how much filesystem overhead is incurred from guest fragmentation as I am about to put back together my test bench with my Asus Hyper M2 x16 card after recently liberating my older SSD's in testing for x8x4x4 bifurcation that will at least allow me to use 3 SSD's (1 dedicated, maybe 2 in raid 0). I think I've demonstrated trying to resolve VM guest fragmentation is a bit of a doubous matter especially if you are not using a passthrough disk but virtual disks instead.

Admittedly I haven't had time yet to do a deep dive with the info chrocoluk provided.

Well they all get worse as the disk gets fuller.

ZFS has some customisations to allow you to tweak its approach to fragmentation and how it behaves once the utilisation is above a set threshold. It also by default has variable cluster sizing in the form of records. So it has a ashift size (basically the block size and the min cluster size) and a record size which is the maximum amount of data able to be processed at once, effectively a cluster limit. The ashift is locked in on pool creation, but the record size is customisable live and will effect any new files written. It also compresses by default which will help on compressible files. It has no defrag tool at all. It does report fragmentation though, example output below.

# zpool list
NAME    SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP  HEALTH  ALTROOT
zroot  1.78T   603G  1.19T        -         -    40%    33%  1.00x  ONLINE  -

EXT4 can be heavily customised at creation time again to manipulate its vulnerability to fragmentation, can adjust inode limit, can tell it where to store metadata (I like to put it all at start of filesystem), can adjust its recovery blocks, can enable clusters, and more.