Defragmenting my drives manually is one of those things like closing apps on my phone I am not using. Everyone says there is no need to do it. It might even be slightly worse overall. I even have enough self control to avoid doing it and have for years...
But deep down in my heart, I truly FEEL like if I did it would improve things... somehow.
This is on a slow device but it might be worse on a fast one as the extra allocation of small extents start to hit harder on the host side:
(fragmented relative to baseline) bandwidth: 100.65 -> 94.74 MiB/s (-5.86%) elapsed time: 339.550 -> 360.699 s (+6.23%) mean latency: 9935.7 -> 10554.6 us (+6.23%) p50 latency: 6715.2 -> 6838.9 us (+1.84%) p95 latency: 19394.4 -> 20599.9 us (+6.22%) p99 latency: 131498.2 -> 131988.3 us (+0.37%) max latency: 183713.2 -> 624392.9 us (+239.87%) jitter stddev: 17823.1 -> 19806.0 us (+11.13%) jitter CV: 179.38 -> 187.65% (+4.61%)
Also check your nvme granularity with fstrim -D if you are on 4KB your nvme is so fine grain that it doesn't matter much but if it is 64KB like my main one. Ouch, those small files in the middle of the 64KB won't magically go away.
I too am nearly certain that the positive benefits are approaching nil, but I still feel it should be helpful.
I'd be curious to know the reqson why she would care on her own phone and why it would be an issue on someone else's phone?
The purpose is to have a sense of when ZFS may developing trouble quickly finding contiguous blocks of free space to dump new data into. Mostly only a concern for high active and highly full pools. When you start watching the number climb past double digits, you may start finding performance issues but it’s a very contextual thing, rather than “at x value it’s bad”.
Basically it has nothing to do with written file fragmentation, or potential read speed.
Because ZFS is a copy on write filsystem, attempts at defragmentation an active pool are generally not effective. An exception would be a pool that is full of large files that was written in a non contiguous way (like torrents), in which case copying the data to a new dataset and deleting the old roughly achieves the purpose. Or you could use something like https://github.com/salesforce/zfs_defrag
It's like those copper bangles with magnets that people swear makes their joints less creaky.
My example is a daith piercing, that's when you pierce a ring through a fold of cartilage in a certain spot inside your ear. It's supposed to stop you getting migraines, but there's no sensible mechanism for this to work. It's all woo and bunkum, apparently.
But I've had four migraines in eight years as opposed to four every month.
So, if it feels faster, it might well be.
> Fragmentation and extent allocation were adding measurable variance, even on NVMe,
Why exactly would there be a measurable variance on NVMe? I understand there could be some impact on magnetic hard drives. This sounds like some sort of coincidence due to other factors and that this defragmentation won’t achieve much (except for wearing out your SSD even faster in certain cases).
The current analogy I am using when vibe coding is it's like defragging a hard drive in terms of time wasted.
Years of sitting there as the bar crept closer to 99 in the vain hope I could turn the computer off and go to bed, just to have to sit there for another hour or so until finished.
So many late nights wasted doing, I'm not sure what.
And now I'm sat there waiting for the agent to come back..
Running for 17m.. "dealing with a complex response"..
Codex (GPT5.6) has me hooked on that loop moreso as it is, in practice, so much quicker than Claude (Opus or Fable), so I don't quite ever walk away.
Nostalgia.
I suspect the noise is caused by the power supply emitting a tiny bit of 'coil whine' when put under more load
The "Fast, Cheap, or Good... choose any two..." joke is very real. =3
First some kind of before and after benchmark, so you can actually see if defragging made a difference. Second is a recommendation for defragging based on things like the fragmentation, free space, and whether it’s an HDD or SSD.
Especially the second one feels useful since defragging doesn’t necessarily mean improvement to the system as you said yourself.
Just as an idea for a feature your defragmenter could have.
(Also this is a side thought; but I wonder if AWS AMIs and EBS volumes will boot faster if also organized sequentially.)
Probably a measurable, but small, difference. I would expect it's not worth the effort unless you're spending a lot of time booting, and even then, there's probably better things to work on in the boot process, such as reducing the amount of things that run or reducing the size of them.
Second, is it actually true that it doesn't need it? The readme implies that this defragmenter (for Linux) produces actual benefits.
Third, it says the benefits exist even on NVMe, i.e. on SSDs? I thought defragging was relevant for spinny disks only?
Thanks
Of course just having a compressed filesystem image to restore for each test, or a collection of images to cover various cases instead of only optimising for ideal, would be more efficient…
But I love that this now exists instead!
I love it!
fat seemed particularly prone to spreading files into very small chunks across the disk surface; it's not so big of a deal when fragmented files are made up of large contiguous chunks.
but check my other comments, it makes a difference, even on a slow nvme. Probably the overhead of the wasted small extent, and maybe prefetching?
Chop wood, defragment
After enlightenment
Chop wood, defragment
Now HNers here don't even question the soundness of this software and just assume it works as it was suspiciously built in 4 days.
One side-effect of the AI mania is that developers have lost the ability to reason around the verification stage of vibe coded software.
Oh dear.
Like the cleaning lady who ignored my DO NOT RESTART note on the PS1 at the after school daycare and nuked all my progress.
(fragmented relative to baseline) bandwidth: 100.65 -> 94.74 MiB/s (-5.86%) elapsed time: 339.550 -> 360.699 s (+6.23%) mean latency: 9935.7 -> 10554.6 us (+6.23%) p50 latency: 6715.2 -> 6838.9 us (+1.84%) p95 latency: 19394.4 -> 20599.9 us (+6.22%) p99 latency: 131498.2 -> 131988.3 us (+0.37%) max latency: 183713.2 -> 624392.9 us (+239.87%) jitter stddev: 17823.1 -> 19806.0 us (+11.13%) jitter CV: 179.38 -> 187.65% (+4.61%)
Also check your nvme granularity with fstrim -D if you are on 4KB your nvme is so fine grain that it doesn't matter much but if it is 64KB like my main one. Ouch, those small files in the middle of the 64KB won't magically go away.
If you're at the point where you're optimizing for this, you've got some really high performance requirements though. And you'll have to do your own testing, because rules of thumb won't do.
Indeed, most Linux setups supporting trim, already defer these operations to a weekly schedule to reduce wear, and most fs will optimize in 10MB or 25MB chunks given unlike HDD... the SSD seeks are nearly constant time. Logging fs like f2fs, are content aware so will auto re-locate hot and cold (rarely modified) file types, and despite the log-structure... on an SSD performance losses are often surprisingly negligible.
Most modern NVMe with dram cache and SLC buffer areas also defer committing pages to low-endurance flash areas. And most kernel tweakers will set swapiness to 1 on SSD/NVMe machines to try to keep stuff buffered in dram as long as reasonably possible. It is a space-time tradeoff that can boost a desktop machine performance especially with preload daemon active.
If people want ludicrous speed... than just run ext4 with a separate 128GB journal NVMe drive on a split PCIe x4 bus.
Defrag on most modern drives usually just fills these buffer areas full, and things grind to the slowest i/o choke point. =3
Indeed torrents were the most obvious cause of the fragmentation I saw.
contiguous fragmented
-------------------------------------------------------------------
PCIe 4.0 NVMe Read Speed ~5,000 – 7,000 MB/s ~80 – 250 MB/s
SATA 3.0 SSD Read Speed ~500 – 550 MB/s ~30 – 60 MB/s
> You are giving up virtually 0% to 3% of real-world performance. In daily development, media editing, and standard system use, defragmenting your SSD will yield no human-noticeable speedup.
Otherwise the claimed defragmentation here is not actually resulting in sequential data.
Which is rather a case for home use, not practical for highly parallel access industrial use cases. That's why the issue became a research topic. For the first time I saw a proof that the "fact" SSDs do not need defragmentation is actually a myth. Well, you still want to avoid explicit defragmentation (waste of time and SSD lifetime) by filesystem driver submitting additional hints (using new NVMe extensions) to the SSD controller about what blocks belong to the same file, so that SSD can place them for an optimal sequential access with properly interleaving (not necessary strictly physically consecutively whatever this means on SSD).
I haven't measured it but the SSD firmware should be able to look up where the next block is speculatively in order to be able to immediately start sending it if the host tries and read the next block (as opposed to a random one).
Despite this, many of my relatives have somehow learned this habit of opening the switcher and closing all apps when they are done with them.
That being said, I dislike having 3 dozen apps open because I simply can’t quickly find and switch to the ones I’m using. So when I enter a new situation, I swipe all the way left, and Clear All apps, to start over with a tabula rasa.
Open it, and it opens a list of recently searched for apps. The focus is on the top search bar (and the keyboard has also popped up), and I can type to find any app I have installed.
But the switcher shows every app ever opened and tries to pretend these apps aren't being closed. Right now for instance, i just checked and as best i can tell, the last 3 were open. going back any further and i could tell the app had to reload. but i can scroll to the left back forever. the one on the furthest left i easily haven't opened for months