Their filesystem goals are in some ways consistent with Apple's (marketing) vision: Users would never have terabyte libraries of anything, as the various iServices would (should) be hosting that stuff in the cloud (where one presumes it is stored on a filesystem that actually includes data integrity). Since users won't be storing much of anything locally, Apple needn't care too much about data integrity. This is of course, nonsense.

The idea that Apple's storage devices are error-free is arrogant--but even assuming that were true, there can still be bit errors in the SATA/PCI bus, errors in memory, race conditions, gamma rays, etc. Apple uses ECC memory on their Mac Pro, so obviously someone still believes that sort of thing is possible.

For example, https://www.google.com/patents/US6289356 was filed in 1998, so I presume it's expiring fairly soon. Given that some of the original lawsuits were Network Appliance suing Sun/Oracle, I'm wondering how much of a role this played in the timing of the release of these features? After all, Apple could pretty much pick a window to release a new file system - nothing special about 2016, that they couldn't have done this in 2015 or 2017...

Which makes me wonder if there are data integrity patents that will expire, and at such time, Apple can now drop the functionality into APFS. After all, they did say during their presentation, that the flexibility of the data format is one of the key design features of APFS.

It's much easier to pretend that this is the case when the file system isn't verifying it.

Checksumming would probably expose problems that would otherwise go unnoticed by users or be blamed on computer gremlins. It's hard to say if doing the "correct" thing here would improve the subjective user experience. Maybe putting on airs of infallibility is the more profitable route.

Good checksumming to detect bit rot is exactly what is needed since as an owner of said laptop I have NO idea whether any of my data was affected.

If Apple want to say 'the majority of our devices are mobile and checksumming puts a large performance overhead' then that's one thing. But to claim it's not needed is just plain wrong and makes me worry that Apple's product managers sit in an echo chamber hearing only what they want to hear.

Holy shit.

I guess that explains why my Mac recently had a bunch of daemons burning all cpu crashing repeatedly in a tight loop when getting sqlite errors on a db in ~/Library. Cause disk corruption never happens.

But on another level, I guess if hardware fails, then well, you buy more hardware, which is good for Apple. Presumably people who bought in the past from Apple won't turn around and buy an Acer or HP laptop. They'll still buy Apple.

I recommend sending every file system engineer on a year-long journey as a traveling system integrator.

same deal with some heap protections. say you are running a kernel which doesn't have byte patterns to detect heap overflows or reuse after free. maybe you have some heap overflows which because of their nature never cause any corruption but now you turn on heap protections and peoples kernels are getting more panics :/

APFS has file level encryption, so you would in theory be able to detect a flip by selecting an encryption algorithm that gives error upon decrypting modified data. I could see this being worked into apps_fsck at some point.

A similar case could be made for adding it into the compression algorithm, which the OP thinks will be coming to APFS later, popular algorithms such as deflate already have this built in.

And in the Sun era you might be prepared to bet your business on not being on the wrong end of a lawsuit from the owner of the various patents and copyrights around Sun IP.

Only a completely insane person would argue that's a good idea now.

There are other FSes that allow the behavior that APFS is demonstrating - look at OCFS2 and Btrfs, both of which allow you to do cp --reflink.

I was referring to the Linux kernel modules implementing HFS+ and other Apple FSes.

> Who else actually "supports" HFS+ ? Sure there are linux "ports" based on the spec but nobody claims them as being "supported".

Yes, by support I meant other developers who want to be able to read and write to devices in APFS format.

> Apple would have had to continue supporting HFS+ whether they chose to implement ZFS, btrfs or HAMMER.

Yes, Apple would have to continue supporting HFS+, but other kernel developers would not have to port yet another filesystem (APFS) with all of its own quirks; and, who knows, maybe it would be less work for Apple to inherit ZFS/btrfs/HAMMER/some other filesystem's solutions to some of the same problems they're trying to solve from scratch here. My point was more that by reinventing the wheel to implement some of these features, they've created not just more work for themselves potentially, but more for the open source kernel development community as well in the long run.

> I don't know how Apple writing their own filesystem is wasteful of anybody else's time ( except possibly Apple's and/or Disk utility software for vendors for OS X)

APFS will find its way to external HDDs/SSDs/flash drives, etc., then in order to read those filesystems someone else will have to port it to any other devices/readers of that device/FS.

> The standard is the interface ( POSIX / SUS ) and unless APFS breaks that how is this applicable ?

I didn't mention POSIX, VFS, or filesystem _interfaces_. The analogy to the XKCD strip was that we already had N filesystems that have a large subset of (or in some cases superset of) the features of APFS as of right now, now we have N+1 complex filesystems to contend with and port and interoperate with in other kernels/OSes (mainly Linux + non-Darwin BSDs).

This may just be the price of progress, which is fine. I think it'll be fantastic if Apple makes progress in this area and improves upon the work of others. The developer seemed to be ignoring history so as not to "taint" himself (did he mean IP/legally tainted?), which is slightly worrying to me.

I hope Apple open sources their implementation under a BSD/GPL dual license to make it easier for others to port it directly into other kernels, rather than having to reimplement it themselves.

I was reacting to the idea of APFS for macOS, as well as having yet another filesystem to deal with on external media that interacts with multiple computers (HDDs/SSDs/USB flash drives/etc.).

From Freebsd Mastery: ZFS Pg 135

"For a rough-and-dirty approximation, you can assume that 1 TB of deduplicated data uses about 5 GB of RAM. You can more closely approximate memory needs for your particular data by looking at your data pool and doing some math. We recommend always doing the math and computing how much RAM your data needs, then using the most pessimistic result. If the math gives you a number above 5 GB, use your math. If not, assume 5 GB per terabyte."

Otherwise I think this is like the myth that ZFS requires expensive ECC ram whereas ECC ram is recommended for any filesystem and zfs needs it no more nor less.