Userspace FUSE for macOS(fuse-t.org) |
Userspace FUSE for macOS(fuse-t.org) |
https://github.com/buildbarn/bb-adrs/blob/master/0009-nfsv4....
Fortunately, fuse-t doesn't make any of my work unnecessary. Buildbarn uses go-fuse, which talks to the FUSE character directly instead of using libfuse. fuse-t would thus not be a drop-in replacement. Phew!
PS: A bit unfortunate that fuse-t isn't Open Source. :-(
I do wonder how this library deals with some of the fundamental differences beween FUSE and NFSv4. For example, with FUSE the kernel and server share intimate knowledge on which part of the file system lives in the kernel's inode cache. Only when the kernel issues a FORGET call, may the server drop information corresponding to a given nodeid.
As NFSv4 is designed to be stateless, servers may need to be able to process requests containing arbitrary file handles, regardless of how long ago they were returned as part of some prior request. You therefore see that in-kernel implementations of NFS servers rely on special file system methods to resolve objects by file handle, in addition to being able to resolve by path.
This means that if you implement FUSE on top of NFSv4, you will most likely not be able to purge any state. Your FUSE file system's FORGET method will probably never be called. This means that memory usage of fuse-t will most likely just keep on growing as time progresses? Or it announces itself as using file handle mode FH4_VOLATILE_*, but UNIX-like NFSv4 clients hardly ever know how to deal with that.
I would assume this person is just doing an in-memory "NFS server" that would keep all of that state around. So it's more like a FUSE-compatible layer that speaks NFSv4 as a "front end" (since NFS clients are better than "ha ha, surprise! This FUSE backend is networked and can fail on weird ways").
I'm not sure how they chose to translate things like DELEGATE, SEQUENCE, and so on (probably just reply with an error). But for basic OPEN, READ, WRITE, etc. it's all fairly straightforward.
I had hoped they used nfs-ganesha for the NFS server / frontend, but the attributions file suggests they probably rolled their own (and thus is going to be full of bugs and immature for quite some time).
I went down the NFS road long ago - I somewhat (??) remember that coaxing macOS to mount_nfs on loopback is actually a PITA (I remember having to activate 127.0.0.* and mount 127.0.0.2).
Maybe, some years later, Windows' WebDAV client has improved. And I can't remember whether macOS WebDAV was better or not.
Also, one will want to be aware of the currently unsupported features <https://github.com/macos-fuse-t/fuse-t/wiki#unsupported-feat...> since the submitted article appears to be aspirational
I am curious though why a NFSv4 server was chosen over wrapping Apple's File Provider API [1], which seems to be the native method for providing virtual file systems from user space on macOS since macOS 11.5. After glancing over the API I guess it's because FPEs are too high-level to implement FUSE properly, but I'd be glad if you can share any details to satisfy my curiosity.
[1] https://developer.apple.com/documentation/fileprovider
EDIT: Errata, I assumed it was open-source, but it is not. Too bad :( -- but at least this will eventually provide a more stable FUSE experience on macOS.
https://github.com/macos-fuse-t/fuse-t/blob/main/License.txt
Apple is introducing user-space file system technology in macOS – LiveFS/UserFS/com.apple.filesystems.lifs – some of the infrastructure turned up in Monterey, in Ventura it is actually being used for mounting FAT/exFAT filesystems.
It looks like for now Apple is keeping the API Apple-internal only (although I haven't looked at the Ventura SDK, so I could be wrong about that). But if Apple made the API public, it could be the death-knell of all these commercial FUSE-alternatives for macOS offerings. (Even if Apple's API isn't FUSE-compatible, if it is close enough, someone could easily open-source a translation layer – likely to be a lot simpler than bridging FUSE to an NFS server.)
Hopefully they’ll open those APIs up for 3rd party usage on the Mac next year.
--Apple
The design was obviously tailored for providers that implement real disk files, like Dropbox and Apple's own iCloud Drive.
Btw, Apple also ships with an undocumented 9P implementation. It seems to be used for mounting the host filesystem in virtualized guests. It is unclear if it can be made to work over a normal (non-PCI) transport like TCP.
I started on a program to mount a website resource directory on your computer using this api, but I gave up due to the restrictions of the api: https://github.com/mathiasgredal/Itslearning
The only problem I ran into was not being able to bind to 127.0.0.1:445, or connect to a different port, on Windows. I ended up writing a small pcap program that would look for packets going out to some other ip and replay them back at localhost on the port samba was running on, so Windows thought it was connecting to a remote machine. It was a ridiculous solution and I’m sure there’s a better way, but it worked for what I needed.
Such a shame that it's not really open source. The idea of using NFSv4 is awesome though.
I have had the same idea myself before, but with a more cross-platform scope (Linux and Windows too, not just macOS). Whatever it flaws, SMB/CIFS has the advantage of being supported out-of-the-box on more platforms than NFSv4 is.
Things that support SMB out of the box: Windows, Mac
Things that support NFSv4 out of the box: Windows, Mac, Linux, FreeBSD, OpenBSD, NetBSD, illumos
Windows has an NFS client, but it is an additional OS feature which isn't installed by default (and I believe its NFS version support is somewhat outdated?)
Whether Linux distributions install the NFS and/or SMB client support by default, or require an additional package install for either or both, is really going to depend on the distribution (and its installation options)
FreeBSD, NetBSD and Illumos also include SMB client support, but I don't know how easy it is to use them as compared to their NFS clients. (From what I understand, OpenBSD pointedly has first-class support for NFS but not SMB: NFS has an in-kernel client, for SMB you have to use a user-space NFS-to-SMB translation daemon, which is in their ports tree.)
Also, https://github.com/macos-fuse-t/fuse-t/blob/main/License.txt appears to be "playing lawyer"
Do you have any specific concerns with the text of the license? The text seems clear enough that there don't appear to be any potential liabilities from using the software in a personal capacity.
As I understand it, the worst that could happen is that the license/software author could unknowingly incur liabilities from statutory or implied warranties - i.e. they left out some important exclusions that could be brought to bear if a plaintiff successfully construed the license as a contract.
(IANAL, but I've read a lot about software licenses over the years and have dealt with a legal challenge related to dual commercial/OSS licensing)
Do you expect Apple has any consideration that things like this are happening inside the usermode processes when its System suspends and compresses or decompresses their memory of or throttles them by flinging them back and forth to different classes of CPU cores or analyzes them to look for other efficiencies that they don’t talk about publicly (network, thermal, sandbox, etc). If Apple wants FUSE, it will build a carve-out (in the form of a framework) like it did for virtualization. If you still want to use FUSE, maybe try System76’s Pop!_OS or find The [Apple] Way to accomplish your client’s goal.
> The drivers you build with DriverKit run in user space, rather than as kernel extensions, which improves system stability and security. You create your driver as an app extension and deliver it inside your existing app.
makes it seem like every consumer who wants to use something FUSE-y would have to ship their own impl, right?
and I shudder to think of the heartache required with signing or whatever Apple gatekeeping is going on nowadays
In my experience it's because Windows and mac developers aren't aware of local file sockets. Windows API, in particular, doesn't have a similar concept if I recall.
I guess full stack developers means people who can read the top 5 lines of a stack trace
If Linux and *BSD did it (especially if they adopted a mutually compatible implementation), the POSIX standardisation team (Austin Group) would likely be interested in adding it to POSIX, and Windows/macOS/AIX/etc will likely follow their example sooner or later.
It’s not well known enough, looking at the recent thread where people are amazed that the i notation is simply a 32bit number van can be used like that. Or even http://0xd1d8e6f0
For example, even if you do vind to localhost, anyone on your system can access your service. Let’s say an electron is running on port 5555. A guest user on your system can just access the electron app. If this app happens to be vscode, you now have full access.
It’s just plain stupid. You basically nuked multi user security. Better run dos then
While off-topic for this thread, I don't think that's going to do what you expect. If you snapped your fingers and XNU suddenly had cgroups and other namespace trickery required to make containers operate, you'd still have the grave problem of "containers are not virtual machines" and thus an XNU container would only run Darwin binaries, so you're back to the old days of "run one exe locally, run another in prod"
I could probably get by with the chroot support that macOS has, but I never manage to find the motivation.
Really FUSE should be implemented in the kernel itself, it was meant to do so, a kernel-side layer to allow implementing a filesystem userspace efficiently. When they created fuse NFS already existed, and if they decided to create a new protocol and add it to the kernel, rather that something on top of NFS or directly exposing a NFS server in applications, a reason maybe existed right?
Not having a stable FUSE implementation is one of the many reasons that made me abandon macOS and return to the combination of Windows and Linux. Even Windows has a mostly decent open-source FUSE implementation!
IANAL-either but my mental model is that one should not "blaze trails" in making up licenses. I find it does not pass the straight-face test that no other license in the known world captures the author's intentions, so they had to make one up on the fly, typos and all
We all are in our own ways ;-). I pulled that comment out of my ass so I'm ok with being corrected as appropriate.
First, many of the systems that local-use-only-but-served-via-TCP-on-localhost apps use are not multi user. VS Code is a good example; I'd hazard that the vast majority of installs thereof are on systems that don't have simultaneous users logged in.
Second, many localhost-tcp apps do use authentication of a sort; this is simple to set up via a secret that is pre-shared at application installation time.
Third, it's easily possible to use ip[tables] to restrict loopback traffic based on conditions that include user ID or group ID. I'm not sure how many people take advantage of this capability, since doing so reliably would probably imply the "server" component having root so it could impose firewally restrictions on loopback users at startup time.
> it's not almost enough... to not
That is not a standard English structure and I can't unwrap all the negatives and follow it.
> i notation
(?)
> van be used like that
(?) "can be"
> vind to localhost
(?) "bind to"
> an electron is running on port
(?)
Anyway, FUSE is more efficient, since the application directly talks to the kernel, without passing trough the TCP/IP stack and the NFS driver. It cannot be possibly implemented more efficiently without implementing the FS as a proper kernel module.
Also NFS introduces a security problem: authentication is tricky to implement, and I don't think it's used in this application: that allows any user on the computer to access, via the NFS API trough an userspace implementation the filesystem, bypassing any permission of the filesystem itself.
Finally NFS cannot support (as far as I know) all the features that are possible with a FUSE filesystem. And since it's based on a network protocol (meaning client and server may be on different machines with a non reliable network connecting them) something will probably never be supported/work reliably (for example inotify).
Windows has both NFS server and client. It's not enabled by default, but it can be enabled. It has supported NFSv4 since Windows 8.
Only as a server. https://docs.microsoft.com/en-us/windows-server/storage/nfs/...
It has never supported nfsv4 as a client.
> Things that support SMB out of the box: Windows, Mac, Linux, FreeBSD, NetBSD, illumos
> Things that support NFSv4 out of the box: Windows, Mac, Linux, FreeBSD, OpenBSD, NetBSD, illumos
OpenBSD is the only real difference here. (And if "install something from ports" counts as "out of the box", there is no difference.)
There is still somewhat of a difference in ease-of-use though. For NFS, Windows users have to enable an OS feature which is not enabled by default. (I suppose one could automate that in the installer, run a PowerShell script for example.) Not sure how true that is for other platforms.
It still raises the question of why one might prefer NFSv4 vs SMB.
C'mon. Do I really need to quote the guidelines to you?
Regardless of whether it's an in-memory or persistent solution, the problem remains: fuse-t has little choice but to leak resources of the underlying FUSE file system, as there is no valid point in time in which you can issue FORGET operations. This means that any file system with some form of churn rate will leak memory.
Also note that delegation is effectively optional. If a server simply always replies with OPEN_DELEGATE_NONE, the client has to interact with the file as if it's stored remotely.
There is no need to implement SEQUENCE, by the way. macOS implements NFSv4.0, while SEQUENCE is part of NFSv4.1 and later.
Whoa. That's a real mistake. v4.0 is widely considered a "oops, we shouldn't have released this". TIL!
As long as the file the NFSv4 file handle refers to is still usable (i.e., linked into the file system), the NFSv4 file handle must remain usable. Note that this is not a universal requirement, but at least one that the macOS NFSv4 client enforces. It only implements FH4_PERSISTENT. This doesn't seem to be documented explicitly, but is somewhat revealed by this printf():
https://github.com/apple/darwin-xnu/blob/main/bsd/nfs/nfs_vf...
> Besides, how many FUSE filesystems implement FORGET?
Any file system that wants to remove files in the background (meaning: not by calling unlink() through the FUSE mount) must likely do proper refcounting on such files, and provide an implementation of FORGET.
For example, a file system that can give information on live football matches may want to remove files/directories belonging to matches that have already ended. In that case you want the node IDs to remain valid, but refer to files that have already been unlinked from the file system hierarchy. The FORGET operation allows you to determine when those files can be removed from the FUSE server's bookkeeping entirely.
Here is an implementation of FORGET that I wrote for Buildbarn:
https://github.com/buildbarn/bb-remote-execution/blob/master...
Maybe they also shouldn't ship garbage that doesn't work and expect others to debug it?
Maybe this should also happen before you deprecate and make it essentially impossible to install kext's? (It's possible, but trying to walk users through it is a great way to lose 99% of your user base instantly)
(My experience with driverkit is identical to the parent's)
The whole "we removed it for security reasons" is also a hilarious facade.
The linux kernel has a much more expansive (heck, crazy!) driver interface and number of drivers.
Yet, the rate of system compromise due to them vs applications/servers is probably 99 to 1 in favor of applications/servers.
Since working directory is per-process not per-thread, this seems a great way to introduce race condition bugs. It also basically rules it out for anything meant to be used as a library or framework.
One could fork a subprocess, chdir()+socket() there, then pass the socket back to parent over another socket (opened maybe with socketpair().) Should work on any Unix-like which supports SCM_RIGHTS (which is almost everybody, apparently even obscure platforms like AIX, IBM i, z/OS). But not Windows, which doesn't (at least not yet, they may add it at some point.)
Makes one really wish there was a bindat() call:
int bindat(int sockfd, const struct sockaddr *addr, socklen_t addrlen, int dirfd);
int funixsockat(int type, int dirfd, const char * name);
As for the latter, macOS actually does have what they call Containers (https://developer.apple.com/library/archive/documentation/Se...) but as best I can tell such a thing requires opt-in from the app, which kind of defeats the purpose of running untrusted software IMHO. I actually only learned about that Containers stuff from trying to find where in the hell 1Password 8 stores its actual sqlite file: `$HOME/Library/Group Containers/2BUA8C4S2C.com.1password/Library/Application Support/1Password/Data/1password.sqlite`
Nope, you get the same thing with jails, just easier. Jails weren't developed by a company living off selling support for it, you know :-)
Anyone who builds software for MacOS, iOS, or iPadOS targets Darwin as their production environment. This includes end user applications and tools used by other devs.
No one used that word
$ sudo mandb
$ man -K UDS
brew install php@7.2 php@7.3 php@7.4 php@8.0 php@8.1 # and soon php@8.2
brew install node@18 node@16 node@14 node@12 node@10 $ mkdir foo
$ cd foo
$ rmdir ../foo
$ stat .
16777221 401089477 drwxr-xr-x 2 ed staff 0 64 "Sep 6 06:41:05 2022" "Sep 6 06:41:05 2022" "Sep 6 06:41:05 2022" "Sep 6 06:41:05 2022" 4096 0 0 .
$ touch .
$ stat .
16777221 401089477 drwxr-xr-x 2 ed staff 0 64 "Sep 6 06:45:34 2022" "Sep 6 06:45:34 2022" "Sep 6 06:45:34 2022" "Sep 6 06:41:05 2022" 4096 0 0 .
I think it's important to realise that a FUSE nodeid _does not_ represent an identifier of an inode in the file system. Instead, they are identifiers of the inode in the kernel's inode cache. Every time the object is returned by LOOKUP, MKDIR, MKNOD, LINK, SYMLINK or CREATE, should their reference count be increased. FORGET is called to decrease it again.
#define SIO_AF_UNIX_GETPEERPID _WSAIOR(IOC_VENDOR, 256) // Returns ULONG PID of the connected peer process
#define SIO_AF_UNIX_SETBINDPARENTPATH _WSAIOW(IOC_VENDOR, 257) // Set the parent path for bind calls
#define SIO_AF_UNIX_SETCONNPARENTPATH _WSAIOW(IOC_VENDOR, 258) // Set the parent path for connect calls
// NOTE: setting the parent path is not thread safe.
What does the "NOTE: setting the parent path is not thread safe" comment mean? Not thread safe if multiple threads are sharing the same socket? (Which seems like an acceptable limitation.) Or something worse than that?
As a result, it's useful to keep them in non-tmpfs paths that can survive a reboot. That way, very simple programs can use them as sort of a config file: `$XDG_HOME/myprogram/do_xyz.sock`.
Additionally, persistent sockets created once at program install time can help coordinate multiple launches of whatever uses them (e.g. by having servers flock(2) the socket or fight over binding to it as a mutex). For programs whose "server" component isn't managed by a service manager, but can instead be launched many times in response to some user action, that can simplify things.
/run/user/$UID/openssh_agent
Several things place sockets there, some other stuff places the socket in a subdirectory.