Files are the interface humans and agents interact with(madalitso.me) |
Files are the interface humans and agents interact with(madalitso.me) |
We spend ours on our laptops and assume others do too. But many people don't even own a laptop. They handle all their computing needs from their phones, using proprietary apps, with the data living in WhatsApp messages.
If you've got a coding convention, enforce it using a linter. Have the LLM write the rules and integrate it into the local build and CI tool.
Has noone ever thought about how – gasp – a future human collaborator would be onboarded?
Ah yes - I hate that. Yes it "works", but I don't want things to only work on my machine: I want them to work everywhere.
I was wondering why Google's Jules wasn't more popular, and I guess this is why. My preference for my code to work in different environments is unusual.
It's about how filesystems as they are (and have been for decades) are proving to be powerful tools for LLMs/agents.
Now I tend to skim through it to see if a title looks like it may bring interesting discussions, and then I skim through the discussions. Because there are very knowledgeable people who sometimes share valuable insights.
Interestingly, last time I asked a question, hoping to get interesting people to share insights, I was answered that I "should learn how to use an LLM instead of asking questions" :-).
Most people end up routing these back-to-human calls through Discord/Slack, where the agent shows up as a bot. It works, but agents are structurally second-class there — manual setup, limited API surface, no agent-native identity. The async file interface and the real-time decision channel are solving two different things.
(This is actually what I've been working on: a platform where agents aren't bolted onto human chat as bots, but provision themselves as first-class participants. Files handle state; what handles decisions? That's the gap we're trying to close.)
I've been researching and building with a different paradigm, an inversion of the tool calling concept that creates contextual agents of limited scope, but pipelines of them, with the user in triplicate control of agent as author, operator of an application with a clear goal, and conversationally cooperating on a task with one or more agents.
I create agents that are inside open source software, making that application "intelligent", and the user has control to make the agent an expert in the type of work that human uses that software. Imagine a word processor that when used by a documentation author has multiple documentation agents that co-work with the author. While that same word processor when used by a, for example, romance novelist has similar agents but experts in a different literary / document goal. Then do this with spreadsheets, and project management software, and you get an intelligent office suite with amazing levels of user assistance.
In this structure, context/task specific knowledge is placed inside other software, providing complex processes to the user they can conversationally request and compose on the fly, use and save as a new agent for repeated use, or discard as something built for the moment. The agents are inside other software, with full knowledge of that application in addition to task knowledge related to why the user is using that software. It's a unified agent creation and use and chain-of-thought live editing environment, in context with what one is doing in other software.
I wrap the entire structure into a permission hierarchy that mirrors departments, projects, and project staff, creating an application suite structure more secure than this Filesystems approach, with substantially more user controls that do not expose the potential for malicious application. The agents are each for a specific purpose, which limits their reach and potential for damage. Being purpose built, the users (who are task focused, not developers) easily edit and enhance the agents they use because that is the job/career they already know and continue to do, just with agent help.
Yes, this! Please label AI generated content. Pull request written by an AI? Label it as ai generated. Blog post? Article generated with AI? Say so! It’s ok to use AI models. Especially if English is your second language. But put a disclaimer in. Don’t make the reader guess.
Eg:
> This content was partially generated by chatgpt
Or
> Blog post text written entirely by human hand, code examples by Claude code
The problem I think with AI generated posts is that you feel like you can't trust the content once it's AI. It could be partly hallucinated, or misrepresented.
> That's not a technical argument. It's a values argument. And it's one that the filesystem, for all its age and simplicity, is uniquely positioned to serve. Not because it's the best technology. But because it's the one technology that already belongs to you.
That's a bit vague. Was the article written without the aid of LLMs? Yes or no.
There are a lot of unique aspects of the writing in this post that LLMs don't typically generate on their own.
And there's not a "delve" or "tapestry" or even a bullet point to be found.
Also, accusations and complaints like this are off-topic and uninteresting.
We should be talking about filesystems here, not your gut instinct AI detector that has a sky-high false-positive rate.
I swear there needs to be some convention around throwing wild accusations at people you don't know based exclusively on vibes and with zero actual evidence.
It's not a website you go to — it's a little spirit that lives on your machine.
Not a chatbot. A tool that reads and writes files on your filesystem.
That's not a technical argument. It's a values argument."
I personally use a graph like format but organized like a simple text file, each node prefixed with [id] and inline referencing other nodes by [id], this works well with replace, diff, git and is navigable at larger scales without reading everything. Every time I start work I have the agent read it, and at the end update it. This ensures continuity over weeks and months of work. This is my take on file system as memory - make it a graph of nodes, but keep it simple - a flat text file, don't prescribe structure, just node size. It grows organically as needed, I once got one to 500 nodes.
"I spent money writing "a prompt* and this is what I learned".
It is easy to spot the compacted token distribution unique to each model, but search engines still seem to promote nonsense content. =3
"Bad Bot Problem - Computerphile"
https://www.youtube.com/watch?v=AjQNDCYL5Rg
"A Day in the Life of an Ensh*ttificator "
Are you saying this post is a few edits away from becoming a New York Times bestseller?
But you're right, it did hit the front page, and that says more about my sensibilities not lining up with whoever is voting the article up.
The problem today is that we build specific, short-lived apps that lock data into formats only they can read. If you don't use universal formats, your system is fragile. We can still open JPEGs from 1995 because the files don't depend on the software used to make them. Using obscure or proprietary formats is just technical debt that will eventually kill your project. File or forget.
It’s proven several times over that it’s the correct approach. Abstractions (formerly Google photos, currently Immich) should just be built on top - but these proprietary databases are only for convenience.
For work, I’m having the same experience as the author and everything is just markdown and csv files for Claude Code (for research and document writing).
Ostensibly, things like MacOS Spotlight can bring real utility and value to the file system, and extended attributes through the sidecar indexing, etc. But Spotlight is infamous for its unreliability.
The other issue with file systems is simply that the user (potentially) has "direct access" to them, in that they can readily move files in and up and around whimsically. The "structure" is laid bare for them to potentially interfere with, or, such as the case with the extended attributes, drag a file to a USB fob, and then copy it back -- inadvertently removing those attributes.
And thats how we end up with everything being stuffed into a SQLite DB.
What are your thoughts on XMP sidecar files? I'm torn right now between digital negative + external metadata versus all-in-one image with mutable properties. Portability vs. Durability etc.
It is convenient to be able to undo crops or filters, but I wish the industry would standardize so these changes are portable.
We should recognize files as an essential pillar of digital liberty, on par with FOSS licensing.
They make it seem like data is bound up with apps, and should not have an independent existence. They also make it hard to import/export data, except via backups, which a very crude way of taking control of your personal data.
I'm working on a tool to work around these issues to the extent possible, allowing users to extract their information, as granular files, from device backups into their personal digital library. For immutable data, archival is ok, but for editable data, the biggest challenge is how to make the extracted data "live", i.e., available and editable on the devices again, preferably in the same apps used to create them. There seems to be no good solutions.
It pulled back Plan 9, and I was shocked: this is exactly what we need today, as I'm convinced we need to think about minimizing agent permissions the exact same way companies do. Plan 9 was just too early.
Then again, perhaps in this era of ever expanding storage and compute, maybe someone can make it work even better?
The article gets some fundamentals completely wrong though: file systems are full graphs, not strict trees and are definitely not acyclic
You could sorta bolt these features on with FUSE, but to see real benefits you'd want something closer to Inferno, which is like an OS/application runtime that runs on top of another OS host.
In my mind, the security model is the closest thing to a killer feature it has. Because everything is a file(system) and the fork/rfork and bind syscalls let you precisely control what resources/files/services/etc. a child process has access to via easily understandable shell commands (or using libc functions if you want), it means you don't need special APIs for namespacing (ie. containers) and access controls. It's very clean. When a parent process forks or spawns a child process, it can chose whether that process inherits the namespace or gets a clean slate that it can then bind filesystems onto, controlling precisely what it has access to.
You don't even need root permissions to do tons of stuff in 9front with filesystems. You don't even need a root account.
Instead of reading multi-meg data into memory to determine what to do, I used the file system and the program would store data related to the key in sub directories instead. The older people saw what I did and thought that was interesting. With development time factored in, doing it this way ended up being much faster and avoided memory issues that would have occurred.
So with AI, back to the old ways I guess :)
The challenge is how to structure messy data as a filesystem the agent can use. That is a lot harder than querying a vector db for a semantic query.
The code bases we’ve been using agents in had been pruned and maintained over years, we’ve got principles like DRY that pushed us to put the answer in one place… implicitly building and maintaining that graph with all the actors in the system invested in maintaining this. This is not the case for messy data, so while I see the authors point and agree that a filesystem is a better structure for context over time, we haven’t supplanted search yet for non-code data.
My life's data, including all the official stuff (bank statements, notary acts, statements made to the police [witness, etc.], insurance, property titels), all my coding projects, all the family pictures (not just the ones I took) and all the stuff I forgot, is in files, not in a dedicated DB. But these files are a definitely a database.
And because I don't want to deal with data corruption and even less want to deal with synching now corrupted data, many of my files contains, in their filename, a partial cryptographic checksum. E.g. "dsc239879879.jpg" becomes "dsc239789879-b3-6f338201b7.jpg" (meaning the Blake3 hash of that file has to begin with 6f338201b7 or the file is corrupted).
At any time, if I want to, I can import these in "real" dedicated DBs. For example I can pass my pictures as a read-only to "I'm Mich" (immich) and then query my pictures: "Find me all the pictures of Eliza" or "Find me all the pictures taken in 2016 on the french riviera".
But the real database of my all my life is and shall always be files on a filesystem.
With a "real" database, a backup can be as simple as a dump. With files backuping involve... Making sure you keep a proper version of all your files.
I'd say files are even more important than the filesystem: a backup on a BluRay disc or on an ext4-formatted SSD or on an exfat formatted SSD or on a tape... Doesn't matter: the files are the data.
A filesystem is the first "database" with these data: a crude one, with only simple queries. But a filesystem is definitely a database.
The main advantage of this very simple database is that as long as the data are accessible, you know your data is safe and can always use them to populate more advanced databases if needed.
Production grade systems might be written by agents running on filesystem skills, but the production systems themselves will run on consistent and scalable data structures.
Meanwhile the UI of AI agents will almost certainly evolve away from desktop computers and toward audio/visual interfaces. An agent might get more context from a zoom call with you, once tone and body language can be used to increase the bandwidth between you.
Voice assistants, AI pins, and whatever other speech-based interfaces they come up with next will always be "nice to have", but I don't think anybody should be throwing away their keyboards anytime soon. We may have transformed how we make computers work for us, yet the ways we interact with them are much harder to revolutionize, because they are grounded in the physical, neurological, and habitual constraints of human existence. All of which is to say, when I look at the future, I still see a lot of typing.
Saw this video recently, by an AI company working to get contextual cues from tone and body language. I think they're converting it to text and feeding it into a LLM, so not natively multimodal, but I still thought it was really cool.
Relational is better. Hell, and kind of unique identifier would be nice. So many better ways to organize data stores.
Files are practical now - but I think we should more analyze this from first principles. Here is my attempt at that: https://zby.github.io/commonplace/notes/a-good-agentic-kb-ma... (just entry point - the whole kb is about this).
Where is the source for that?
File systems are not a good abstraction mechanism for remote procedure calls, though. I think it's important to distinguish between the two, since I find there are a lot of articles conflating both - comparing MCPs to SKILLs, which are completely different things.
I think the confusion comes from the fact that MCP came before SKILLs, and there's a mental model where SKILLs are somehow "better than" MCPs. This is like saying local Word documents are better than a fully integrated collaborative office suite. It's just not the same thing.
The reason SKILLs work so well is because there's 50 years of accumulated knowledge of how to run rudimentary Unix tools.
the TLDR
File systems - organising information MCP/APIs - remote procedure calls
Which is mainly to say, trust me, this is a temporary state, the god of complexity is coming. It is utterly inevitable. The people who created React, Kubernetes, all those Java frameworks you hated etc didn't go away. They are right now thinking about how amazing it would be if you if you stacked ten different tools together with brand new structured file formats and databases. We already have "beads" and "gastown" where this is starting. Enjoy these times because a couple of years from now it will already be the end of the "fun" part I think.
Configuration files are the greater than a central repository like Windows Registry. They allow temporary changing and ease of sharing how an application operates. They also create a well defend meaning of the settings.
This is why I do not like how Windows has been implemented, they treat files like a 3rd class citizen versus a 1st class.
The original, much more ambitious version of Vista, Longhorn, was shifting more responsibility for metadata from files to a DB.
Were Haiku mor mature/stable would have been a nice fit for the OS for the LLM/Ai personal use cases.
[1] https://arstechnica.com/information-technology/2018/07/the-b...
I have used rhash (https://github.com/rhash/RHash) before to do something similar - but with all the hashes for a directory stored in a single file (which I can diff against past or future versions).
Backing up private data and verifying it for corruption is crucial.
The best compromise is what modern OSs have: a tree-like structure to store files but a database index on top for queries.
So often we want to look up 'the last file I printed' or 'that message I got from Bob'. Instead of just creating that lookup, we have to go spelunking.
Hell, every major app creates it's own abstractions because the OS/Filesystem doesn't have anything useful. Email systems organize messages and tags; document editors have collections of document aspects they store in a structured blob. Instead of asking the OS to do that.
All files are represented in a table with rows and columns. "Directories" simply have a special "directory = true" attribute in a row (simplified).
The hierarchy is for you, the human.
Like many file systems, NTFS also contains a log for recoverability/rollback purposes.
It's not quite relational but it doesn't make sense to be relational. Why would you need more than one 'table' to contain everything you need to know about a file? Microsoft experimented with WinFS, which wasn't a traditional file system (it was an MSSQL database with BLOB storage which sat ontop of a regular NTFS volume). Performance was bad and Skydrive replaced the need for it (in the view of MSFT).
One thing directories solve: they're great grouping mechanisms. "All the Q3 stuff lives in this directory"
I bet we move towards a world where files are just UUIDs, then directory structures get created on demand, like tags.
All good.
Thanks for starring the repo and let me know if you need any help.
Yeah, that's what I want to abstract way. There are tools that do this to some extent, but they stop at backup extraction, without helping you to integrate the extracted data into a proper personal digital library with easy retrieval.
Please elaborate.
NTFS is still the better choice for common desktop usage. ReFS goals are centered around data integrity but it comes at the cost of performance.
You can have several versions of the same set of data object at once - an entire source set for a build, all the names duplicate but tagged with 'revision' so they can be distinguished.
Hard to do that without a UUID at root, to use for unique identification of the particular 'particle' of the particular data set.
Also the API is much simpler than POSIX.
Hurd with settrans made things much easier than the classical Unix, but it's still in alpha and it still has to implement POSIX for convenience.
Consider: you want a certain data object, with a given UUID. You can find it anywhere, even from a malicious server. Then look up the hash in a trusted database, verify it. Impossible then for the MITM to fool you. No more virus scanning executables.
https://stackoverflow.com/a/68284922
Here is my source file for it. It was so long ago I don’t recall all the details but you can retrieve this information using exif commands.
https://github.com/jmathai/elodie/blob/master/configs/ExifTo...