LLMs are breaking 20 year old system design(zknill.io) |
LLMs are breaking 20 year old system design(zknill.io) |
Correct, but we solved this a long time ago when we started sending files to servers to be converted, for example. We either got a 'job_id' or a call to a webhook when the job was finished."
> Long running work: an agent doing a 10 minute task isn’t a ‘request’, it’s a long-running async process.
it should not be stateful at the database/storage level:
> Stateful compute: an agent might run multiple turns of a conversation, might process multiple tool calls, and relies on accumulated context. That state is not really ‘database state’, it’s the agents memory.
According to the author, the problem is already solved, but implemented with the wrong design assumptions.
(Uploading a file for conversion could be framed as a slightly different problem from the author's, though, due to size constraints)
Also, has teh author never heard of long running queued jobs? Or long running scheduled jobs? They ultimately report back into the DB (updating their status etc).
This article reeks of someone using AI to make huge leaping jumps of logic. The "single source of truth" rule has survived this long for a reason. It works!
Please, please, please don't say "polling". Because you've clearly missed the entire argument of the article if you say polling.
This post doesn’t seem to understand how these systems work at the core of agent harnesses.
This theme keeps popping up everywhere. Lots of things were "the way we did things" because a lot of reasons. LLMs just amplify some things and they get enhanced visibility. It can be a good thing, if you're able to understand what/why/how changed, or it can be a bad thing if you insist that "this is how we do things, because this is how we've always done things".
Or... maybe... just maybe... it can be a bad thing, because it's a bad thing.
When was it not? We've been doing this for decades. Something usually sticks.
I would suggest that there is a much, much older technology than pub/sub that can be used for such kind of data transfer: it's UDP, documented in 1980.
I can't stop thinking how overcomplicated our software engineering reality is so we need to reinvent layers and layers of stuff on top of the other stuff. We must make applications for browsers; browsers disallow basic network communication for the code they execute; so sending a chunk of data from a client to a server becomes a real adventure.
Then you'll be reimplementing host discovery (i.e. how do clients find the host that has context on their request), retransmissions, flow control, congestion control, and many other things on top of it, and suddenly it doesn't sound so simple anymore.
The "cloud native" (as the author refers to it) idea that app servers should be stateless is actually the new idea.
The industry eventually reached consensus on sticky sessions being a bad idea a lot of the time. That's why stateless app servers became the norm.
So you can be notified by the database, but you can't (with the stateless HTTP + loadbalancer design explained in the article) get that notification back to the client. Because the client isn't connected anymore; so how does the client know that there's new information?