Principles for Building One-Shot AI Agents(edgebit.io) |
Principles for Building One-Shot AI Agents(edgebit.io) |
Not at all what one-shot means in the field. Zero-shot, one-shot and many-shot means how many examples at inference time are needed to perform a task
Zero shot: "convert these files from csv to json"
One shot: "convert from csv to json, like "id,name,age/n1,john,20" to {id:"1",name:"tom",age:"20"}
This is probably a case where some educational training could have saved the engineer(s) involved a lot of frustration.
at the end its fine if the agent self corrects amongst many shots too
This is the largest issue : using LLMs as a black box means for most goals, we can't rely on them to always "converge to a solution" because they might get stuck in a loop trying to figure out if they're stuck in a loop.
So then we're back to writing in a hardcoded or deterministic cap on how many iterations counts as being "stuck". I'm curious how the authors solve this.
This is what I’ve done working with smaller model: if it fails validation once, I route it to a stronger model just for that tool call.
the problem the GP was referring to is that even the large model might fail to notice it's struggling to solve a task and keep trying more-or-less the same approaches until the loop is exhausted.
EDIT: not as to creating an agent that can do anything but creating an agent that more reliably represents and respects its reality, making it easier for us to reason and work with seriously.
Because here I'm getting "YouTuber thumbnail vibes" at the idea of solving non-deterministic programming by selecting the one halting outcome out of a multiverse of possibilities
This issue will likely always require a monitor “outside” of the agent.
Curious what folks are seeing in terms of consistency of the agents they are building or working with – it's definitely challenging.
Sounds tautological but you want to get as far as possible with the one-shot before iterating, because one-shot is when the results have the most integrity
Humans don’t encounter an infinite loop problem because we are not “process-bound” - we may spend too long on a task but ultimately we are constantly evaluating whether or not we should continue (even if our priorities may not be the same as whoever assigned us a task). The monitoring is built-in, by nature of our cross-task processing.
E.g. imagine an arxiv paper from French engineer sebastiennight:
Using quantum chips to mitigate halting issues on LLM loops
Thumbnail: (SHOCKED FACE of Youtuber clasping their head next to a Terminator robot being crushed by a flaming Willow chip)
Title: French Genius SHOCKS the AI industry with Google chip hack!And even if you could simulate and measure multiple things in parallel, that still wouldn't let you solve the halting problem, which would require simulating and measuring infinite things in parallel.
Another way of saying it: everything that can be done on a quantum computer can also be done on a classical computer. It's just that some specific algorithms can be done much faster on a quantum computer, and in the case of integer factorization, a quantum computer could factor numbers larger than would ever be practical on a classical computer. But that's really it. There's nothing magical about them.
Both of them are quantum "systems", in that both require quantum physics to work, if we're considering modern CPU gate sizes. Just, classical computers expose binary bits, and quantum computers expose qubits.
What I think you're picturing is a quantum "system", like a blob of quantum goo, that you can toss some "state" into and...something. But, that's not what a quantum computer is, any more than a classical computer is something you could throw into a blob of electrical goop and expect it to do anything.