Inner Loop Agents

14 points by tkellogg 1 year ago | 3 comments

bob1029 1 year ago |

> The software you’re using to run your LLM, e.g. Ollama, vLLM, OpenAI, Anthropic, etc., is responsible for running this loop.

There's currently always this same man behind the curtain.

To me these are all effectively the same picture. The differences merely shuffle things around in the code that is invoking the LLM and handling its response.

Until we have something like CLR integration for ChatGPT, I don't see the significance of an inner loop agent.

Has anyone considered this model yet? E.g., shipping a DLL to the LLM provider that contains the actual implementations of the desired tool calls? Imagine how much easier it would be to provide your debugging symbols and XML doc files directly rather than re-documenting everything for some ever-shifting tool calling API surface.

rahimnathwani 1 year ago | |

Right. The core premise of this article makes no sense:

  what if LLMs executed tool calls without going back to the client

An LLM can't execute tools. The only thing an LLM can do is output tokens. It's up to the calling code to decide what to do with those tokens. If an API provider enables its own tools, that doesn't mean the LLM is executing its own tools. It just means the API provider is executing tools instead of the API caller.

jasonjmcghee 1 year ago |

Is this the widely used term? Do you know of any open source models fine-tuned as an "inner loop" / native agentic llm? Or what the training process looks like?

I don't see why any model couldn't be fine-tuned to work this way - i.e. tool use doesn't need to be followed by an EOS token or something - it could just wait for an output (or even continue with the knowledge there's an open request, and to take action when it comes back)