It is counterproductive to force cache on all tools of an agent

1 points by istvan_intrnd 159 days ago | 1 comment

To make agentic cache work, LLM providers force frameworks to add all tools into the cached prompt. This is the main reason why I cannot properly use cache in my own agentic framework, where the set of tools changes in function of objectives. It confuses most base models, and however many times they say it doesn't my observations contradict this. How about yours?

sshadmand 159 days ago |

Don't they have to cache in order to access the tools quickly enough to run the agents? I imagine not doing slow would slow down the progress considerably. Or, maybe I misunderstand which part of the flow/agent/work you mean by "cache on tools"