I built a system that lets LLMs automatically learn and improve problem-solving strategies over time, inspired by Andrej Karpathy's idea of a "third paradigm" for LLM learning.

The basic idea: instead of using static system prompts, the LLM builds up a database of strategies that actually work for different problem types. When you give it a new problem, it selects the most relevant strategies, applies them, then evaluates how well they worked and refines them.

For example, after seeing enough word problems, it learned this strategy:

1) Read carefully and identify unknowns,

2) Define variables with units,

3) Write equations,

4) Solve step-by-step,

5) Verify the answer.

All strategies are stored as human-readable JSON that you can inspect and edit.

I tested it on math benchmarks and saw decent improvements - 8.6% better on Arena Hard, 6.67% on AIME24. After 500 queries, the system had created 129 strategies and refined 97 of them.

The implementation is an open-source plugin for optillm (our inference optimization proxy). It works with any OpenAI-compatible API - you just add "spl-" to your model name. Has two modes: inference-only (uses existing strategies) and learning mode (creates and refines strategies).

What's interesting is that it bridges the gap between the sophisticated system prompts that production AI uses and the basic prompts most of us work with. Your model literally gets better at the types of problems you throw at it.

Built it because I noticed ChatGPT, Claude etc. have incredibly detailed system prompts with problem-solving frameworks, but most developers use basic prompts and miss out on those performance gains. The approach is inspired by Andrej Karpathy's tweet about a "third paradigm" for LLM learning beyond just pretraining and fine-tuning: https://x.com/karpathy/status/1921368644069765486

The strategies are completely transparent - you can see exactly what the system learned and why it's making certain decisions. No black box learning.

https://github.com/codelion/optillm/tree/main/optillm/plugin...

Would love feedback on the approach. Has anyone else experimented with LLMs learning from their own experience?

user:	pratikk10
created:	September 3, 2018
karma:	1