A verification loop 4x'd DeepSeek's intelligence, matching Opus at 1/7 the cost

A verification loop 4x'd DeepSeek's intelligence, matching Opus at 1/7 the cost(ironbee.medium.com)

15 points by sozal 2 hours ago | 7 comments

bel8 1 hour ago |

From what I understood, the gist of it is that uses a bunch of skills in a loop to help LLMs solve the tasks better via browser-testing, debugging, visual-testing, etc..

The skills can be found here:

https://github.com/ironbee-ai/ironbee-devtools-skills/tree/m...

Question that was left unanswered:

How much longer does a 4x DeepSkeep loop took compared to Opus?

_aavaa_ 2 hours ago |

Seems interesting, but buried under a mountain of written slop.

rightbyte 1 hour ago | |

It is marketing for some SaaS thing. However, I think something like this, "verification loops with some sort of tricks", is what Anthropic and "OpenAI" do behind the scene with their interfaces.

sozal 2 hours ago | |

Fair on the length. The verification loop took DeepSeek from ~20 to ~80 on Web-Bench, matching Opus at ~7x less cost. Curious if the method holds up for you.

_aavaa_ 1 hour ago | | |

Well I don’t know what the method is, there’s a mention of ironbee, but I’m not willing to spend the time digging through this to figure out what’s needed and how to set it up.