Maker here. Favur is a multi-agent harness written in Python: a team of 14 specialized agents — planner, architect, coder, tester, reviewer, builder among them — coordinated by Favur itself, not by an LLM. It takes a written statement of work and produces completed code, with no hand-holding or constant guidance along the way. Favur Evals scores those runs across models, using the same standardized SOW every time. The fastest way to get it: https://favur.dev/drive/top_run — an interactive replay of whatever run currently tops the board. Every run is captured end-to-end, so you can watch the agents plan, hand work to each other, write the tests before the code exists, go through review, and ship — at your own pace, exactly as it happened. (Faithful playback of the record, not a live run.) Every run on the board has its replay linked from its details page. Scoring: each run gets a composite across eight engineering subjects — code quality, test quality, cost efficiency, velocity, tool discipline, effort efficiency, process discipline, deliverables — computed from the run's own artifacts (lint, complexity, its pytest results, tool telemetry). Click any score and it expands into its formula. Favur is a multi-model harness — different models can take different seats in the same run — and what we keep finding is that no single model leads every part of the job. Currently the top composite is an all-Meta Muse Spark run, while the strongest test suites, the best value-per-dollar, and the cleanest zero-failure execution belong to three other configs. The per-seat behavior fingerprints (cache utilization, reasoning depth, tool cadence) are my favorite way to compare models. Caveats: every model runs inside our scaffolding, so treat the scores as relative rankings inside this harness, and some configs only have a run or two so far. Favur itself is closed-source and invite-only — please sign up for email updates to get news and future access. Happy to answer anything about the harness, the scoring, or the replays. |