Cost per outcome: measuring the real economics of AI workflows Hi HN, I’m the technical founder of botanu (https://www.botanu.ai
). I started building this after repeatedly running into the same problem on AI teams: we could see total LLM spend, but we couldn’t answer a simple question: “What did one successful outcome actually cost?” In real systems, a single business event often requires multiple attempts before it succeeds — retries, fallbacks, tool calls, escalations, async workers, etc. Most tooling measures individual model calls or sometimes a single workflow run, which hides the real cost. The unit that matters to the business is the outcome, not the individual call. The approach I’m exploring in botanu: An event_id represents the business intent (e.g., resolve support ticket, generate report) Each attempt is a run with its own run_id All runs share the same event_id A final outcome is emitted for the event (success / failure / partial) Cost per outcome = sum of all runs for that event, including failed attempts Run context propagates across services using W3C Baggage (OpenTelemetry) so the event can be traced across distributed systems. The idea is to make AI economics measurable at the outcome level, not just tokens or model calls. On the engineering side, teams can use this to: experiment with models and workflows in a dev playground compare architectures and retries optimize the cost of producing a successful outcome On the business side, it helps teams understand: unit economics of AI features cost per customer action how to support outcome-based pricing models. I’m curious how others here are thinking about AI unit economics and measuring outcomes in production systems. Happy to answer technical questions or get critical feedback. Deborah deborah [at] botanu dot ai |