Ask HN: How are you estimating API costs before committing to an architecture?

1 points by sarthakaggarwal 75 days ago | 2 comments

I've been building agent workflows on top of Claude's API and keep running into the same problem: I can't predict what a feature will cost until I've already built it and run it at some scale.

The input side is manageable — you can count tokens before sending. But output tokens are essentially unknowable upfront, and with agents that chain multiple calls (tool use, multi-turn reasoning, retries on failure), a single user action might be 3 API calls or 40. Multiply that by prompt caching behavior (which is great when it hits, but you can't always guarantee it), and the cost variance per task can easily be 10-20x.

This makes it really hard to do basic things like: set pricing for an AI-powered feature, decide whether an approach is even economically viable before building it, or give finance any kind of credible forecast.

What I've tried/looked at so far:

- Anthropic's token counting endpoint gives you exact input token counts pre-flight, which helps, but doesn't solve the output/chaining problem

- Logging everything post-hoc and building up averages per workflow — works but you're already committed by that point

- Setting hard spend caps at the API level — blunt instrument, doesn't help with per-feature attribution

- Looked at various OSS tools (ccusage, Langfuse, Helicone) — mostly retrospective dashboards, good for what did I already spend but not what will I spend

How are you handling this, especially if you're running agent-heavy workloads or building products where AI cost is a meaningful part of COGS. Are you doing any kind of pre-flight estimation? Cost-aware routing between models? Or just building first and optimizing later?

gdhaliwal23 63 days ago |

You're right that Langfuse and Helicone are retrospective — they show traces but don't roll up to "feature X costs $0.12/invocation." A middle ground that's worked: tag every API call with a feature ID during prototype runs and compute per-call cost from token counts using current pricing. Within a day or two of testing you have a real cost distribution per feature — enough to decide if the architecture is viable before building it out fully.

We open-sourced the cost calculation part for free: `ai-cost-calc` (PyPI and npm).

If you want to cap the budget you can also wrap your AI calls with a guarded call method, the budgets can be configured for customers and/or features on https://margindash.com

verdverm 75 days ago |

I stay relatively human-in-the-loop to keep costs down and quality up. My custom agent setup also has some "weird tricks" to keep context size down. Starting with a cheaper model like gemini-3-flash has good ROI too.