Ask HN: Best AI Gateway?

2 points by petemilly 8 days ago | 3 comments

What AI gateways work well? We currently use Anthropic's router directly for our AI agent (on val.town) but are considering adding OpenRouter or the Vercel AI Gateway, or something else entirely.

I know little about gateways except that they seem to be the new VS Code forks (i.e. everyone is doing it). My cursory research tells me that OpenRouter has the most models/providers, but charges a 5.5% pay-as-you-go platform fee (we pass-through inference costs to customers) and doesn't support features like prompt caching.

We're mostly a Cloudflare shop, and I learned minutes ago that they, too, have an AI gateway--but so far the "Best LLM Gateway 2026" type articles I've found are published by...companies with a gateway

crawdog 6 days ago |

I have been working on a project that is built off of Cloudflare's pingora project - https://sbproxy.dev. This provides all of the features of a LiteLLM, but can also support your API traffic too. I worked on the edge infrastructure for Walmart, and took those learnings and build a fully open source gateway.

Take a look - should be able to address your needs.

santiago-pl 5 days ago |

I’ve created an honest, reproducible benchmark of self-hosted AI gateways literally two days ago. I posted it on HN here: https://news.ycombinator.com/item?id=48688213

To be totally transparent, I’m the author of GoModel, but the benchmark is fully reproducible. It might be helpful in making your mind.

verdverm 8 days ago |

Check out OpenCode, they are pretty good, have no markup, and an enterprise sales pipeline. They support Big Ai and Open (Weight) Ai models