For a B2C AI SaaS product, the need to route every request via the backend (to protect keys) is one of the hurdles in offering low-latency experiences. The only solution is to ask the user to bring their own keys, which is a poor UX requiring them to sign up with each separate provider.

Anyone's got a better idea?