Many teams connecting LLMs to external tools eventually encounter the same architectural issue: as more tools and agents are added, the integration pattern becomes an N×M mesh of direct connections. Each agent implements its own auth, retries, rate limiting, and logging; each tool needs credentials distributed to multiple places and observability becomes fragmented. We built LLM gateway with this goal to provide a single place to manage authentication, authorization, routing, and observability for MCP servers, with a path toward a more general agent-gateway architecture in the future. The system includes a central MCP registry, support for OAuth2/DCR integration, Virtual MCP Servers for curated toolsets, and a playground for experimenting with tool calls. Resources - Architecture Blog – Covers the N×M problem, gateway motivation, design choices, auth layers, Virtual MCP Servers, and the overall model. https://www.truefoundry.com/blog/introducing-truefoundry-mcp... Tutorial – Step-by-step guide to writing an MCP server, adding Okta-based OAuth, and integrating it with the Gateway. https://docs.truefoundry.com/docs/ai-gateway/mcp-server-oaut... Feedback on gaps and edge cases is welcome. |