Ferrogate: Self-Hosted AI Gateway

Ferrogate is an open-source AI gateway and reverse proxy built in Rust on top of Pingora. It gives you production-grade LLM traffic control for self-hosted deployments: OpenAI-compatible Chat/Responses endpoints, multi-provider routing with fallback, virtual API keys, per-key budgets, exact-match caching, MCP tool execution, observability, an admin dashboard, cluster operations, and automatic HTTPS.

• You need a self-hosted alternative to LiteLLM or Portkey with stronger performance characteristics.
• You want provider routing and fallback without sending traffic through a third-party SaaS.
• You need per-team or per-project API key management with budget enforcement.
• You want to cache LLM responses at the gateway level to reduce costs.
• You need MCP tool execution proxied through your gateway.

• Rust + Pingora performance. Built on Cloudflare's Pingora framework — async, zero-copy, high throughput.
• OpenAI-compatible API. Drop-in base URL for agents and tools that speak OpenAI protocol.
• Provider routing. Route requests to different providers with fallback chains and load balancing.
• Virtual API keys. Issue scoped keys with per-key budgets, rate limits, and usage tracking.
• Exact-match cache. Cache identical requests to avoid redundant inference costs.
• MCP tool execution. Proxy and execute MCP tool calls through the gateway.
• Observability. Built-in request tracing and an admin dashboard for monitoring.
• Automatic HTTPS. TLS termination handled by the gateway itself.
• Cluster ops. Designed for multi-node deployments, not just single-instance.

• New project (launched mid-2026) — smaller community than LiteLLM or Portkey.
• Rust expertise helpful for deep customization or contribution.
• Documentation is still maturing compared to established gateways.
• No managed cloud offering — strictly self-hosted.

Clone the repository, build with Cargo, and configure providers via YAML. The admin dashboard provides a UI for key management and monitoring.

• GitHub: lianluo-esign/ferrogate
• Language: Rust
• License: See repository