LiteLLM Proxy for Multi-Provider Routing

Agents and tools speak OpenAI-style APIs. Providers do not. LiteLLM proxy gives you a single base URL and model aliases across clouds and local servers.

• One endpoint for mixed providers
• Model aliases like smart, cheap, local-coder
• A place to attach logging and spend limits later

1. Install LiteLLM proxy via the current upstream recommended method.
2. Create a config mapping model aliases to provider model ids.
3. Export provider keys in the environment (never commit them).
4. Start the proxy on a local port.
5. Point one agent client at the proxy base URL and call an alias.

Illustrative config shape:

model_list:
  - model_name: cheap
    litellm_params:
      model: openai/gpt-4o-mini
  - model_name: smart
    litellm_params:
      model: anthropic/claude-opus-4-8

• curl chat completion against an alias
• Confirm the upstream provider dashboard shows the request
• Fail closed when a key is missing (do not silently fall through to the wrong model)

• 401/403: wrong key or org permissions
• Model not found: alias vs upstream id mismatch
• Tool calls fail for one provider only: schema differences — test tool-use per provider

A proxy is not governance. Approval gates, allowlists, and spend caps still belong in the agent broker or gateway policy layer.