LiteLLM

What it is

LiteLLM is an open-source Python SDK and proxy server (“AI Gateway”) from Berri AI that calls 100+ LLM APIs in OpenAI format with cost tracking, guardrails, load balancing and logging (GitHub repo description, read 2026-10-08). Its docs now also present an MCP Gateway (agents get governed access to tools) and an A2A Agent Gateway next to the LLM gateway.

Maker and licence

• Maker: BerriAI (repo BerriAI/litellm). Licence: MIT for everything outside the enterprise/ directory; content under enterprise/ has its own licence (LICENSE file, read 2026-10-08). GitHub reports the licence as “NOASSERTION” because of this split.
• Stars: 60,373 on 2026-10-08 (GitHub API). Latest release read that day: v1.104.2 (2026-10-08).

Position

LLM gateway first, with an MCP gateway and A2A agent gateway added to the same proxy. Self-hosted proxy plus admin UI.

MCP gateway capabilities (per docs.litellm.ai/docs/mcp, read 2026-10-08)

• Register MCP servers in config.yaml or the admin UI; transports include streamable HTTP, SSE and stdio (stdio needs LITELLM_ENABLE_MCP_STDIO=true in the proxy environment). Servers can be given aliases.
• Upstream auth options listed in the docs nav: OAuth (discovery, dynamic client registration, PKCE), OAuth passthrough, on-behalf-of (OBO), Okta ID-JAG, AWS SigV4 (for Bedrock AgentCore hosted servers), static headers, per-user and per-key upstream credentials, zero-trust JWT signer.
• Governance pages in the docs: permission management, granting MCP server access to keys and teams, MCP cost tracking, MCP guardrails, tool policies, toolsets, semantic tool filter, tool search, server submissions.
• The LLM side adds budgets and rate limits, routing and fallbacks, caching, spend tracking and logging callbacks.

MCP spec support

Not stated on the pages read; see Open items.

Maturity

60,373 GitHub stars and a release dated 2026-10-08 (both read that day). Individual MCP features are documented but their GA/beta status is not marked on the overview page read.

Fit and limits (opinion)

• Strong fit if you already route LLM traffic through LiteLLM and want one control plane for models and tools, with per-key and per-team budgets.
• Python proxy with a database for the full feature set; the docs also mention a Rust core. Some governance features sit under the enterprise/ licence, so check which you need before relying on them.

Related notes

• agentgateway, Arcade MCP Gateway, Portkey, OmniRoute: other gateway notes in this folder
• MCP security with a gateway, Model Context Protocol, A2A protocol
• HyperTool MCP (tool-subset proxy) and Tool Platforms (managed tool/integration platforms) cover adjacent ground and are not repeated here

Self-learning

• Docs home: https://docs.litellm.ai/docs/
• MCP gateway overview: https://docs.litellm.ai/docs/mcp
• Source: https://github.com/BerriAI/litellm

Sources

• https://github.com/BerriAI/litellm (API: description, stars, release; fetched 2026-10-08)
• https://raw.githubusercontent.com/BerriAI/litellm/main/LICENSE (fetched 2026-10-08)
• https://docs.litellm.ai/docs/mcp (fetched 2026-10-08)

Open items

• MCP spec revision supported: not stated on the pages read.
• Which MCP features sit under the enterprise/ licence: not checked.
• No currency prices recorded; see the vendor site for plans.