litellm
litellm is an open source AI gateway that lets developers call 100+ AI model providers through one OpenAI-style interface with cost tracking and load balancing.
What it is good for
- Switching between AI model providers without rewriting code
- Tracking spend on AI model calls across a team
- Routing AI requests across several models for reliability
- Connecting MCP tools to any AI model
At a glance
- Who it is for
- Setup difficulty
- Technical ยท Guided command-line setup
Level 4 of 5. You copy and paste commands into a terminal by following the instructions. Comfortable territory for vibe coders.
- Cost
- Free and open source
- Type
- Framework or SDK
- Tasks
- Works with
- OpenAI, Anthropic, Bedrock, Azure, Vertex AI, Cursor
How to get started with litellm
- Install the gateway with
uv tool install 'litellm[proxy]'. - Start it with a model using
litellm --model gpt-4o. - Point an OpenAI client at
http://0.0.0.0:4000as the base URL. - Add your provider API keys, such as
OPENAI_API_KEY, as environment variables.
From the project's own documentation. If a step does not match what you see, the official pages have the latest instructions.
Before you try it
- You need API keys from each model provider you want to call.
- The Python SDK requires writing code to use it.
- The README mentions an enterprise tier and a hosted proxy alongside the open source version.
Momentum
- Rank this week
- Not ranked (last #45, week 45, 2026)
- Best rank
- #45 week 45, 2026
- GitHub stars
- 63,335
GitHub numbers as of Nov 16, 2026. Updated weekly.
Ranking history
Official links
Questions
- Is litellm free?
- Yes. It is free and open source.
- Do I need to know how to code to use litellm?
- You do not need to write code, but setup means copying and pasting commands into a terminal by following its instructions.
- What does litellm work with?
- According to its own documentation, it works with OpenAI, Anthropic, Bedrock, Azure, Vertex AI, and Cursor.
- How do I get started with litellm?
- 1. Install the gateway with uv tool install 'litellm[proxy]'. 2. Start it with a model using litellm --model gpt-4o. 3. Point an OpenAI client at http://0.0.0.0:4000 as the base URL. 4. Add your provider API keys, such as OPENAI_API_KEY, as environment variables.
- What should I know before trying litellm?
- You need API keys from each model provider you want to call. The Python SDK requires writing code to use it. The README mentions an enterprise tier and a hosted proxy alongside the open source version.
- What is litellm good for?
- Switching between AI model providers without rewriting code, tracking spend on AI model calls across a team, routing AI requests across several models for reliability, and connecting MCP tools to any AI model.
Similar agents to litellm
- New API
Also a self-hosted gateway that routes to many model services and tracks costs, but without a Python SDK focus.
- ClawRouter
Focuses on sending each request to the cheapest capable model, rather than a general gateway with guardrails and logging.
- LangChain
Builds agents and LLM apps on top of models, while this unifies access to the models themselves.
- Stripe AI
A set of SDKs for connecting Stripe payments to AI apps, not a model gateway.
- crewAI
CrewAI is a Python framework that lets developers build teams of AI agents that split up and complete multi-step work.
- Conductor
Conductor is a workflow engine that runs multi-step jobs and AI agents reliably, so steps survive crashes and retries, for developers.