litellm

litellm is an open source AI gateway that lets developers call 100+ AI model providers through one OpenAI-style interface with cost tracking and load balancing.

What it is good for

  • Switching between AI model providers without rewriting code
  • Tracking spend on AI model calls across a team
  • Routing AI requests across several models for reliability
  • Connecting MCP tools to any AI model

At a glance

Who it is for
Setup difficulty
Technical ยท Guided command-line setup

Level 4 of 5. You copy and paste commands into a terminal by following the instructions. Comfortable territory for vibe coders.

Cost
Free and open source
Type
Framework or SDK
Tasks
Works with
OpenAI, Anthropic, Bedrock, Azure, Vertex AI, Cursor

How to get started with litellm

  1. Install the gateway with uv tool install 'litellm[proxy]'.
  2. Start it with a model using litellm --model gpt-4o.
  3. Point an OpenAI client at http://0.0.0.0:4000 as the base URL.
  4. Add your provider API keys, such as OPENAI_API_KEY, as environment variables.

From the project's own documentation. If a step does not match what you see, the official pages have the latest instructions.

Before you try it

  • You need API keys from each model provider you want to call.
  • The Python SDK requires writing code to use it.
  • The README mentions an enterprise tier and a hosted proxy alongside the open source version.

Momentum

Rank this week
Not ranked (last #45, week 45, 2026)
Best rank
#45 week 45, 2026
GitHub stars
63,335

GitHub numbers as of Nov 16, 2026. Updated weekly.

62.5k 62.9k 63.4k Nov 2, 2026 Nov 16, 2026
GitHub stars for litellm: 62,543 on Nov 2, 2026, 63,335 on Nov 16, 2026. Source: GitHub.

Ranking history

First ranked in week 45, 2026.

Questions

Is litellm free?
Yes. It is free and open source.
Do I need to know how to code to use litellm?
You do not need to write code, but setup means copying and pasting commands into a terminal by following its instructions.
What does litellm work with?
According to its own documentation, it works with OpenAI, Anthropic, Bedrock, Azure, Vertex AI, and Cursor.
How do I get started with litellm?
1. Install the gateway with uv tool install 'litellm[proxy]'. 2. Start it with a model using litellm --model gpt-4o. 3. Point an OpenAI client at http://0.0.0.0:4000 as the base URL. 4. Add your provider API keys, such as OPENAI_API_KEY, as environment variables.
What should I know before trying litellm?
You need API keys from each model provider you want to call. The Python SDK requires writing code to use it. The README mentions an enterprise tier and a hosted proxy alongside the open source version.
What is litellm good for?
Switching between AI model providers without rewriting code, tracking spend on AI model calls across a team, routing AI requests across several models for reliability, and connecting MCP tools to any AI model.

Similar agents to litellm

  • New API

    Also a self-hosted gateway that routes to many model services and tracks costs, but without a Python SDK focus.

  • ClawRouter

    Focuses on sending each request to the cheapest capable model, rather than a general gateway with guardrails and logging.

  • LangChain

    Builds agents and LLM apps on top of models, while this unifies access to the models themselves.

  • Stripe AI

    A set of SDKs for connecting Stripe payments to AI apps, not a model gateway.

  • crewAI

    CrewAI is a Python framework that lets developers build teams of AI agents that split up and complete multi-step work.

  • Conductor

    Conductor is a workflow engine that runs multi-step jobs and AI agents reliably, so steps survive crashes and retries, for developers.

Also compared with litellm on these pages:

Last updated Nov 16, 2026. Tracked since Nov 2, 2026. Spotted an error? Tell us.