LiteLLM config.yaml — The Complete Configuration Playbook for Production LLM Proxy Deployment

If you're running multiple LLM providers through LiteLLM, your config.yaml is the single file that controls routing, pricing, fallback chains, rate limits, and observability. Get it right once and every API call that flows through your proxy inherits production-grade resilience. Get it wrong and you'll spend nights debugging silent failures.

This playbook covers every critical section of config.yaml with actionable configurations you can copy, adapt, and deploy immediately.

Key Takeaways

• One file, full control: config.yaml manages model routing, pricing overrides, fallback chains, rate limits, permissions, and logging — all in a single declarative file.
• Model mapping is the foundation: Every request starts with the model_map section. Misconfigured aliases cause silent 404s and billing confusion.
• Fallback chains prevent outages: Define ordered fallback providers so your application degrades gracefully instead of throwing errors.
• Use params for defaults: Global params let you set default temperature, max_tokens, and stop sequences without touching application code.
• Cost tracking requires explicit config: Without per-model cost entries, LiteLLM cannot calculate spend or enforce budget limits.
• Start minimal, iterate fast: A working config.yaml with 5 models beats a perfect one with 50 that never ships.

What Is LiteLLM config.yaml?

config.yaml is the primary configuration file for the LiteLLM Proxy Server. It is a YAML file that tells the proxy how to route requests, which models are available, what pricing to apply, which providers to fall back to on failure, and how to log and monitor traffic.

When you start the proxy with litellm --config config.yaml, LiteLLM reads this file once at startup and uses it for the lifetime of the process. Changes require a restart or a configuration reload.

Core Sections of config.yaml

1. model_info — Registering Your Models

The model_info section is the model registry. Each entry defines how LiteLLM treats a specific model alias, including its provider, pricing, and metadata.

model_info:
  - model_name: gpt-4o
    litellm_params:
      model: openai/gpt-4o
      api_key: os.environ/OPENAI_API_KEY
    model_id: "gpt-4o-001"
    base_model: openai/gpt-4o
  - model_name: claude-sonnet-4
    litellm_params:
      model: anthropic/claude-sonnet-4-20250514
      api_key: os.environ/ANTHROPIC_API_KEY
    model_id: "claude-sonnet-4-001"
    base_model: anthropic/claude-sonnet-4-20250514
  - model_name: deepseek-chat
    litellm_params:
      model: deepseek/deepseek-chat
      api_key: os.environ/DEEPSEEK_API_KEY
    model_id: "deepseek-v3"
    base_model: deepseek/deepseek-chat
  - model_name: gemini-flash-2.0
    litellm_params:
      model: gemini/gemini-2.0-flash
      api_key: os.environ/GEMINI_API_KEY
    model_id: "gemini-flash-201"
    base_model: gemini/gemini-2.0-flash

Pro tip: Always set model_id explicitly. Without it, LiteLLM generates internal IDs from the alias name, which can cause cache misses and logging ambiguity across environments.

2. model_group_alias — Flexible Model Renaming

Use model_group_alias to map multiple incoming aliases to the same underlying model. This is essential when different teams or applications reference the same model by different names.

model_group_alias:
  gpt4o: openai/gpt-4o
  gpt4o-mini: openai/gpt-4o-mini
  sonnet: anthropic/claude-sonnet-4-20250514
  "ds": deepseek/deepseek-chat

3. router_settings — Load Balancing and Routing

The router_settings section controls how requests are distributed across models and providers. This is where you define retry logic, fallback chains, and load-balancing strategies.

router_settings:
  num_retries: 3
  request_timeout: 60
  retry_policy: ["request_timeout", "context_length_exceeded", "rate_limit"]
  fallbacks:
    - gpt-4o:
        - gpt-4o-mini
        - deepseek-chat
      claude-sonnet-4:
        - gemini-flash-2.0
  allowed_fails: 3
 健康的_provider:
    - openai
    - anthropic
    - deepseek
    - google

Key behavior: When allowed_fails is set, LiteLLM automatically removes a provider from rotation after consecutive failures exceed the threshold. This prevents cascading failures from saturating your error budget.

4. general_settings — Proxy Behavior Controls

general_settings:
  proxy_port: 4000
  proxy_hostname: "0.0.0.0"
  pass_through_endpoints:
    - "/v1/chat/completions"
    - "/v1/completions"
    - "/v1/embeddings"
  default_model: "openai/gpt-4o"
  health_check_interval: 30
  server_read_timeout: 600
  server_host: "0.0.0.0"

5. env_file — Externalizing Secrets

Never hard-code API keys in config.yaml. Use the env_file directive to point LiteLLM at a separate environment file.

env_file: .env

Your .env file should contain only secrets:

OPENAI_API_KEY=sk-proj-xxxxxxxxxxxxxxxx
ANTHROPIC_API_KEY=sk-ant-xxxxxxxxxxxxxxxx
DEEPSEEK_API_KEY=sk-proj-xxxxxxxxxxxxxxxx
GEMINI_API_KEY=AIzaxxxxxxxxxxxxxxxx

6. budget_manager — Per-Model and Per-User Spend Limits

The budget manager prevents cost overruns by enforcing limits at the model, user, or project level.

budget_manager:
  model_budget:
    gpt-4o: 500.00
    claude-sonnet-4: 300.00
    deepseek-chat: 100.00
    gemini-flash-2.0: 50.00
  user_budget:
    team-alpha: 1000.00
    team-beta: 500.00
  project_budget:
    production: 5000.00
    staging: 500.00

7. litellm_settings — Global Request Parameters

These settings apply defaults to every request that passes through the proxy, eliminating the need to specify them in each application call.

litellm_settings:
  max_retries: 3
  default_headers:
    X-Request-Source: "litellm-proxy"
  temperature: 0.7
  max_tokens: 2048
  top_p: 1.0
  timeout: 60.0
  drop_params: true
  set_verbose: false

Critical setting: