LiteLLM

Call over 100 Large Language Models using a single, consistent OpenAI input/output format.

Ensures text responses are always available at a standard path, simplifying integration across providers.

Automatically translates requests and responses to match the specific requirements of various LLM providers.

Implements automatic retries and fallbacks across multiple LLM deployments for enhanced reliability.

Monitor LLM expenditures and set budget limits per project to manage costs effectively.

Enforce usage limits per user or project via the proxy to prevent abuse and manage resource allocation.

Manage access and track spend for individual users or projects using customizable virtual keys within the proxy.

Easily log LLM inputs and outputs to popular observability tools like Lunary, MLflow, Langfuse, and Helicone.

Maps exceptions from all supported providers to OpenAI's exception types for consistent error management.

Provides an endpoint to see how LiteLLM normalizes and transforms LLM API requests internally for debugging.

Monitor real-time metrics for streaming responses via custom callback functions.

Deploy a central LLM Gateway to unify access, load balance, and track costs across multiple LLMs for platform teams.

Integrate directly into Python applications for calling 100+ LLMs with built-in load balancing and cost tracking.