Overview
LiteLLM is an open-source platform designed to streamline large language model (LLM) integration and management, offering developers and platform teams a unified gateway for accessing multiple AI models with minimal complexity.
Key Features
- Support for 100+ LLM providers (OpenAI, Azure, Gemini, Bedrock)
- OpenAI-compatible API format
- Automatic spend tracking and budgeting
- Cost attribution by key/user/team
- Comprehensive logging and observability
- Virtual keys and rate limiting
- LLM fallback mechanisms
- Prometheus metrics integration
Use Cases
- Enterprise LLM platform management
- Multi-model AI application development
- Cost control and usage tracking
- Simplified model access for development teams
- Standardizing AI model interactions across organizations
Technical Specifications
- Open-source Python SDK
- Docker support
- Proxy gateway architecture
- Integrations with Langfuse, OpenTelemetry
- Supports tag-based spend tracking
- Compatible with major cloud platforms
