LiteLLM

As soon as an agent lives longer than a weekend, the provider question shows up: a different model for the expensive steps, a local one for the cheap ones, a fallback for when an API stalls. Building that straight into the agent spreads provider knowledge across the whole codebase.

LiteLLM pulls that layer out. The agent keeps speaking the OpenAI chat format and the proxy translates it to whatever actually runs behind it. The translation itself is the less interesting part; what grows around it matters more: one place where keys live, where calls are logged, where cost and limits per user become visible. Exactly the things you skip in a first prototype and painfully miss in first real operation.

Useful as pure reading material too: the provider adapters show very concretely where the APIs differ - in tool calls, in streaming and in error messages.