Project
Multi-Provider LLM Proxy
One endpoint for every LLM provider, with automatic failover and cost control built in.
The problem
Wiring a product directly to one model provider is a standing liability. When that provider has an outage, raises prices, or changes a model, the whole product feels it, and nobody can say what a single request actually costs.
What I built
A FastAPI gateway that puts one messages-first endpoint in front of every provider. It routes each call, fails over automatically, normalises responses, estimates cost per request, and logs everything, with new providers added through a small registry. It ships in Docker with test coverage.
The result
Switching or combining providers becomes a config change instead of a rewrite, and every call is observable and costed.
Stack
PythonFastAPIOpenAIAnthropicDockerpytest