One Chat Window for Every Model I Use
I was bouncing between Claude, GPT, DeepSeek and local models in four different apps. So I put my own router in front of all of them.
I use a lot of models: Claude for code, GPT for some tasks, DeepSeek because it's cheap, and local Qwen models on my RTX 5070 Ti when I want privacy or I'm offline. That meant four interfaces and a habit of pasting the same prompt into several of them.
The setup now is one Open WebUI in front of everything. Requests go to a small FastAPI router I wrote, which forwards them to LiteLLM, and LiteLLM handles the actual providers. The router decides where each message goes. It doesn't ask a model to classify the prompt, which would cost a call and add latency. It uses plain rules: code-heavy prompts go one way, quick questions go to something cheap, and an explicit model choice always wins.
Two details made it feel solid. Failures escalate down a fallback chain, and a provider that fails three times in a row gets benched for five minutes, so one bad API key doesn't turn every message into a timeout. And a VRAM guard skips local models when the GPU is busy, which on this machine usually means a game is running. Strict local-only modes never fall back to the cloud.
The whole thing is plain asyncio and a YAML routing file, with no agent framework. It has tests, logs every routing decision to SQLite, and has quietly become how I talk to every model I use.