An LLM gateway is one front door to multiple providers — routing, keys, fallbacks, and usage accounting in one place.
In my daily workflow
- I route by role: copilot vs embed vs batch eval.
- I configure fallbacks when a provider rate-limits or errors.
- I centralize API keys in env — never in client bundles.
- I review cost and latency dashboards weekly.
How it makes me work smarter
Gateways stop provider lock-in from spreading across the codebase. When Anthropic is slow or OpenAI is cheaper for a subtask, I change routing in one module. Token accounting becomes possible — important on a portfolio with public assistant traffic.
My setup
- Provider router in `src/features/ai` with role-based model resolution
- Settings table for copilot provider/model in admin
- Structured logging of model ID and token usage
- Optional AI Gateway (Vercel) for unified billing
On this portfolio
Admin settings pick the copilot provider and model; server code resolves the primary model through a single router. Swapping models for experiments does not require hunting through every API route.
