AI costs stay manageable by routing, caching, smaller models, and not sending the kitchen sink into every prompt.
In my daily workflow
- I review token usage by route weekly.
- I downgrade model tier for classification and routing steps.
- I cache embeddings and stable retrieval results where invalidation is clear.
- I trim tool outputs before the next model call.
How it makes me work smarter
Public assistant traffic scales with attention, not revenue — cost discipline matters. Smart routing often cuts bills 40% without quality loss because most steps are easy. Cost optimization pairs with eval gates so savings do not become regressions.
My setup
- Role-based models via provider router
- Embedding reuse on unchanged content
- Aggressive context pruning
- PostHog or provider dashboards for spend alerts
On this portfolio
Admin picks copilot models; fast models handle routing while strong models write. Knowledge embeddings regenerate only on publish, not every read — compute on write, not on browse.
