LLM Selection by Task
Overview
One flagship model for everything is expensive and slow. LLM selection by task assigns each step the smallest model that clears the bar: fast models for routing and classification, stronger models for synthesis and reasoning.
When to Use
- Designing a multi-step pipeline (retrieve → classify → generate)
- Copilot feels slow or costly on simple requests
- Eval shows a cheap tier passes on subset of tasks
- Skip when a compliance rule mandates one approved model
Quick Reference
| Task type | Typical tier |
|---|
| Intent routing / classification | Fast, cheap |
| JSON extraction from short text | Fast with schema validation |
| Long-form synthesis, multi-doc reasoning | Strong |
| Code generation across files | Strong + IDE agent |
| Embeddings | Dedicated embedding model |
Implementation
- List pipeline steps — each gets its own model decision.
- Define pass criteria per step — not one global "quality" score.
- Benchmark 2 tiers per step on 30+ real inputs.
- Assign models in router config — role-based, not hardcoded per route file.
- Log model ID per request — so you can prove the split works in production.
- Re-eval when providers ship new tiers — the map changes quarterly.
On This Portfolio
Admin settings pick copilot provider/model; server code resolves via a single router (copilot role). Assistant retrieval and generation can use different cost profiles — the task picks the tier, not habit.
Common Mistakes
- Using the strongest model for boolean classification
- Selecting models from leaderboard hype, not your eval set
- Changing model in one file while ten routes still call the old default
Related Skills
- Cost-latency-accuracy tradeoff analysis
- Provider-agnostic abstraction layer design
- Latency benchmarking across providers