Model Selection Based on Project Constraints
Overview
Benchmarks are useless if the model was never eligible. Project constraints — budget, latency SLA, data residency, team ops — shrink the shortlist before you spend time on bake-offs.
When to Use
- Enterprise or regulated client with data-handling rules
- Fixed monthly AI budget
- p95 latency SLA under 2s for user-facing chat
- Skip when prototyping locally with no compliance or spend cap
Quick Reference
| Constraint | Effect on selection |
|---|
| PII cannot leave VPC | Self-host or approved private endpoint |
| <$500/mo AI spend | Cap context, smaller models, aggressive caching |
| p95 < 2s | Avoid huge context + slow tier on hot path |
| No GPU ops team | Rule out self-hosted unless managed offering |
Implementation
- Gather hard constraints — legal, finance, product SLOs (not preferences).
- Eliminate ineligible providers/models — document why ruled out.
- Benchmark only the remainder on your eval set.
- Record decision matrix — constraint → choice for audits later.
- Revisit when constraints change — new BAA, budget bump, new region.
On This Portfolio
Vercel-hosted Next.js + API keys in env means hosted providers with server-side-only keys. Public assistant never sends admin data to models — constraint shaped auth boundaries before model choice mattered.
Common Mistakes
- Running a two-week bake-off on a model legal already rejected
- Ignoring latency SLA until UX testing fails
- Letting "we like Claude" override residency requirements
Related Skills
- Build vs buy (API vs self-host)
- Token cost estimation & budgeting
- Open-source vs closed-source model evaluation