---
name: context-window-cost-tradeoff
description: Use when deciding how much text to put in the prompt versus retrieve or summarize separately.
---

# Context Window vs Cost Tradeoff

## Overview

Bigger context windows tempt you to paste everything. **Cost and latency scale with tokens** — stuffing 200k tokens because you can often burns budget without improving answers. Retrieval, summarization, and selective @-mentions usually win.

## When to Use

- Prompts growing past a few thousand tokens
- Monthly token bill climbing faster than traffic
- Model supports huge context and team wants "just include all docs"
- Skip when the entire corpus truly fits cheaply and static (rare)

## Quick Reference

| Situation | Prefer |
|---|---|
| Large doc corpus | RAG / chunk retrieval |
| Long chat history | Summarize older turns |
| Repeat static instructions | Cache system prompt |
| One-off deep dive on 3 files | @-mention or attach only those |
| Need full contract text once | Large window for that request only |

## Implementation

1. **Measure current tokens per request** — input breakdown by section.
2. **Identify redundant context** — duplicated docs, full thread, verbose tool dumps.
3. **Replace with retrieval or summary** — keep only what changes the answer.
4. **A/B cost and quality** — smaller context + RAG vs mega-prompt.
5. **Set soft caps** — max input tokens per route with trim strategy.
6. **Monitor p95 tokens** in observability — alerts on drift.

## On This Portfolio

Assistant injects retrieved chunks with titles and URLs, not full pages. Copilot graph builds state per node instead of one mega-prompt. Cursor skills and @-files beat pasting the whole repo.

## Common Mistakes

- Pasting entire knowledge bases because the model "supports it"
- Sending full tool JSON responses into the next turn without compression
- Optimizing context size without measuring retrieval quality drop

## Related Skills

- RAG pipeline design
- Context engineering
- Caching strategy for LLM calls