---
name: failure-mode-mapping
description: Use before shipping any LLM-backed feature, especially ones that take actions or show output directly to users.
---

# Failure Mode Mapping Before Build

## Overview

LLM systems fail in predictable ways: hallucination, bad retrieval, malformed JSON, provider timeout, runaway agent loops, and silent cost spikes. **Failure mode mapping** lists these before build so you design guardrails instead of incident-driven patches.

## When to Use

- Any user-facing assistant or action-taking agent
- Features that write to DB, send email, or change production state
- Skip for personal one-off scripts with no blast radius

## Quick Reference

| Failure mode | Guardrail |
|---|---|
| Hallucinated facts | RAG + citations; refuse without source |
| Wrong retrieval | Eval retrieval separately; metadata filters |
| Invalid structured output | Zod/schema validation + retry |
| Provider outage | Fallback model or graceful message |
| Runaway tool loop | Max iterations, timeouts, allowlists |
| Cost spike | Rate limits, token caps, caching |

## Implementation

1. **Brainstorm failures** — user, model, retrieval, tool, infra layers.
2. **Rate severity** — user harm, data harm, financial harm.
3. **Assign strategy** — prevent, detect, or accept explicitly.
4. **Build prevent/detect** before launch for high-severity items.
5. **Add eval cases** for each mapped failure — regression forever.
6. **Document accepted risks** — signed tradeoffs, not surprises.

## On This Portfolio

Public assistant refuses without corpus support; copilot tools are auth-scoped and schema-validated. PostHog and Sentry feed back production failures into eval ideas — mapping continues after ship.

## Red Flags

| Thought | Reality |
|---|---|
| "We'll harden after launch" | First failure may be public |
| "Good model won't hallucinate" | All models do under some inputs |
| "Retries fix everything" | Retries without caps multiply cost |

## Common Mistakes

- No provider fallback on a user-critical path
- Validating only the happy path in CI
- Letting agents call destructive tools without human gates

## Related Skills

- Eval harness design
- Guardrails & agent safety design
- Human-in-the-loop checkpoints