Failure Mode Mapping Before Build

Overview

LLM systems fail in predictable ways: hallucination, bad retrieval, malformed JSON, provider timeout, runaway agent loops, and silent cost spikes. Failure mode mapping lists these before build so you design guardrails instead of incident-driven patches.

When to Use

• Any user-facing assistant or action-taking agent
• Features that write to DB, send email, or change production state
• Skip for personal one-off scripts with no blast radius

Quick Reference

Implementation

1. Brainstorm failures — user, model, retrieval, tool, infra layers.
2. Rate severity — user harm, data harm, financial harm.
3. Assign strategy — prevent, detect, or accept explicitly.
4. Build prevent/detect before launch for high-severity items.
5. Add eval cases for each mapped failure — regression forever.
6. Document accepted risks — signed tradeoffs, not surprises.

On This Portfolio

Public assistant refuses without corpus support; copilot tools are auth-scoped and schema-validated. PostHog and Sentry feed back production failures into eval ideas — mapping continues after ship.

Red Flags

Common Mistakes

• No provider fallback on a user-critical path
• Validating only the happy path in CI
• Letting agents call destructive tools without human gates

Related Skills

• Eval harness design
• Guardrails & agent safety design
• Human-in-the-loop checkpoints