---
name: data-availability-audit
description: Use before committing to a RAG, fine-tuning, or eval approach that depends on a specific dataset existing.
---

# Data Availability Audit

## Overview

RAG, fine-tuning, and eval harnesses all assume data exists, is legal to use, and is clean enough to work. A **data availability audit** inspects real samples before you commit to an approach — schemas lie; rows do not.

## When to Use

- Before RAG (need retrievable corpus)
- Before fine-tuning (need labeled examples)
- Before promising accuracy from "we have logs somewhere"
- Skip for pure prompt engineering with no external data

## Quick Reference

| Approach | Verify |
|---|---|
| RAG | Corpus size, chunkability, access rights, update cadence |
| Fine-tuning | Label quality, class balance, held-out set |
| Eval harness | Ground truth or calibrated judge exists |
| Tool calling | APIs authenticated and rate limits understood |

## Implementation

1. **Pull 50–100 real records** — not ER diagrams.
2. **Check volume** — enough for retrieval recall or training generalization?
3. **Check quality** — duplicates, OCR garbage, wrong language, broken HTML.
4. **Check PII and licensing** — can this train or be shown to a model vendor?
5. **Check freshness** — stale corpus → confident wrong answers.
6. **Write go/no-go** — if data fails, change approach before build.

## On This Portfolio

Assistant retrieval only indexes **published** knowledge — admin draft/archived status is the data gate. Job-fit analytics assumes recruiter feedback exists; without labels, the eval loop cannot close.

## Common Mistakes

- Trusting a schema export without reading documents
- Discovering GDPR or contract blocks mid-sprint
- Embedding admin-only drafts into a public RAG index

## Related Skills

- RAG vs fine-tuning decision
- Chunking strategy selection
- Instruction-tuning dataset curation