The Production RAG Reference Architecture

Here is the whole system on one page — the blueprint that turns the previous nine articles into something you can deploy.

The ingestion pipeline (offline)

1. Clean source docs (strip boilerplate, fix encoding).
2. Chunk structurally, 300–600 tokens, ~15% overlap.
3. Enrich each chunk with metadata (source, section, date, url).
4. Embed with a versioned model.
5. Index into Postgres/pgvector with an ANN index + a keyword index.

The query pipeline (online)

1. (Optional) Rewrite the query with a small model.
2. Hybrid retrieve — vector + keyword, fused with RRF, top 30–50.
3. Re-rank with a cross-encoder; keep top 3–8.
4. Confidence gate — if the top score is weak, return "I don't know."
5. Generate grounded, with citations, from the kept chunks.
6. Faithfulness check the output; cache the answer.

The loop that keeps it honest

Wrap it in evaluation + observability: run the eval set on every change (recall, faithfulness, relevance), and log real queries with their retrieval scores so you can grow the eval set from production.

That's production RAG: measurable retrieval, grounded generation, honest under uncertainty, and affordable at scale. You now have the map and the mechanics.