The complete RAG pipeline, stage by stage

Nine stages, two paths, one shared artefact — the index — that connects them:

════════════ OFFLINE: INGESTION (runs on corpus change) ════════════

  source documents
        │
   [1] PARSE ──────────► text + structure + metadata          (06-01)
        │                 PDFs, tables, scans; failures are SILENT
        │
   [2] CHUNK ──────────► retrievable units with overlap        (06-02)
        │                 + dedup and boilerplate strip        (06-04)
        │                 + metadata attached                  (06-03, 07-05)
        │
   [3] EMBED ──────────► one vector per chunk                  (07-02)
        │                 model + version + prefix convention PINNED
        │
   [4] INDEX ──────────► sparse inverted index  (07-01)
                       + dense ANN index        (07-04)
                       + metadata store, filterable
                              │
                              ▼
                    ┌──────────────────┐
                    │   THE INDEX      │  ← the only shared state
                    └──────────────────┘
                              │
════════════ ONLINE: QUERY (runs per request) ══════════════════════
                              │
   [5] QUERY PREP ◄─── user question + conversation history     (12-11)
        │                 rewrite to a self-contained query
        │                 resolve the caller's entitlements     (07-05)
        │
   [6] RETRIEVE ───────► sparse leg + dense leg, both filtered
        │                 fuse with RRF                         (07-06)
        │                 → N candidates
        │
   [7] RERANK ─────────► cross-encoder scores each candidate    (07-07)
        │                 → reordered, thresholded
        │
   [8] ASSEMBLE ───────► edge-loaded context with headers       (07-08)
        │                 token budget respected                (04-06)
        │
   [9] GENERATE ───────► answer grounded in context, WITH        (07-11)
                         CITATIONS — or an honest decline

Why nine and not five. Most published diagrams show four or five boxes: embed, store, retrieve, generate. That compression is exactly why RAG systems fail in ways their builders cannot explain. Parsing is not part of chunking. Reranking is not part of retrieval. Assembly is not part of generation. Each of the nine has its own failure mode, its own measurable output, and its own owner. A stage you have not named is a stage you cannot attribute a failure to, and attribution is the whole game (07-10).

Mapping to NVIDIA's stated pipeline. The [NVIDIA-DOC] description covers the query path at a coarser granularity, and the mapping is exact:

Learn the NVIDIA ordering as an ordering; component-order questions are an explicit drill emphasis. Learn the nine-stage decomposition as your debugging vocabulary.

The NVIDIA products that sit on this pipeline [NVIDIA-DOC]:

Note carefully: NIM and Triton are deployment for the models this pipeline calls, not pipeline stages. 12-13 covers them properly, and confusing "how a model is served" with "what stage it serves" is a real distractor family.

L1 — The intuition you can carry into an exam

Two paths. Offline you turn documents into a searchable index. Online you turn a question into a search, narrow the results, arrange them, and answer with citations. The quality of the whole thing is the quality of its worst stage.

Three facts to carry:

• Nine stages: parse, chunk, embed, index, query-prep, retrieve, rerank, assemble, generate-and-cite.
• NVIDIA's query path: query → embedding model → vector match → retrieve and decode → synthesise and cite [NVIDIA-DOC].
• Chain, not sum. Find the worst stage.

L2 — What each stage takes in, puts out, and how it fails

This table is the lesson's core asset. Every row is a stage you can measure independently, and the "silent?" column is the one that decides how much monitoring each deserves.

Read the "silent?" column as a monitoring budget. Stages 1, 2, and 3 fail without any error, anywhere, ever. A parser that drops a table returns text. A chunker that splits an answer returns chunks. An embedder with the wrong prefix returns vectors. These three stages need assertions, because they will never volunteer a problem — and they are also the three that sit earliest, which means their failures propagate through everything downstream and get misattributed to whatever stage the team happens to be looking at.

The corresponding assertions, each cheap:

The self-retrieval assertion at stage 3 is the highest-value single test in the entire pipeline. It is one embedding call and one search, it catches nearly every encoder-consistency failure, and it belongs in CI.

L3 — The two paths' different lifecycles, and where they couple

The ingestion path and the query path run on completely different schedules, and most operational RAG problems live in the seam between them.

Where they couple, and what breaks:

The embedding model is shared state. Stage 3 encoded the corpus; stage 6's dense leg encodes the query. They must use the same model, version, prefix convention, and pooling. This is the coupling that produces the module's signature failure: mismatched encoders yielding plausible nonsense with high scores and no error (07-02). The engineering answer is a single module owning the embedding convention, imported by both paths.

Changing the embedding model invalidates the whole index. Vectors from two models are not comparable even at equal dimensionality. A model upgrade is therefore a full re-embed and re-index — a migration project, not a config change (12-12).

Metadata schema is shared state. Stage 2 attaches it; stages 6 and 8 depend on it. Adding a filterable field means re-ingesting, which is precisely why permission metadata has to be designed in before the pipeline is built (07-05) and why this module put access control at 07-05 rather than later.

Index freshness is a correctness property, not a performance one. Between a document changing and the index reflecting it, the pipeline confidently serves stale content (07-03). Permissions changing is the acute version of the same problem, and it is why ACL sync must be decoupled from content re-embedding (07-05).

Two things must be versioned together. The corpus snapshot and the embedding-model version form one logical unit. If you cannot say "this index is corpus snapshot X embedded with model Y at settings Z", you cannot reproduce a result, roll back, or explain a regression.

Four readings that exam items probe.

RAG and fine-tuning solve different problems. RAG injects facts with provenance and freshness; fine-tuning changes behaviour, style, and format. The confusable pair is on the study guide's explicit list and 11-08 owns the full decision rule. The one-line version: if the answer changes when the documents change, that is RAG; if the answer changes when you want a different kind of answer, that is fine-tuning.

Minimal RAG is a legitimate starting point and a bad ending point. Four stages is the right shape for a prototype. Its problem is not that it is wrong but that it is unattributable: when quality disappoints, there are four boxes and the real cause is usually in a stage the diagram does not name.

Long-context stuffing is a real alternative at small scale. If the corpus fits in the window, sending it whole eliminates every retrieval failure in this module. It does not scale, it is expensive per query, and it maximises lost-in-the-middle exposure (07-08). Considering it seriously is part of not over-applying RAG (07-12).

Only RAG gives you citations and access control. These are the two properties that most often make RAG the right architecture in an enterprise, and neither is available from a fine-tuned model — model weights cannot forget a fact, cannot cite a source, and cannot be filtered per user (13-05).

This is a constructed end-to-end trace with invented numbers, built so every stage boundary is inspectable. It is not a measurement of any real system.

The system. An internal support assistant over 12,000 chunks drawn from product documentation, release notes, and an archived support forum. Because 12,000 chunks is small, the dense index is exact in-memory search rather than an ANN index — the 07-04 argument applied.

The question. A support engineer authenticated as u_4417 (groups: support-tier2, all-staff) asks:

"customer on 4.2 is hitting ERR_CERT_AUTHORITY_INVALID after we rotated their CA — is this expected?"

Stage 1 — Parse

Source: 340 PDFs, 1,100 HTML pages, one CSV of release notes.

Extracted:      1,441 documents
Empty output:   3 documents      ← scanned PDFs with no OCR layer
Table-bearing:  61 documents     ← extracted as pipe-delimited text, not flattened prose
Assertion:      character count within 0.8×–1.2× of source estimate → 3 failures, flagged

The three empty documents are the stage's silent failure caught by an assertion. Without that assertion they would simply be absent from every future answer, and nobody would ever know why the certificate-rotation runbook could not be found.

Stage 2 — Chunk

Recursive chunking at ~400 tokens with 50-token overlap (06-02), boilerplate stripped and near-duplicates removed (06-04), metadata attached (06-03, 07-05).

Chunks before dedup:      13,884
Near-duplicate removed:   1,412    ← mostly a repeated legal footer
Boilerplate-only removed:   472    ← nav blocks
Final chunks:            12,000
Max chunk length:           487 tokens   ← under both the embedder's and the reranker's limits ✓
Assertion: known answer sentence wholly inside exactly one chunk → pass

That 1,412-chunk dedup is doing more work than it looks. A footer repeated across thousands of pages becomes the nearest neighbour of every query, because its vector sits at the centroid of the corpus. Removing it is a retrieval-quality intervention disguised as a hygiene task.

Stage 3 — Embed

Model:      pinned by name AND version
Prefix:     "passage: " for chunks, "query: " for queries   ← ONE module owns this
Pooling:    mean, as the model requires
Normalise:  L2, so cosine == dot product
Output:     12,000 × 768 float32 = 30.7 MB      (M0.4)

SELF-RETRIEVAL ASSERTION:
  embed chunk_0041's own text as a query
  → returns chunk_0041 at rank 1, similarity 0.998  ✓

That assertion is the pipeline's single highest-value test. Had the prefix been omitted on the query side, this check would have returned rank 1 at similarity ~0.72 or a different chunk entirely — and every downstream stage would have continued reporting success.

Stage 4 — Index

Sparse:  inverted index, one analyser shared by index and query paths   (07-01)
Dense:   exact in-memory NumPy matrix — 12,000 rows, recall 1.0 by construction  (07-04)
Meta:    acl_groups, classification, product_version, doc_date, status, source_type, doc_id, chunk_pos
Index identity recorded: corpus snapshot 2026-07-28 + embedding model vX.Y + prefix convention A

No vector database. At 30.7 MB and 7.7 million multiply-adds per query, one is unnecessary — and exact search means index recall is not a variable that can confound anything downstream.

Stage 5 — Query prep

Raw:      "customer on 4.2 is hitting ERR_CERT_AUTHORITY_INVALID after we rotated their CA
           — is this expected?"
History:  none (first turn) → no rewrite needed                (12-11)
Identity: u_4417 → groups [support-tier2, all-staff]           (07-05)
Filter:   acl_groups ∈ {support-tier2, all-staff}
          AND status = 'current'                               ← recency, per 07-03
Eligible: 9,120 of 12,000 chunks

Two filters, two different jobs. The ACL filter is a security boundary and must be inside the retrieval query (07-05). The status = current filter is a quality control addressing recency blindness (07-03). Both are pre-filters; with exact search both are a boolean mask applied before the dot product.

Stage 6 — Retrieve and fuse

Both legs run over the eligible 9,120, in parallel, top-50 each.

Sparse leg (BM25). ERR_CERT_AUTHORITY_INVALID is maximally rare, so it dominates:

rank 1  c8801   11.42   "ERR_CERT_AUTHORITY_INVALID: the presented chain terminates in an untrusted root"
rank 2  c2170    9.88   release note 4.2: "ERR_CERT_AUTHORITY_INVALID now includes the chain in the log"
rank 3  c5512    3.10   "certificate rotation checklist"
...

Dense leg (cosine). Recovers the mechanistic and procedural chunks that share no rare terms:

rank 1  c5512   0.81   "certificate rotation checklist"
rank 2  c8801   0.78   the direct explanation
rank 3  c6033   0.76   "after replacing a CA, clients must be restarted to reload the trust store"
...

Fusion (RRF, k_rrf = 60). For the four documents that matter:

c8801:  sparse 1 → 1/61 = 0.016393  +  dense 2 → 1/62 = 0.016129  =  0.032522
c5512:  sparse 3 → 1/63 = 0.015873  +  dense 1 → 1/61 = 0.016393  =  0.032266
c2170:  sparse 2 → 1/62 = 0.016129  +  absent          = 0        =  0.016129
c6033:  absent            = 0       +  dense 3 → 1/63 = 0.015873  =  0.015873

Fused: c8801 > c5512 > c2170 > c6033. Note the defect already familiar from 07-06: c6033 — the chunk stating that clients must be restarted to reload the trust store, which is the actual answer to the user's real problem — sits fourth, because only one leg found it.

Stage 7 — Rerank

The fused pool of 74 unique candidates goes to the cross-encoder. Scores for our four:

c6033   0.93   "after replacing a CA, clients must be restarted to reload the trust store"  ← the answer
c8801   0.91   the error's meaning
c5512   0.62   rotation checklist
c2170   0.28   a release note about log formatting

Reranked: c6033 > c8801 > c5512 > c2170. The reranker promoted c6033 from rank 4 to rank 1, because joint encoding let it see that "after replacing a CA, clients must be restarted" answers "we rotated their CA and now they get this error" — a connection with almost no lexical overlap and only one leg's vote. It also collapsed c2170 from rank 3 to last: a release note that mentions the error string in the context of log formatting is not an answer, which is the mention-versus-explanation distinction from 07-07.

Relevance floor of 0.5 keeps three candidates and drops c2170 and the other 70.

Stage 8 — Assemble

Edge-loaded, best first and second-best last, with rendered headers (07-08):

[SYSTEM] Answer using only the numbered sources below. Cite the source id after
         each claim. If the sources do not contain the answer, say so.

[SOURCE 1 · doc:ops-118#c14 · 2026-05-02 · current · official]
after replacing a CA, clients must be restarted to reload the trust store …

[SOURCE 2 · doc:ref-441#c07 · 2026-03-11 · current · official]
certificate rotation checklist …

[SOURCE 3 · doc:err-009#c01 · 2026-01-20 · current · official]
ERR_CERT_AUTHORITY_INVALID: the presented chain terminates in an untrusted root …

[QUESTION] customer on 4.2 is hitting ERR_CERT_AUTHORITY_INVALID after we rotated their CA
           — is this expected?

Token accounting:
  instructions          120
  3 chunks + headers    430
  question               35
  ─────────────────────────
  input                 585
  reserved for output   500
  total                1,085  ≪ context window            ✓  (02-03, 04-06)

Three chunks, not fifty. 585 input tokens, not 8,000. The two strongest chunks at the two strong positions. Every chunk carries a date, a status, a source tier, and a citable id.

Stage 9 — Generate and cite

Yes, this is expected. Replacing a certificate authority does not take effect on a
client until the client is restarted so it reloads its trust store [SOURCE 1].
ERR_CERT_AUTHORITY_INVALID specifically indicates the presented chain terminates in
a root the client does not trust [SOURCE 3], which is what happens while the old
trust store is still loaded. The rotation checklist [SOURCE 2] lists the restart as
a required step.

Every claim cites a source. Every source is current, official, and one this user is entitled to read. The answer addresses the user's real question — is this expected — rather than merely defining the error code.

What the trace proves

Now remove one stage and watch the whole thing fail:

Nine stages, and removing any one of them degrades or breaks the answer. That is what "quality equals the worst stage" means concretely — and it is why a diagram with four boxes cannot support a debugging conversation.

The build order that avoids wasted work, which is not the same as the pipeline order:

1. Eval set first (01-08). Twenty real questions with labelled answer chunks. Without this every later decision is a guess.
2. Parse and chunk with assertions. Silent stages first, because their failures masquerade as everyone else's.
3. BM25 baseline (07-01). Free, and the denominator for every later claim.
4. Dense retrieval with the self-retrieval assertion (07-02).
5. Exact index (07-04). Do not adopt infrastructure yet.
6. Fuse (07-06), measure.
7. Rerank (07-07), measure. Often the largest single gain.
8. Assemble properly (07-08). Often the largest gain per line of code.
9. Ground and cite (07-11).
10. Only now consider a vector database, a bigger model, or fine-tuning.

Teams routinely invert steps 1 and 10, and that inversion is the most expensive mistake in applied RAG.

This is the highest-yield lesson in the module by objective coverage. The blueprint mentions RAG three times across its objectives [OFFICIAL], and this lesson serves:

• 1.3 / 4.2 — Build LLM use cases such as retrieval-augmented generation (RAG), chatbots, and summarizers. The pipeline is the build.
• 1.4 — Curate and embed content datasets for RAGs. Stages 1–3.
• 1.6 / 4.3 — Familiarity with the capabilities of Python natural language packages (spaCy, NumPy, vector databases, etc.). Stage 4.
• 1.8 — Select and use models to create text embeddings. Stage 3.
• 1.9 — Use prompt engineering principles to create prompts to achieve desired results. Stages 8 and 9.
• 4.4 — Identify system data, hardware, or software components required to meet user needs. The component inventory is this lesson's §1.
• 4.1 / 1.1 — Assist in deployment and evaluation of model scalability, performance, and reliability. Two paths with different latency and cost profiles.
• 4.5 — Monitor functioning of data collection, experiments, and other software processes. The silent-stage assertions.
• 5.1 / 5.2 / 5.3 — Trustworthy AI. Citations serve Transparency; retrieval filtering serves Privacy and Safety and Security; grounding is a hallucination control [NVIDIA-DOC], and 13-01 covers the four named pillars.

Memorise NVIDIA's stated pipeline order [NVIDIA-DOC]: query → embedding model → vector match against indexed knowledge base → retrieve and decode → LLM synthesises and cites sources. Component-order questions are an explicit drill emphasis, and this sequence is the most likely thing to be asked about RAG.

Also memorise the product-to-stage map: NeMo Retriever for retrieval, NeMo Curator for curation, NeMo Guardrails for safety rails, the AI Blueprint for RAG as the reference workflow, NIM and Triton for serving [NVIDIA-DOC]. The [FIELD] calibration that NVIDIA-branded options tend to be keyed when two answers are technically defensible makes this map worth knowing cold; 12-13 and 13-02 go deeper.

Question phrasings you should recognise:

Distractor families:

• Stages out of order — most commonly retrieval before embedding the query, or reranking before retrieval.
• NIM or Triton offered as a pipeline stage. They are how models are served (12-13).
• "Fine-tuning is required to add new documents." No — RAG's whole value is that adding facts does not require retraining (11-08).
• "RAG eliminates hallucination." It reduces it by grounding and makes it detectable via citations. It does not eliminate it (09-12, 07-11).
• The embedding model conflated with the generator. Different models, different stages.
• The vector database credited with embedding. It stores and searches; the embedding model encodes.
• A four-box pipeline offered as complete. Parsing, reranking, and assembly are the omissions that matter.
• "Improve the LLM to fix retrieval." Wrong stage.

One calibration note to carry as uncertainty: candidate reports converge on a heuristic that in scenario questions, when one option proposes building a RAG solution, it is usually the keyed answer [FIELD]. This is field calibration, not official NVIDIA guidance. Use it only as a tie-breaker on genuinely ambiguous items, and check the counter-cases first — no corpus to retrieve from, a need to change style or format rather than facts, and a hard latency floor. 07-12 is entirely about those counter-cases.

Mistakes 1, 2, and 3 are the same mistake at three levels of severity, and they share one root: the earliest stages are the silent ones, and the interesting stages are the late ones. Human attention flows to the reranker and the LLM; the failures live in the parser and the embedder. Instrumenting the boundary of every stage is what breaks that pattern, and it is the practice that separates a RAG system you can operate from one you can only hope about.

In practice, the earliest ones — and that is exactly why they get the least attention.

I am offering this as a reasoned ordering rather than a measured ranking; I have no sourced distribution of RAG failure causes to quote. The reasoning is structural and I think it holds:

The pattern: the stages that fail silently and early dominate, and the stages that are fun to tune are usually fine. The corrective is mechanical rather than intuitive — instrument every boundary, assert the silent stages, and measure the eval set at each boundary rather than only end to end.

The single most valuable diagnostic remains the one 07-07 and 07-10 both point at: for a failing question, retrieve the top-50 and find where the correct chunk actually is.

Four questions, ten minutes, and a stage name. That is the whole method, and 07-10 formalises it.

By measuring at stage boundaries, not only at the end. An end-to-end score tells you whether the system is good; boundary metrics tell you which stage to fix.

Two metrics matter most, and they must be reported separately. Retrieval recall answers "was the evidence available?" Faithfulness answers "did the model use the evidence?" A system with high recall and low faithfulness has a generation problem. A system with low recall and high faithfulness is faithfully answering from bad evidence. Collapsing them into one score makes both diagnoses impossible — this is the decomposition 07-10 and 09-07 are built on.

Re-run the eval set at every stage change. The evalRerun discipline (01-08, 03-04) exists because a change that improves one stage can degrade another: shorter chunks improve retrieval precision and may split answers; a deeper reranker pool improves ordering and raises latency; a stricter currency filter improves accuracy and may cause empty results.

Two things beyond quality metrics.

Cost and latency are pipeline metrics too. Track tokens sent per query, embedding calls, reranker calls, and p95 end-to-end latency (12-09, 12-10). A quality gain that triples cost is a decision someone must make explicitly.

Put it in CI. A retrieval-recall threshold and a self-retrieval assertion that fail the build are worth more than a dashboard nobody reads (10-04). The week-5 deliverable of this course is exactly that: an eval set, four RAG metrics, and a CI gate that fails the build.

They sit at two different levels: NeMo Retriever is a component family for the retrieval stages, and the AI Blueprint for RAG is a reference workflow for the whole pipeline [NVIDIA-DOC].

The rest of the NeMo family maps onto the pipeline cleanly, and this mapping is worth memorising as a table because it is exactly the kind of tool-to-problem matching the exam rewards:

Two distinctions the exam cares about, both flagged as reported confusables:

NIM is packaged deployment, not a training tool [NVIDIA-DOC]. It is how a model becomes a stable, secure, low-latency API. It is not a pipeline stage and it does not train anything (12-13).

TensorRT and TensorRT-LLM are different products. TensorRT is a general-purpose inference compiler; TensorRT-LLM adds LLM-specific machinery — KV cache, paged attention, in-flight batching, speculative decoding (12-08, 12-05, 12-07). Both sit beneath serving, not inside the RAG pipeline's logic.

The value of the Blueprint framing for a course like this one: it is an explicit acknowledgement that RAG is a workflow with named stages rather than a library call. That is the same claim this lesson makes, and it is the claim that makes debugging possible.

• Nine stages, two paths. Offline: parse, chunk, embed, index. Online: query prep, retrieve, rerank, assemble, generate-and-cite.
• NVIDIA's stated query path, in order [NVIDIA-DOC]: query → embedding model → vector match against the indexed knowledge base → retrieve and decode → the LLM synthesises and cites sources. Memorise the sequence.
• Quality equals the worst stage. A flawless reranker cannot repair a parser that dropped every table. Find the worst stage, not the interesting one.
• Parsing, chunking, and embedding fail silently. They are also the earliest, so their failures get misattributed downstream. Assert on all three.
• The worked example's headline result: the reranker promoted the chunk that actually answered the question from fused rank 4 to rank 1, and the assembled context was 3 chunks and 585 input tokens — not 50 chunks and 8,000.
• The self-retrieval assertion is the single highest-value test in the pipeline. One embed, one search, catches nearly every encoder-consistency failure.
• The embedding model is shared state between the two paths. Changing it invalidates the entire index and requires a full re-embed (12-12).
• Build in a different order than you run: eval set, assertions on silent stages, BM25 baseline, dense, exact index, fuse, rerank, assemble, ground — infrastructure last.
• Measure at stage boundaries, not only end to end. Retrieval recall and faithfulness must be reported separately or neither diagnosis is possible.
• The NVIDIA map: NeMo Curator curates, NeMo Retriever retrieves, NeMo Guardrails guards, the AI Blueprint for RAG is the reference workflow, and NIM and Triton serve the models rather than being stages [NVIDIA-DOC].

You now have a nine-stage object and a set of boundary metrics. What you do not yet have is the procedure — the specific sequence of questions that turns "the answer was wrong" into "stage 7 is the problem" in under ten minutes. There is one question that resolves roughly half of all RAG debugging time on its own, and it must be asked before any other. Next: 07-10 names it — was the correct passage retrieved, or was it retrieved and misused? — and builds the full decision tree from that single split, because asking anything else first means you may spend a week improving a stage that was never at fault.