How the Pieces Fit: One NVIDIA Agentic Stack End to End

Identity statement: a typical NVIDIA agentic stack is not six interchangeable "AI tools" but six distinct jobs, chained together in a specific order, where a request is orchestrated, served, optimized, hosted, grounded, and safety-checked by six different pieces of software, each doing exactly one of those jobs and none of the others. [GROUND TRUTH] (Sources/ncp-aai/domain-7-nvidia-platform-implementation.md) states the composition directly: "A typical NVIDIA agentic stack: the NeMo Agent Toolkit orchestrates the agent (framework-agnostic, with MCP), calling NIM microservices for inference (optimized by TensorRT-LLM, served by Triton), grounded via RAG over a vector database, and wrapped by NeMo Guardrails for safety."

Read that sentence as a sequence, not a list, because the order it names is the order a request genuinely passes through in a working system: the toolkit decides what to do next and calls the model; NIM exposes that model's standard API endpoint; underneath NIM, TensorRT-LLM has already optimized the specific engine being called, and Triton is what actually serves it, managing batching and concurrency at request time; a vector database, queried as part of a RAG pipeline, supplies the retrieved context the model's response is grounded in; and NeMo Guardrails, sitting between application code and the LLM on both the inbound and outbound path, checks the request and the response at the boundaries where the rest of the stack does not.

The trap this whole module has been building toward

[GROUND TRUTH] (Sources/ncp-aai/domain-7-nvidia-platform-implementation.md) states the module's central trap in one sentence, worth repeating exactly as written because every prior lesson in this module has been building a defense against it: "Don't conflate the tools. Toolkit = orchestration/dev; NIM = serving microservice; TensorRT-LLM = optimization; Triton = general server; Guardrails = safety. Each has a distinct job." Five names, five jobs, and the sixth piece — the vector database — adds a distinct sixth: grounding. A scenario question that swaps any one of these six labels for another, describing Triton's batching as if it were TensorRT-LLM's job, or describing the toolkit's orchestration as if it were NIM's serving, is testing exactly this conflation, and it is the single most consequential fact this closing lesson exists to reinforce across all six pieces at once rather than one pair at a time.

L1 — Intuition: a relay race with six specialized legs, not one runner doing everything

Picture a relay race with six legs, each run by a specialist trained for that specific leg and no other: one leg is run by a strategist who decides the race plan (the toolkit), one by a runner whose only job is to carry the baton across a fixed, standard handoff point (NIM), one by a coach who tuned that runner's stride for this exact track months before race day (TensorRT-LLM), one by an official who manages several runners crossing the same stretch of track at once without collision (Triton), one by a scout who has already mapped the terrain and hands the runner a shortcut through it (the vector database), and one by a safety officer stationed at both the start and the finish line, checking that nothing dangerous enters or leaves the track (Guardrails). No single leg wins the race alone, and a race plan that skipped any one of the six specialists and asked a different one to cover two jobs would be slower, less safe, or both.

L2 — Mechanism: what happens, in order, for one real request

Constructed scenario, illustrative only. Trace a single user's question — "what is our return policy for a damaged item, and can you check whether my specific order qualifies?" — through a support agent built on all six pieces.

Step 1 -- INPUT RAIL (NeMo Guardrails):
  The incoming message is checked before anything else runs.
  It is on-topic (a policy and order-status question), not
  malicious, and passes through to the toolkit's workflow.

Step 2 -- ORCHESTRATION (NeMo Agent Toolkit):
  A ReAct-style loop (M7-01) decides this question needs two
  things: the general return policy text, and this specific
  order's status. It plans two sub-actions.

Step 3 -- GROUNDING (vector database, via RAG):
  The toolkit's workflow queries a vector database holding the
  company's policy documents, retrieving the specific passage
  about damaged-item returns -- the canonical RAG pipeline this
  course covers in Module 6, referenced here generically.

Step 4 -- INFERENCE (NIM, optimized by TensorRT-LLM, served by
  Triton):
  The toolkit calls the language model through its NIM endpoint
  to reason over the retrieved policy passage and the order
  lookup result together. Underneath that endpoint, TensorRT-LLM
  had already compiled the engine for fast execution on this
  GPU (M7-03), and Triton is the server actually handling this
  request alongside whatever other concurrent requests are
  arriving at the same moment, batching them dynamically.

Step 5 -- OUTPUT RAIL (NeMo Guardrails):
  Before the response reaches the user, a factuality rail checks
  that the stated policy detail actually matches what was
  retrieved in step 3, rather than a plausible-sounding but
  unsupported claim the model generated on its own.

Step 6 -- RESPONSE:
  The user receives an answer grounded in the real policy
  document and the real order lookup, checked on the way in
  and the way out.

Reading the six steps in order is the entire point of this lesson: no single piece of software did more than one job, and no job was done by more than one piece. The toolkit never touched the vector database's retrieval mechanics directly — it called a retrieval step as one composable function among several, exactly the composability property M7-01 names. Guardrails never reasoned about the return policy's content — it only checked whether the input was in scope and whether the output was grounded in what was actually retrieved. NIM never decided how to batch concurrent requests — that is Triton's job, running underneath NIM's standard API. And TensorRT-LLM never ran at request time at all — its work was finished before this specific request ever arrived, back when the engine was compiled.

L3 — Objective 7.3: the toolkit is the connective tissue, not a seventh tool

[GROUND TRUTH] (Sources/ncp-aai/domain-7-nvidia-platform-implementation.md) names objective 7.3 — "optimize workflows with the toolkit" — as "the connective tissue" holding the rest of the stack together. This is worth stating precisely because it is easy to misread as implying a seventh distinct tool sitting alongside the other six. It is not. Objective 7.3 is a property of how the toolkit (already counted as piece one of six) is used: the toolkit's own composability, the same "build once, reuse" property M7-01 develops in full, is what lets a workflow call NIM, query a vector database, and route through Guardrails' checks as sequenced steps in one coherent pipeline, rather than as five disconnected calls a developer has to wire together by hand for every new workflow. The connective-tissue framing describes the toolkit's function in the stack, not an additional piece of the stack beyond the toolkit itself.

⭐ THE EARNED INSIGHT Every one of this module's five earlier lessons taught a version of the same warning — don't conflate this tool's job with that tool's job — applied to a different pair each time. Reading all six pieces together in one traced sequence is what finally shows why the warning kept recurring: it is not five unrelated facts about five unrelated products, it is one fact about the shape of the stack, restated once per pair because the stack itself is built from single-purpose pieces chained together rather than from a few multipurpose ones. A system built this way is precisely as strong as its weakest conflation — mistake any one piece's job for a neighbor's, in design or in diagnosis, and the mistake costs you exactly as much whether it is TensorRT-LLM mistaken for Triton, NIM mistaken for the toolkit, or Guardrails mistaken for the vector database. Holding all six jobs distinct, at once, is the actual skill this entire module has been training — the six-name list is just where that skill finally gets exercised in full.

The "what happens if it is missing" column is worth sitting with directly, because a scenario question describing a stack with one piece silently absent — a system that serves a model with no rail layer, or an agent that reasons without ever grounding in retrieved facts — is testing whether you can name the specific failure that piece's absence produces, not just whether you can recite the six names.

Constructed scenario, illustrative only. A team reports that their support agent regularly produces confident-sounding answers about return policy details that turn out to be wrong — not garbled or off-topic, just factually incorrect about specifics like time windows or refund percentages. Diagnose which of the six pieces is most likely missing or misconfigured, using only the symptom described.

Symptom: confident, coherent, factually wrong on specific details.
Not: off-topic, malicious, or unsafe content -- ruling out an
     input-rail gap.
Not: garbled or slow responses -- ruling out a serving or
     optimization problem (NIM, TensorRT-LLM, Triton all
     functioning normally, since responses arrive promptly and
     are well-formed).
Is:  the model appears to be reasoning without a grounded source
     for the specific facts it states confidently.

Most likely missing piece: the vector database / RAG grounding
step, or an output rail (NeMo Guardrails) that would have caught
an ungrounded factual claim before it reached the user.

This diagnostic exercise is the practical payoff of holding all six pieces distinct rather than treating "the stack" as one undifferentiated system: a symptom this specific — right tone, right topic, wrong facts — points at exactly two candidate gaps (missing grounding, or a missing factuality check on output) and rules out the other four pieces entirely, because none of the other four's failure modes produce this particular symptom shape. A team that responded to this report by re-tuning TensorRT-LLM's batching configuration would be optimizing a piece that was never the problem.

NVIDIA Platform Implementation is Domain 7 of the NCP-AAI blueprint at 7%, and [GROUND TRUTH] (Sources/ncp-aai/domain-7-nvidia-platform-implementation.md) closes the domain's own material with exactly this composition, under the heading "How the Pieces Fit," followed immediately by the trap sentence this lesson has repeated in full: don't conflate the tools, because each has a distinct job. That structural choice — ending the domain's own source material with the composed picture rather than a seventh isolated fact — signals that the composition itself, not merely the five (or six) individual identities, is something the exam expects a candidate to hold together at once.

Expect the question to arrive in two recurring shapes that differ from the single-tool identification items this module's earlier lessons prepared for. A sequence item describes a request's journey in scrambled order — say, describing Guardrails checking the output before describing the toolkit ever deciding to call the model — and asks which ordering is correct, or asks which piece would catch a specific kind of problem given where each piece sits in the sequence. A diagnostic item, the shape this lesson's two worked examples were built to prepare for directly, describes a symptom (ungrounded answers, slow responses under concurrent load, unsafe content reaching a user) and asks which piece is most likely responsible, requiring a candidate to rule out the pieces whose failure modes do not match the described symptom before landing on the one that does.

What the distractors typically look like

Expect a described symptom paired with a fix targeting the wrong piece — retuning TensorRT-LLM's precision when the actual problem is missing grounding, or adjusting Guardrails' rail configuration when the actual problem is Triton's batching window under concurrent load. Expect the toolkit's connective-tissue role (objective 7.3) offered as if it named an additional, seventh tool distinct from the toolkit itself, rather than a property of how the toolkit is used. And expect a stack description that quietly omits one piece — describing NIM, TensorRT-LLM, and Triton in a chain with no mention of Guardrails or the vector database at all — with the question asking what capability that omission costs the system, testing whether a candidate notices the absence rather than simply confirming that whatever pieces were named are described correctly.

The vector database and the RAG grounding step it supports, followed closely by an output-side NeMo Guardrails factuality check as the second line of defense. Neither the toolkit's orchestration, NIM's serving, TensorRT-LLM's optimization, nor Triton's batching produces an ungrounded-answer symptom on its own — all four of those pieces can be functioning perfectly and a response can still be confidently wrong if the model never retrieved the actual facts it needed before reasoning about them. A scenario describing this exact symptom — coherent, on-topic, but factually wrong on specifics — is testing whether a candidate can rule out the serving-and-optimization pieces and correctly land on the grounding-and-safety pieces instead.

Not necessarily every deployment at every stage, but NVIDIA's own framing describes this six-piece composition as the typical stack, and a candidate should be able to name what is lost when any one piece is absent rather than assuming all six are always mandatory in every scenario. A simple internal tool with no need for retrieved factual grounding might reasonably skip the vector database; a low-stakes prototype might defer a full Guardrails configuration until closer to production. What does not change across deployments is the six distinct jobs themselves — even a system that omits one piece is still missing a specific, nameable capability (grounding, or safety) rather than getting that capability from a different piece by default. The exam's own framing treats the six-piece composition as the reference architecture to reason from, not a claim that every real deployment includes every piece unconditionally.

• A typical NVIDIA agentic stack chains six distinct jobs: the toolkit orchestrates, NIM serves, TensorRT-LLM optimizes, Triton hosts, a vector database grounds, and NeMo Guardrails secures the request path on both directions.
• Each piece's job is exclusive to it — no two of the six do the same job, and swapping any one piece's name for another's in a description is the module's standing trap.
• Objective 7.3 is the toolkit's connective-tissue property, not a seventh tool — it describes how the toolkit's own composability sequences the other five pieces together.
• A described symptom (ungrounded answers, slow responses, unsafe content passing through) points at a specific, nameable piece's absence or misconfiguration, and ruling out the pieces whose failure modes do not match the symptom is the fastest diagnostic path.
• This module's five earlier lessons — the toolkit, NIM's throughput tuning, TensorRT-LLM/Triton's optimize-versus-serve split, Guardrails' platform role, and multimodal routing — are each one piece of this same six-piece picture, examined individually before being assembled here.

Domain 7 has answered "which NVIDIA tool does which job." Next: the next domain in this course turns from architecture to operations — once a stack like this one is actually live, serving real traffic, how do you know it is still working, and what turns a quietly degrading response quality into an automated, traceable response rather than a surprise discovered by an unhappy user? Monitoring, logging, and maintaining a deployed agentic system over time is where this course goes from here.