Beyond Vector RAG: GraphRAG, HybridRAG, and Agentic RAG Compared

Consider a question like "which vendors supply components that appear in products recalled in the last two years, and which of those vendors also supply our top three current product lines?" This is not a question about which passage is semantically similar to the question text. It is a question about tracing a chain of relationships — recall → product → component → vendor → current product line — through structured facts, several hops deep. A vector search over embedded document chunks has no native way to perform that chase. It can retrieve a passage that mentions a specific recall, and a separate passage that mentions a specific vendor, but it has no mechanism for connecting the dots between them the way the question requires, because "semantically similar to this question" and "connected to this fact by a two-hop relationship" are simply different properties, and vector similarity only measures the first one.

This is the precise gap that motivates everything else in this lesson: plain vector RAG retrieves by semantic similarity, and semantic similarity is not the same operation as relational, multi-hop reasoning over explicit connections between entities. Vector search wins at finding text that is about the same topic as a query, but it does not natively win at answering questions whose real content is a chain of relationships, because nothing in an embedding vector represents a specific connection between two entities as a queryable, traceable fact — an embedding only represents that two pieces of text mean roughly the same thing.

Knowledge graphs and vector search are not interchangeable: graphs win at relational, multi-hop reasoning, while vector search wins at semantic similarity [GROUND TRUTH] (Sources/ncp-aai/domain-6-knowledge-integration.md). That split, echoed from the agent-memory side of the outline, shows up wherever an agent needs to ground itself in facts, whether that grounding is long-term memory (M1-06's subject) or document retrieval (this lesson's subject).

L1 — The intuition you can carry into an exam

GraphRAG answers the multi-hop problem by changing what gets retrieved from: instead of retrieving nearby vectors, it retrieves paths and neighborhoods within a knowledge graph — a structure of entities (nodes) connected by explicit, named relationships (edges). Where a vector index answers "what's semantically close to this query," a knowledge graph answers "what is this entity connected to, by what relationship, and what is that connected to in turn." That second question is exactly the shape of the recall-to-vendor chain above, and a graph traversal can walk it directly, hop by hop, following explicit edges rather than hoping semantic similarity happens to connect the right passages.

L2 — Why a graph structure enables what a vector index cannot

The mechanism a knowledge graph provides that a vector store does not is explicit, typed connectivity. In a knowledge graph, "Product X" is a node, "Component Y" is a node, and there is an edge between them explicitly labeled something like "contains-component," created when the data was structured into the graph. A query asking about products containing a given component does not need to guess based on textual similarity — it follows the "contains-component" edges directly from the component node, and it can keep following further edges (component → vendor, vendor → other products) as many hops as the question requires, with each hop being an exact, verifiable traversal rather than an approximate similarity match. This is what "relational, multi-hop reasoning" means concretely: the graph doesn't need to have ever seen the specific multi-hop question before, because the answer emerges from composing individually simple, exact traversals.

This structure also buys something vector RAG cannot offer at all: verifiability. A graph traversal that produces an answer can show its work as an explicit path — node, edge, node, edge, node — that a human or a downstream system can audit and confirm hop by hop. A vector-retrieved passage can be shown as evidence too, but it cannot show a chain of reasoning the way a graph path can, because similarity retrieval never represented the relationship as a discrete, inspectable fact in the first place — it only ever represented that two pieces of text are alike.

NVIDIA's own comparison of these approaches found GraphRAG leading specifically on correctness for exactly this class of relational question [GROUND TRUTH] (Sources/ncp-aai/domain-6-knowledge-integration.md), which tracks with the mechanism: correctness on multi-hop questions is what explicit, followable relationships are built to deliver, and it is exactly what unstructured semantic similarity is not built to deliver.

L3 — What GraphRAG costs, and where it does not help

None of this comes free, and the costs are the honest reason GraphRAG is not simply "the better version of RAG" that should replace vector retrieval everywhere. Building a knowledge graph in the first place requires extracting entities and relationships from source material — a nontrivial preprocessing step in its own right, and one whose quality caps everything the graph can subsequently answer, in exactly the same spirit as M6-04's broader point about data quality capping retrieval quality. A knowledge graph is also only as complete as the relationships someone bothered to extract and encode into it; if the vendor-to-component relationship was never captured as an edge, the graph cannot traverse a connection that was never represented, no matter how well-built the traversal engine is.

And critically, GraphRAG does not help with the class of question plain vector RAG is actually good at. "What does our refund policy say about late requests" is not a multi-hop relational question — it's a single-topic semantic-similarity question, and building a graph specifically to answer it would be solving a problem that was never posed. The two approaches are strong in complementary, non-overlapping regions of question-space, which is exactly the setup that motivates the next section.

L1 — The intuition you can carry into an exam

The most commonly tested misconception in this cluster of material, stated as directly as the source material states it: GraphRAG and vector RAG are not either/or — HybridRAG exists precisely to combine them [GROUND TRUTH] (Sources/ncp-aai/domain-6-knowledge-integration.md). If a scenario question frames GraphRAG and vector RAG as two competing options where you must pick exactly one, and neither "combine both" nor "HybridRAG" appears as a choice, re-read the question — the framing itself is very likely the trap, because the entire reason HybridRAG has a name is that the field converged on "you should not have to choose" as the right answer once both approaches' failure modes became clear.

L2 — What HybridRAG actually retrieves, and how the two results get used together

HybridRAG runs both retrieval mechanisms — semantic similarity search over a vector index, and relational traversal over a knowledge graph — against the same underlying knowledge, and combines what each returns before generation [GROUND TRUTH] (Sources/ncp-aai/domain-6-knowledge-integration.md). The value of doing this rather than picking one comes directly from the fact that the two approaches fail on different classes of question: a query that is mostly a semantic-similarity question with a small relational component gets its similarity-shaped part answered well by the vector leg and its relational part answered well by the graph leg, and the combination covers more of the question than either leg alone would.

Concretely, this looks like retrieving a set of semantically relevant passages from the vector index and a set of relevant entities, relationships, or graph paths from the knowledge graph, for the same incoming query, and then assembling both into the context handed to the generation stage. The generation stage — unchanged from M6-01's stage 5 in its mechanics — now has both kinds of evidence available: passages that are topically close to the question, and explicit relational facts that trace connections the passages alone might never make clear. A well-built HybridRAG system does not simply concatenate both retrieval outputs blindly; it typically has some logic for recognizing which parts of a question are relational versus similarity-shaped and weighting each leg's contribution accordingly, though the exam-relevant fact is the combination itself, not any particular weighting scheme.

L3 — Why combination is structurally necessary rather than merely convenient

It's worth being precise about why this is a "must combine" situation rather than a "nice to have both available" situation. A vector-only system cannot answer a genuinely multi-hop relational question well regardless of how much better its embeddings get, because the operation it performs — similarity — is not the operation the question requires. A graph-only system cannot answer a genuinely open-ended semantic question well regardless of how complete its graph is, because most real-world knowledge that matters for answering a broad question was never going to be exhaustively modeled as explicit typed relationships — a company's onboarding documentation, written in prose, is not naturally a graph, and forcing it into one loses far more than it gains. Neither approach dominates the other across the space of realistic questions an agent actually encounters, which is exactly the condition under which "combine both" stops being a compromise and becomes the correct engineering answer.

L1 — The intuition you can carry into an exam

Everything covered so far in this lesson and the two before it — plain vector RAG, GraphRAG, HybridRAG — shares one structural property: each treats retrieval as something that happens once per question. A query comes in, retrieval runs (against a vector index, a graph, or both), the results get handed to generation, and the pipeline is done. Agentic RAG breaks that structural assumption [GROUND TRUTH] (Sources/ncp-aai/domain-6-knowledge-integration.md). Instead of one retrieval pass per question, an agent using agentic RAG treats the query as something to reason about first: it plans focused sub-questions, retrieves separately for each one, and — this is the property most worth memorizing precisely — reformulates and retries when a retrieval attempt comes back thin, rather than accepting whatever the first lookup happened to return.

L2 — The mechanism: planning, retrieving per sub-question, and retrying on thin results

Walk through what "agentic" actually changes, mechanically, compared to the single-shot pipeline every previous section assumed. A single-shot system takes the user's question, embeds it (or extracts entities from it, for the graph leg), retrieves once, and generates. Agentic RAG inserts a planning step before any retrieval happens at all: given the incoming question, the agent first decides whether it is actually one question or several smaller questions bundled together, and if it's several, it breaks the original question down into those focused sub-questions explicitly. Each sub-question then gets its own retrieval call — its own trip to the vector index, the graph, or both — rather than forcing one broad retrieval call to somehow cover everything the original compound question needed.

The second mechanical change is what happens after a retrieval call returns. A single-shot pipeline takes whatever came back and moves straight to generation, whether the retrieved chunks were a strong match or a weak one. An agentic RAG loop instead evaluates the retrieved result for a specific sub-question and, if that result looks thin — too few relevant chunks, low similarity scores across the board, a graph traversal that came up empty — the agent reformulates the sub-question (rephrasing it, narrowing it, or trying a different retrieval strategy entirely) and retries, rather than passing a weak result straight through to generation and hoping the model compensates. This retry-on-thin-results behavior is the single most exam-relevant mechanical detail in this entire lesson, because it is exactly the behavior a scenario question tends to probe directly [GROUND TRUTH] (Sources/ncp-aai/domain-6-knowledge-integration.md): when initial retrieval returns thin results in an agentic RAG setup, the correct behavior is reformulating the sub-question and retrying before synthesizing an answer — not returning an empty answer, and not simply generating from the thin evidence anyway.

Once all sub-questions have been retrieved for — each with its own retry cycle as needed — the agent synthesizes a final answer from the accumulated evidence across every sub-question, rather than generating from a single retrieval pass's results. This synthesis step is doing real work: it is combining evidence gathered across potentially several separate retrieval calls into one coherent answer to the original question, which is a strictly harder generation task than summarizing a single retrieval pass, and it is the reason agentic RAG systems tend to cost more in latency and API calls than single-shot pipelines — a cost that is worth paying specifically when the original question genuinely needed decomposing, and wasted overhead when it didn't.

L3 — Why this is the agent-native pattern, and what it costs

This lesson's opening framed RAG as one tool an agent calls partway through a reasoning loop rather than the entire shape of an application, and agentic RAG is the fullest expression of that framing carried to its natural conclusion. A single-shot RAG pipeline, however good its retrieval mechanics, is fundamentally a fixed procedure: embed, search, generate, done, every time, regardless of what the question actually needed. Agentic RAG puts the agent's own reasoning in charge of how many times to retrieve, what to retrieve for at each step, and whether a given retrieval attempt was good enough to build on — decisions that a fixed single-shot procedure cannot make, because a fixed procedure has no branch point at which to make them.

The cost of this flexibility is real and worth naming honestly rather than treating agentic RAG as a strictly better replacement for everything covered earlier in this lesson. Planning sub-questions, retrieving multiple times, and potentially retrying several of those retrievals means more model calls, more retrieval calls, and more latency than a single-shot pipeline running once. For a question that a single well-phrased query would have answered perfectly well on the first try, agentic RAG's extra machinery buys nothing and costs real time and expense. The judgment call — is this question complex enough, or ambiguous enough, or does it need enough sub-questions decomposed, to justify agentic overhead — is itself a decision an agent's higher-level planning has to make, echoing the same "is retrieval even the right move right now" question M6-01 raised about invoking RAG in the first place, now one level more specific: not just whether to retrieve, but how much retrieval reasoning the current question actually warrants.

⭐ THE EARNED INSIGHT: GraphRAG, HybridRAG, and agentic RAG look, from a distance, like three competing upgrades to plain vector RAG, each newer and more elaborate than the last — and that "newer must be better" reading is precisely what the exam's traps are built to punish. None of the three is a strict upgrade. GraphRAG trades preprocessing cost for relational correctness it cannot get any other way; HybridRAG's entire reason to exist is that neither retrieval mechanism alone covers realistic question-space; and agentic RAG's reasoning-and-retry loop is worth its latency only when the question actually needed decomposing. The right mental model is not a ladder from "basic" to "advanced" — it is a set of tools, each matched to a specific shape of question, with plain vector RAG remaining the correct, cheapest choice for the large share of questions that are genuinely just semantic-similarity questions.

The row worth internalizing above the others: agentic RAG is not a fourth alternative sitting alongside the first three in the same category — it is a reasoning layer that can sit on top of any of them. An agentic loop could plan sub-questions and retry using plain vector retrieval for every sub-question, or using GraphRAG, or using a HybridRAG combination for each sub-question's retrieval step. The "agentic" property is about when and how often to retrieve and re-retrieve; it does not specify which underlying retrieval mechanism answers any individual retrieval call.

This is a constructed scenario with an invented company and invented numbers, built to make the differences concrete.

A procurement agent is asked: "Which of our approved suppliers had a quality incident in the last 18 months, and are any of them the sole supplier for a component used in our flagship product?"

PLAIN VECTOR RAG ATTEMPT

  Query embedded, compared against a knowledge base of supplier
  contracts, incident reports, and product specs.

  Top 3 retrieved chunks:
    - "Quality incident report: Supplier C, Q2, packaging defect"
    - "Approved supplier list, current fiscal year"
    - "Flagship product bill of materials, component list"

  Generation attempt: produces a plausible-sounding but UNVERIFIED
  answer, because nothing in the retrieved text explicitly states
  whether Supplier C is a SOLE supplier for any flagship component —
  that fact exists only as an implicit relationship the model would
  have to infer across three separately-retrieved documents, which
  it may get right by chance and may not.

GRAPHRAG ATTEMPT

  Query decomposed by the graph query layer into a traversal:
    Supplier --[had-incident]--> Incident (filtered: last 18 months)
    Supplier --[sole-supplies]--> Component
    Component --[used-in]--> Product (filtered: "flagship")

  Traversal result: Supplier C --[had-incident]--> Q2 packaging defect
                    Supplier C --[sole-supplies]--> Component X
                    Component X --[used-in]--> Flagship Product

  This is an EXACT, inspectable path — not an inference — because
  the "sole-supplies" and "used-in" relationships were explicit
  edges in the graph, extracted and encoded ahead of time.

HYBRIDRAG ATTEMPT

  Same graph traversal as above, PLUS a vector search for
  supporting narrative context:
    - graph leg confirms: Supplier C, sole supplier, flagship
      component, recent incident (as traced above)
    - vector leg retrieves: the actual incident report narrative,
      giving qualitative detail (what the defect was, how it was
      resolved) that the graph's edges don't carry on their own

  Combined answer: the exact relational fact from the graph, backed
  by the narrative detail from the vector leg — neither leg alone
  produced this complete an answer.

The plain vector attempt is not wrong because retrieval failed mechanically — it retrieved genuinely relevant chunks. It is wrong because the question's real content was a relationship ("sole supplier for a flagship component") that no single retrieved passage stated explicitly, and semantic similarity has no mechanism for verifying a relationship that was never written down as one connected fact in any single chunk. The GraphRAG attempt succeeds specifically because that relationship existed as explicit, traceable edges. The HybridRAG attempt is the version that would actually ship in most production systems, because a procurement team wants both the verified relational fact and the human-readable narrative explaining what happened.

This cluster is objective 6.3 of Domain 6, and the domain's own scope note calls out exactly this material by name [GROUND TRUTH] (Sources/ncp-aai/domain-6-knowledge-integration.md): candidates should be able to distinguish plain vector RAG from GraphRAG, HybridRAG, and agentic RAG.

The domain explicitly flags this as containing some of its most common exam traps [GROUND TRUTH] (Sources/ncp-aai/domain-6-knowledge-integration.md), worth restating precisely because each one is attractive in a specific way: GraphRAG and vector RAG are not either/or — HybridRAG exists precisely to combine them; agentic RAG reasons and retries, it is not a single-shot retrieval; and knowledge graphs give relational, multi-hop reasoning while vector search gives semantic similarity.

Expect a scenario question that describes a multi-hop relational question and asks which approach best supports it — the correct answer is GraphRAG, and the domain's own self-check material states this directly [GROUND TRUTH] (Sources/ncp-aai/domain-6-knowledge-integration.md). Expect a second question shape describing initial retrieval that came back thin in an agentic context, asking what the agent should do next — the correct behavior is reformulating and retrying before synthesizing, never returning an empty answer and never simply generating from weak evidence. And expect at least one question where "combining graph-based and vector-based retrieval" is offered as one of four options for describing HybridRAG, alongside distractors describing only-graph or only-vector behavior, or no retrieval at all [GROUND TRUTH] (Sources/ncp-aai/domain-6-knowledge-integration.md).

Reach for GraphRAG specifically when the questions your system needs to answer require tracing a relationship across multiple entities rather than finding text that is merely about the same topic as the question. A useful test: if answering the question correctly requires connecting two or more separately stated facts through a relationship that was never written down together in any single source passage, that is a multi-hop relational question, and GraphRAG's explicit, traceable edges are built for exactly that. If the question is answerable by finding the one passage that already directly addresses it, plain vector RAG remains the simpler, cheaper, equally effective choice, and building a knowledge graph for that case adds preprocessing cost without a corresponding benefit.

The difference is not that agentic RAG uses an agent and plain RAG doesn't — an agent can call retrieval once and stop, and that is still single-shot retrieval regardless of the fact that an agent made the call. What makes agentic RAG distinct is the reasoning wrapped around the retrieval call itself: planning the original question into focused sub-questions before retrieving anything, evaluating whether each sub-question's retrieved evidence is actually sufficient, and reformulating and retrying a sub-question when the evidence looks thin, rather than accepting whatever the first attempt returned. A single retrieval call, however sophisticated the underlying vector or graph mechanism behind it, is not agentic RAG unless that reasoning-and-retry loop is present around it.

• Plain vector RAG retrieves by semantic similarity and structurally cannot reliably answer multi-hop relational questions, because similarity and explicit relational connectivity are different properties.
• GraphRAG retrieves over a knowledge graph instead of a vector index, following explicit, typed, traceable edges — and NVIDIA's own comparison found it leading on correctness for relational questions specifically.
• HybridRAG combines graph-based and vector-based retrieval; the two approaches are not competitors, and a question framing them as an either/or choice is very likely testing that exact trap.
• Agentic RAG plans sub-questions, retrieves per sub-question, and reformulates and retries when results come back thin — it is defined by reasoning-and-retry, not merely by an agent making a retrieval call.
• When initial retrieval in an agentic setting returns thin results, the correct behavior is reformulating and retrying before synthesizing, never returning an empty answer.
• None of the three extensions in this lesson replaces plain vector RAG outright — each is strong precisely where vector RAG (or the previous extension) is weak, and each carries its own added cost in preprocessing, maintenance, or latency.
• Agentic reasoning is a layer that can sit on top of any underlying retrieval mechanism — vector, graph, or hybrid — rather than being a fourth, separate retrieval mechanism itself.

Everything in this lesson has assumed the underlying knowledge being retrieved — whether from a vector index or a knowledge graph — was already clean, current, and well-structured by the time retrieval touched it. That assumption is the one this module has deferred the longest, and it stops being safe to make the moment real source data enters the picture. Next: M6-04 covers the ETL and data-quality work that determines whether any of the retrieval sophistication described in this lesson has good material to work with in the first place — because a perfectly designed HybridRAG system built over duplicated, stale, or malformed source data still produces confidently wrong answers, for reasons no amount of retrieval-mechanism cleverness can fix.