Structuring Multi-Step Reasoning: Logic Trees, Prompt Chains, and Adaptable Architecture

A single LLM call, however capable the model behind it, answers one question with one response. It has no built-in mechanism for saying "first do this, then check the result, then depending on what you found, do one of these two different next things." A task that genuinely requires several coordinated decisions — where an early choice constrains what is even possible later, exactly the gap M1-01 identified in reactive architectures — needs the sequence itself represented somewhere outside any single model call, because no single call can hold a multi-step sequence's branching and ordering on its own.

That external representation is what "structuring multi-step reasoning" means concretely: building something — a data structure, a script, a state object — that tracks where the task currently stands, what has already happened, and what should happen next, and that persists across multiple separate calls to a model or a tool. The three mechanisms NVIDIA names are three different shapes that external representation can take, each suited to a different kind of multi-step structure.

L1 — Intuition

A logic tree structures multi-step reasoning as a branching decision structure: at each node, some condition is evaluated, and the outcome determines which branch — which next step — the agent follows. [GROUND TRUTH] (Sources/ncp-aai/domain-1-agent-architecture-design.md): NVIDIA's material describes a logic tree as branching on conditions, which is the load-bearing phrase — the defining property is conditional branching, not sequencing.

L2 — Mechanism

Concretely, a logic tree for a customer-support agent might look like: "Is the customer asking about billing or a technical issue? If billing, is the account past due? If past due, offer a payment plan; if current, check for a duplicate charge. If technical, is the issue reproducible? If reproducible, escalate to engineering; if not, request more diagnostic detail." Each diamond in that description is a branch point, and the agent's behavior at any given moment depends on which path through the tree the current conversation has taken — not on a fixed sequence of steps that runs the same way every time.

L3 — The edge case that trips people up

The trap worth flagging is assuming a logic tree and a prompt chain are the same thing because both involve "multiple steps." A prompt chain (the next mechanism) is fundamentally about sequencing sub-prompts one after another; a logic tree is fundamentally about branching based on a condition. A system that always executes step A, then step B, then step C, in that fixed order regardless of what happens at each step, is not using a logic tree even if it has three steps — there is no conditional branch anywhere in it. Conversely, a system with only one decision point and two possible outcomes is a (very small) logic tree even though it has far fewer total steps than a five-stage prompt chain. Step count and branching are independent properties, and a scenario question testing this distinction is checking whether you notice which one the described system actually has.

L1 — Intuition

A prompt chain structures multi-step reasoning as a sequence of sub-prompts, where each sub-prompt's output feeds into constructing the next sub-prompt. [GROUND TRUTH] (Sources/ncp-aai/domain-1-agent-architecture-design.md): the source material's own phrasing is that a prompt chain "sequences sub-prompts" — sequencing, not branching, is the defining property here, the mirror image of the logic tree's defining property.

L2 — Mechanism

A document-drafting agent might chain: prompt one, "extract the five key facts from this source material"; prompt two, built using the output of prompt one, "given these five facts, draft an outline"; prompt three, built using the output of prompt two, "given this outline, write the full draft section by section." Each link in the chain is a separate call to a model, and the chain's structure — which prompt comes after which, and how one prompt's output is folded into constructing the next prompt's input — is fixed in advance by the person or system that designed the chain, rather than decided dynamically based on a condition evaluated at runtime.

L3 — The edge case that trips people up

The subtlety worth sitting with is that a prompt chain can be static (the same three prompts, in the same order, every single time) or it can incorporate light branching without becoming a logic tree in the full sense — for instance, a chain that runs steps one and two identically every time, but where step three's exact wording depends on whether step two's output crossed some length threshold. Where the line actually falls, for exam purposes, is whether the branching is incidental prompt-construction detail (mostly-fixed sequence, with the exact wording of one step varying) or whether the path itself through multiple genuinely different next steps depends on a condition (a logic tree). A chain with one minor conditional wrinkle inside an otherwise fixed sequence is still best described as a prompt chain; a system whose entire structure is organized around "which of several different next steps do we take" is better described as a logic tree, even if each individual step happens to be a single prompt.

Dynamic prompt chains that branch on intermediate results at runtime, and the closely related question of refining agent decision-making by measuring how well the agent chose, get a full standalone treatment later in this course in M2-01 — this lesson's job is only to establish the static-chain baseline and the boundary against logic trees, not the full branching-and-refinement picture.

L1 — Intuition

Neither a logic tree nor a prompt chain says anything, on its own, about where the accumulated context of the task lives as the agent moves through multiple steps. Stateful orchestration is the mechanism that answers that question: it carries context and state across the multi-step task, so that step five of a task can still see and use information gathered in step one, without that information having to be manually re-passed into every single step along the way.

L2 — Mechanism

[GROUND TRUTH] (Sources/ncp-aai/domain-1-agent-architecture-design.md): the source material names stateful orchestration as the third of the three mechanisms, alongside logic trees and prompt chains, specifically for the "carries context across steps" job. Concretely, this might be an orchestration layer that maintains a task-state object — the customer's account ID once looked up, the outline once drafted, the list of documents already checked — and makes that object available to whichever step, tool call, or sub-prompt runs next, rather than requiring each individual step to independently re-derive or re-request information a previous step already established.

L3 — The edge case that trips people up

The trap here is assuming stateful orchestration is a competitor to logic trees and prompt chains rather than a layer underneath both of them. A logic tree needs somewhere to record which branch was taken, so that later branch points can condition on earlier ones rather than each decision being made in isolation from the ones before it. A prompt chain needs somewhere to hold the outputs of earlier sub-prompts so later sub-prompts can be constructed using them. Stateful orchestration is that "somewhere" — it is the state-carrying substrate that makes both of the other two mechanisms actually work across more than one step, rather than a third, mutually exclusive alternative to choose between. A real multi-step agent design typically uses all three together: stateful orchestration holding the accumulating context, with a logic tree or a prompt chain (or both, at different points in the task) determining what actually happens with that context at each step.

The problem an architecture has to solve before launch day, not after

[GROUND TRUTH] (Sources/ncp-aai/domain-1-agent-architecture-design.md): objective 1.8's framing is direct — the architecture must survive change (new tools, new models, more load) without a rewrite, and the recommended approach is favoring modular, composable designs where an individual agent or tool can be swapped without retraining the whole system. This is not a nice-to-have layered on after a system ships; it is a property the architecture either has from the initial design or does not, and retrofitting it after the fact is close to as expensive as rebuilding.

The reasoning behind why this matters is straightforward once stated: the tools an agent calls, the models it runs on, and the load it needs to handle are all things that change on a timeline the architecture did not get to choose. A better search API becomes available; a cheaper or faster model is released; the product succeeds and traffic triples. An architecture tightly coupled to today's specific tool, today's specific model, and today's specific load level treats every one of those ordinary events as an emergency requiring a rewrite, rather than as routine maintenance.

What modular, composable actually means in a multi-step design

Concretely, a modular design for a multi-step reasoning system means each step in a logic tree, each link in a prompt chain, and each piece of state in the orchestration layer is defined by a contract — what it needs as input, what it produces as output — rather than by hard-coded knowledge of exactly which model or tool implementation currently sits behind that step. If the "extract five key facts" step in a document-drafting prompt chain is defined as "takes source text, returns five facts as a structured list," then swapping the specific model executing that step for a different one is a matter of pointing the same contract at a different implementation, not rewriting the chain. If a logic tree's "check for duplicate charge" branch is defined as "takes an account ID, returns a boolean," then that check can be backed by a different billing-system tool tomorrow without touching the tree's branching logic at all.

The alternative — a design where a prompt chain's second step directly embeds assumptions about the exact model that ran the first step, or where a logic tree's branch condition is hard-coded against one specific tool's exact response format — is what "cannot survive change without a rewrite" looks like concretely. The moment that specific model or tool changes, the coupling built into the design breaks, and the fix requires touching code that has nothing to do with the actual change being made, because the change and the fix are in different places purely due to how tightly everything was wired together.

Constructed scenario, illustrative only. Consider a research agent that answers a user's question by chaining three steps: search the web for relevant sources, summarize the top three results, and synthesize a final answer citing those sources. Trace what happens to two different implementations of this same three-step prompt chain when the underlying search API is swapped.

Design A: tightly coupled implementation.
Step 1 (search): calls Search-Provider-X's API directly, and the code that
  builds Step 2's prompt reaches into Search-Provider-X's specific JSON
  response shape (result["snippet"], result["source_url"]) to build the
  "summarize the top three results" prompt.
Step 2 (summarize): assumes exactly three results exist in that shape,
  because that is what Provider X always returned in testing.
Step 3 (synthesize): built the same way, reaching directly into Provider
  X's field names to build citations.

Change event: Search-Provider-X is deprecated; the team switches to
Search-Provider-Y, whose response JSON uses different field names
(result["extract"], result["link"]) and sometimes returns two results
instead of three for narrow queries.

What breaks: Step 2's prompt-construction code throws a key error looking
for a field name ("snippet") that no longer exists in Provider Y's
response. Even once that is patched, Step 3's citation-building code has
the same hard-coded field-name problem, and the "always exactly three
results" assumption silently produces malformed citations whenever
Provider Y returns two. Three separate pieces of code, scattered across
three chain steps, all need locating and fixing before the chain works
again — this is the rewrite the source material warns is the cost of a
tightly coupled design.

Design B: modular implementation, same three logical steps.
Step 1 (search): defined by a contract — "given a query, return a list of
  {text, url} results," regardless of count. A thin adapter translates
  whatever a given search provider's raw response looks like into that
  fixed shape; the adapter is the only place that knows Provider X's or
  Provider Y's specific field names.
Step 2 (summarize): built only against the {text, url} contract, with no
  assumption about which provider produced it or exactly how many results
  came back — it handles two results exactly as readily as three.
Step 3 (synthesize): built the same way, citing whatever {text, url}
  entries Step 2 actually summarized.

Change event: same swap, Search-Provider-X to Search-Provider-Y.

What breaks: nothing in Steps 2 or 3. A new adapter is written that
translates Provider Y's {extract, link} shape into the same {text, url}
contract Step 1 always exposed, and every downstream step keeps working
unmodified, because none of them ever depended on Provider X's specific
response shape or its specific result count in the first place.

The difference between the two designs is not that Design B "knew in advance" a provider swap was coming — it is that Design B defined each step by a contract rather than by a specific implementation's specific details, which is exactly the modular, composable property objective 1.8 asks for, and exactly what lets a tool swap stay a small, localized change instead of a chain-wide emergency.

⭐ THE EARNED INSIGHT Logic trees, prompt chains, and stateful orchestration answer "how is this task's reasoning sequenced and tracked," while adaptability answers a completely different question — "what happens to that sequencing and tracking the day one of its parts changes" — and treating the two as the same design problem is exactly how a well-structured multi-step agent still ends up needing a full rewrite eighteen months later. A logic tree can be perfectly branched and a prompt chain perfectly sequenced while every step inside them is hard-wired to one specific tool's response format; the structure is correct and the coupling is still fatal, because structure describes the shape of the reasoning and coupling describes whether that shape survives the reasoning's ingredients changing underneath it.

The same modular, contract-based reasoning objective 1.8 asks for inside a single agent's multi-step structure turns out to be exactly what makes multi-agent systems scale, which is why this domain's material draws the connection explicitly rather than treating adaptability as a single-agent-only concern. [GROUND TRUTH] (Sources/ncp-aai/domain-1-agent-architecture-design.md): the benefits named for multi-agent systems include modularity and scalability, described as the ability to replace or update one agent instead of retraining the whole system — which is the identical property this lesson has been building up from the single-step level, just applied at the level of a whole agent instead of a single step inside one agent's chain.

The parallel is worth making explicit because it means the modular-contract discipline this lesson develops for logic trees, prompt chains, and orchestration state is not a narrow technique for one design pattern — it is the same underlying principle showing up at every level of an agentic system's structure. A single step in a prompt chain defined by an input/output contract can be swapped for a different tool implementation without touching the rest of the chain. A single agent in a multi-agent system, if it exposes a similarly well-defined contract for what it accepts and what it returns to the rest of the team, can likewise be replaced or upgraded — a better-performing summarization agent swapped in for a weaker one — without redesigning the orchestration topology (M1-04) that coordinates the team around it. The concrete mechanism differs (a step-level contract versus an agent-level contract), but the design discipline behind both is the same commitment to defining a boundary by what crosses it, not by which specific implementation currently sits behind that boundary.

This is also where "more load" — the third change event objective 1.8 names alongside new tools and new models — gets a concrete answer inside the multi-step-reasoning material specifically. A prompt chain or logic tree built around stable step contracts can have any individual step's implementation swapped for a faster or more parallelized version to absorb more traffic, exactly as it can be swapped for a different provider's API, because "handle more load" and "handle a different underlying tool" are, from the contract's point of view, the same kind of change: a new implementation behind an unchanged boundary. A tightly coupled chain has no such option — scaling it up means touching the same brittle, implementation-specific code that a tool swap would have required touching anyway, because the coupling problem and the scaling problem trace back to the identical root cause.

Agent Architecture and Design carries 15% of the NCP-AAI blueprint, tied for the heaviest weight in the exam, and objectives 1.6 (structuring multi-step reasoning) and 1.8 (adaptability and scalability) sit inside it as named, separately numbered objectives rather than being folded into a single generic "design an agent" item. [GROUND TRUTH] (Sources/ncp-aai/domain-1-agent-architecture-design.md): the domain's scope note is explicit that this is a professional exam expecting reasoning about why a structure fits a scenario, and the logic-tree/prompt-chain/stateful-orchestration vocabulary exists precisely so a question can name a specific mechanism and ask you to recognize it, or name a scenario and ask which mechanism it is using.

Expect two recurring question shapes. The first names a described multi-step behavior — branching on a condition, a fixed sequence of sub-prompts, context persisting across steps — and asks which of the three mechanisms it demonstrates, testing the branching-versus-sequencing-versus-state-carrying distinction directly. The second describes a change event (a tool deprecated, a new model available, traffic tripling) against a described architecture, and asks whether that architecture handles the change gracefully or requires a rewrite — testing whether you can spot hard-coded coupling versus a genuinely modular, contract-based design, exactly as the worked example above walks through. A candidate who can name all three mechanisms correctly but cannot recognize tight coupling in a described architecture will pass the vocabulary check and fail the scenario check, which is where this domain's professional framing bites hardest.

Yes, and most real multi-step agents do exactly this rather than choosing one mechanism exclusively. A customer-support agent might use a logic tree to route an incoming ticket to one of several specialist paths (billing, technical, retention), and then, once routed, execute a fixed prompt chain specific to that path (for the billing path: look up the account, check for a duplicate charge, draft a resolution message). The tree decides which chain runs; the chain then executes as a fixed sequence once selected. Recognizing that combination in a scenario — a branch point followed by a fixed sequence — means correctly identifying both mechanisms are present rather than forcing the whole system into a single label.

No — stateful orchestration describes a property (context persists and is available across steps), not a specific product or library you have to adopt to get that property. It can be implemented as something as simple as a shared object passed by reference through a sequence of function calls, or as something as elaborate as a dedicated orchestration framework with persistence, retries, and observability built in. What makes an implementation count as stateful orchestration is that later steps can access what earlier steps established without that information being manually threaded through every intermediate step by hand — the specific technology used to achieve that is an implementation detail the exam's own framing treats as secondary to the property itself.

• Logic trees branch on conditions, prompt chains sequence sub-prompts, and stateful orchestration carries context across steps — three distinct mechanisms, not interchangeable synonyms for "multi-step agent."
• Stateful orchestration is the substrate underneath the other two, not a competing alternative: a tree needs somewhere to record which branch was taken, and a chain needs somewhere to hold prior outputs.
• Step count and branching are independent properties — a fixed five-step sequence with no decision points is a prompt chain, not a logic tree.
• Adaptability and scalability mean an individual agent, tool, or model can be swapped without retraining or rewriting the whole system, and that property has to be designed in from the start via modular, contract-based components.
• Hard-coding a step against one specific tool's exact response format is what turns an ordinary tool swap into a full rewrite; an adapter translating that tool's shape into a stable contract is what prevents it.
• A well-structured logic tree or prompt chain can still be fatally brittle if its individual steps are tightly coupled to specific implementations — structure and adaptability are separate design questions, both required.
• Real multi-step agents typically combine all three structuring mechanisms rather than choosing one exclusively, and the exam rewards recognizing which mechanism (or combination) a described scenario is actually using.

Structuring the sequence of steps and making each step swappable is only half the picture, because a sequence of steps still has to do something at each step — reason about the current situation, decide on an action, and learn from what that action reveals. M1-03 turns to exactly that: the ReAct pattern, which interleaves reasoning with real tool actions and observations, and why that interleaving — not just having a multi-step structure at all — is what actually curbs hallucination in a long-running agent.

Next: M1-03 — ReAct: interleaving reasoning and acting, including a full worked implementation with tool-failure handling.