Adapting Reasoning from Prior Experience and Feedback

[GROUND TRUTH] (Sources/ncp-aai/domain-5-cognition-planning-memory.md): the domain's own framing states the mechanism directly — "An agent that logs outcomes (episodic memory) and reflects (a planning direction) can adjust its approach over time." Read that sentence as naming two necessary ingredients, not one. The first ingredient, logging outcomes, is M5-02's episodic memory doing its ordinary job: recording a specific past event — what was attempted, what happened, what the result was — in a form that can be retrieved later. The second ingredient, reflecting, is M5-04's Reflection direction doing its ordinary job: taking an observed outcome and using it to revise behavior going forward, rather than proceeding as though the outcome had not occurred.

Neither ingredient alone produces adaptation. Logging without reflection produces a growing archive that never changes anything — the classic failure mode of a system that "has all the data" but never acts on it. Reflection without a log to reflect on has nothing to work from beyond the current task's own immediate outcome, which limits it to reacting within a single task rather than adapting across many. Adaptation, in the sense this lesson uses the word, specifically means the combination: a durable record of what happened across potentially many past tasks, retrieved and reasoned over, producing a change in how a future task gets approached — not just how the current one gets finished.

It is worth being precise about a distinction that is easy to blur: Reflection, as M5-04 covered it, can revise a plan mid-task in response to an outcome from earlier in that same task — this is a real, valuable behavior, but it is not yet what this lesson means by adaptation. Adaptation specifically refers to a change that persists past the task in which the triggering outcome occurred, informing how a later, different task gets approached. The difference is exactly the episodic-memory ingredient: a mid-task Reflection can happen with no durable log at all, using only the current task's own in-progress state (M5-05's orchestration state) as its input. Adaptation requires that outcome to survive past the task's end, written into a durable store, so that a future, unrelated task's planning can retrieve and benefit from it.

L1 — Intuition

A single meeting where you notice an approach isn't working and change tactic mid-meeting is Reflection within that meeting. Writing a note afterward — "this kind of approach tends to fail when the stakeholder is skeptical up front; try leading with data instead" — and actually consulting that note before your next similar meeting is adaptation. The first happens once, inside one event, and leaves nothing behind. The second requires the note to exist, to be findable later, and to actually get read and acted on before the next relevant meeting starts.

L2 — Mechanism

Mechanically, the adaptation loop this lesson describes has four stages, each doing distinct work: (1) an outcome occurs — a task's plan succeeds, fails, or produces an unexpected result; (2) that outcome is logged as an episodic entry, capturing enough context (what was attempted, under what conditions, with what result) to be useful later, not just that something happened; (3) a future, potentially unrelated task's planning process retrieves relevant past episodes — using the Memory planning direction from M5-04 — as an input to how it structures its own approach; (4) the retrieved episode changes something concrete about that future task's plan, rather than being retrieved and then ignored. All four stages have to actually occur for the loop to close. A design missing stage 2 has nothing to retrieve later. A design missing stage 3 has a log nobody consults. A design missing stage 4 retrieves the past episode and generates a plan that looks exactly like it would have without that retrieval — which is functionally identical, from the outside, to not having adapted at all.

L3 — The exam-relevant edge case: adaptation is not retraining

A specific, high-value distinction to hold onto: adapting reasoning from experience, in the sense this domain describes, does not mean updating the underlying model's weights. ⚠️ UNVERIFIED — whether some production agent systems additionally incorporate weight updates as part of a broader adaptation strategy is plausible but not something the domain-5 source material itself asserts, so the specific mechanism this lesson's adaptation loop describes should be understood as memory-and-reflection-based, not necessarily model-retraining-based, unless a scenario explicitly states otherwise. The adaptation this lesson covers happens at the level of what gets retrieved and reasoned over before a decision, not at the level of the model's parameters — an agent can adapt meaningfully across thousands of tasks using nothing more than an accumulating episodic and semantic memory store, with the exact same underlying model weights the entire time. Conflating "the agent got better at this over time" with "the model was retrained" is a natural but incorrect inference, and a scenario testing this distinction is checking whether you default to the retraining explanation when the source material's own framing points specifically at logged outcomes and reflection instead.

Outcomes an agent observes directly (did the task succeed or fail) are one source of signal for this loop, but they are not the only one. Structured feedback — explicit signal from a human or another system about whether a given response or action was good — feeds the identical adaptation loop through the identical mechanism: it gets logged (often as its own episodic entry, or as an annotation attached to an existing one), retrieved by a future task's planning process, and used to inform that future task's approach. A rating a user gives an agent's response, a correction a human reviewer makes to an agent's draft, or a flag raised during a human-oversight review are all structured feedback in this sense, and all three close the same loop: log the feedback, make it retrievable, let it actually change a future decision.

The reason to name this as a separate input rather than folding it silently into "outcomes" is that feedback and outcomes are not always the same signal. A task can produce an outcome that looks, by the agent's own criteria, like a success — the task completed, the format was correct, nothing errored — while a human reviewer's feedback flags it as unsatisfactory for a reason the agent's own success criteria never captured, such as a tone that reads as dismissive even though the factual content was correct. An adaptation loop that only logs self-assessed outcomes and never incorporates externally supplied feedback will miss exactly this class of gap, however faithfully it logs and reflects on everything it can measure about itself.

This closing lesson's adaptation loop shares real structure with two ideas that belong to a different module of this cert, and naming the connection — without collapsing the distinction — is worth doing explicitly, since the exam expects both. This cert's Module 3 treats structured feedback and what it calls the data flywheel as their own subject: feedback informing refinement, generally in prose terms rather than this domain's specific memory-and-reflection mechanism. The data-flywheel idea, broadly, is that experience feeds refinement — outcomes and feedback accumulate, get incorporated, and the resulting behavior improves, which then generates more experience to feed the same cycle again. This module's adaptation loop is, at the mechanism level, one concrete way that broader flywheel idea actually gets implemented for an individual agent's reasoning: episodic logging is the accumulation step, Reflection and retrieval are the incorporation step, and a changed future plan is the improved-behavior step the flywheel's next turn then builds on.

The distinction worth holding onto is that Module 3's flywheel framing is broader and platform-level — it also covers, for instance, a repeatable evaluation harness confirming that a change actually helped rather than merely feeling like an improvement — while this lesson's adaptation loop is specifically about the memory-and-reflection mechanism inside a single agent's reasoning that this domain names. A scenario invoking "the data flywheel" broadly is likely drawing on Module 3's framing; a scenario asking specifically how an agent's reasoning changes based on logged outcomes and reflection is squarely this lesson's territory. Recognizing which of the two a question is actually asking about — the platform-level cycle, or the reasoning-level mechanism — is more useful than trying to memorize one universal definition that covers both equally well.

Because the loop has four distinct stages, it also has (at least) four distinct ways to fail, and diagnosing which stage broke is more useful than treating "the agent didn't adapt" as one undifferentiated problem. Working through each stage's specific failure mode makes the diagnosis concrete rather than a vague sense that "something in memory isn't working."

A stage-1 failure (the outcome itself) happens when an agent cannot actually tell whether a given task succeeded or failed — for instance, a refund gets auto-approved and the agent's own success criteria mark it "done" with no visibility into whether the customer disputed it again a week later, because nothing in the agent's design ever checks back on a resolved task's downstream fate. Without a reliable signal about what actually happened, there is nothing worth logging in the first place, and every later stage is working from bad or missing input no amount of retrieval sophistication can fix.

A stage-2 failure (logging) happens when an outcome is known but never gets written into a durable, retrievable form — it might be visible in a dashboard a human glances at, or briefly present in a task's orchestration state, but if nothing writes it into the episodic store described in M5-02, it is gone the moment the task's temporary state is discarded. This is the single most common failure mode in systems built primarily for execution rather than for learning: the information genuinely existed at some point, but no one built the write path that would have made it available later.

A stage-3 failure (retrieval) happens when outcomes are logged correctly but a later task's planning process never queries them — the episodic store exists, is populated, and is simply never consulted, either because no retrieval step was wired into the planning process at all, or because the retrieval query is specified narrowly enough that relevant past episodes never actually surface. §6's task 41 retrieval, for instance, depended on querying by a dollar range and an outcome pattern; a retrieval query scoped only to exact-dollar-amount matches would have missed the broader pattern entirely, technically "retrieving" without ever surfacing the generalization that mattered.

A stage-4 failure (change) is the subtlest and the one most likely to go unnoticed, because everything upstream of it looks correct: outcomes are logged, retrieval runs, a relevant past episode or rule genuinely surfaces in the current task's context — and the plan proceeds exactly as it would have without that retrieval. This can happen when a retrieved episode is presented to a planning step as informational context with no explicit instruction or mechanism forcing the plan to actually incorporate it, so it sits in the input unused, the same way any other unused piece of context might. This is the failure §6's worked example was built specifically to avoid: task 42's plan had to visibly differ from task 1 through 40's for the loop to count as closed, and confirming that difference actually occurred — not just confirming that retrieval happened — is the necessary final check.

Consider a support agent handling refund requests across many separate sessions over several months, tracing the adaptation loop end to end.

Task 1 (month 1): Agent auto-approves a $180 refund with no escalation. Customer disputes
the charge again a week later, indicating the auto-approval did not actually resolve the
underlying issue.
  -> Logged as an episodic entry: {amount: 180, action: "auto-approved",
     outcome: "disputed again within a week", flag: "auto-approval did not resolve root cause"}

Tasks 2-40 (months 1-3): Similar pattern recurs across several dozen refund requests in
the $150-220 range -- auto-approval followed by a repeat dispute within roughly a week,
each logged as its own episodic entry.

Task 41 (month 3): Before deciding how to handle a new $195 refund request, the planner's
Memory step (M5-04) retrieves past episodes in a similar dollar range and outcome pattern.
Retrieval surfaces the recurring "disputed again" flag across enough prior entries that a
semantic rule gets generalized and written to long-term memory (M5-02): {rule: "Refunds
in the $150-220 range that are auto-approved without a root-cause note have an elevated
repeat-dispute rate; escalate for a root-cause check before approving."}

Task 42 (month 3, later the same day): A new $200 refund request arrives. The planner
retrieves the newly written semantic rule, and -- this is the stage-4 test from L2 -- the
plan actually changes as a result: instead of auto-approving, the agent routes the request
through a root-cause check before approving, which task 1 through 40's plans never did.

Constructed scenario — the dollar amounts, task counts, and specific rule are illustrative, not drawn from a real support system. Every one of the four stages from §2's L2 is visible here: outcomes were logged (stage 1-2) across many tasks, a later task's planning retrieved the accumulated pattern (stage 3), and — critically — task 42's actual plan differed from task 1's because of that retrieval (stage 4). Had task 42 retrieved the same semantic rule and still auto-approved without a root-cause check anyway, the loop would not have closed, regardless of how faithfully the logging and retrieval steps had worked.

To make §3's distinction between self-observed outcomes and structured feedback concrete rather than abstract, trace a case where the two disagree.

Task: An agent drafts a decline letter for a loan application, citing the applicant's
debt-to-income ratio as the reason, and sends it.

Self-observed outcome: The letter was sent successfully, used the correct template,
and cited a factually accurate debt-to-income figure. By the agent's own success
criteria (did it send, was the content technically accurate), this task is logged as
a success.

Structured feedback (added a day later, from a compliance reviewer): The letter is
flagged as non-compliant -- it did not include a required specific-reasons disclosure
statement mandated for this loan category, a requirement the agent's own drafting
process was never checking for.

Without the feedback layer: only the self-observed "success" gets logged. A future,
similar decline-letter task retrieves this episode as a positive precedent and repeats
the same drafting pattern -- including the same missing disclosure -- because nothing
in the log ever indicated a problem existed.

With the feedback layer: the compliance flag gets logged as its own entry, linked to
the same task. A future similar task's retrieval surfaces both the self-observed
outcome AND the compliance flag, and the resulting plan adds an explicit disclosure-
statement check before sending -- a change self-observed outcomes alone would never
have triggered, because the agent's own criteria never measured the thing that was
actually wrong.

Constructed scenario — the loan category and specific disclosure requirement are illustrative, not a citation of any real regulation. This is the concrete version of §3's abstract point: an outcome that looks like success by an agent's own internal measure can still be missing something only an external reviewer would catch, and an adaptation loop that only logs self-assessed outcomes has a permanent, structural blind spot exactly where its own success criteria happen to be incomplete.

A self-observed outcome is generated by the agent's own success criteria — did the task complete, was the output well-formed, did any error occur — and can only ever detect problems the agent's own criteria were designed to check for. Structured feedback is supplied externally, by a human reviewer or another system, and can surface problems entirely outside what the agent's own success criteria measure, such as a compliance gap, a tone issue, or a subtly wrong recommendation that nonetheless executed without error. Both feed the identical four-stage adaptation loop — logged, retrieved, reflected on, and used to change a future plan — but only structured feedback can catch the specific class of failure where a task looks successful by every internal measure while still being wrong in a way nothing internal was checking for.

Cognition, Planning, and Memory carries 10% of the NCP-AAI blueprint, and objective 5.5 — adapting reasoning strategies "based on prior experiences and feedback" — is this domain's closing objective, deliberately positioned to require every other piece of this module already in place: memory to log with, Reflection to act with, and orchestration state to carry the triggering outcome from execution into a durable record in the first place. [GROUND TRUTH] (Sources/ncp-aai/domain-5-cognition-planning-memory.md) states this closes the loop "with evaluation (Domain 3) and the data flywheel: experience feeds refinement" — which is the specific cross-domain connection §4 worked through, and a likely source of scenario questions that test whether you can name the mechanism (memory plus reflection) without either conflating it with model retraining or dismissing the Module 3 connection entirely.

Expect a question that describes an agent behaving identically across many repeated failures with no change, and asks what is missing from its design — the correct diagnosis is almost always a broken link somewhere in the four-stage loop from §2 (most often stage 3 or 4: the log exists, but nothing retrieves it, or something retrieves it and ignores it), not "the agent needs a better base model." A second likely shape offers "the model was retrained on the failure data" as a tempting but incorrect explanation for an agent's improved behavior over time, when the scenario's own details (accumulating episodic entries, a generalized rule, a changed future plan) point specifically at this lesson's memory-and-reflection mechanism instead.

⭐ THE EARNED INSIGHT

"The agent learned from its mistakes" describes the same observable outcome whether the mechanism underneath it is a retrained model or an unchanged model retrieving its own accumulated episodic memory — and the exam is betting that most people reach for the retraining explanation by default, because it is the more familiar story about how machines improve. Every mechanism this lesson actually describes leaves the weights untouched: the improvement lives entirely in what gets logged, what gets retrieved, and whether retrieval ever reaches far enough to change the plan in front of it. A model that never learns anything can still belong to an agent that visibly gets better every week, and telling those two apart is the whole point of this closing objective.

Four things, all of which have to be present: an outcome has to occur, it has to be logged as a durable episodic entry (not just held in the current task's temporary orchestration state), a future task's planning has to actually retrieve that entry, and that retrieval has to change something concrete about the future task's plan. Logging without retrieval produces an archive nothing acts on; retrieval without a resulting plan change produces a lookup that changes nothing. Adaptation specifically means all four links close into one loop — any one of them missing, and the agent will behave identically to how it would have without any of this machinery at all.

Not in the sense this domain describes. The mechanism this lesson covers is memory-and-reflection-based: outcomes are logged as episodic entries, retrieved by future tasks' planning processes, and used to inform those future plans — all without changing the underlying model's weights. An agent can adapt meaningfully across thousands of tasks with the exact same model the entire time, purely by accumulating and acting on its own memory. Attributing improved behavior to retraining, when a scenario's details point at logged outcomes and retrieval instead, is one of this lesson's most commonly tested traps.

Yes, and this is a real design risk rather than a purely theoretical one: an agent that generalizes a semantic rule from a handful of episodic entries risks encoding a pattern that was actually coincidental rather than reliably predictive, the same overfitting concern that applies to any process generalizing from a limited sample. §6's task-41 example generalized its rule from several dozen prior episodes spanning multiple months — a sample size deliberately chosen to make the pattern credible rather than a fluke drawn from two or three unlucky cases. A design that writes a semantic rule after a single bad outcome, with no check on how many similar past cases actually support the generalization, risks an agent that adapts confidently to noise. This is not a reason to avoid adaptation, but it is a reason the retrieval and reflection stages of the loop should weigh how much evidence actually backs a pattern before letting it change future behavior, rather than treating every single logged outcome as equally strong grounds for a rule.

• Adaptation requires a full four-stage loop — log an outcome, retrieve it later, reflect on it, and change a future plan — not any single stage alone.
• Mid-task Reflection and cross-task adaptation are related but distinct: Reflection can happen using only in-progress orchestration state, while adaptation specifically requires the triggering outcome to survive into a durable, retrievable record.
• Structured feedback from a human or external system feeds the same loop as self-observed outcomes and can catch gaps a purely self-assessed outcome would miss.
• Adaptation, in this domain's framing, does not mean retraining the underlying model's weights — it means logged experience changing what gets retrieved and acted on in future reasoning.
• This lesson's adaptation loop connects to, but is distinct from, this cert's Module 3 treatment of structured feedback and the data flywheel: the loop is the reasoning-level mechanism, the flywheel is the broader platform-level practice it instantiates.
• On the exam, expect a scenario describing an agent that fails to improve despite accumulating data, with the correct diagnosis usually being a broken retrieval or plan-change link rather than a model or data quality problem.

Module 5 opened with the fact that a model has nothing of its own to carry forward between calls, and every lesson since has been an answer to the same underlying question, asked at a different scale: memory named what can be retained, reasoning frameworks and planning named what can be done with it, orchestration named where it lives while a task runs, and this closing lesson has named what it takes for a retained outcome to actually reach forward and change a future decision rather than sit unread. None of these pieces functions as a substitute for any other — a rich memory taxonomy with no planning direction to act on it is inert, a sophisticated planner with nothing durable to retrieve from is starting over every time, and a full adaptation loop with no orchestration state to capture the triggering outcome in the first place has nothing to log. The module's guiding question — what does it mean for an agent to remember, reason, and plan, and which of the five planning moves are you actually making at each step — is answerable now precisely because each lesson supplied one piece the others assumed already existed.

This closing lesson finishes Module 5's answer to what it means for an agent to remember, reason, and plan — memory to draw on, frameworks to reason with, five directions to plan through, a shared state to coordinate all of it, and a loop that lets experience actually change future behavior.

Next: the next module turns from an agent's internal cognition to what it grounds that cognition in — how an agent retrieves knowledge it does not already have baked into its weights, starting with the canonical retrieval pipeline and the distinction between plain vector lookup and richer retrieval architectures.