Memory as an Architectural Concern: Short-Term vs. Long-Term

[GROUND TRUTH] (Sources/ncp-aai/domain-1-agent-architecture-design.md): objective 1.4 frames memory as something an architect manages, because LLMs cannot remember on their own — memory is an explicit architectural component you add. The mechanism behind that fact is worth stating plainly: a language model's forward pass computes an output from whatever tokens are currently in its input context and nothing else. There is no persistent internal counter, no side-channel notebook, no hidden state surviving from one API call to the next independent of what gets explicitly passed back in. If information from a prior turn is going to influence the current turn's output, that information has to physically appear in the current turn's input — either because the surrounding application included it in the prompt, or because it was never actually gone in the first place (still sitting inside the same context window the model is reading right now).

This is the fact that makes "does the agent remember X" always, at bottom, a question about the surrounding system's architecture rather than a question about the model's intelligence or capability. A more capable model does not remember more inherently; it can make better use of whatever memory the surrounding system hands it, which is a different and more limited claim. An architect who treats memory as something the model "just handles" because it is a sufficiently advanced model has skipped the actual design work objective 1.4 is asking for, and the result is an agent that behaves as though it remembers, right up until the specific mechanism quietly holding that illusion together (usually an un-truncated context window) runs out.

L1 — Intuition

Short-term memory is the rolling buffer of recent inputs and outputs the model can currently see — its context window — and its defining property is that it does not persist beyond the current session. Once the session ends, or once older content is pushed out of the context window by newer content, that information is gone from the model's reach unless something else, outside short-term memory entirely, captured it first.

L2 — Mechanism

[GROUND TRUTH] (Sources/ncp-aai/domain-1-agent-architecture-design.md): the source material describes short-term memory as a rolling buffer / context window holding recent turns, overwritten and non-persistent past a session. Concretely, in a chat-style agent, short-term memory is the literal sequence of prior user and assistant messages included in the current prompt, up to whatever window size the model supports. As the conversation grows, either the whole history keeps being resent every turn (fine for short sessions, increasingly expensive as the session grows) or older turns get trimmed, summarized, or dropped to stay within a token budget — at which point whatever was trimmed is no longer part of short-term memory, and the agent's behavior with respect to that trimmed content reverts to not having it at all, exactly as if it had never been said.

L3 — The edge case that trips people up

The trap worth naming directly: a model that correctly references something said fifteen messages ago in a long conversation has not demonstrated long-term memory — it has demonstrated that fifteen messages still fit inside the current context window. ⚠️ UNVERIFIED: whether a specific deployed conversation has actually exceeded its context window at any given point is not something a candidate can determine from the model's apparent recall alone, which is exactly why this appears as an exam trap rather than an obvious distinction — the visible behavior (the model "remembering") looks identical whether the mechanism underneath it is short-term (still-present context) or long-term (a deliberately persisted store being re-injected into the prompt). The only reliable way to tell the two apart in a scenario question is to check whether the description states that information persists across sessions (a new conversation, a new day, the same user returning after the context window would have long since been cleared) — if it does, the mechanism cannot be short-term memory alone, because short-term memory by definition does not survive past its session.

L1 — Intuition

Long-term memory is whatever a system builds specifically to survive past the current session — knowledge or context that is available not because it is still sitting in this conversation's context window, but because it was deliberately written somewhere durable and can be retrieved again later, in a different session, potentially by a different instance of the agent entirely.

L2 — Mechanism

[GROUND TRUTH] (Sources/ncp-aai/domain-1-agent-architecture-design.md): the source material names three typical implementations for long-term memory — databases, knowledge graphs, or vector embeddings — and notes that RAG (retrieval-augmented generation) is a common technique built on top of this layer. A database implementation stores structured facts (a user's stated preferences, a record of prior transactions) that can be looked up by key and injected into a future prompt when relevant. A knowledge graph implementation stores entities and the relationships between them, which — as M1-06 covers in full — supports relational, multi-hop queries a flat lookup cannot answer on its own. A vector embeddings implementation stores content as numeric vectors positioned so that semantically similar content sits close together, letting a new query retrieve whatever stored content is most similar to it, even if no exact keyword match exists.

L3 — The edge case that trips people up

The trap here is assuming these three implementations are three competing options where you pick exactly one for an entire system. In practice, they typically layer: a database might hold structured user-preference facts, a vector store might hold semantically searchable transcripts of past interactions, and a knowledge graph might hold the relational structure connecting entities across both — and a single agent's long-term memory can legitimately draw on more than one of these simultaneously, depending on what kind of question is being asked of it. The exam-relevant point is not "which one is correct" but recognizing which implementation a described retrieval mechanism actually is, and knowing that "long-term memory" is the umbrella term covering all three, not a synonym for any single one of them.

Why richer memory is not free

[GROUND TRUTH] (Sources/ncp-aai/domain-1-agent-architecture-design.md): the source material states directly that how you lay out memory shapes latency, cost, and how much relevant context reaches a decision — which is a compact way of saying that adding memory is never a strictly free improvement, even when it strictly improves what the agent knows. A richer long-term memory store — more history retained, more granular embeddings, a denser knowledge graph — genuinely can improve decision quality by making more relevant context available. It also genuinely adds retrieval latency (a vector search or a graph traversal takes real time, scaling with the store's size and the query's complexity) and real infrastructure cost (storage, compute for embedding and indexing, ongoing maintenance as the store grows).

Where the trade-off actually shows up in a design decision

This trade-off is not abstract — it shows up directly in choices like how large a short-term context window to maintain before trimming, how aggressively to summarize older turns rather than keep them verbatim, and how much long-term retrieval to run before responding to a given query. A support agent that runs an exhaustive long-term-memory search across a customer's entire multi-year history before answering every single message is spending latency on retrieval that may add nothing for a routine "what's my order status" question, while a support agent with no long-term memory at all will repeatedly ask a returning customer to re-explain context they have already provided in a prior session, degrading the experience in the opposite direction. Neither extreme is correct in general; the right amount of memory, and the right implementation for it, is a function of what the specific task actually needs to know and how much latency the task can tolerate paying to know it.

Retrieval-augmented generation is worth naming explicitly here because it is easy to mistake for a fourth long-term memory implementation sitting alongside databases, knowledge graphs, and vector embeddings, when it is actually a technique built on top of whichever of those three a system uses. [GROUND TRUTH] (Sources/ncp-aai/domain-1-agent-architecture-design.md): the source material names RAG as a common technique for implementing long-term memory, not as a distinct fourth storage mechanism in its own right. What RAG concretely does is retrieve relevant content from a long-term store — most commonly a vector store, though a knowledge-graph-backed retrieval step or a database lookup can play the identical structural role — and inject that retrieved content into the current prompt before generation, so the stateless model call in question has, for this one turn, exactly the external information it needs sitting inside its context window.

Seen this way, RAG is really long-term memory's retrieval half, paired with generation as the consumption half: a long-term store holds the persisted information, and a retrieval step (the "R" in RAG) is one specific, well-established pattern for deciding what subset of that stored information is relevant enough to surface into a given prompt. This also clarifies why "does this agent use RAG" and "does this agent have long-term memory" are related but not identical questions — an agent could have a long-term memory store that it queries by exact key lookup rather than by retrieval-based similarity search, in which case it has long-term memory without using RAG's specific retrieval pattern; conversely, describing a system as "using RAG" already implies some long-term store exists behind the retrieval step, because there is nothing to retrieve from otherwise.

Short-term and long-term is the core split this lesson has worked through, and it is the split objective 1.4 names directly. A closely related, finer-grained taxonomy — splitting memory further into episodic (specific past events), semantic (generalized facts), and procedural (learned skills and behaviors) categories — exists in this cert's material and gets its own full, dedicated treatment in Domain 5's memory-taxonomy lesson, because that finer split is Domain 5's own named objective rather than Domain 1's. The short version worth carrying forward from this lesson, without duplicating that later treatment, is that all three of those finer categories are functional subdivisions within long-term memory as this lesson has defined it — a specific past event, a generalized fact, and a learned skill are all things that, if they need to survive past the current session, get stored via one of the same three implementations (database, knowledge graph, or vector embeddings) this lesson already covered. The short-term/long-term split is about whether something persists past a session; the episodic/semantic/procedural split is about what kind of thing is being persisted once you have already decided it needs long-term storage. Keeping those two splits separate in your own head is what prevents the common error of treating "episodic" as a third option alongside "short-term" and "long-term," when it is instead a category living entirely inside long-term.

Constructed scenario, illustrative only. A travel-planning agent handles four distinct pieces of information over the course of a single multi-turn conversation and one later, separate session. Sorting each into short-term or long-term memory — and, for the long-term cases, into a specific implementation — is the exact judgment call a scenario question tests.

Item 1: "I'd like a window seat" -- stated in message 3 of the current
  conversation, referenced again by the agent when booking a seat in
  message 9 of the SAME conversation.
  -> Short-term memory. It never needs to survive past this session; the
  context window still holding message 3 is sufficient for message 9 to
  use it, and nothing about this preference needs a durable write.

Item 2: "I always prefer window seats on every future booking, not just
  this trip" -- an explicit request to apply this preference to
  bookings made in FUTURE sessions.
  -> Long-term memory, database implementation. This is a small,
  structured fact (seat preference) attached to a specific customer
  record, looked up by key on every future booking regardless of
  session -- exactly the database-backed case worked through in SS6's
  example.

Item 3: "Find me a trip similar to the anniversary trip we planned for
  me last year, but somewhere I haven't been" -- a request that
  requires searching an unstructured history of past conversations for
  one that resembles this new request semantically, with no exact key
  to look it up by.
  -> Long-term memory, vector-embedding implementation. There is no
  structured field called "anniversary trip" to query by key; this
  needs a semantic-similarity search over past interaction content,
  which is exactly the vector-embeddings case this lesson named and
  exactly the kind of query a database's exact-key lookup cannot serve.

Item 4: "Book me a flight connecting through whichever city my
  colleague usually flies through when visiting the same client, since
  I want the same airline lounge access" -- this requires knowing a
  RELATIONSHIP: this customer, that colleague, the shared client, and
  the colleague's typical routing -- not a single fact and not a
  semantically similar past conversation.
  -> Long-term memory, knowledge-graph implementation (the specific
  mechanism `M1-06` develops in full): relational, multi-hop reasoning
  over entities (customer, colleague, client, city, airline) and their
  relationships is exactly the query shape a graph serves and the other
  two implementations do not serve natively.

The pattern worth extracting from all four items is that the shape of the query a future request will need to make against stored information is what determines the right implementation — an exact-key lookup wants a database, a semantic-similarity search wants vector embeddings, and a multi-hop relational query wants a graph — and that determination has to happen at design time, when the information is first being written to storage, not retroactively once a request the storage cannot serve well finally arrives.

Constructed scenario, illustrative only. Consider a customer-support agent handling a returning customer across two separate sessions, a week apart.

Session 1 (Monday):
  Customer: "I'd like all future shipping notifications sent to my work
             email instead of my personal one."
  Agent: "Done — I've noted that preference for your account."
  [Session ends. The conversation's context window is discarded once the
  session closes, exactly as short-term memory's definition predicts.]

Design A: no long-term memory implementation.
  The "I've noted that preference" response was true only in the sense
  that the model said something plausible-sounding inside the
  conversation -- no database write, no persisted record, actually
  happened behind that sentence.

Session 2 (the following Monday):
  Customer: "Where's my shipping notification for the order I placed
             yesterday? I never got it."
  Agent (Design A): has no access to Session 1's content at all -- it is
  gone, and no long-term store was ever written to capture it. The
  notification went to the personal email exactly as originally
  configured, because the preference change was never actually
  persisted anywhere durable. The agent cannot even explain why,
  because it has no record the preference change was ever requested.

Design B: a database-backed long-term memory implementation.
  Session 1's "send to work email instead" request triggers an actual
  write to a customer-preferences database record, keyed to the
  customer's account ID -- not just a conversational acknowledgment.

Session 2, Design B:
  Before generating a response, the agent's orchestration layer looks up
  the customer's preference record from the database (a long-term
  memory read, independent of anything in the current session's short
  context window) and finds "shipping notifications: work email,
  updated Monday." The agent can now correctly explain: "Your
  notification preference was updated to your work email last Monday --
  can you confirm that address is still correct, since the notification
  should have gone there rather than to your personal email."

The gap between Design A and Design B is not a difference in model capability — the same underlying model could power both. The gap is entirely architectural: Design B added a persistence step (an actual database write, triggered by the preference-change request) and a retrieval step (an actual database read, triggered at the start of the next session) around the model, and Design A did not. [GROUND TRUTH] (Sources/ncp-aai/domain-1-agent-architecture-design.md) — this is exactly the objective 1.4 framing in miniature: the model saying something that sounds like a memory commitment is not memory; memory is the explicit component that makes the commitment durable, and Design A's failure is precisely what happens when a team assumes the model's confident-sounding "I've noted that" substitutes for actually building that component.

⭐ THE EARNED INSIGHT A model producing a sentence that sounds like a memory commitment — "I've noted that," "I'll remember this for next time" — is not evidence that any memory mechanism actually exists behind it; it is evidence that the model correctly predicted what a helpful assistant would plausibly say next, which is a claim about language modeling, not about persistence. The only way to know whether an agent's apparent memory is real is to ask what specific component — a context window still holding the relevant turn, or a database/graph/vector write that actually happened — is responsible for the information being available later, and an architecture review that skips that question is trusting the model's confident tone as a substitute for verifying the actual write path exists.

Memory sits inside Agent Architecture and Design as objective 1.4, in a domain tied for the heaviest weight in the whole blueprint at 15%. [GROUND TRUTH] (Sources/ncp-aai/domain-1-agent-architecture-design.md): the objective's own framing — manage short-term and long-term memory for context retention — places memory design squarely among the architectural decisions this domain expects candidates to reason about, not merely define, matching the domain's own scope note that this is a professional exam testing why a design choice fits a scenario.

Expect the question shape to work in two recurring forms. The first is direct classification: a described mechanism (a rolling context buffer, a customer-preference database, a semantic search over past support tickets) mapped to short-term or long-term, and — for the long-term case — to which of the three named implementations it actually is. The second, and the one this lesson's worked example was built to prepare for, is a scenario testing the trap directly: a model appearing to "remember" something, with the question asking whether that is evidence of a real memory component or simply an unfilled context window, which requires noticing whether a session boundary was actually crossed rather than reasoning from how confidently the model phrased its response.

No — a vector store is one of three typical implementations of long-term memory, not a synonym for the category itself. Treating "long-term memory" and "vector store" as interchangeable is a narrower version of the trap this lesson has repeatedly flagged: it silently forgets that databases and knowledge graphs are equally valid long-term memory implementations, suited to different query shapes than the semantic-similarity search a vector store provides. A system that persists customer preferences in a structured database, with no vector store anywhere in its design, still has genuine long-term memory — it simply implemented it via exact-key lookup rather than similarity search, exactly as Item 2 in the worked example above did. The habit of reaching for "vector store" as shorthand for "long-term memory" in casual conversation is understandable, given how common RAG-style architectures have become, but a scenario question testing this material is specifically checking whether you know the umbrella term has three implementations under it, not one.

No — a longer context window increases how much a single session can hold before short-term memory starts trimming older content, but it does nothing for information that needs to survive past that session, however long the session itself can run. A model with an enormous context window can maintain a very long single conversation's worth of short-term memory without ever trimming, and that is a genuine improvement for tasks confined to one long session. It still provides nothing for a customer who returns a week later, in a fresh session, expecting the agent to know what was discussed the previous week — that gap is a session boundary, not a context-window-size problem, and no amount of window expansion closes it. Long-term memory exists specifically for information that needs to cross that boundary, and a bigger window changes where the boundary sits within one session, not whether the boundary exists between sessions at all.

There is also a cost dimension to this that a "just make the window bigger" instinct tends to overlook: resending an ever-larger short-term context on every single turn of a long session is not free, even before any session boundary is reached. Every additional token carried forward inside the context window is a token the model has to process again on the next call, and a session that has grown to fill a very large window is paying that processing cost on every subsequent turn, whether or not most of that carried-forward content is still relevant to the current question. A well-designed short-term memory layer often trims or summarizes proactively, well before the window's hard limit, specifically to keep that per-turn cost bounded — which means "how big is the window" and "how much of the window am I actually using at any given moment" are two different design levers, and a bigger window does not obligate a system to use all of it on every turn.

• An LLM call is stateless by design — memory is always an explicit architectural component someone adds, never a property the model provides on its own.
• Short-term memory is the session-scoped context window; it does not persist past the session, and content trimmed to fit a token budget is gone exactly as if it had never been said.
• Long-term memory is whatever is deliberately built to survive past a session, typically via a database, a knowledge graph, or vector embeddings — three implementations that commonly layer rather than compete.
• Adding memory is never strictly free: richer long-term memory improves decision quality at the cost of real retrieval latency and infrastructure cost, and matching the amount of memory to what a task actually needs is itself a design decision.
• A model producing language that sounds like a memory commitment ("I've noted that") is not evidence a persistence write actually happened — verify the write path, not the model's tone.
• A longer context window extends short-term memory's reach within one session; it does nothing to close the gap across a session boundary, which only a genuine long-term mechanism can bridge.

Knowing that long-term memory can be backed by a knowledge graph raises an immediate follow-up question this lesson deliberately left open: what does a knowledge graph actually buy an agent that a vector store does not, and when does that difference matter enough to reach for one over the other? M1-06 picks up exactly there, going deep on relational, multi-hop reasoning as the specific capability a flat vector lookup structurally cannot provide.

Next: M1-06 — knowledge graphs for relational, multi-hop reasoning, and why graphs and vector search are not interchangeable despite both counting as long-term memory implementations.