Metadata in RAG: what to embed versus what to return

RAG metadata is a set of key-value fields stored with each chunk in the index, next to that chunk's vector and its text. Concretely, an indexed record looks less like a bare vector and more like this:

record
  id             doc-4471#chunk-12
  vector         [0.0182, -0.0431, ... ]        ← what similarity search compares
  text           "Study leave is granted at the manager's discretion ..."
                                                ← what the model reads
  metadata
    source_file        hr-policy-2024-v3.pdf
    page               41
    section            "7.4 Study and examination leave"
    doc_title          "Employee Handbook 2024"
    doc_type           policy
    effective_date     2024-04-01
    version            3
    supersedes         2
    department         hr
    acl                ["employee", "manager"]
    chunk_index        12
    chunk_of           58
    ingested_at        2024-05-02
Illustrative record shape, constructed for teaching.

Three of those items do three genuinely different things and it is worth separating them before anything else:

• The vector decides whether the chunk is retrieved, by similarity to the query.
• The metadata fields decide which chunks are eligible to be retrieved (filtering), and what the reader is told about the chunk afterwards (citation, provenance, freshness).
• The text decides what the model actually reads and can therefore say.

The embed-versus-return decision is the question of which facts should participate in the first of those and which in the second. It is not a question of which facts to keep — you keep all of them.

L1 — Two channels, one record

Think of every chunk as having two channels into the retrieval system.

The semantic channel is the embedded string. Whatever text you embed is what the vector represents, so anything in that string can pull the chunk towards a query. Prepend the section heading 7.4 Study and examination leave to a chunk body and a query about "study leave" now matches on the heading as well as the body — a real improvement when the body uses different words than the reader does.

The structured channel is the metadata fields. These are not embedded and contribute nothing to similarity. They are exact, filterable, sortable, and returnable. A query restricted to doc_type = policy AND effective_date >= 2024-01-01 never even considers chunks that fail the predicate, however similar their vectors are.

The reason both channels exist is that embeddings are bad at exactly the things structured fields are good at. Vector similarity cannot reliably represent "after March 2024", cannot represent "not superseded", cannot represent "only if the user is in the finance group", and cannot represent negation at all — a limit taken up properly in 07-03. Embedding a date as text does not give you date comparison; it gives you a vector that is faintly more similar to other text containing similar-looking digits. That is not a filter. It is a rumour of a filter.

L2 — Metadata filtering and how it interacts with the index

Filtering on metadata happens in one of three places, and knowing which is a genuine architecture question:

Post-filtering is the trap. It looks equivalent and is not: it silently reduces your result count and, in the worst case, returns nothing for a query whose answer is sitting in the index behind a filter the ranker never knew about. Pre-filtering costs more and gives the right answer. Approximate-nearest-neighbour structures such as HNSW and IVF interact with filtering in ways that matter here, which is why the vector-database lesson 07-04 follows this one rather than preceding it.

L3 — Enrichment: metadata that becomes text

The advanced move is deliberate context enrichment: constructing the embedded string from the chunk body plus a small amount of structural context, while keeping the same facts as fields too.

Chunk body as cut by the chunker
  "The limit rises to 90 days for employees in this category."

Embedded string after enrichment
  "Employee Handbook 2024 — 7.4 Study and examination leave
   The limit rises to 90 days for employees in this category."

Metadata fields, unchanged, still stored separately
  doc_title, section, page, effective_date, version, acl ...
Constructed example.

The same facts appear twice on purpose, in two channels, for two jobs. The heading in the embedded string makes the chunk findable by someone searching for study leave. The heading in the metadata field makes the citation renderable and the results filterable by section. Duplication between the channels is not redundancy to be eliminated; it is the design.

The discipline is knowing when to stop. Enrichment is additive to a fixed-length vector, so every token of prepended context is a token of the body's influence diluted. Prepend a document title and a section heading and you have improved the vector. Prepend the title, the section, the subsection, the date, the version, the department, the author, and a confidentiality banner, and the body of a 300-token chunk is now a minority of its own embedded string — and worse, every chunk in the document now shares a large identical prefix, so all their vectors have been pulled towards each other. You have reduced the index's ability to distinguish its own contents. That is the boilerplate mechanism from 06-04, arriving through a door you opened yourself.

The changing it later row is the practical one. Metadata that lives in the embedded string is baked into the vector, so correcting it means re-embedding. Metadata that lives in fields can usually be updated in place. That asymmetry alone argues for keeping the embedded string minimal and the field set generous: fields are cheap to fix and vectors are not.

A constructed example. Take one chunk of an HR handbook and index it three ways, then ask what each version can and cannot do. Constructed scenario.

The chunk body, exactly as the chunker produced it:

"The limit rises to 90 days for employees in this category, subject to
 approval by the department head. Applications must be submitted at least
 six weeks before the leave begins."

Version A — body only, no metadata at all.

embedded string: the body above
metadata:        none

query "how much study leave can a manager approve?"
  → the chunk contains neither "study" nor "leave" nor "manager"
  → similarity is weak; the chunk does not enter the top 10
answer: the system reports it cannot find the policy
citation: impossible — nothing records which file or page this came from

The chunk is in the corpus, it is the correct answer, and it is unreachable. This is a decontextualisation failure of exactly the kind 06-02 predicted, and no chunking parameter fixes it because the words the reader searches for are not in the span.

Version B — everything crammed into the embedded string.

embedded string:
  "hr-policy-2024-v3.pdf | page 41 | Employee Handbook 2024 |
   7.4 Study and examination leave | policy | effective 2024-04-01 |
   version 3 | department hr | acl employee,manager | CONFIDENTIAL —
   INTERNAL USE ONLY — DO NOT DISTRIBUTE |
   The limit rises to 90 days for employees in this category, ..."
metadata: none (it is all in the text)

query "how much study leave can a manager approve?"
  → now matches on "Study and examination leave" and "manager" — better
but:
  → every chunk of every 2024 policy shares ~40 tokens of identical prefix,
    so all their vectors have been dragged towards a common point
  → the confidentiality banner is now part of the semantic content of every
    chunk in the corpus, and a query about confidentiality retrieves
    everything, ranked essentially arbitrarily
  → "effective 2024-04-01" as text supports no date comparison whatsoever;
    a query for "current policy" cannot exclude the superseded version 2
  → "acl employee,manager" is a string in a vector, not an access control;
    it restricts nothing and is trivially retrievable by anyone
Constructed illustration.

Version B is what "add metadata to improve retrieval" degenerates into when the two channels are not distinguished. It improved one thing and broke three, and its ACL is security theatre.

Version C — split across the two channels deliberately.

embedded string:
  "Employee Handbook 2024 — 7.4 Study and examination leave
   The limit rises to 90 days for employees in this category, subject to
   approval by the department head. Applications must be submitted at least
   six weeks before the leave begins."

metadata fields:
  source_file  hr-policy-2024-v3.pdf     page 41
  doc_type     policy                    section  "7.4 Study and examination leave"
  effective_date 2024-04-01               version  3      supersedes 2
  department   hr                         acl      ["employee","manager"]
  chunk_index  12                         chunk_of 58

query "how much study leave can a manager approve?"
  filter: doc_type = policy AND version = current AND acl ∋ caller's roles
  → superseded version 2 chunks are excluded before search
  → chunks the caller may not see are excluded before search
  → similarity search runs over the eligible set and matches on the heading
    plus the body
  → answer returned with citation: Employee Handbook 2024, §7.4, page 41,
    effective 1 April 2024
Constructed illustration.

Version C costs one extra design decision and buys three capabilities Version B cannot have at any price: real date and version filtering, a real permission boundary, and a renderable citation. Note also what stayed out of the embedded string — the filename, the page number, the date, the version, the ACL. None of them helps a reader's query match, and all of them help after the match is made.

The never embed an ACL row is the one to carry away as a rule rather than a preference. Text in an embedded string is a contributor to a similarity score. It is not a gate, it is not checked, and it cannot deny anything. A chunk whose embedded text says acl: finance-only is retrievable by every caller, and the string itself makes it more likely to surface for a query mentioning finance. Access control has to be enforced by the retriever as a filter applied before results are returned, and it depends on how the index was built — which is precisely why the course teaches it as its own lesson after vector databases. 07-05 is where that argument is completed.

Metadata serves objective 1.4, "Curate and embed content datasets for RAGs" — the phrase "curate and embed" is doing double duty, and the embed-versus-store split is the most concrete form of that distinction in the whole pipeline. It serves objective 1.6, "Familiarity with the capabilities of Python natural language packages (spaCy, NumPy, vector databases, etc.)," because metadata filtering is a named capability of a vector database and knowing that a vector store holds filterable fields alongside vectors is exactly the level of familiarity the objective asks for. It serves objective 1.3 on building RAG use cases, and objective 4.4 on identifying required components — "we need date filtering, therefore the store must support metadata predicates" is that objective in one sentence. [OFFICIAL] on objective wording.

There is also a trustworthy-AI thread here that pays off in module 13. NVIDIA's published account of RAG makes citing sources part of what RAG is: the model retrieves, synthesises, and cites. [NVIDIA-DOC] Citation is not a feature bolted on at the end — it is only possible if provenance metadata was captured at ingestion and carried through retrieval. Transparency is one of NVIDIA's named trustworthy-AI pillars, and a RAG answer with no source is an answer with no transparency. [NVIDIA-DOC] The decision that makes citation possible is made here, three stages before anyone asks for it.

Question phrasings to expect:

• "Which of the following should be stored as metadata rather than embedded in the chunk text?" — dates, ids, filenames, ACLs. Anything requiring exact comparison or enforcement.
• "A RAG system must answer only from documents published after a given date. What capability is required?" — metadata filtering in the vector store, not a better embedding model.
• "Why can a vector search not reliably restrict results to the most recent policy version?" — embeddings represent similarity, not recency or ordering. 07-03 is the full treatment.
• "What must be captured during ingestion to allow a RAG system to cite its sources?" — document identity and location: source file, page, section.
• "A team adds the document title and section heading to each chunk before embedding. What is the likely effect?" — improved matching for topic-scoped queries, with a risk of vector convergence if the prefix grows long or is shared by every chunk.
• "Where should per-user access control be enforced in a RAG pipeline?" — at retrieval, as a filter on indexed permission metadata; never as text in the embedded chunk.

Distractor families:

They answer different questions and they compose rather than compete.

The right composition is filter first, then rank. Narrow to what the caller is allowed to see and what is current and in scope, then let similarity choose among those. Doing it the other way — rank the world, then drop the ineligible — is the post-filter mistake, and it converts a governance requirement into a silently degraded result count.

Usually yes, and it is the single highest-return metadata decision available at ingestion, for a reason worth stating plainly: a section heading is the document author's own summary of the span, written in the vocabulary a reader is likely to search with. A body paragraph says "the limit rises to 90 days". The heading says "Study and examination leave". A user searching for study leave matches the heading, and the heading is free — it already exists, it was written by someone who understood the content, and it costs a handful of tokens.

Three cautions keep it from becoming Version B of the worked example. First, keep it short: heading, optionally the document title, and stop. Second, watch for shared-prefix convergence — if every chunk in a 4,000-page corpus is prefixed with the same corporate document title, that prefix contributes nothing distinguishing and a lot of dilution; prefer the most specific heading available over the most general. Third, keep the field copy: the embedded prefix is for matching and the field is for filtering and display, and you want both.

For deeply nested documents, a breadcrumb of the two or three nearest heading levels usually beats either the full path or the leaf alone. Employee Handbook — 7 Leave — 7.4 Study and examination leave carries useful hierarchy; a nine-level path carries mostly punctuation.

At minimum, four fields, and each one answers a question the reader will ask:

Two additions turn a citation into something auditable rather than merely decorative. Ingestion lineage — which pipeline run and which parser and chunker version produced this record — lets you answer "why does this chunk look wrong?" without re-deriving the corpus. And the retrieval trace — which chunks were returned, with what scores, under what filters — is what turns a bad answer into a diagnosable one, the discipline 07-10 is built on.

All of it has to be captured upstream. There is no query-time recovery of a page number the parser knew and the chunker dropped: that fact was available for a few milliseconds during ingestion and, once discarded, it is gone until you re-ingest. This is why provenance is an ingestion concern and not a presentation concern, and why it appears in this module rather than in the module about answering.

• Metadata has two jobs, and they use two different channels. Embedded text changes what a chunk matches. Stored fields decide what is eligible and what gets shown.
• Embed short descriptive context only — document title and section heading. The heading is the author's own summary in the reader's vocabulary and is the highest-return enrichment available.
• Never embed dates, ids, filenames, or permissions. They need exact comparison or enforcement, and an embedding provides neither.
• Never treat embedded text as access control. A chunk whose text names its ACL is readable by everyone and is more likely to surface for queries mentioning it.
• Pre-filter, do not post-filter. Post-filtering silently returns fewer results than requested and can return none while the answer sits in the index.
• Long shared prefixes converge vectors. Enrichment is additive to a fixed-length representation, so every prepended token dilutes the body's influence and pulls sibling chunks together.
• Fields are cheap to fix; vectors are not. Correcting embedded metadata means re-embedding, which is a strong argument for a minimal embedded string and a generous field set.
• Citation is an ingestion decision. Source, location, date, version, and a resolvable pointer must be captured at parse and chunk time, or the answer can never be verified — and NVIDIA's own framing makes citing sources part of what RAG is. [NVIDIA-DOC]

You now have chunks that carry their context and their provenance. What you do not yet have is a corpus where each distinct fact appears once. Real corpora are full of repetition you did not intend: the running footer the parser extracted from every page, the confidentiality banner on every slide, the standard clause pasted into 300 contracts, the same policy PDF sitting in four shared drives under four filenames, the quoted reply chain repeated down an email thread, and the near-duplicate chunks you manufactured yourself by setting a generous overlap in 06-02. Repetition is not merely wasted storage. A short string that appears on every page of a 4,000-page corpus becomes, in embedding space, a dense cluster sitting near the centre of everything — and a cluster near the centre of everything is close to every query.

Next: 06-04 Deduplication and corpus cleaning for RAG — why a repeated footer can become the nearest neighbour of every query, and how to find and remove it before it does.