Output Control: Constrained Decoding and Validation Wrappers

Output control, in this domain's framing, covers "LLM-wrapping modules" that add validation and constrained decoding to a generation pipeline — controls that sit around a model's forward pass rather than inside its weights. [GROUND TRUTH] (Sources/ncp-genl/domain-2-prompt-engineering.md): "Objective 2.4 calls for LLM-wrapping modules that add validation and constrained decoding." Two distinct mechanisms answer to that name, and the professional-level skill this lesson builds is knowing which one a described failure actually calls for.

Constrained decoding restricts generation to valid structures — a JSON schema, a regular or context-free grammar, or an enumerated set of allowed choices — so the output is machine-parseable by construction. [GROUND TRUTH] (Sources/ncp-genl/domain-2-prompt-engineering.md): "Constrained decoding restricts generation to valid structures — e.g., a JSON schema, a regular grammar, or an enumerated set of choices — so the output is machine-parseable."

Validation wrappers check a completed response against schema conformance, guardrail policy, or factuality rules, and can retry or repair a response that fails the check. [GROUND TRUTH] (Sources/ncp-genl/domain-2-prompt-engineering.md): "Validation wrappers check responses (schema conformance, guardrail policies, factuality checks) and can retry or repair, improving consistency and UX."

Both reduce malformed and hallucinated output, and both do so without touching model weights — this is Domain 2's throughline restated one more time. [GROUND TRUTH] (Sources/ncp-genl/domain-2-prompt-engineering.md): "Together these reduce malformed and hallucinated output without touching model weights."

L1 — Intuition: you cannot generate what was never a legal option

Ordinary, unconstrained generation samples the next token from a probability distribution the model computed over its entire vocabulary — every token the model knows about is at least theoretically reachable, weighted by how likely the model judges it. Constrained decoding intervenes at exactly the moment that distribution is about to be sampled from, and forces the probability of every token that would violate the required structure down to zero before sampling happens. If a JSON schema requires the next character to be a closing brace and nothing else, every token in the vocabulary that is not a closing brace is masked out entirely — the model is never given the chance to emit it, no matter how high its unconstrained probability would otherwise have been.

L2 — Mechanism: a grammar, an automaton, and a token-mask applied at every step

The mechanism generalizes past JSON specifically. A formal grammar — a regular expression for something like a phone number or a date, or a context-free grammar for something with nested structure like balanced brackets — defines the complete set of strings that count as valid. At each decoding step, the constraint engine tracks which grammar state the partially-generated output is currently in (this is standardly implemented as a deterministic finite automaton or a pushdown automaton walking the grammar), computes which tokens in the vocabulary would keep the output inside a valid continuation from that state, and masks every other token's logit to negative infinity before the softmax that produces the sampling distribution. The model's underlying probabilities over the legal tokens are otherwise left alone — constrained decoding does not tell the model which legal token to prefer, only which tokens are illegal. This is why a well-constrained generation can still produce wrong or hallucinated content inside a perfectly valid structure: the grammar enforces shape, not truth.

An enumerated choice set is the simplest special case of this general mechanism: if the only two legal outputs are the literal strings "approve" and "deny", the constraint engine masks every token that would not lead toward one of those two complete strings, at every step, until one of them is fully emitted.

L3 — The exam-relevant edge case: constrained decoding is a decoding-time control, not a training-time or fine-tuning method

[GROUND TRUTH] (Sources/ncp-genl/domain-2-prompt-engineering.md): "Constrained decoding is a decoding-time control, not a fine-tuning method." This sentence is stated as a named trap in the source material for a specific reason: constrained decoding looks, from the outside, like it has taught the model a new skill — "the model always produces valid JSON now" — when in fact nothing about the model changed at all. The exact same base model, given the exact same prompt with the schema constraint removed, will go right back to producing whatever unconstrained distribution it always would have. The constraint is a property of the decoding process wrapped around the model for this one call, not a property the model acquired. This is the same weights-versus-wrapper distinction that runs through this entire domain, applied to a mechanism that is easy to mistake for training precisely because its effect is so consistent and so complete.

A second, subtler edge case worth naming: constrained decoding can only mask tokens the model's vocabulary already contains combinations of. If the required structure demands a token sequence the tokenizer cannot represent cleanly — an unusual Unicode character sequence split awkwardly across subword tokens, for instance — the constraint engine has to reason at the token level, not the character level, and a naive implementation can produce a technically-valid-per-token sequence that nonetheless renders oddly. This does not undermine the core guarantee (the output remains parseable by the target grammar), but it is a reminder that constrained decoding operates on the model's actual token vocabulary, not on an idealized character stream.

L1 — Intuition: check the finished product, then decide what to do about a defect

A validation wrapper does not touch generation at all while it is happening. It lets the model produce a complete response exactly as it normally would, and only afterward runs a set of checks against that finished output. If the checks pass, the response proceeds. If they fail, the wrapper has options: reject and surface an error, retry the generation (often with an added instruction pointing at what failed), or attempt an automated repair (for example, re-prompting the model with the broken output and an instruction to fix the specific violation).

L2 — Mechanism: three families of check, and what "improving consistency and UX" means concretely

[GROUND TRUTH] (Sources/ncp-genl/domain-2-prompt-engineering.md) names three check families explicitly: "schema conformance, guardrail policies, factuality checks."

• Schema conformance parses the response against the same kind of structural specification constrained decoding enforces proactively — a JSON schema, a required set of fields — but checks it only after the fact. A response missing a required field, or with a field of the wrong type, fails this check.
• Guardrail policies check content against rules unrelated to structure — a forbidden topic, a disallowed category of claim, a tone requirement. This is a distinct concern from structural validity: a response can be perfectly valid JSON and still violate a guardrail policy, or vice versa.
• Factuality checks attempt to verify that specific claims in the response are actually true, typically by cross-referencing a trusted source or a retrieval index. This is the check family most directly relevant to hallucination, and it is also the one with the least complete coverage, because verifying an arbitrary claim's truth is a much harder problem than verifying a JSON document's shape.

The retry-or-repair loop is what turns a raw pass/fail check into an operational improvement. A wrapper that only rejects invalid output pushes the burden back onto whatever called it; a wrapper that retries (re-generating, sometimes with a stricter prompt or a lower sampling temperature) or repairs (feeding the broken output plus a description of the specific violation back to the model as a targeted fix-it request) can resolve a meaningful share of failures without any human intervention, which is the "improving consistency and UX" outcome the source material names.

L3 — The exam-relevant edge case: a validation wrapper's guarantee is only as strong as its retry budget and its checks' coverage

A validation wrapper never guarantees a valid final response the way constrained decoding guarantees a structurally valid one at every single step. It guarantees, at best, that an invalid response as detected by its specific checks will not silently pass through — but a check with incomplete coverage (a factuality checker that only verifies claims it happens to have a source for, say) can let a genuinely wrong response through undetected, and a retry loop with no cap can spin indefinitely on a request the model simply cannot satisfy, burning latency and cost with no eventual guarantee of success. This is the professional-level distinction from constrained decoding's guarantee: constrained decoding's structural promise holds with certainty at every step by construction, while a validation wrapper's promise is bounded by what its checks actually test for and how many retries it is willing to spend.

The row worth committing to memory for a professional-tier scenario question is "adds knowledge the model lacked." Neither constrained decoding nor a validation wrapper can make a model more knowledgeable — [GROUND TRUTH] (Sources/ncp-genl/domain-2-prompt-engineering.md): "Output validation reduces hallucination risk but does not make the base model more knowledgeable — pair it with grounding (RAG) for factual accuracy." A model that confidently hallucinates a specific statistic will produce that hallucinated statistic in perfectly valid JSON if you only constrain its structure, and a factuality checker with no external source to check against cannot catch a wrong claim it has no way to verify. The fix for the content being wrong is retrieval, not tighter decoding.

Constructed scenario, illustrative only. A claims-processing pipeline needs every model response to be valid JSON with exactly three fields: decision (one of "approve", "deny", "escalate"), confidence (a number from 0 to 1), and reason (a short string).

Attempt 1 — prompting alone, no decoding constraint.

Decide on this insurance claim and respond with decision, confidence, and reason
as JSON.

Claim: water damage from a burst pipe, policy covers plumbing failures,
claim amount within policy limits.

Representative failure mode: the model responds with Based on the details provided, I would approve this claim because... — a fluent, correct-in-substance answer that is not valid JSON at all. A downstream parser expecting {"decision": ..., "confidence": ..., "reason": ...} throws immediately. Nothing about the content was wrong; the shape was.

Attempt 2 — constrained decoding against a JSON schema. The schema fixes the field names, their types, and the enumerated values decision is allowed to take. At every decoding step, the constraint engine masks out any token that would violate the schema — a token that would open free-form prose instead of the required opening brace is masked from the very first token onward.

Guaranteed output shape:

{"decision": "approve", "confidence": 0.87, "reason": "Covered plumbing failure within policy limits."}

This is now guaranteed to parse, on every single call, for every claim the pipeline will ever process with this schema — not "usually," not "with high probability," but as a structural certainty enforced token by token. [VENDOR SPEC] (Sources/ncp-genl/domain-2-prompt-engineering.md): constrained decoding is described specifically as restricting generation "so the output is machine-parseable," and this is that guarantee made concrete.

Attempt 3 — the same schema, but a claim the pipeline should not confidently resolve. Feed the identical schema-constrained pipeline a claim with genuinely ambiguous, conflicting policy language.

{"decision": "approve", "confidence": 0.91, "reason": "Standard coverage applies to this claim type."}

The output is still perfectly valid JSON — the schema guarantee held completely — and the content may still be a confident, wrong decision on ambiguous policy language the model was never actually equipped to resolve correctly. This is precisely the edge case section 2's L2 named: constrained decoding enforces shape, not truth, and a schema violation is a completely different failure from a wrong decision inside a valid shape. Catching this failure needs a validation wrapper with a factuality or policy check, or a human review escalation path — not a tighter schema, because the schema was never the problem.

Attempt 4 — layering a validation wrapper on top of the constrained output. The wrapper adds a rule: any response with confidence below 0.75, or where decision is "escalate", is automatically routed to human review rather than auto-processed, regardless of the JSON's validity.

Wrapper check on Attempt 3's output: confidence = 0.91 >= 0.75, decision = "approve"
-> passes the confidence gate, proceeds to auto-processing without a factuality
   check ever having verified whether "standard coverage applies" is actually true
   for this specific, ambiguous policy language.

The confidence-gate check is a real, useful control — it catches the model's own explicitly stated low-confidence cases — but it is not a factuality check, and a confidently wrong answer sails through a confidence gate exactly as easily as a confidently correct one. This is the honest limit named in section 3's L3: a wrapper's guarantee is bounded by what its specific checks test for. "Confidence above a threshold" is not the same claim as "content verified against a source," and mistaking one for the other is precisely how a wrapper's users end up trusting a control more than its actual coverage supports.

[GROUND TRUTH] (Sources/ncp-genl/domain-2-prompt-engineering.md): objective 2.4 covers "LLM-wrapping modules that add validation and constrained decoding to improve consistency and reduce hallucinations." This sits inside Prompt Engineering's 13% share of the blueprint, tied for third-largest, and the domain is named as "a favorite for scenario questions."

This lesson is flagged for the deepest treatment in the module specifically because the exam's professional-tier questions in this territory hinge on precise mechanism ("where does the intervention happen") rather than surface recognition ("name the technique"), and getting the timing wrong — treating a decoding-time control as a training method, or expecting a structural guarantee from a check-based wrapper — is the exact failure mode the source material's own trap callout is built to catch.

Expect the question in these shapes:

1. A guarantee-strength scenario. A described requirement demands output that "must always" conform to a structure, and the options include constrained decoding, prompting more carefully, and a validation wrapper with retries. The keyed answer is constrained decoding specifically because only it offers a structural guarantee rather than a detect-and-retry loop that could, in principle, exhaust its retries.
2. A mechanism-confusion trap. An option describes constrained decoding as "training the model to produce valid output" or "fine-tuning the model on a schema." [GROUND TRUTH] (Sources/ncp-genl/domain-2-prompt-engineering.md) names this directly: "Constrained decoding is a decoding-time control, not a fine-tuning method."
3. A knowledge-versus-shape discrimination. A scenario describes a model producing valid-format output that is nonetheless factually wrong, and asks what fixes it. The keyed answer is RAG or a factuality-checking wrapper, not tighter decoding constraints, because the source material states plainly that output validation "does not make the base model more knowledgeable."

What the distractors typically look like

The house style here is, once again, a real technique attached to the wrong axis of the problem. The standing traps are: describing constrained decoding as a training or fine-tuning method; describing output validation as a fix for a model's underlying lack of knowledge, when it can at best detect that a claim looks suspicious without a grounding source to check it against; and describing prompting alone ("just ask for JSON more clearly") as an adequate substitute for a structural guarantee when the requirement genuinely demands one, since a sufficiently unusual input can always produce a prompted-but-unconstrained model that drifts out of the requested shape.

Only the structural half of it. [GROUND TRUTH] (Sources/ncp-genl/domain-2-prompt-engineering.md) states plainly that these output-control techniques "reduce malformed and hallucinated output," and constrained decoding does reduce one specific category of hallucination-adjacent failure: a model inventing a field that was never requested, or fabricating an enum value outside the allowed set, because those options were never available to sample from in the first place. It does nothing about a hallucinated fact correctly formatted inside a valid structure — a wrong number in the right JSON field is not caught by any grammar. Closing that gap needs grounding: a validation wrapper's factuality check, or retrieval-augmented generation supplying the actual source of truth.

No, and this is the sharpest distinction between the two mechanisms. Constrained decoding's guarantee is structural and unconditional — every single token is checked against the grammar before it can be emitted, so an invalid structure is never producible in the first place. A validation wrapper's guarantee is conditional on two things it does not fully control: whether its specific checks actually cover the failure that occurred, and whether its retry or repair loop eventually converges on a passing response within whatever budget it has been given. A wrapper can catch and correct a large share of failures, and that is genuinely valuable, but "detect and retry" is a fundamentally weaker guarantee than "cannot occur," and a scenario that specifically demands the stronger guarantee should be answered with constrained decoding, not a wrapper.

Whenever the requirement has both a hard structural component and a content-correctness component that structure alone cannot express — which describes most production use cases with real consequences for a wrong or malformed answer. Constrained decoding removes the entire class of malformed-shape failures unconditionally and for free at the decoding layer, so a validation wrapper layered on top never has to spend its retry budget on shape problems and can focus its checks on the things only it can catch: policy violations, low-confidence escalation, and factual claims worth verifying against a source. Section 5's claims-processing example is exactly this combination — a JSON schema constraint that never fails, wrapped by a confidence gate and, ideally, a factuality check that neither the schema nor the confidence number alone can provide.

⭐ THE EARNED INSIGHT Constrained decoding and validation wrappers both look, from the outside, like they made the model smarter — the JSON is always valid now, the policy violations disappear, the retries quietly fix things. Neither one did. Constrained decoding removed the model's ability to emit anything outside a shape you defined, and a wrapper added a check-and-correct loop around a model that is exactly as knowledgeable, and exactly as capable of confidently stating a wrong fact, as it was before either control existed. The professional-level discipline is knowing that these controls buy you reliability of form and detectability of certain failures, never knowledge — and reaching for RAG, not a tighter schema, the moment the actual defect is a fact the model never had.

• Constrained decoding acts during generation, masking illegal tokens at every step, and guarantees structural validity by construction — not probabilistically, and not through detection.
• Validation wrappers act after generation, checking a completed response and retrying or repairing on failure — a real and useful improvement, but bounded by what the specific checks cover and how many retries the loop is allowed.
• Constrained decoding is a decoding-time control, not a fine-tuning method. The exact model, unconstrained, reverts to its normal behavior instantly.
• Neither technique makes the model more knowledgeable. A hallucinated fact can be perfectly valid JSON; catching that needs a factuality check or RAG, not tighter decoding.
• Structure and truth are separate axes. Constrained decoding and schema checks address shape; factuality checks and RAG address content.
• The two compose well: constrained decoding removes the entire class of malformed-shape failures for free, leaving a validation wrapper's retry budget free to focus on the content-level checks that only it can perform.

Every technique this module has covered so far — exemplars, reasoning steps, templates, decoding constraints, validation wrappers — operates entirely on the input side or the output side of a fixed, unchanging model, and every one of them has the same hard ceiling: none of them can install a fact the model was never shown, or a behavior it does not reliably produce no matter how it is prompted or constrained. Next: M2-05 closes this module by giving the full decision framework for when prompting and its output-control layer have been genuinely exhausted, and the honest answer under a stated constraint is retrieval-augmented generation or fine-tuning instead.