Probability basics for LLMs: distributions, variance, and expectation

A probability distribution over a finite set of outcomes is an assignment of a number to each outcome, satisfying exactly two rules:

1. Every number is non-negative — no outcome gets negative belief.
2. The numbers sum to exactly 1 — all the belief is accounted for.

That is the whole definition. Anything satisfying both is a valid distribution; anything violating either is not, however plausible it looks.

When a language model processes text, its final layer emits one raw score per token in its vocabulary. These raw scores are called logits, and they are unbounded real numbers — they can be negative, they do not sum to anything in particular, and they are not probabilities. Softmax converts them:

softmax(z)ᵢ = exp(zᵢ) / Σⱼ exp(zⱼ)

Exponentiating makes every value positive; dividing by the sum makes them total 1. So the output is a genuine distribution over the vocabulary. If a model has a 50,000-token vocabulary, every single generation step produces a 50,000-element probability vector, and the model then either takes its largest element or samples from it.

The vocabulary of probability, with what each term means specifically in an LLM context:

That last row is worth pausing on. A language model computes a conditional distribution: the probability of each possible next token given the tokens so far. Everything else in this course — training objectives, decoding controls, evaluation metrics — is a statement about that conditional distribution or about how well it matches reality.

L1 — Intuition: shares of a fixed pie

Picture a fixed amount of belief — one whole pie — that must be divided among all the tokens in the vocabulary. The distribution is how the pie is cut. Two facts drop out immediately and both matter:

• Belief is zero-sum. Raising one token's probability necessarily lowers others'. This is why suppressing an unwanted token redistributes its share to everything else rather than making the model "less certain" in some absolute way.
• A confident model has a lopsided cut — one giant slice and a long tail of crumbs. An uncertain model has a flatter cut. "How lopsided" is exactly what perplexity later measures.

Expectation is the intuition of a weighted average. If you will win £10 with probability 0.2 and £0 with probability 0.8, your expected winnings are 0.2 × 10 + 0.8 × 0 = £2. You will never actually win £2 — that outcome does not exist — which is the first thing to internalise about expectation: it is a summary of the distribution, not a prediction of any single event.

Variance is the intuition of spread. Two distributions can share an expectation and behave completely differently: winning £2 with certainty and winning £1,000,000 with probability 0.000002 have the same expectation and nothing else in common. Variance is what distinguishes them, and it is the reason a single measured number tells you almost nothing without knowing how much it moves.

L2 — Mechanism: the formulas, and the temperature knob

Expectation of a discrete random variable, over outcomes x₁…xₙ with probabilities p₁…pₙ:

E[X] = Σᵢ pᵢ xᵢ

Recognise the shape: that is a dot product between the probability vector and the value vector, exactly the operation from M0.1. Expectation is a weighted sum, nothing more.

Variance, two equivalent forms:

Var(X) = E[(X − E[X])²]  = Σᵢ pᵢ (xᵢ − E[X])²
Var(X) = E[X²] − (E[X])²                          ← the computational shortcut

Standard deviation is σ = sqrt(Var(X)), and it is the form you should quote, because it is in the same units as the thing you measured. A variance in "squared percentage points" is not interpretable; a standard deviation in percentage points is.

Softmax with temperature. The generation control called temperature divides the logits before softmax:

softmax_T(z)ᵢ = exp(zᵢ / T) / Σⱼ exp(zⱼ / T)

The effect of T, which is one of the highest-value facts in this lesson because temperature is a Tier-1 exam topic:

The mechanism to remember in one sentence: temperature does not add randomness, it rescales the differences between logits. Low temperature exaggerates the model's existing preferences; high temperature suppresses them. Nothing new is invented at high temperature — the model's ranking is unchanged, only how sharply it is enforced.

Two further properties of softmax that explain real behaviour:

• It is shift-invariant. Adding the same constant to every logit changes nothing, because exp(z+c)/Σexp(z+c) = exp(z)/Σexp(z). Implementations exploit this by subtracting the maximum logit before exponentiating, to avoid numerical overflow. This is why you will see z - z.max() in real softmax code.
• It is never exactly zero. exp of any finite number is positive, so every token in the vocabulary always has some non-zero probability, however absurd. This is precisely why truncation strategies exist: top-k keeps only the k highest-probability tokens, and top-p (nucleus) keeps the smallest set whose probabilities sum to at least p. Both exist because softmax's long tail is real and occasionally gets sampled.

L3 — Sampling variance, and why small evaluation sets lie

This tier is the one that changes how you work, so it gets the most space.

Everything you measure about a model is an estimate from a sample. You run 20 examples and get 14 right; you report 70%. But 70% is not the model's accuracy — it is one draw from a distribution of possible outcomes, and if you had drawn a different 20 examples you would have got a different number.

The relevant machinery is the binomial setting: n independent trials, each succeeding with unknown probability p. For the count of successes:

E[successes]   = n p
Var(successes) = n p (1 − p)

For the proportion — which is what you report as accuracy — divide by n:

E[proportion]   = p
Var(proportion) = p (1 − p) / n
σ(proportion)   = sqrt( p (1 − p) / n )

That last line is the single most useful formula in this lesson, and everything about small-eval-set scepticism follows from it. Note what it says: the standard deviation of your measured accuracy shrinks as sqrt(n), which means to halve your uncertainty you need four times the data.

Work it for the sizes people actually use, taking p = 0.7 as an illustrative true accuracy:

These are computed from the formula, not measured from any real system, and the ±2σ interval is a rough normal approximation that is itself poor at small n and near the 0/1 boundaries. But the shape of the conclusion is robust and it is brutal: on a 20-item evaluation set, a true 70% model routinely measures anywhere from 50% to 90%. A prompt change that moves you from 14/20 to 16/20 has moved you well inside the noise. It is not evidence.

This does not make small eval sets useless, and it is important not to overcorrect. A 20-item hand-written set is excellent for finding categorical failures — the model ignores your format instruction, it hallucinates a citation, it refuses. Those are visible in one example, not twenty. What a small set cannot do is support a claim that configuration A is 3 points better than configuration B. Use small sets to find bugs; use large sets, or paired comparisons, to rank options.

One important refinement, because it rescues a lot of small-sample work: paired comparison has much lower variance than comparing two independent proportions. If you run both configurations on the same items and count only the items where they disagree, the shared difficulty of the items cancels out. Ten disagreements split 9–1 is much stronger evidence than two independent 70%-vs-75% measurements on twenty items each. This is the same logic that makes A/B tests on identical traffic more sensitive than before-and-after comparisons, and it recurs in the experimentation module.

Two other variance sources compound the sampling problem and are worth naming now:

• Decoding variance. At any temperature above 0, the same prompt gives different outputs on different runs. So even with a fixed eval set, re-running produces different scores. Fixing the seed reduces this but, as M0.2a noted, does not guarantee bitwise reproducibility on GPU.
• Judge variance. If a metric involves a model or a human judging outputs, the judge has its own variance and its own biases. An LLM-as-judge score is an estimate produced by an estimator that is itself unreliable.

The honest summary: any single number you measure about an LLM has error bars, usually wider than you would like, and the discipline of asking "how wide" before believing a difference is what separates real experimentation from confirmation bias.

Five terms that get used interchangeably and are not interchangeable. This table is high-value because several of them appear as exam distractors for each other.

Three distinctions worth being precise about:

Probability vs likelihood. These are the same arithmetic read in opposite directions. Probability fixes the model and asks how likely the data is. Likelihood fixes the data and asks how good the model is. Crucially, likelihood is not a probability distribution over models — it does not sum or integrate to 1 over parameters — which is why you can compare likelihoods between models but cannot read one as "the probability this model is correct".

Log probabilities exist for a reason. Multiplying 500 probabilities each around 0.01 gives a number around 10^-1000, which underflows to exactly zero in floating point and destroys the computation. Adding 500 log probabilities each around −4.6 gives about −2300, which is perfectly representable. Sequence scoring is therefore always done in log space, and this is why cross-entropy loss is defined with a logarithm rather than as a product — the log is not decoration, it is what makes the arithmetic possible.

Perplexity is an exponentiated average log loss. If the average negative log probability per token is H, perplexity is exp(H). Its interpretation is "the effective number of equally likely choices the model felt it had at each step". Perplexity 1 means perfect certainty and correctness; perplexity equal to vocabulary size means the model was as good as uniform guessing. Lower is better. The trap, which 09-02 covers properly, is that perplexity is measured in tokens, so two models with different tokenizers do not have comparable perplexities, and perplexity says nothing at all about whether an answer is true or useful.

One more confusable pair, on the decoding side, because it belongs with the distribution discussion:

The distinction that gets tested: top-k keeps a fixed count, top-p keeps a variable count chosen by cumulative probability. When the model is confident, top-p may keep only one or two tokens; when it is uncertain, it may keep dozens. Top-k keeps k regardless, which is either too permissive on confident steps or too restrictive on uncertain ones. All of this is 04-05 and the decoding-parameters material; what you need from this lesson is that every one of these strategies is an operation on a probability vector you now know how to read.

Five vocabulary tokens, so the arithmetic fits on a page. Suppose the model's final layer emits these logits for the next token:

token   :   "the"   "a"   "cat"   "quantum"   "zzz"
logit z :    3.0    2.0    1.0      0.0        -1.0

Step 1 — softmax at T = 1. Exponentiate each logit, then divide by the total.

exp(3.0)  = 20.086
exp(2.0)  =  7.389
exp(1.0)  =  2.718
exp(0.0)  =  1.000
exp(-1.0) =  0.368
                        sum = 31.561

p("the")     = 20.086 / 31.561 = 0.636
p("a")       =  7.389 / 31.561 = 0.234
p("cat")     =  2.718 / 31.561 = 0.086
p("quantum") =  1.000 / 31.561 = 0.032
p("zzz")     =  0.368 / 31.561 = 0.012
                                 -----
                          total = 1.000  ✓

Check both rules: all non-negative, sums to 1. Note that "zzz" — the absurd token — still has probability 0.012, roughly one chance in eighty. Over a thousand generation steps you would expect it about a dozen times. This is why truncation strategies exist, and it is the concrete answer to "why did the model emit that".

Step 2 — the same logits at T = 0.5 (sharper). Divide logits by 0.5, i.e. double them, then softmax.

z/T     :  6.0     4.0     2.0     0.0     -2.0
exp     : 403.4   54.60    7.389   1.000    0.135     sum = 466.5

p("the")     = 403.4 / 466.5 = 0.865
p("a")       = 54.60 / 466.5 = 0.117
p("cat")     =  7.389/ 466.5 = 0.016
p("quantum") =  1.000/ 466.5 = 0.0021
p("zzz")     =  0.135/ 466.5 = 0.0003

"the" has gone from 0.636 to 0.865. "zzz" has gone from 1-in-80 to roughly 1-in-3,500. The ranking is identical — this is the key observation — only the sharpness changed.

Step 3 — the same logits at T = 2.0 (flatter). Divide logits by 2.

z/T     :  1.5     1.0     0.5     0.0     -0.5
exp     : 4.482   2.718   1.649   1.000   0.607      sum = 10.456

p("the")     = 4.482 / 10.456 = 0.429
p("a")       = 2.718 / 10.456 = 0.260
p("cat")     = 1.649 / 10.456 = 0.158
p("quantum") = 1.000 / 10.456 = 0.096
p("zzz")     = 0.607 / 10.456 = 0.058

Now "zzz" is at 0.058 — about one step in seventeen. Raising temperature to 2.0 made a nonsense token roughly five times more likely than at T = 1 and nearly 200 times more likely than at T = 0.5. The whole "high temperature makes models creative and also makes them talk nonsense" observation is visible right there in three numbers.

Side by side, which is the table to remember:

Step 4 — top-k and top-p on the T = 1 distribution. Sorted descending: 0.636, 0.234, 0.086, 0.032, 0.012.

top-k with k=2 : keep {"the", "a"} → renormalise by 0.636+0.234 = 0.870
                 p("the") = 0.636/0.870 = 0.731   p("a") = 0.234/0.870 = 0.269

top-p with p=0.9: cumulative 0.636 → 0.870 → 0.956
                 0.636 < 0.9; 0.870 < 0.9; 0.956 ≥ 0.9 → keep the first three
                 renormalise by 0.956: 0.665, 0.245, 0.090

Note the two gave different sets from the same distribution: k=2 kept two, p=0.9 kept three. On a more confident distribution — the T=0.5 one — p=0.9 would have kept only {"the", "a"} (cumulative 0.865 then 0.982), because the nucleus adapts to the model's confidence and a fixed k does not. That adaptivity is the entire argument for top-p over top-k.

Step 5 — expectation and variance on a concrete quantity. Suppose each of these five tokens costs a different amount of downstream processing, measured in arbitrary units:

token    :  "the"  "a"  "cat"  "quantum"  "zzz"
p (T=1)  : 0.636  0.234  0.086   0.032     0.012
cost x   :   1      1      2       5        10

Expectation — the probability-weighted sum, i.e. a dot product:

E[X] = (0.636×1) + (0.234×1) + (0.086×2) + (0.032×5) + (0.012×10)
     =   0.636   +   0.234   +   0.172   +   0.160   +   0.120
     =   1.322

Variance via the shortcut E[X²] − (E[X])²:

E[X²] = (0.636×1) + (0.234×1) + (0.086×4) + (0.032×25) + (0.012×100)
      =   0.636   +   0.234   +   0.344   +   0.800    +   1.200
      =   3.214

Var(X) = 3.214 − (1.322)² = 3.214 − 1.748 = 1.466
σ      = sqrt(1.466) = 1.211

So the expected cost is 1.32 units with a standard deviation of 1.21 units — a spread almost as large as the mean itself. That is a high-variance quantity: the average is dominated by cheap tokens, but the occasional expensive one moves it a long way. Reporting "average cost 1.32" without the spread would be actively misleading, and this is the general lesson about reporting a mean with no dispersion measure attached.

Step 6 — the same arithmetic at T = 2, to show that a decoding knob moves your cost distribution. Using the flatter probabilities:

E[X] = (0.429×1)+(0.260×1)+(0.158×2)+(0.096×5)+(0.058×10)
     = 0.429 + 0.260 + 0.316 + 0.480 + 0.580 = 2.065

Expected cost has risen from 1.32 to 2.07 — a 56% increase — purely from raising temperature, because flattening the distribution shifts mass onto the rare expensive tokens. This is a constructed example with invented cost units, not a measured result. But the mechanism is real and general: decoding parameters change the distribution, and anything you compute as an expectation over that distribution changes with it — cost, latency, error rate, output length. This is the kind of second-order consequence that makes "just raise temperature for more creative output" a decision with a bill attached.

The row that matters most in practice is the fifth one, because it is the mistake almost everyone makes at least once: seeing a two-example improvement on a twenty-example set and shipping a prompt change on the strength of it. The formula in section 2's L3 tier is the antidote, and it takes ten seconds to apply.

This lesson is load-bearing for two of the five blueprint domains and for one of the exam's most-reported topic clusters.

Core ML and AI (30%). Generation and decoding parameters are a reported Tier-1 topic: temperature, top-k, top-p, greedy, beam search, repetition penalty, and the effect of each on determinism versus diversity. Every one of those is an operation on a softmax distribution. Candidates who learn them as a list of knob names get the easy questions and miss the ones that ask what happens to both diversity and error rate when temperature rises. Candidates who understand section 4's three-column table answer both. Softmax also appears in this domain's activation-function material, where it is the output-layer function for multi-class problems — a Tier-2 topic.

Experimentation (22%). This domain's official scope is performing, evaluating and interpreting experiments. Its content includes statistical versus practical significance, sample size, p-hacking and peeking, guardrail metrics, and reading a result correctly. All of that is variance reasoning. A question describing an A/B test on a small sample and asking what conclusion is warranted is a variance question in disguise, and the answer is usually "not the one the scenario invites".

Loss functions and metrics. Cross-entropy is defined as the negative log probability of the correct token, averaged — which is an expectation over a distribution. Perplexity is exp of that average. R² as the proportion of explained variance, named verbatim in objective 2.2 / 3.2, is a ratio of variances. You do not need to derive any of these, but the vocabulary in this lesson is what makes them one-line facts rather than memorised strings.

Where it feeds forward:

The calibration note applies here as strongly as anywhere: candidate reports describe the exam as general-level, favouring "know at a high level what each thing is and when to use it" [FIELD]. So learn what softmax does, what temperature does to a distribution, and why small samples mislead. Do not learn to derive the variance of a beta-binomial. The arithmetic in section 4 is exactly the depth ceiling.

Two of these are worth a further sentence because they are the ones that survive into professional work.

Peeking is the most respectable-looking of the list. Running an experiment, checking after 30 examples, seeing a promising gap and stopping is a completely natural thing to do and it systematically produces results that do not replicate — because you gave yourself many chances to observe a fluctuation and stopped at the most favourable one. Deciding n in advance costs nothing and removes the problem entirely.

Reading a mean with no spread is the mistake that most often reaches a decision-maker. "Average latency 400 ms" and "average latency 400 ms, 95th percentile 4 s" describe wildly different systems, and only the second one is honest about what a user will experience. In LLM work, where output length and cost are both high-variance, quoting a mean alone is close to meaningless — which connects directly to the next lesson's insistence that a number is not a fact until its assumptions are stated.

It is the model's output at every single generation step: one non-negative number per token in the vocabulary, summing to exactly 1, expressing how likely each token is to come next given everything before it. The raw final-layer outputs are logits — unbounded real numbers that are not probabilities — and softmax converts them into a distribution by exponentiating and normalising. So for a 50,000-token vocabulary, each step produces a 50,000-element probability vector, and the decoding strategy decides whether to take its largest element or sample from it.

Temperature divides the logits before softmax, which rescales the differences between them. Dividing by a number below 1 exaggerates the differences, sharpening the distribution so the top token dominates; dividing by a number above 1 shrinks the differences, flattening it toward uniform. In the worked example, "zzz" — an absurd token — carries probability 0.0003 at T = 0.5, 0.012 at T = 1.0, and 0.058 at T = 2.0. Crucially, the ranking never changes: temperature does not invent new preferences, it only decides how strictly the existing ones are enforced. At T → 0 it becomes deterministic greedy decoding.

Because the standard deviation of a measured proportion is sqrt(p(1−p)/n), and at n = 20 with a true accuracy around 70% that is about 10 percentage points. A rough 95% interval is therefore ±20 points, so a genuinely 70%-accurate system routinely measures between 50% and 90% purely by which 20 examples you happened to pick. A change from 14/20 to 16/20 is well inside that noise. Uncertainty shrinks only as sqrt(n), so halving it takes four times the data. Small sets remain excellent for spotting categorical failures — a broken output format, a hallucinated citation — because those show up in one example, not twenty.

Expectation is the probability-weighted average of a quantity — E[X] = Σ pᵢ xᵢ, which is a dot product between the probability vector and the value vector. Variance is the expected squared distance from that average — Var(X) = E[X²] − (E[X])² — and its square root, the standard deviation, is the readable version because it is in the original units. Expectation tells you the centre; variance tells you how far things routinely stray from it. Two distributions can share an expectation and behave nothing alike, which is why a mean quoted without a spread hides exactly the information a decision needs.

Top-k keeps a fixed number of the highest-probability tokens, renormalises over them, and samples. Top-p — nucleus sampling — keeps the smallest set of tokens whose probabilities cumulatively reach at least p, renormalises, and samples. The number kept by top-p therefore adapts to the model's confidence: on a confident step it may keep one or two tokens, on an uncertain step dozens. Top-k keeps k regardless, so the same k is too permissive when the model is sure and too restrictive when it is not. In the worked example, the same distribution gave two tokens under k=2 and three under p=0.9.

Because multiplying many small probabilities underflows. Five hundred tokens each around probability 0.01 multiply to roughly 10^-1000, which is far below what a double-precision float can represent and becomes exactly zero — destroying the computation. Adding five hundred log probabilities each around −4.6 gives about −2300, which is perfectly representable. This is why sequence scoring is always done in log space, and it is why cross-entropy loss is defined with a logarithm: the log is not a stylistic choice, it is what makes the arithmetic possible at all.

1. A distribution is two rules: non-negative, sums to 1. Check both on any vector claiming to be one.
2. A language model's output is a conditional distribution over the whole vocabulary, produced by softmax from unnormalised logits. Every decoding control is an operation on that vector.
3. Temperature rescales logit differences; it does not add randomness. T→0 is deterministic argmax, T>1 flattens toward uniform, and the ranking never changes.
4. Softmax is never exactly zero, so absurd tokens are always reachable. That is why top-k and top-p exist, and why lowering temperature shrinks but never removes the tail.
5. Top-k is a fixed count; top-p is a cumulative probability budget that adapts to the model's confidence. This is a commonly tested distinction.
6. Expectation is a dot product between probabilities and values — the same operation as M0.1, in different clothing.
7. Variance is what a mean hides. Quote a standard deviation, or a percentile, or at minimum n. A bare mean on a high-variance quantity is close to meaningless.
8. The standard error of a measured accuracy is sqrt(p(1−p)/n). At n=20 that is about 10 points, so a ±20-point swing is ordinary. Two examples out of twenty is not evidence.
9. Uncertainty shrinks as sqrt(n) — four times the data to halve the error bar. Paired comparison on identical items is the cheap alternative.
10. Small eval sets find bugs, not winners. Use twenty hand-written items to catch categorical failures; use hundreds, or pairing, to rank close options.
11. Work in log space for sequences. Multiplying hundreds of small probabilities underflows to zero; the log in cross-entropy is a necessity, not a convention.
12. Perplexity is exp of average negative log probability, is per-token, and is therefore not comparable across tokenizers — and says nothing about truth.
13. Decide n before you look. Peeking until the numbers look good is the most respectable-looking way to produce a result that will not replicate.

You can now read a probability, compute an expectation, and say how much a measurement is allowed to move before it means something. What you cannot yet do reliably is the other half of quantitative honesty: getting the units right. The gap between GB and GiB is about 7% and grows with scale; a "24 GB" card and a "24 GiB" allocation are not the same thing; and an estimate that is right to one significant figure with its assumption written down beats one carried to three decimals with its assumption hidden. Those errors are unglamorous and they are exactly what turns a correct memory calculation into an out-of-memory error.

Next: M0.4 Units and order-of-magnitude estimation (GB vs GiB) — the two byte conventions and where each is used, the powers of two worth knowing cold, how to estimate a memory footprint or a token bill in one line, and why one significant figure with a stated assumption is the professional answer.