GB vs GiB: units and order-of-magnitude estimation for LLM work

There are two families of size prefixes for bytes, and they disagree.

The historical reason for the mess is that computer memory is naturally addressed in powers of two, so 1,024 was close enough to "kilo" that the industry borrowed the prefix. The IEC introduced the -bi- prefixes (kibi, mebi, gibi, tebi) to disambiguate, and adoption is partial: standards-conscious software uses them, marketing does not, and a great deal of software says "GB" while computing in GiB.

Who uses which, in practice:

The one to watch hardest is the last row. Network and interconnect figures are conventionally in bits per second, and storage and memory in bytes. A quoted "10 Gbps" link moves at most about 1.25 GB/s, and confusing bits with bytes is an 8× error — larger than any prefix mismatch and far more common than it should be.

Why this is not pedantry. Three concrete situations where the gap decides the outcome:

1. A memory estimate near the boundary. You calculate that a model needs 23.5 "GB" and the card is advertised at 24 GB. If your 23.5 was in GiB and the card's 24 was decimal GB, the card actually holds about 22.35 GiB, and your model does not fit. You will discover this as an OOM, not as an arithmetic error.
2. A capacity plan across many machines. A 7.4% error compounds nowhere but is quoted everywhere; at the tebi scale it is 10%, and a 10% error in a storage plan is a real budget line.
3. A comparison between two reported numbers. Comparing a framework's bytes / 2**30 figure with a vendor's decimal GB figure produces a discrepancy that looks like a bug in your code. Many hours have been lost to the search for that bug.

The professional habit is simple and it is the actual takeaway of this section: state your unit, always, and prefer the explicit binary prefixes when you mean binary. Writing "≈14 GiB (bf16 weights, 7e9 params × 2 bytes / 2³⁰)" is unambiguous and self-checking. Writing "14 GB" is neither.

L1 — Intuition: the answer you can defend beats the answer with more digits

An order-of-magnitude estimate is a calculation deliberately carried to one significant figure, using round numbers, with every assumption named. Its purpose is not to be precise. Its purpose is to be checkable — by you, thirty seconds later, and by someone else who disagrees with one of your assumptions and can now say which one.

Consider two answers to "will this fit in 24 GB of VRAM?"

Answer A: "Estimated footprint 21.847 GB."

Answer B: "Roughly 20 GiB, assuming bf16 weights, Adam state in fp32, batch 4 at 512 tokens, and no activation checkpointing. Weights dominate. If Adam state is kept in bf16 instead, it drops to roughly 13 GiB."

Answer A is worse in every way that matters. Nobody can check it. Nobody knows which precision it assumed, whether it included optimizer state, or whether the unit is decimal or binary. Its three decimal places imply a precision the underlying assumptions cannot possibly support — the activation term alone varies by more than that with a batch-size change. And when it turns out to be wrong, there is no way to find out why.

Answer B is one significant figure and it is the useful answer. It names the dominant term, so you know which assumption to attack. It gives a sensitivity — what changes if one assumption changes. It states its unit. It is, crucially, falsifiable in parts.

This is the design note this lesson exists to deliver: one significant figure with the assumption stated beats three significant figures with the assumption hidden. False precision is not a harmless stylistic flaw; it actively conceals the information a reader needs in order to disagree with you productively.

L2 — Mechanism: the numbers to know cold, and the one-line method

Powers of two. These are worth genuinely memorising, because they appear as vocabulary sizes, context lengths, batch sizes, hidden dimensions and memory quantities everywhere in this field.

The one approximation that does most of the work: 2¹⁰ ≈ 10³, with an error of 2.4% per application. So 2²⁰ ≈ 10⁶ (4.9% high), 2³⁰ ≈ 10⁹ (7.4% high), 2⁴⁰ ≈ 10¹² (10% high). Memorise the errors rather than the exact values and you can convert in your head: binary is a few percent bigger, and the few percent compounds by roughly 2.4 points per prefix.

Bytes per number, by precision — from M0.2, restated because it is the multiplier in every estimate:

The one-line estimation method, which is the actual transferable skill:

1. Write the formula in words.          "params × bytes-per-param"
2. Round every input to one figure.     7B params, 2 bytes
3. Multiply the powers of ten.          7e9 × 2 = 1.4e10 bytes
4. Convert to the unit you want, and say which.
                                        1.4e10 / 2**30 ≈ 13 GiB
                                        1.4e10 / 1e9   ≈ 14 GB
5. Name the dominant term and the biggest assumption.
                                        "weights dominate; assumes bf16"

Step 4 is where the GB/GiB decision becomes explicit rather than accidental, and step 5 is what makes the estimate worth writing down.

L3 — Which errors actually matter, and the sanity-check reflex

Not all estimation errors are equal, and knowing the hierarchy tells you where to spend care.

Read that table as a priority list. Rounding to one significant figure is the smallest error on it — smaller than the GB/GiB gap, and two orders of magnitude smaller than a bits/bytes slip. This is the quantitative justification for order-of-magnitude estimation: the precision you give up by rounding is dwarfed by the precision you never had in the first place, because you did not know the exact batch size or whether checkpointing was on.

It follows that effort is better spent on structure — did I include all four memory consumers? is this figure in bits or bytes? total or trainable parameters? — than on carrying decimals.

The sanity-check reflex. Four questions, applied to any quantitative claim, including your own, in under thirty seconds:

1. What are the units, exactly? Bits or bytes? Decimal or binary prefix? Per token, per word, or per request? Per second or per hour?
2. Is the magnitude plausible? Compare against an anchor you trust. A number three orders of magnitude from your anchor is almost certainly a units error rather than a surprising discovery.
3. What dominates? In any sum, one or two terms usually carry most of the total. If you cannot name the dominant term, you do not understand the estimate.
4. What would change it most? The assumption with the largest leverage is the one to state and to test. If the answer flips when a plausible assumption changes, say so.

A worked instance of question 2, which is the habit worth building: someone tells you a model needs "500 MB" to fine-tune. Anchor: a 7B model's weights in bf16 are around 14 GB. 500 MB is nearly thirty times smaller than the weights alone. Either they mean an adapter, or a much smaller model, or they have made an error. You have identified the ambiguity in five seconds without any calculation, purely by holding one anchor.

Anchors worth holding, all approximate and all constructed for scale rather than measured:

Those anchors are approximations for scale-checking, not measurements. The tokens-per-word figure especially is a rule of thumb that varies substantially with tokenizer, language, domain vocabulary and formatting — code and technical text tokenize very differently from prose. Use it to sanity-check, then count actual tokens with the actual tokenizer when it matters, which is exactly what 01-06's token-counting material is for.

Three of these deserve expansion because they cause specific, recognisable failures.

Advertised versus usable VRAM. A GPU's stated capacity is what the hardware contains, not what your process may allocate. The driver, the CUDA context, and framework workspaces all take a portion before your first tensor exists. Combined with the GB/GiB issue, this means a nominally "24 GB" card may present something meaningfully under 23 GiB to your framework, and budgeting to the advertised number is how you produce an OOM at a size that "should" have fitted — precisely the failure M0.2 had you provoke on purpose.

Tokens versus words. Every cost, context-window and chunk-size calculation in LLM work is denominated in tokens, and every human-scale intuition is denominated in words. The conversion is not fixed: it depends on the tokenizer, the language, and the text. Rare words, code, URLs, numbers and non-English scripts all tokenize into more pieces per word than plain English prose. So "roughly 1.3 tokens per word" is an estimation anchor and nothing more; when a context window or an invoice is at stake, run the tokenizer.

Parameters versus FLOPs. Parameter count tells you memory. Floating-point operation count tells you compute. They are related but not interchangeable, and the exam-relevant intuition is just that: a model's size constrains what fits, while its compute cost constrains how fast it runs. Scaling-law discussions involve both, and conflating them produces nonsense in both directions.

Everything here is constructed illustrative arithmetic, not a measurement. The point is the method and the unit discipline.

The question: will a 7-billion-parameter model fine-tune on a card advertised as 24 GB?

Step 1 — write the formula in words. Full fine-tuning needs weights, gradients, optimizer state, and activations. Four consumers, from M0.2.

Step 2 — state assumptions explicitly, before any arithmetic.

params           = 7e9        (total, all trainable — this is the key assumption)
weight precision = bf16       → 2 bytes per parameter
gradient prec.   = bf16       → 2 bytes per parameter
optimizer        = AdamW, state in fp32 → 2 values × 4 bytes = 8 bytes per parameter
activations      = not estimated; noted as a separate term

Step 3 — multiply, one significant figure, in bytes.

weights   : 7e9 × 2 = 1.4e10 bytes
gradients : 7e9 × 2 = 1.4e10 bytes
optimizer : 7e9 × 8 = 5.6e10 bytes
                      --------
subtotal            = 8.4e10 bytes

Step 4 — convert, in both conventions, and label them.

decimal : 8.4e10 / 1e9   = 84 GB
binary  : 8.4e10 / 2**30 = 78 GiB

Note that the same footprint is "84 GB" or "78 GiB" depending purely on convention — a 7.4% difference in the number with no change in the physical quantity. If you had computed 78 and compared it against a card advertised as "80 GB", the answer would depend entirely on which unit each figure was in.

Step 5 — the card's side of the comparison, in the same unit.

advertised          : 24 GB (decimal, per marketing convention)
in binary           : 24e9 / 2**30 = 22.4 GiB
minus driver/context/workspace headroom (assume ~10%)
usable, roughly     : ~20 GiB

Step 6 — compare, in one unit, and answer.

needed  ≈ 78 GiB   (before activations)
usable  ≈ 20 GiB
verdict : no, by roughly 4×. Not a margin problem — a structural one.

Step 7 — name the dominant term and the highest-leverage assumption. Optimizer state is 5.6e10 of the 8.4e10 total — two thirds of the bill. So the highest-leverage assumption is "all 7e9 parameters are trainable with Adam", because both the gradient and optimizer terms scale with trainable parameters, not total ones.

Step 8 — re-estimate with that assumption changed. Freeze the base and train an adapter of 0.1% of the parameters — 7e6 trainable. Quantize the frozen base to 4 bits.

frozen base, 4-bit : 7e9 × 0.5 = 3.5e9 bytes  → 3.5 GB  /  3.3 GiB
gradients          : 7e6 × 2   = 1.4e7 bytes  → 0.014 GB / 0.013 GiB
optimizer, fp32    : 7e6 × 8   = 5.6e7 bytes  → 0.056 GB / 0.052 GiB
                                 ---------
subtotal                       ≈ 3.6e9 bytes → 3.6 GB  /  3.4 GiB

needed  ≈ 3.4 GiB + activations + framework overhead
usable  ≈ 20 GiB
verdict : yes, with substantial room for activations.

The answer, written the way it should be written:

Full fine-tuning a 7B model needs roughly 80 GiB before activations — assuming all parameters trainable, bf16 weights and gradients, and AdamW state in fp32. That is about 4× a card advertised at 24 GB (≈20 GiB usable after headroom), so it does not fit and no batch-size reduction closes the gap. Optimizer state is two thirds of the total, so freezing the base and training a ~0.1% adapter over a 4-bit base brings it to roughly 3.4 GiB plus activations, which does fit comfortably. Figures are constructed one-significant-figure estimates for scale, not measurements; confirm against nvidia-smi in the actual environment.

Every claim in that paragraph is checkable. Every assumption is named. Every unit is labelled. The numbers are all one or two significant figures, and that is the point rather than a limitation — you can see from it exactly which assumption to challenge, and challenging it changes the verdict, which is precisely what an estimate is for.

A second estimate in a different currency, because tokens are the other unit LLM work is denominated in.

The question: you have 2,000 internal documents averaging 8 pages each, and you want to know roughly what it costs to embed them for a retrieval index — and whether they fit within a context-window budget if you tried to stuff them into a prompt instead.

Step 1 — assumptions, stated.

documents        = 2,000
pages per doc    = 8            (average)
words per page   = 500          (typographic convention, an anchor not a measurement)
tokens per word  = 1.3          (rule of thumb for English prose; varies by tokenizer)

Step 2 — the estimate, one significant figure at each stage.

words per doc    = 8 × 500        = 4,000
tokens per doc   = 4,000 × 1.3    = 5,200   → call it 5e3
total tokens     = 2,000 × 5e3    = 1e7     → 10 million tokens

Step 3 — convert into the units each decision uses.

Context-window question:
  10 million tokens vs a context window of, say, 2**17 = 131,072 tokens
  ratio = 1e7 / 1.3e5 ≈ 77×
  verdict: not close. Stuffing the corpus into a prompt is off by roughly two
           orders of magnitude, which is why retrieval exists.

Index-size question (embedding storage):
  suppose 512-token chunks with no overlap → 1e7 / 512 ≈ 2e4 chunks
  suppose 768-dimensional embeddings in fp32 → 768 × 4 = 3,072 bytes each
  raw vector storage = 2e4 × 3.07e3 ≈ 6e7 bytes ≈ 60 MB / 59 MiB
  verdict: trivial. Vector storage for a corpus this size is not a constraint.
           Index structures and metadata add to this, but not by orders of magnitude.

Cost question:
  cost scales linearly with tokens, so the whole answer is
  "10 million tokens, once, at whatever the per-token embedding rate is".
  Deliberately not multiplied out: published rates change, and inventing one
  would be a fabricated figure. The estimate that matters is the token count.

Three things this example demonstrates that the memory example did not:

• An order-of-magnitude answer can settle a question decisively. "Off by 77×" needs no refinement. Nobody needs the second significant figure to know that stuffing 2,000 documents into a prompt is not the plan.
• An order-of-magnitude answer can also declare a non-problem. 60 MB of vectors is not worth another minute of anyone's attention. Knowing which questions to stop working on is half the value of estimating.
• Leave the unknown as a variable rather than inventing it. The token count is estimable; the price per token is a published figure that changes. Multiplying by a made-up rate would produce a number that looks authoritative and is fiction. Saying "10 million tokens at the current rate" is both more honest and more durable.

When is precision worth buying? A decision table:

The pattern: estimate to decide what to try; measure to report what happened. An estimate that lands near a hard limit is not a conclusion, it is a warning that you must measure — the 30% band around a VRAM ceiling is exactly where assumptions you did not question decide the outcome.

The blueprint does not have a "units" objective, so it is worth being precise about where this pays off. It pays off in three places, two examinable and one that determines how efficiently you study.

First, hardware and capacity questions. Official objective 4.4 asks you to identify system data, hardware, or software components required to meet user needs, and the Software Development domain — 24% of the blueprint — includes latency, throughput and capacity planning, memory math for weights plus KV cache, and quantization trade-offs. A scenario question that describes a model and asks which deployment target is adequate is an estimation question. The distractors in such questions are frequently off by a factor that a one-line estimate catches instantly: a fp32-versus-bf16 confusion is a 2× error, a weights-only-versus-full-training confusion is a 6–8× error, and both are exactly the kind of plausible-looking wrong answer a multiple-choice item is built from.

Second, token budgeting. Tokenization is a reported Tier-1 topic, and the specific skill named in the course index is token counting and context-window budgeting. That is a units skill: tokens are not words, the ratio is not fixed, and every context-window, chunk-size and cost decision is denominated in tokens. 01-06 and the chunking material both assume you can estimate in tokens rather than words.

Third, exam pacing. The exam is 50–60 questions in 60 minutes, roughly 60–70 seconds per question [OFFICIAL for the format; the per-question pacing is arithmetic from it]. You do not have time to do careful arithmetic. What you have time for is exactly what this lesson trains: round to one figure, spot the dominant term, notice a units mismatch, and eliminate the options that are off by 2× or 8×. Estimation is a test-taking skill here as much as an engineering one.

Where it feeds forward:

That last row is the real reason this lesson closes the prerequisites module. Everything after it involves quoting numbers — accuracy, latency, memory, cost, similarity scores — and the discipline of naming your unit, rounding honestly, and stating what would change the answer is the same discipline that keeps an evaluation honest. False precision in a memory estimate and an unreplicable 2-point improvement on a 20-item eval set are the same error wearing different clothes.

The two that separate a careful engineer from a careless one are the last two rows, and they pull in opposite directions in a way worth stating plainly. Inventing a figure is the failure of overreach — filling a gap in your evidence with a number that feels right, which produces confident nonsense. Estimating when you should measure is the failure of stopping early — treating a back-of-envelope figure as a result. The discipline that avoids both is the same: be explicit about what kind of number you are holding. An estimate labelled as an estimate, with its assumptions attached, is a legitimate and useful artefact. The same number with the label stripped off is a liability.

A gigabyte (GB) is 10⁹ = 1,000,000,000 bytes, the decimal SI convention. A gibibyte (GiB) is 2³⁰ = 1,073,741,824 bytes, the binary IEC convention. The GiB is about 7.4% larger, and the gap grows with each prefix — about 4.9% at mega, 7.4% at giga, 10% at tera, 12.6% at peta. The practical problem is that the two are used by different parties without announcement: hardware and storage vendors quote decimal GB, while operating systems, nvidia-smi and deep-learning frameworks generally report binary units. Comparing a figure from one convention against a figure from the other produces a discrepancy that looks like a bug and is actually a units error.

About 22.4 GiB, because 24 × 10⁹ / 2³⁰ = 22.35. And that is still not the amount your process can allocate: the driver, the CUDA context and framework workspaces take a portion first, so plan on meaningfully less — leaving something like 10% headroom is a reasonable starting assumption, confirmed against nvidia-smi in your actual environment. This is why a memory estimate that lands at 23 "GB" against a "24 GB" card is not a comfortable fit but a coin flip, and why the professional habit is to state the unit and leave real headroom rather than budgeting to the advertised figure.

Because the precision you appear to gain is fictional and the honesty you lose is real. A memory footprint depends on precision, whether gradients and optimizer state are included, batch size, sequence length, whether activation checkpointing is on, and framework overhead — inputs that individually vary the answer by tens of percent. Three decimals on top of that implies a certainty the inputs cannot support. Worse, a precise-looking figure with no stated assumptions cannot be checked or argued with: nobody can tell you which input you got wrong. One significant figure with the assumptions written next to it, plus a note on which term dominates, is falsifiable in parts — which is what makes it useful.

Estimate at roughly 1.3 tokens per English word, and treat that as an anchor rather than a fact. For 2,000 documents of 8 pages, at a conventional 500 words per page, that is 2000 × 8 × 500 × 1.3 ≈ 1e7 — 10 million tokens, to one significant figure. That is enough to answer feasibility questions decisively: 10 million tokens against a context window of around 131,000 is off by roughly 77×, so retrieval rather than prompt-stuffing is the only option. The ratio varies substantially with tokenizer, language and content — code, URLs, numbers and non-English text all tokenize into more pieces per word — so when a context limit or an invoice depends on the answer, run the actual tokenizer instead of the anchor.

Yes, in three specific situations, and no otherwise. It matters when your estimate lands near a hard capacity limit, because 7.4% is larger than the margin people typically leave. It matters when you compare two numbers reported by two different tools, because the discrepancy masquerades as a bug. And it matters at larger scales, because the gap grows to 10% at tebi and above, which is a real line in a storage budget. It does not matter when your answer is off by 4×, as the fine-tuning example in section 4 is — there, the units question is noise against a structural conclusion. Knowing which situation you are in is the skill.

Estimate to decide what to attempt; measure to report what happened. A one-line estimate is the right tool for feasibility ("does this fit at all?"), for choosing between approaches that differ by more than about 2×, and for declaring a non-problem so you stop thinking about it. Measurement is required when you are sizing hardware you will pay for, when the estimate lands within roughly 30% of a hard limit, when you are reporting a benchmark or evaluation result, and when you are comparing two of your own configurations — where, as M0.3 showed, you also need the sample size and the spread before the difference means anything.

1. GB is 10⁹ bytes, GiB is 2³⁰ bytes, and the GiB is about 7.4% larger — growing to 10% at tera and above.
2. Vendors quote decimal, tools report binary. A "24 GB" card is about 22.4 GiB nominally, and less than that in practice after driver, context and workspace overhead.
3. Label every figure with its unit. "≈14 GiB (7e9 params × 2 bytes / 2³⁰)" is self-checking; "14 GB" is not.
4. Bits versus bytes is an 8× error and the largest routine units trap in the field. Network figures are in bits; memory is in bytes.
5. 2¹⁰ ≈ 10³, with 2.4% error per application. That single approximation lets you convert between conventions mentally.
6. Memorise bytes per number: 4 for fp32, 2 for bf16, 1 for int8, 0.5 for 4-bit. It is the multiplier in every footprint estimate.
7. One significant figure with the assumption stated beats three with it hidden. Rounding is a smaller error than every structural mistake above it on the list — including the GB/GiB gap itself.
8. Name the dominant term. If you cannot say which component carries the total, you do not yet understand the estimate. In a full fine-tune with Adam, it is optimizer state.
9. State sensitivity, not just the answer. "Roughly 80 GiB, but 3 GiB if only an adapter is trainable" tells a reader which assumption to attack.
10. Never invent a figure to complete a calculation. Leave the unknown as a variable, or label the whole example as constructed. A fabricated number that looks authoritative is worse than an honest gap.
11. Tokens are not words. About 1.3 tokens per English word is an anchor for estimation; count with the real tokenizer whenever a limit or a bill depends on it.
12. Estimate to decide, measure to report. And when an estimate lands within about 30% of a hard limit, that is not a conclusion — it is an instruction to go and measure.
13. Hold a few anchors. A 7B model's bf16 weights are around 14 GB; Adam state is roughly 8 bytes per parameter in fp32; a page is about 500 words. Anchors turn plausibility checks into five-second reflexes.

You now have the whole prerequisite kit. M0.1 gave you shapes and the dot product; M0.2 gave you the hardware constraint and the four things that fill it; M0.2a gave you somewhere to run code and a way to trust it; M0.3 gave you distributions, expectation and the reason a small measurement can lie; and this lesson gave you the units and the estimation discipline that keeps every number you meet from here honest.

What none of it has told you is what the model is actually for. You know that its output is a probability distribution over a vocabulary, but not why training it to produce that distribution — over and over, on ordinary text, with no labels and no task — yields something that can summarise a document, answer a question, or write code. That is the single idea the rest of the course is built on, and it is where the graded curriculum properly begins.

Next: 01-01 Next-token prediction: what a language model is trained to do — the objective itself, why it needs no labelled data, and how a task nobody explicitly asked for turns out to require learning almost everything about language.