What the words mean
16 terms you will meet on this site and everywhere else, each in one plain sentence.
Where a term is routinely used to mean more than it does — “open weights” is the clearest case — the correction is part of the definition rather than a footnote to it.
- benchmark
- A fixed set of test questions, used to compare one model against another.Old benchmarks leak into training data and stop separating models, which is why the useful ones keep being replaced.
- contamination
- When a test's questions have leaked into the material a model was trained on, so it has effectively seen the exam paper.
- context window
- How much text a model can hold in view at once, counted in tokens — everything you have said plus everything it has replied.Once the conversation is longer than the window, the earliest part falls out of view.
- deprecation
- A provider announcing that a model will stop working on a given date, so anything built on it has to move.
- distillation
- Training a small model to imitate a large one, to get most of the behaviour at a fraction of the running cost.
- Elo
- A rating built from head-to-head comparisons, borrowed from chess: people are shown two answers and pick the better one.It measures which answers people prefer, which is not the same as which are correct.
- hallucination
- A model stating something false with the same confidence it states something true.It is not a malfunction to be patched out — it follows from how these systems generate text, so the useful question is how often, not whether.
- harness
- The software that puts the questions to the model and grades the answers — the exam conditions rather than the exam.
- inference
- Running a model to get an answer, as opposed to training it in the first place.
- open weights
- The trained model file is published, so anyone can download it and run it on their own machine.Not the same as open source: the licence may still restrict what you do with it, and the training data is almost never released.
- parameters
- The numbers inside a model that were adjusted during training — a rough measure of its size, usually quoted in billions.Bigger is not reliably better; a well-trained small model often beats a large one.
- quantisation
- Storing a model's numbers less precisely so it fits on smaller hardware, at some cost to quality.
- saturated
- A test that nearly every good model now passes, so it no longer tells you which is better.
- scaffold
- The set-up around a model during a test: how many attempts it gets, how long it may think, what tools it can use.Changing only the scaffold can move a score by more than the gap between two rival models, which is why every score on this site carries it.
- token
- The unit these systems read and write in — roughly three quarters of a word, so a page of text is about 500 of them.Prices are quoted per million tokens, which is why they look so small.
- training compute
- How much calculation went into building a model — the closest thing the field has to a measure of how much was spent making it.
Missing a word you met on the wire? It should have been glossed where it appeared — that is a defect in the digest, not in this list.