Scoring Answers with Log-Probabilities
A response's probability is a sum over its tokens. Why summing and averaging pick different answers, and what to score.
Jump to a section
Two answers to 17 × 3
Ask a model "What is 17 × 3?" and score two candidate answers by the probability it assigns to them. Answer A, "The answer is 51.", is correct and five tokens long. Answer B, " 41", is wrong and a single token. Suppose the model gives A's tokens probabilities 0.62, 0.81, 0.93, 0.44 and 0.9, and gives B's token 0.3.
Multiply A's probabilities together and the whole answer gets 0.185. B gets 0.3. Ranked by total probability, the model prefers the wrong answer. Switch between the two scoring rules below to see the ranking flip.
The-0.48 answer-0.21 is-0.07 51-0.82.-0.11
41-1.20
The total favors the shorter answer, which is wrong: every extra token can only lower a sum of log-probabilities.
Authored probabilities, not model outputs. Shading shows surprisal, −log₂ p; small numbers are natural-log probabilities.
The model has not made a mistake here. A sequence probability measures how expected a piece of text is, and every extra token costs probability. How you aggregate token scores decides which kind of answer wins, and no aggregation measures correctness. Used with that in mind, log-probabilities rank candidates, flag low-confidence outputs, and score best-of-N samples when no verifier exists.
The chain rule turns a response into a sum
For a prompt and response , the probability of the response factorizes over its tokens, and its log becomes a sum:
Each term is the log-probability of one token given everything before it. Its negative, , is the token's surprisal: near zero for a token the model expected, large for one it did not. In answer A the surprisals are 0.48, 0.21, 0.07, 0.82 and 0.11. The largest is on "51", the token that carries the answer. Shading text by surprisal shows where the model hesitated, and that is often the number or claim that matters.
Every term is at most zero, so adding a token can only lower the total. A's five terms sum to ; B's single term is . Sum scoring prefers short responses whenever the extra tokens are not nearly certain.
What the sum covers matters as much as how it is computed. Without an end-of-sequence token, it is the probability of that token prefix, not the probability that the model stops there. Include EOS when scoring a complete, stopped response, and keep the tokenizer, prompt, answer span and stopping convention identical across the candidates you compare.
Normalizing for length
The length-normalized score divides by :
Per token, A averages and B , so the mean selects the correct answer. In perplexity terms, A has 1.40 and B 3.33: the model was, on average, choosing among fewer than two plausible tokens for A and more than three for B. The mean is a ranking heuristic, not a sequence log-probability, and it has the opposite bias: it can over-reward a long, fluent answer whose many easy tokens dilute one bad one.
Between the two sits a family of partial penalties. Google's neural machine translation system divided the log-probability by during beam search. Setting recovers the plain sum, and larger values weaken the bias against long outputs. At that penalty still picks B here; at it picks A. The right setting depends on how lengths differ among your candidates, so tune it on held-out data.
1from math import exp, log
2
3A = [0.62, 0.81, 0.93, 0.44, 0.9] # "The answer is 51." (correct)
4B = [0.3] # " 41" (wrong)
5
6
7def total(ps):
8 return sum(log(p) for p in ps)
9
10
11def mean(ps):
12 return total(ps) / len(ps)
13
14
15def gnmt(ps, alpha):
16 # GNMT length penalty: lp(T) = ((5 + T) / (5 + 1)) ** alpha
17 return total(ps) / ((5 + len(ps)) / 6) ** alpha
18
19
20assert round(total(A), 3) == -1.688 and round(total(B), 3) == -1.204 # sum picks B
21assert round(mean(A), 3) == -0.338 # mean picks A
22assert round(exp(-mean(A)), 2) == 1.40 and round(exp(-mean(B)), 2) == 3.33
23assert gnmt(A, 0.6) < gnmt(B, 0.6) # a partial penalty still picks B
24assert gnmt(A, 1.0) > gnmt(B, 1.0) # a full penalty picks AChoosing what to score
For a reasoning model, the response before the answer can be long and varied while the final answer is a few tokens. Scoring the whole response mostly measures how expected the reasoning prose was. Scoring only the answer span, conditioned on the model's own reasoning, asks how confident the model is in its conclusion. That is closer to what you want for selection, but it still rewards a confident mistake.
The same tool serves a few other jobs, each with its own trap. To compare fixed answer options, as in multiple choice, score each option's tokens under the same prompt; options that tokenize into different numbers of tokens reintroduce the length problem. To flag an uncertain output, look at the minimum or the mean token log-probability; what counts as low varies by model and task. To evaluate a language model, report perplexity on held-out text, which is only comparable between models that share a tokenizer.
In code
Score a response with one forward pass over prompt plus response. Logits at position predict token , so shift by one and keep only the response positions. Supply a nonempty prompt (or a beginning-of-sequence token) and a nonempty response, and include EOS in the response ids if you want to score stopping.
1import torch
2import torch.nn.functional as F
3
4@torch.no_grad()
5def response_logprob(model, prompt_ids, response_ids, normalize=True):
6 ids = torch.cat([prompt_ids, response_ids])[None] # [1, P + T]
7 logits = model(ids).logits[0, :-1] # predict next tokens
8 logp = F.log_softmax(logits.float(), dim=-1) # float32 for stable log-softmax
9 targets = ids[0, 1:]
10 token_logp = logp.gather(1, targets[:, None]).squeeze(1)
11 response = token_logp[prompt_ids.numel() - 1:] # response tokens only
12 return response.mean() if normalize else response.sum()The .float() matters. Log-softmax subtracts the log-sum-exp of the logits from each logit, two nearly equal numbers for a confident token. In bf16, numbers between 16 and 32 are spaced 0.125 apart, so if that subtraction happens in bf16 a token with log-probability comes out as exactly zero. Summed over a long response, those lost hundredths add up.
Where it goes wrong
Likely is not correct. A high log-probability says the text is expected under the model, and the model's expectations include its systematic errors. A fluent wrong answer can outscore an awkward right one. Use log-probability to rank or route, and pair it with a verifier when correctness matters.
Calibration is conditional. Kadavath et al. found that larger models are well calibrated on multiple-choice and true/false questions when the questions are presented in the right format. That is a result about those formats, not a license to read any token probability as a probability of being right.
Tokenizers differ. Perplexity is an average per token, and a different tokenizer splits the same text into a different number of tokens. Comparing perplexities across tokenizers compares the tokenizers as much as the models.
Off by one. The logits at position are the prediction for token . Forgetting the shift scores each token against the wrong prediction and still returns plausible-looking numbers.
Prefixes scored as answers. Leaving EOS out of the response scores how likely the text is to begin that way, not to end there. A candidate that is a prefix of a longer one will look better than it is.
Sources and further reading
- Raschka, Building Reasoning Models from the Ground Up: scoring candidates with token and sequence log-probabilities, then self-refinement.
- Wu et al., Google's Neural Machine Translation System: length normalization in beam search scoring.
- Kadavath et al., Language Models (Mostly) Know What They Know: when model probabilities are calibrated, and when they are not.