Meaning and formula
For a binary outcome, denote the event by 1, its absence by 0 and the forecast probability by p. The educational error score is the square of the difference between p and the outcome. Averaging over the same series of questions makes it possible to compare probabilistic approaches; a lower value means a lower squared error. The choice of loss function is part of forecast-evaluation methodology.[1](#source-rm-forecast-evaluation)
Example
A probability of 80% gives an error of 0.04 when the event occurs and 0.64 when it does not. An overly confident incorrect forecast is penalized more heavily. The numbers come from the formula, not from financial-market data.
Limits of comparison
Do not compare authors’ final scores on different sets of questions as if they were one tournament. Simple and difficult events have different base rates. Use a common set, probabilities recorded in advance and a comparable information-access time.
The metric does not measure monetary profit and does not replace analysis of dependence between observations. An example with several events and a baseline forecast appears in the [forecast calibration guide](/guides/forecast-calibration/).
[Forecast Evaluation and Combination](https://www.nber.org/papers/t0192)