Calibrated Probabilities

Question

Your model outputs the probability that a quote, if filled, is adversely selected, and this probability is plugged directly into the expected-value calculation that sets your bid and ask.

Explain the difference between discrimination and calibration, why a model with excellent AUC can still lose money in this setup, how to measure calibration (reliability diagram, Brier score and its decomposition), and how to fix a miscalibrated model.

Accepted Answer

Two different qualities of a probabilistic modelDiscrimination is the ability to rank: do adversely selected fills receive higher scores than benign ones? It is measured by ROC-AUC. Calibration is the numerical truthfulness of the probabilities: among all events to which the model assigns probability 0.2, do about 20% actually turn out positive? Formally, calibration requires \begin{equation} P(y = 1 \mid \hat{p} = p) = p \quad 	ext{for all } p \end{equation} The two are independent: squashing every prediction from $[0, 1]$ into $[0.45, 0.55]$ leaves the ranking — and hence the AUC — untouched, but makes the numbers useless as probabilities. AUC is invariant under any monotone transformation of the scores; expected-value arithmetic is not.

Why miscalibration loses money hereThe quoting engine computes something like \begin{equation} EV = (1 - \hat{p}) \cdot 	ext{spread capture} - \hat{p} \cdot 	ext{adverse loss} \end{equation} and quotes where the EV is positive. If the model says 10% when the truth is 20%, the engine systematically overestimates the EV of quoting, quotes too tightly in toxic conditions, and bleeds — even though the model's ranking of conditions, and therefore its AUC, may be superb. Whenever probabilities are consumed as numbers rather than as a ranking (pricing, bet sizing, expected-value gating), calibration is not optional.

Measuring calibrationThe reliability diagram: bin the predictions (e.g. into deciles), and for each bin plot the empirical positive frequency against the mean predicted probability. A calibrated model sits on the diagonal; an overconfident model is too flat at the ends. The Brier score summarises probabilistic quality in one number: \begin{equation} BS = \frac{1}{N}\sum_{i=1}^{N} (\hat{p}_i - y_i)^2 \end{equation} and over the bins it decomposes into \begin{equation} BS = \underbrace{\frac{1}{N}\sum_k n_k (\bar{p}_k - \bar{y}_k)^2}_{	ext{reliability}} - \underbrace{\frac{1}{N}\sum_k n_k (\bar{y}_k - \bar{y})^2}_{	ext{resolution}} + \underbrace{\bar{y}(1 - \bar{y})}_{	ext{uncertainty}} \end{equation} Reliability (want: 0) measures miscalibration — the gap between predicted and realised frequency per bin. Resolution (want: large) measures how much the model's bins separate the outcomes — this is the discrimination component. Uncertainty is the irreducible base-rate term: even the perfect forecaster cannot go below it. Both the Brier score and log loss are proper scoring rules — their expectation is minimised by reporting the true probability — so training and model selection on them at least pushes towards calibration, unlike accuracy or AUC.

Fixing a miscalibrated modelKeep the model's scores and learn a mapping from score to probability on held-out data:

 	Platt scalingfit $\sigma(a s + b)$ on validation scores $s$ — two parameters, robust on small samples, assumes a sigmoid-shaped distortion.
 	Isotonic regressionfit the best monotone step function — no shape assumption, more flexible, needs more data and can overfit small validation sets.

Always fit the recalibrator on data the model was not trained on, and refit it after every retrain. Useful background facts: well-regularised logistic regression tends to come out naturally calibrated (it optimises log loss directly); boosted trees and modern neural networks are notorious for miscalibration, typically overconfidence. And the operational point for a trading desk: calibration is the first casualty of distribution shift — a model calibrated in a calm regime will be confidently wrong in a stressed one — so reliability should be monitored live and per regime, not checked once at validation time and forgotten.

Title	Category	Subcategory	Difficulty	Status
Reading the ROC Curve	Machine Learning	Model Evaluation	Medium	Unlock
The Accuracy Trap	Machine Learning	Model Evaluation	Easy	Unlock

The Interview Guide for Quants and Traders

Calibrated Probabilities

What do you think of this page?