Probability Calibration & Predictive Uncertainty

A binary predictor is calibrated when cases assigned probability $p$ are positive about fraction $p$ in the relevant population:

$$P(Y=1\mid \hat p=p)=p.$$

Calibration is a population property, not a guarantee for an individual case. A weather model can be calibrated at 70% if rain occurs on 70% of comparable 70% forecasts; it still cannot say which particular day will rain.

Discrimination and calibration are independent #

A score can rank every positive above every negative and still map scores to wrong probabilities. Any strictly increasing transform preserves ROC-AUC but changes calibration. Conversely, a constant predictor equal to prevalence is perfectly calibrated in aggregate and has no discrimination.

Suppose 100 cases all receive 0.8, but only 60 are positive. This group is overconfident: predicted 80%, observed 60%. Replacing every prediction with 0.6 fixes aggregate calibration but provides no ranking within the group.

Good probabilities need both refinement/discrimination and reliability/calibration.

Proper scoring rules evaluate the whole probability #

The Brier score is

$$\frac1n\sum_i(\hat p_i-y_i)^2.$$

Log loss is

$$-\frac1n\sum_i[y_i\log\hat p_i+(1-y_i)\log(1-\hat p_i)].$$

Both are proper: in expectation, reporting the true probability minimizes the score. Log loss punishes confident wrong predictions especially strongly. Neither alone locates the problem, so inspect calibration curves, score distributions, and decision-region performance.

A reliability diagram bins predictions and plots mean predicted against observed frequency. Binning creates bias/variance choices: too few bins hide local failures; too many are noisy. Always show counts or uncertainty per bin. Equal-frequency bins improve support but have unequal widths.

Post-hoc calibration #

Platt/sigmoid calibration fits a logistic mapping from raw score to probability. It is stable with modest data but assumes a sigmoid-shaped correction. Isotonic regression fits a monotone step function, allowing more shape but overfitting small calibration sets. Multiclass calibration must also preserve probabilities summing to one.

Calibration data must not be the same cases used to fit the base model's scores. Use a separate calibration split or cross-validated out-of-fold predictions. Evaluate the calibrated result on another untouched set. If the base model is refit on more data afterward, its score distribution can change and invalidate the learned calibrator unless the procedure accounts for that refit.

Prevalence and policy change probabilities #

Under prior probability shift, class-conditional feature distributions may stay similar while $P(Y)$ changes. The same likelihood ratio then corresponds to different posterior probability. A model calibrated on a 50/50 case-control sample is not calibrated for a 1% deployment population without correction.

Action policies also create selective labels. If high-risk patients are treated, observed bad outcomes among them can fall; naive recalibration may conclude their original risks were exaggerated. Outcomes must be interpreted under intervention and label-observation policy.

What “uncertainty” can refer to #

• Aleatoric uncertainty: outcome variability remaining even with the correct model and inputs.
• Epistemic/model uncertainty: uncertainty about the fitted function from finite data, model choice, or regions without support.
• Distributional uncertainty: the deployment case may not belong to the training population.
• Measurement uncertainty: inputs and labels are themselves estimates with a measurement process.

A softmax value near 0.5 may reflect class overlap, but a value near 1 can occur far outside training support. Network output entropy alone is not epistemic uncertainty. Ensembles and Bayesian approximations can reveal some fit variability, but agreement among identically biased models is not safety evidence.

Intervals and prediction sets need a coverage statement #

For regression, a confidence interval for a mean response is narrower than a prediction interval for a new outcome because the latter includes irreducible outcome variation. Quantile regression estimates conditional quantiles; distributional models estimate a full conditional law.

Conformal procedures can wrap a fitted model to provide finite-sample marginal coverage under exchangeability: a nominal 90% prediction set covers about 90% of future exchangeable cases. This is not automatically 90% coverage for every subgroup or every $x$, and temporal/distribution shift breaks the premise.

Operational checks #

Evaluate calibration at the thresholds where actions change, across important groups, and over time. Record expected calibration error only as a coarse summary; its value depends on bins and can hide offsetting errors. Monitor proper scores when labels arrive, prevalence, action rate, and coverage of any prediction intervals/sets.

Calibration answers whether numerical confidence matches frequency under a population and policy. It does not prove causal validity, in-distribution status, or individual certainty.