Classification Accuracy & Decision Thresholds

A classifier answers “which label should I choose?” A probability forecaster answers “how uncertain am I?” Turning every probability above 0.50 into YES discards information and silently assumes equal decision costs.

Accuracy and its blind spot #

For predicted labels $\hat y_i$ and outcomes $y_i$,

$$Accuracy=\frac1n\sum_{i=1}^n\mathbf1(\hat y_i=y_i).$$

Predictions 0.51 and 0.99 both become YES at a 0.50 threshold, although their confidence and Brier losses differ greatly. In an imbalanced population with 95% NO, always predicting NO reaches 95% accuracy while detecting nothing.

Decisions require payoffs #

For a YES contract priced at $p$ with per-share cost $c$, buying has expected net profit

$$EV_{net}=q-p-c.$$

The action threshold is therefore

$$q>p+c,$$

not necessarily $q>0.50$. At $q=0.65$ and $p=0.70$, YES is the more likely outcome but buying has negative expected value. At $q=0.40$ and $p=0.20$, YES is less likely than NO but buying YES has positive gross EV of 0.20.

Life example #

A smoke alarm should trigger well below a 50% fire probability because the cost of missing a fire is enormous relative to checking a false alarm. Decision thresholds encode consequences.

Application to no-hedge #

No-hedge logic should compare estimated maker-side fair value with the executable price plus every cost and safety buffer. A hard-coded 0.50 direction rule ignores the quoted payoff economics and can systematically buy overpriced favorites.