Hypothesis Testing

Hypothesis testing formalizes “would data this extreme be surprising if a baseline model were true?” It is a compatibility check, not a machine that assigns probabilities to theories.

Null, statistic, and p-value #

Let $H_0:q=q_0$ be the null hypothesis. For a large independent binary sample, a standardized statistic is

$$z=\frac{\hat q-q_0}{\sqrt{q_0(1-q_0)/n}}.$$

The p-value is the probability, assuming $H_0$ and all model assumptions, of a statistic at least as extreme as observed. A small p-value is evidence against that complete null model; it is not $P(H_0\mid data)$.

Numeric example #

With 65 wins out of 100 and $H_0:q=0.50$,

$$z=\frac{0.65-0.50}{\sqrt{0.25/100}}=3.$$

That is unusual under an independent fair-coin model. But trading does not ask only whether $q>0.50$. If contracts cost 0.70, even a real 0.65 win rate is economically negative.

Life example #

A medicine can have a statistically detectable average effect that is too small to matter clinically. Conversely, a valuable effect can fail to reach significance in a small study. Effect size, interval uncertainty, decision cost, and power belong beside the p-value.

Application to no-hedge #

The relevant null is often net edge at fill time, not headline accuracy. Test whether realized or properly scored outcomes support $q-p-costs>0$, using event-level clusters and out-of-sample periods. Reusing one event through many fills inflates apparent significance.