Law of Large Numbers & Central Limit Theorem

Repeated observations turn probability into visible frequencies, but two different theorems explain two different things. The law of large numbers (LLN) says an average converges toward its expectation. The central limit theorem (CLT) describes the approximate shape and scale of the remaining error.

The two results #

For independent, identically distributed variables with mean $\mu$,

$$\bar X_n=\frac1n\sum_{i=1}^n X_i \xrightarrow[]{P}\mu.$$

With finite variance $\sigma^2$, the CLT adds

$$\frac{\sqrt n(\bar X_n-\mu)}{\sigma}\xrightarrow[]{d}N(0,1).$$

The standard error of the average therefore shrinks roughly as $\sigma/\sqrt n$, not $\sigma/n$. Four times as many independent observations roughly halve the standard error.

Life example #

One café visit says little about average waiting time. Across many comparable days, the sample average stabilizes. But a new staffing policy changes the process: more old observations no longer guarantee relevance because identical distribution has failed.

One hundred 65% outcomes #

For $n=100$ independent binary events with $q=0.65$,

$$E[K]=nq=65, \qquad SD(K)=\sqrt{nq(1-q)}\approx4.77.$$

Seeing 60 wins is ordinary fluctuation, not immediate proof that 0.65 was wrong. Likewise, “approximately 65 wins” does not say which trades win or prevent losing streaks.

Application to no-hedge #

A no-hedge edge can be realized only across a sufficiently large set of comparable positions. If fills arrive precisely when news is adverse, or many contracts share one event, the observations are neither representative nor independent. Counting 100 correlated fills as $n=100$ exaggerates how much the LLN and CLT protect you.