Probability Distributions

A probability distribution is not merely a named curve. It is the rule that assigns probability to every event expressible in terms of a random variable. The curve or table is one representation; the underlying object answers questions such as “what is the probability of a value in this interval?”, “which threshold is exceeded 1% of the time?”, and “how does a transformation change uncertainty?”

A useful model begins with the mechanism that produces outcomes. Counts, waiting times, bounded proportions, measurement noise, and maxima arise from different mechanisms and generally need different distributions.

Random variable, outcome, and observation #

A sample space $\Omega$ contains possible elementary outcomes. A random variable is a function

$$X:\Omega\to\mathbb R.$$

It maps an outcome to a number. Before observation, $X$ denotes an uncertain quantity governed by a distribution; after one trial, $x$ denotes one realized value. A dataset $x_1,\ldots,x_n$ is not the distribution itself—it is finite evidence about a process that may or may not be stable enough to model with one distribution.

The CDF is the universal representation #

The cumulative distribution function (CDF) of any real-valued random variable is

$$F_X(x)=P(X\le x).$$

Every CDF is nondecreasing, right-continuous, and satisfies

$$\lim_{x\to-\infty}F_X(x)=0, \qquad \lim_{x\to+\infty}F_X(x)=1.$$

Interval probabilities follow from its jumps and differences. In particular,

$$P(a<X\le b)=F_X(b)-F_X(a).$$

The CDF works for discrete, continuous, and mixed distributions. A probability mass function or density may not exist in the simple form a beginner expects, but the CDF always does.

The generalized quantile function is

$$Q(p)=\inf\{x:F_X(x)\ge p\},\qquad 0<p<1.$$

This definition handles jumps. It also explains simulation by inverse transform: if $U$ is uniform on $(0,1)$, then $Q(U)$ has CDF $F$ under the usual generalized-inverse construction.

PMF and PDF mean different things #

For a discrete random variable, the probability mass function (PMF) is

$$p_X(x)=P(X=x),\qquad p_X(x)\ge0,\qquad \sum_xp_X(x)=1.$$

For a continuous random variable with density $f_X$,

$$P(a\le X\le b)=\int_a^b f_X(x)\,dx, \qquad f_X(x)\ge0, \qquad \int_{-\infty}^{\infty}f_X(x)\,dx=1.$$

A density is probability per unit of $x$, not probability at $x$. Therefore $P(X=x)=0$ for a continuous variable even where $f_X(x)$ is large, and a density may exceed 1 when concentrated over a narrow interval. Units expose the distinction: if $X$ is measured in seconds, $f_X$ has units $1/\text{second}$ while $F_X$ and probabilities are dimensionless.

Distribution families encode mechanisms #

Named families are reusable model shapes with parameters. The parameterization must be stated: libraries disagree about whether a gamma or exponential parameter is a rate or a scale, and whether a normal distribution's second argument is a variance or standard deviation.

These are candidate mechanisms, not labels assigned by histogram resemblance. Real counts often show overdispersion relative to Poisson; waiting times often have nonconstant hazard; positive measurements may have truncation or censoring; mixtures can look heavy-tailed or multimodal even when each component is simple.

Worked example: count versus waiting time #

Suppose arrivals follow an ideal Poisson process at rate $\lambda=3$ per hour. The count $N(t)$ in $t$ hours is Poisson:

$$P(N(t)=k)=e^{-\lambda t}\frac{(\lambda t)^k}{k!}.$$

For a 30-minute interval, $\lambda t=1.5$, so

$$P(N(0.5)=0)=e^{-1.5}\approx0.223.$$

The waiting time $T$ to the next arrival is exponential with survival function

$$P(T>t)=e^{-\lambda t}.$$

Thus $P(T>0.5)=e^{-1.5}$ is the same event expressed in two views: no arrival in the next half hour. The Poisson distribution models a count; the exponential distribution models a waiting time. Sharing a parameter does not make them interchangeable.

Transformations move probability, not just axes #

If $Y=g(X)$ and $g$ is differentiable and one-to-one, densities transform by

$$f_Y(y)=f_X\bigl(g^{-1}(y)\bigr) \left|\frac{d}{dy}g^{-1}(y)\right|.$$

The Jacobian factor is necessary because density is mass per unit length. Stretching the axis spreads the same probability over more $y$-space; compressing it concentrates the density.

For $Y=aX+b$ with $a\ne0$,

$$f_Y(y)=\frac1{|a|}f_X\left(\frac{y-b}{a}\right).$$

For a non-monotone transformation, all inverse branches contributing to $y$ must be summed. Blindly substituting variables without the Jacobian loses normalization.

Joint, marginal, and conditional distributions #

A joint distribution describes several random variables together. Marginalization removes variables by summing or integrating:

$$f_X(x)=\int f_{X,Y}(x,y)\,dy.$$

Independence is the factorization

$$f_{X,Y}(x,y)=f_X(x)f_Y(y)$$

in density form, with analogous event or PMF statements. Zero correlation is weaker and does not generally imply independence. Conditioning changes the distribution after learning an event or variable value; Bayes' rule governs that update.

Fit is not validity #

Parameters may be known by design, estimated from data, or assigned from prior knowledge. Those are different epistemic situations. A fitted normal curve does not prove observations are independent, identically distributed, uncensored, or generated by one stable population.

A defensible distribution model needs checks at several layers:

1. Support: can the model generate impossible negative or out-of-range values?
2. Mechanism: do independence, stationarity, rate, or hazard assumptions match the process?
3. Shape: do empirical CDFs, quantiles, and tail plots agree where decisions are sensitive?
4. Dependence: are observations serially, spatially, or hierarchically related?
5. Selection: were values censored, truncated, rounded, or sampled conditionally?
6. Use: is the model adequate for the probability, quantile, test, or simulation being computed?

NIST's statistical handbook makes the operational point explicit: a distribution need not be the best-fitting imaginable family, but assumptions must be adequate for the interval, test, or simulation that consumes it. Tail adequacy matters far more for a 99.9th-percentile alarm threshold than for estimating a central median.

The right question is not “which named curve looks familiar?” It is “which probability model preserves the mechanism and the decision-relevant uncertainty, and what evidence would falsify it?”

Binary outcomes and the 65% example #

A YES/NO event with $P(YES)=q$ follows a Bernoulli distribution:

$$P(Y=y)=q^y(1-q)^{1-y},\qquad y\in\{0,1\}.$$

For $q=0.65$, the distribution assigns mass 0.65 to 1 and 0.35 to 0. It does not assign the realized profit distribution until a purchase price is supplied. At price $p$, subtracting $p$ moves the two outcomes to $1-p$ and $-p$.

Application to no-hedge #

Each unhedged contract has a binary settlement distribution, but a portfolio is not automatically binomial. Common references and shared resolution events create dependence, so the portfolio loss distribution must preserve those clusters.

Self-check: is a density value of 1.4 an impossible probability?

No. A continuous density is probability per unit and can exceed 1; only integrated probability must lie between 0 and 1.