Orthonormal Bases & Hilbert-Space Expansions

An orthonormal basis is not usually a Hamel basis. Infinite-dimensional vectors are represented by convergent infinite expansions, not finite linear combinations. Orthogonality makes those coordinates unique, norm-stable, and independently measurable.

Orthogonal systems and coefficients #

For an orthonormal family $(e_\alpha)_{\alpha\in A}$ in a Hilbert space $H$, the coefficient of $x$ in direction $e_\alpha$ is $\langle x,e_\alpha\rangle$. Projection onto any finite subset $F\subset A$ and Pythagoras give Bessel's inequality:

$$\sum_{\alpha\in F}|\langle x,e_\alpha\rangle|^2\le\|x\|^2.$$

Taking the supremum over finite $F$ defines the possibly uncountable sum. For each fixed $x$, only countably many coefficients can be nonzero: for every $n$, only finitely many may exceed $1/n$ in magnitude.

What makes an orthonormal set a basis #

For an orthonormal family $(e_\alpha)$, these conditions are equivalent:

• it is maximal orthonormal;
• its closed linear span is all of $H$;
• the only vector orthogonal to every $e_\alpha$ is zero;
• Parseval holds for every $x$: $\|x\|^2=\sum_\alpha|\langle x,e_\alpha\rangle|^2$;
• every $x$ has the norm-convergent expansion $x=\sum_\alpha\langle x,e_\alpha\rangle e_\alpha$.

The error after retaining a finite set $F$ is exactly the omitted coefficient energy:

$$\left\|x-\sum_{\alpha\in F}\langle x,e_\alpha\rangle e_\alpha\right\|^2 =\|x\|^2-\sum_{\alpha\in F}|\langle x,e_\alpha\rangle|^2.$$

This is both a convergence theorem and an error budget.

Separable Hilbert spaces look like $\ell^2$ #

A Hilbert space is separable iff it has a countable orthonormal basis. Given a countable ONB $(e_n)$, the coordinate map

$$U:H\to\ell^2,qquad Ux=(\langle x,e_n\rangle)_{n\ge1}$$

is unitary. Thus every infinite-dimensional separable Hilbert space is abstractly isomorphic to $\ell^2$. The choice of basis still matters for localization, sparsity, smoothness, and computational structure.

Fourier series are one instance #

In $L^2[-\pi,\pi]$, the functions

$$e_n(x)=\frac{1}{\sqrt{2\pi}}e^{inx},\qquad n\in\mathbb Z,$$

form an ONB. Fourier coefficients are Hilbert coordinates, mean-square convergence is norm convergence, and Parseval is exactly Pythagoras. Pointwise convergence requires extra regularity because evaluation at a point is not a bounded functional on $L^2$.

Gram–Schmidt constructs an ONB for a span #

Given linearly independent $v_1,\ldots,v_n$, define

$$u_k=v_k-\sum_{j<k}\langle v_k,e_j\rangle e_j, \qquad e_k=\frac{u_k}{\|u_k\|}.$$

In exact arithmetic this preserves every partial span and produces an orthonormal family. If $u_k=0$, the input vector lies in the previous span.

In floating-point arithmetic, classical Gram–Schmidt can lose orthogonality when columns are nearly dependent because the residual is formed by subtracting large nearly equal quantities. Modified Gram–Schmidt updates the residual one projection at a time and is usually more stable; Householder QR is the standard robust choice for dense least-squares problems. Reorthogonalize when measured loss $\|I-Q^*Q\|$ exceeds the problem's tolerance.

Basis diagnostics #

• Check orthonormality and completeness separately.
• State whether convergence is in norm, pointwise, or another topology.
• Use Parseval to validate normalization constants.
• For truncated expansions, inspect omitted energy and not only coefficient count.
• In numerical QR, measure orthogonality and residual $\|A-QR\|$; a plausible-looking $Q$ is not evidence of stability.