Vectors, Matrices & Linear Maps

Vectors and matrices are not special kinds of tables. A vector is a state, direction, or collection of quantities described in a declared coordinate system. A matrix is a rule that turns one vector into another while preserving addition and scaling. Its entries acquire meaning only after the input order, output order, basis, and units are named.

Matrix multiplication looks arbitrary when taught as a recipe, but it is forced by one requirement: the matrix for two rules composed in sequence must produce the same result as running those rules one after another.

Vectors need semantics and units #

A coordinate vector

$$x=\begin{bmatrix}x_1\\x_2\\\vdots\\x_n\end{bmatrix}$$

is an ordered list only after its coordinates have been declared. The same array $[3,2]^{\mathsf T}$ could mean a point at three metres east and two metres north, three units of product A and two of product B, or two sensor readings. Those are different vectors in different models even though the stored numbers match.

Vector addition combines like coordinates:

$$x+y= \begin{bmatrix}x_1+y_1\\\vdots\\x_n+y_n\end{bmatrix}.$$

Multiplication by a positive scalar changes magnitude without changing direction or coordinate proportions. A negative scalar reverses direction, and zero collapses the vector to the zero vector. These operations are meaningful only when the coordinate systems and units are compatible. Adding a position vector to a temperature vector is not repaired by the fact that both arrays have length three.

Dot products are weighted totals and alignment #

For vectors $a,b\in\mathbb R^n$, their dot product is

$$a^{\mathsf T}b=\sum_{k=1}^n a_kb_k.$$

It has two complementary meanings.

• As a weighted total, $a_k$ is the weight applied to input $b_k$.
• For nonzero vectors, as alignment, $a^{\mathsf T}b=\|a\|\,\|b\|\cos\theta$ measures how much one vector points along the other.

For example, if quantities are $q=(3,2)$ units and unit prices are $p=(4,20)$ euros per unit, then

$$p^{\mathsf T}q=4(3)+20(2)=52\ \text{euros}.$$

The multiplication pairs corresponding coordinates; the addition accumulates their contributions into one output. Matrix multiplication repeats this same weighted-total mechanism for many outputs at once.

A matrix is a linear map #

An $m\times n$ matrix $A$ accepts an $n$-coordinate input and produces an $m$-coordinate output:

$$y=Ax, \qquad y_i=\sum_{j=1}^n a_{ij}x_j.$$

Each row of $A$ is the weighted recipe for one output coordinate. Each column is the output produced by one input basis vector. This second view follows because

$$Ax=x_1A_{:1}+x_2A_{:2}+\cdots+x_nA_{:n}.$$

The map is linear because it preserves combinations:

$$A(x+y)=Ax+Ay, \qquad A(\alpha x)=\alpha Ax.$$

For example,

$$R=\begin{bmatrix}0&-1\\1&0\end{bmatrix}$$

rotates the plane by $90^\circ$ counterclockwise. Its first column says where the input basis vector $(1,0)$ goes; its second says where $(0,1)$ goes. Thus

$$R\begin{bmatrix}3\\1\end{bmatrix} =3\begin{bmatrix}0\\1\end{bmatrix} +1\begin{bmatrix}-1\\0\end{bmatrix} =\begin{bmatrix}-1\\3\end{bmatrix}.$$

Translation is not linear because it does not keep the zero vector fixed. A rule $x\mapsto Ax+b$ is affine; homogeneous coordinates let graphics and robotics package that extra offset into a larger matrix.

Why multiplication is row by column #

Let $B\in\mathbb R^{p\times n}$ turn $n$ inputs into $p$ intermediate quantities, and let $A\in\mathbb R^{m\times p}$ turn those quantities into $m$ outputs. If $y=Bx$ and $z=Ay$, then

$$z_i=\sum_k a_{ik}y_k =\sum_k a_{ik}\sum_j b_{kj}x_j =\sum_j\left(\sum_k a_{ik}b_{kj}\right)x_j.$$

Therefore the one matrix that performs the combined map is $AB$, with

$$(AB)_{ij}=\sum_{k=1}^{p}a_{ik}b_{kj}.$$

The shared index $k$ is not bookkeeping: it names the intermediate channels created by $B$ and consumed by $A$. Along each path $j\to k\to i$, effects multiply; parallel paths add. That wiring is why the rule is row by column. An elementwise product cannot represent composition.

Equivalently, column $j$ of $AB$ is $A$ applied to column $j$ of $B$:

$$(AB)_{:j}=A(B_{:j}).$$

The dimensions encode the same contract:

$$(m\times p)(p\times n)=(m\times n).$$

The two occurrences of $p$ are the common intermediate coordinate system.

A product–resource–cost pipeline #

Suppose

$$x=\begin{bmatrix}3\\2\end{bmatrix}$$

means three units of product A and two of product B. Let

$$B=\begin{bmatrix}2&1\\1&3\end{bmatrix}$$

describe kilograms of material per product unit in its first row and labour-hours per product unit in its second. Then

$$Bx=\begin{bmatrix}8\ \mathrm{kg}\\9\ \mathrm h\end{bmatrix}.$$

If material costs €4/kg and labour costs €20/h, use

$$c^{\mathsf T}=\begin{bmatrix}4&20\end{bmatrix}.$$

The composed row, measured in euros per product unit, is

$$c^{\mathsf T}B=\begin{bmatrix}28&64\end{bmatrix},$$

so

$$c^{\mathsf T}Bx=28(3)+64(2)=212\ \mathrm{EUR}.$$

The row-by-column rule has composed two real pipelines: products into resources, then resources into cost. Shapes and units expose a reversed or meaningless multiplication before arithmetic begins.

Composition order matters #

Let

$$S=\begin{bmatrix}2&0\\0&1\end{bmatrix}, \qquad R=\begin{bmatrix}0&-1\\1&0\end{bmatrix}, \qquad x=\begin{bmatrix}1\\2\end{bmatrix}.$$

Stretching horizontally and then rotating gives

$$RSx=R\begin{bmatrix}2\\2\end{bmatrix} =\begin{bmatrix}-2\\2\end{bmatrix}.$$

Rotating first and then stretching gives

$$SRx=S\begin{bmatrix}-2\\1\end{bmatrix} =\begin{bmatrix}-4\\1\end{bmatrix}.$$

In $RSx$, the rightmost map $S$ acts first. Matrix multiplication is generally not commutative because resizing and rotating an image, calibrating and mixing sensor channels, or passing through two physical stages depends on order. It is associative:

$$(AB)C=A(BC),$$

because either parenthesization represents the same three-stage composition.

The operation and its algorithms are different objects #

For $A\in\mathbb R^{m\times p}$ and $B\in\mathbb R^{p\times n}$, the direct three-loop algorithm performs $mnp$ scalar multiplications plus a comparable number of additions, so its arithmetic work scales as $O(mnp)$. Reordering loops or multiplying blocks can greatly improve memory locality without changing the definition of $AB$. Special algorithms may reduce arithmetic for sufficiently large matrices, but they still owe the same mathematical product, subject to floating-point rounding.

This distinction recurs throughout mathematics: a formula specifies the object; an algorithm specifies the steps, work, storage, and numerical path used to obtain it. The DFT and FFT are a particularly clear example.

Linear systems are inverse questions #

Forward multiplication asks: given the input $x$, what output $b=Ax$ follows? A linear system asks the inverse question: which input could have produced an observed output $b$?

Suppose $x_1$ standard and $x_2$ premium tickets satisfy

$$x_1+x_2=7, \qquad 2x_1+x_2=11.$$

In matrix form,

$$\begin{bmatrix}1&1\\2&1\end{bmatrix} \begin{bmatrix}x_1\\x_2\end{bmatrix} =\begin{bmatrix}7\\11\end{bmatrix}.$$

Subtracting the first equation from the second gives $x_1=4$, then $x_2=3$. The matrix records two measurement rules; solving reconciles both at once.

A square matrix has an inverse only when different inputs cannot collapse to the same output. More generally, a system may have no solution, one solution, or infinitely many. Elimination exposes which case applies and is normally preferable to explicitly forming $A^{-1}$ in numerical work.

Where the same operation appears #

Matrices recur because all these models use weighted sums and composable linear channels. This does not mean every rectangular dataset is intrinsically a linear map. The interpretation, units, and chosen input/output spaces create the mathematics.

Diagnostic checklist #

• Name every vector's coordinate order, basis, and units.
• For $Ax$, verify that the number of columns of $A$ matches the input length.
• Read rows as output recipes and columns as basis-vector images.
• For $AB$, say which map acts first and name the intermediate space.
• Use a tiny vector to test composition order; do not assume $AB=BA$.
• Distinguish matrix multiplication from elementwise multiplication.
• Treat $Ax=b$ as a question about existence and uniqueness, not as an order to compute an inverse blindly.
• Separate the mathematical operation from the algorithm, data layout, and floating-point implementation used to compute it.
• Check whether an offset makes the model affine rather than linear.

The row-by-column rule becomes memorable only after its job is clear: it is the unique bookkeeping that makes a composed chain of linear input–output rules behave like the chain itself.