Matrices as linear maps

Summary

A matrix describes a linear map. Its columns say where the input basis vectors land. The same weights that build an input also combine its column images.

Consider $A = (1122)$ and the nonzero input $x = (2, - 1)$ . Can its output be zero? Can we predict that output from the two column arrows?

The geometry

Let $e_{1} = (1, 0)$ and $e_{2} = (0, 1)$ be the input basis vectors. Their images are columns $a_{1} = (1, 1)$ and $a_{2} = (2, 2)$ . The second column is twice the first. Both point along one output line.

Learning objective

A nonzero input can collapse to zero when matrix columns are dependent

Derivation

Read it this way: use the input's weights on the column arrows. Twice the first column equals the second, so subtraction reaches zero. The input was nonzero; this map loses one direction.

Check: How can x be nonzero while 2a1-a2 is zero? What does that reveal about A?

Read it this way: If both matrix columns are (1,1), input coefficients (1,-1) produce zero. A nonzero input need not survive the linear map.
Can a nonzero coefficient vector disappear under dependent columns?

Original coordinate construction checked against MIT 18.022, Peter Selinger's CC BY 4.0 Matrix Theory and Linear Algebra, and Boyd and Vandenberghe's Introduction to Applied Linear Algebra.

Work one output

Write $x = 2 e_{1} - e_{2}$ . Linearity gives

A x = 2 A e_{1} - A e_{2} = 2 a_{1} - a_{2} = 2 (1, 1) - (2, 2) = (0, 0) .

For any input $(x_{1}, x_{2})$ , the output is $(x_{1} + 2 x_{2}) (1, 1)$ . Thus the range is the line spanned by $(1, 1)$ . The null space contains every multiple of $(2, - 1)$ . Check $(1, 1)$ : column weighting and ordinary multiplication both give $(3, 3)$ .

General form and shapes

With $n$ input coordinates and $m$ output coordinates, $A \in R^{m \times n}$ and $f (x) = A x$ . Linearity means $f (α x + β z) = α f (x) + β f (z)$ for inputs $x, z$ and scalar weights $α, β$ . Since $x = \sum_{j = 1}^{n} x_{j} e_{j}$ , we get $A x = \sum_{j = 1}^{n} x_{j} a_{j}$ .

The column space, or range, contains all outputs. It lies in $R^{m}$ .
Each row computes one output coordinate as a linear functional. The row space lies in $R^{n}$ .
The null space, or kernel, is ${x : A x = 0}$ . These input directions disappear.
Rank is the column-space dimension, equal to the row-space dimension.

Rank plus null-space dimension equals $n$ . Here $1 + 1 = 2$ . Independent columns prevent input information loss. A square matrix with independent columns is also onto and invertible.

Change the map

1. Produce three coordinates

New task: send the same two input coordinates to a three-coordinate output.

Picture and change: choose columns $(1, 0, 1)$ and $(0, 1, 1)$ . Keep the input weights $2, - 1$ .

Check: $2 (1, 0, 1) - (0, 1, 1) = (2, - 1, 1)$ . The columns are independent, so no nonzero input collapses. The map is not onto all of $R^{3}$ .

Use when: a layer changes feature count. Inputs count columns; outputs count rows.

2. Add an output offset

New task: compute $f (x) = A x + b$ with $b = (1, 0)$ .

Picture and change: combine columns as before, then translate the result by $b$ . The complete map is affine, not linear.

Check: $f (2, - 1) = (1, 0)$ and $f (0) = b \neq = 0$ . Its values at basis vectors are no longer simply the columns of $A$ .

Use when: a fixed bias displaces the origin.

Composition and multiplication

For $A$ of shape $m \times n$ and $B$ of shape $k \times m$ , applying $A$ then $B$ gives $B A$ . The result has shape $k \times n$ . Associativity follows from composition: $C (B A) x = (C B) A x$ . Reversing the maps generally changes the result.

A projection need not be perpendicular

Let $P = (1010)$ . It satisfies $P^{2} = P$ , but sends $(0, 1)$ to $(1, 0)$ . Its kernel is the line spanned by $(- 1, 1)$ , not the perpendicular complement of its range. For an orthogonal projection, also require $P^{⊤} = P$ .

Try reversing two maps

Let $A = (1011)$ and $B = diag (2, 1)$ . Compare $B A e_{2}$ with $A B e_{2}$ .

Check your answer

$A e_{2} = (1, 1)$ , so $B A e_{2} = (2, 1)$ . But $B e_{2} = (0, 1)$ , so $A B e_{2} = (1, 1)$ . The order changes the output.

Special families

Matrix	Geometric action
Orthogonal $Q$ ( $Q^{⊤} Q = I$ )	Rotation or reflection (preserves length and angle)
Diagonal	Independent scaling along each axis
Symmetric	Has real eigenvalues; orthogonal eigenvector basis
Positive definite	Symmetric + all eigenvalues > 0; defines an inner product
Permutation	Reorders coordinates
Projection ( $P^{2} = P$ )	Maps onto its range along its kernel; orthogonal only when also symmetric

Common pitfalls

Treating matrix multiplication as element-wise. Use Hadamard ( $⊙$ ) for element-wise; matrix multiplication is composition.
Forgetting that shapes determine the map. $A \in R^{3 \times 5}$ is a map $R^{5} \to R^{3}$ , not the other way around.
Confusing column space with row space. Both have dimension = rank, but they live in different spaces ( $R^{m}$ vs $R^{n}$ ).

A fixed-weight dense operation $A x$ is linear. Adding bias makes it affine; applying a nonlinear activation changes the map again. Neural networks and input-dependent attention are not generally linear functions of their inputs.