On Matrices a matrix learned to multiply a vector. This page teaches it to multiply a whole matrix, and then asks the question every engineer asks about a system $Ax = b$ sooner or later: can I just divide by $A$? Sometimes you can. The matrix that does the dividing is the inverse $A^{-1}$, and whether it exists comes down to one number you already know how to find, the number of pivots, which gets its name here: the rank.
On Matrices a matrix became a machine that moves arrows: $Ax$ is where the arrow $x$ lands. This page chains two of those machines together, then asks when a machine can be run backward, so that every landing spot leads back to exactly one starting arrow. The answer comes down to one number, the rank, which counts how many directions a matrix keeps. A matrix that keeps every direction can be undone. A matrix that flattens space onto a plane or a line can't, because flattening forgets where things came from.
The practice is mixed into the lesson: each question sits right after the idea it needs, and the Mixed Practice at the bottom takes that hint away.
You already know how $A$ acts on one vector. Give it several vectors at once, packed side by side as the columns of a matrix $B$, and it acts on each of them. That's the whole definition.
You already know how $A$ moves one arrow. A matrix $B$ is a team of arrows, its columns. Let $A$ move every arrow on the team, and the arrows where they land make a new team: that's $AB$.
Let $A$ be an $m \times n$ matrix and $B$ an $n \times p$ matrix with columns $b_1, \ldots, b_p$. The product $AB$ is the $m \times p$ matrix whose columns are $Ab_1, \ldots, Ab_p$: $$AB = \begin{pmatrix} Ab_1 & Ab_2 & \cdots & Ab_p \end{pmatrix}.$$ Entry by entry, $(AB)_{ij}$ is row $i$ of $A$ times column $j$ of $B$: $(AB)_{ij} = A_{i1}B_{1j} + A_{i2}B_{2j} + \cdots + A_{in}B_{nj}$. The product is defined only when $A$ has as many columns as $B$ has rows.
Let $A$ and $B$ be matrices of size at most $3 \times 3$, where $A$ has as many columns as $B$ has rows. The product $AB$ is the matrix whose columns are $A$ times the columns of $B$, in order: if $B$'s columns are $b_1, b_2, b_3$, then $AB$'s columns are $Ab_1, Ab_2, Ab_3$. Entry by entry, $(AB)_{ij}$ is row $i$ of $A$ dotted with column $j$ of $B$.
Watch it happen to two arrows in the plane. $B$'s columns are $b_1 = (2, 1)$ and $b_2 = (-1, 1)$, and $A = \begin{pmatrix} 1 & 1 \\ 0 & 1 \end{pmatrix}$ slides every point sideways by its height:
The arrows land on $Ab_1 = (3, 1)$ and $Ab_2 = (0, 1)$, so $AB = \begin{pmatrix} 3 & 0 \\ 1 & 1 \end{pmatrix}$. In practice you compute entries, not columns: row times column, one entry at a time.
Let $A = \begin{pmatrix} 1 & 2 & -1 \\ 0 & 3 & 1 \end{pmatrix}$ and $B = \begin{pmatrix} 2 & 1 \\ 0 & -1 \\ 1 & 4 \end{pmatrix}$. Find $AB$ and $BA$.
$A$ is $2 \times 3$ and $B$ is $3 \times 2$, so $AB$ is $2 \times 2$. Each entry is a row of $A$ times a column of $B$: $$AB = \begin{pmatrix} 2 + 0 - 1 & 1 - 2 - 4 \\ 0 + 0 + 1 & 0 - 3 + 4 \end{pmatrix} = \begin{pmatrix} 1 & -5 \\ 1 & 1 \end{pmatrix}.$$ The other order is also defined, but $B$ is $3 \times 2$ and $A$ is $2 \times 3$, so $BA$ is $3 \times 3$: $$BA = \begin{pmatrix} 2 & 7 & -1 \\ 0 & -3 & -1 \\ 1 & 14 & 3 \end{pmatrix}.$$ For instance, its middle entry is row 2 of $B$ times column 2 of $A$: $(0)(2) + (-1)(3) = -3$. The two products aren't even the same size.
When both products are square and the same size, they still usually disagree. Here are a slide and a quarter turn, $A = \begin{pmatrix} 1 & 1 \\ 0 & 1 \end{pmatrix}$ and $B = \begin{pmatrix} 0 & -1 \\ 1 & 0 \end{pmatrix}$: $$AB = \begin{pmatrix} 1 & -1 \\ 1 & 0 \end{pmatrix}, \qquad BA = \begin{pmatrix} 0 & -1 \\ 1 & 1 \end{pmatrix}.$$ $(AB)x = A(Bx)$, so $AB$ turns first and slides second, and $BA$ does it the other way around. Do it to a letter F and you can see the difference:
That one picture is worth a warning of its own.
Never swap the order of a matrix product to make the arithmetic easier. $AB$ and $BA$ are different matrices (when both even exist), so "$AB = BA$" is the one rule from ordinary algebra you have to unlearn. Everything else on the list below still holds.
One matrix leaves every other matrix alone, like the number $1$.
The $n \times n$ identity matrix $I_n$ has $1$s down its diagonal and $0$s everywhere else: $$I_2 = \begin{pmatrix} 1 & 0 \\ 0 & 1 \end{pmatrix}, \qquad I_3 = \begin{pmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{pmatrix}.$$ We write just $I$ when the size is clear.
With the identity in hand, here is the list of rules that do carry over from numbers.
Whenever the sizes make the products defined, and $c$ is a number:
(a) $(AB)C = A(BC)$;
(b) $A(B + C) = AB + AC$ and $(A + B)C = AC + BC$;
(c) $c(AB) = (cA)B = A(cB)$;
(d) $IA = A$ and $AI = A$;
(e) $(AB)^t = B^tA^t$.
First, $(AB)x = A(Bx)$ for every vector $x$. Write $Bx = x_1b_1 + \cdots + x_pb_p$ (the column picture). Since $A$ acts linearly on vectors, $A(Bx) = x_1Ab_1 + \cdots + x_pAb_p$, and that is the combination of $AB$'s columns with weights $x_1, \ldots, x_p$, which is $(AB)x$.
(a) Column $j$ of $(AB)C$ is $(AB)c_j = A(Bc_j)$, and column $j$ of $A(BC)$ is $A$ times column $j$ of $BC$, which is $A(Bc_j)$ too.
(b) Column $j$ of $A(B + C)$ is $A(b_j + c_j) = Ab_j + Ac_j$, column $j$ of $AB + AC$. For the other one, row $i$ of $(A + B)C$ is (row $i$ of $A$ + row $i$ of $B$) times $C$, and the entry rule splits over that sum.
(c) Each entry of all three is $c$ times the same sum $\sum_k A_{ik}B_{kj}$.
(d) Column $j$ of $AI$ is $Ae_j$, which is column $j$ of $A$. Row $i$ of $IA$ takes $1$ times row $i$ of $A$ and $0$ times the others.
(e) The $(i, j)$ entry of $(AB)^t$ is $(AB)_{ji} = \sum_k A_{jk}B_{ki} = \sum_k (B^t)_{ik}(A^t)_{kj}$, the $(i, j)$ entry of $B^tA^t$. $\blacksquare$
This is a visual argument for (a), not a formal proof. It leans on one fact: $(AB)x = A(Bx)$, which is the definition at work, since $AB$ is "move the arrows with $B$, then with $A$". So a chain of three matrices is three moves done in a row, rightmost first. Here $C$ turns the F, $B$ slides it, and $A$ stretches it sideways:
Brackets only say which two moves you combine into one first. They never change the order the moves happen in, so $(AB)C$ and $A(BC)$ land the F in the same place. Rules (b) through (d) are the column picture and a little bookkeeping, and (e) is an entry-by-entry check. The full proof is in the n version.
Rule (a) is why you can write $ABC$ with no brackets at all. Rule (e) turns the order around, and you'll see the same reversal again for inverses.
"Move with $B$, then with $A$" is a composition of two moves. On Composition & Matrix Multiplication that becomes the definition's reason for being: the matrix of "do one map, then another" is the product of their matrices.
Your turn: one product, or one piece of it.
Loading practice engine...
On Systems & Row Reduction you row reduced and counted pivots. That count has been doing quiet work all along, so it gets a name.
The rank of a matrix $A$, written $\rank(A)$, is the number of pivots in a row echelon form of $A$.
Different row operations can give different echelon forms, but they all have their pivots in the same columns: finishing any echelon form off to the reduced form only scales the pivot rows and clears above the pivots, which never moves a pivot, and the reduced form is unique. So the rank doesn't depend on how you got there.
Find the rank of $A = \begin{pmatrix} 1 & 0 & 1 \\ 2 & 1 & 3 \\ 1 & -1 & 0 \end{pmatrix}$.
Clear the first column with $R_2 \to R_2 - 2R_1$ and $R_3 \to R_3 - R_1$: $$\begin{pmatrix} 1 & 0 & 1 \\ 0 & 1 & 1 \\ 0 & -1 & -1 \end{pmatrix}.$$ Row 3 is now $-1$ times row 2, so $R_3 \to R_3 + R_2$ makes a zero row: $$\begin{pmatrix} 1 & 0 & 1 \\ 0 & 1 & 1 \\ 0 & 0 & 0 \end{pmatrix}.$$ Two pivots, so $\rank(A) = 2$. The columns tell the same story: column 3 is column 1 $+$ column 2, since $(1, 2, 1) + (0, 1, -1) = (1, 3, 0)$.
Draw the three columns as arrows. The first two lay out a flat sheet through $0$, and the third one lies in it:
That last check isn't a coincidence. The number of pivots also counts how many of the columns are independent, and how many of the rows are. Both facts need tools from a few pages ahead, so for now rank is "count the pivots", and the columns are a way to double-check.
That last picture isn't a coincidence. The rank also counts the directions the columns fill: rank 1 means they all lie on one line through $0$, rank 2 a plane, rank 3 all of space (and rank 0 means every column is the zero arrow). This page uses that as a picture; a later page proves it. Try guessing a few:
On Bases & Dimension, rank becomes the dimension of the span of the columns, and on The Four Fundamental Subspaces it turns out to be the dimension of the span of the rows too. Rows and columns of a non-square matrix don't even live in the same space, and yet they agree on this one number.
Two limits come straight from the definition, and the second one counts the free variables you met on Systems.
Let $A$ be an $m \times n$ matrix. Then $\rank(A) \le m$ and $\rank(A) \le n$. The homogeneous system $Ax = 0$ has exactly $n - \rank(A)$ free variables, so it has a solution other than $x = 0$ exactly when $\rank(A) < n$.
In an echelon form every pivot has a row of its own and a column of its own, so there are at most $m$ pivots and at most $n$. Reduce $(A|0)$: the last column stays $0$, and the left part is a reduced form of $A$. Its $n$ columns split into $\rank(A)$ pivot columns and $n - \rank(A)$ columns without a pivot, whose variables are the free variables. By the reading-RREF theorem, a consistent system has exactly one solution when there are no free variables and infinitely many otherwise. $Ax = 0$ is consistent ($x = 0$ works), so it has a nonzero solution exactly when $n - \rank(A) > 0$. $\blacksquare$
This is a visual argument, not a formal proof. Each pivot needs a row and a column of its own, so a $2 \times 3$ matrix can't have more than $2$. For the free variables, take three equations in $x$, $y$, $z$ with every right side $0$: three planes through the origin. On Systems you saw that planes through one point meet in a point, a line, or a whole plane. Here are three such systems, with rank 3, 2, and 1:
Rank 3 leaves no free variable, and the planes meet only at $0$. Rank 2 leaves one, and they share a line: one number to choose. Rank 1 leaves two, and all three are the same plane: two numbers to choose. Each time, free variables $= 3 - \rank$. The full proof is in the n version.
"Fewer equations than unknowns means a nonzero solution" is the case $m < n$: then $\rank(A) \le m < n$, so some variable is free.
Now put a right-hand side back. On Systems, a system had no solution exactly when its reduced augmented matrix had a pivot in the last column, a row that says $0 = 1$. In the language of rank:
Let $A$ be an $m \times n$ matrix. The system $Ax = b$ has a solution if and only if $\rank(A) = \rank(A|b)$. When it has one, it has exactly one if $\rank(A) = n$, and infinitely many, with $n - \rank(A)$ free variables, if $\rank(A) < n$.
Reduce $(A|b)$ to its reduced form $R$. The first $n$ columns of $R$ are a reduced form of $A$, so they hold $\rank(A)$ pivots. The last column adds one more pivot, or none: $\rank(A|b)$ is $\rank(A) + 1$ exactly when the last column has a pivot, which is a row $(0 \ \cdots \ 0 \mid 1)$. By the reading-RREF theorem, that row is exactly what makes a system inconsistent. So $Ax = b$ has a solution exactly when the ranks agree. In that case the free variables are the $n - \rank(A)$ columns of $A$ without a pivot, and the same theorem gives one solution when there are none and infinitely many otherwise. $\blacksquare$
This is a visual argument for the first sentence, not a formal proof. It leans on the column picture: $Ax = x_1a_1 + x_2a_2 + x_3a_3$, so the landing spots $Ax$ are exactly the combinations of the columns. Take the rank-2 matrix from the example above, whose columns fill a plane, and watch $b$:
While $b$ sits on the columns' plane, some combination reaches it, and adding $b$ as a fourth column adds no new direction: the ranks agree. Lift $b$ off the plane and no combination of the columns can reach it, and the extra column brings a new direction, so $\rank(A|b)$ jumps to 3. The counting part is the free-variable theorem above. The full proof is in the n version.
Here it is on a system with more equations than unknowns.
Does the system $x + y = 1$, $x - y = 3$, $2x = 5$ have a solution?
Reduce $(A|b)$ with $R_2 \to R_2 - R_1$ and $R_3 \to R_3 - 2R_1$, then $R_3 \to R_3 - R_2$: $$\left(\begin{array}{rr|r} 1 & 1 & 1 \\ 1 & -1 & 3 \\ 2 & 0 & 5 \end{array}\right) \to \left(\begin{array}{rr|r} 1 & 1 & 1 \\ 0 & -2 & 2 \\ 0 & -2 & 3 \end{array}\right) \to \left(\begin{array}{rr|r} 1 & 1 & 1 \\ 0 & -2 & 2 \\ 0 & 0 & 1 \end{array}\right).$$ The left part has 2 pivots, so $\rank(A) = 2$, but the last row puts a pivot after the bar, so $\rank(A|b) = 3$. The ranks differ: no solution. (That last row says $0 = 1$.)
Three lines in the plane. The first two cross at $(2, -1)$, and the third one, $x = 2.5$, walks right past it:
Put the two theorems together and you can count solutions without solving anything.
To count the solutions of an $m \times n$ system, compare ranks, not $m$ and $n$. $\rank(A) \ne \rank(A|b)$: none. $\rank(A) = \rank(A|b) = n$: exactly one. $\rank(A) = \rank(A|b) < n$: infinitely many. So with more unknowns than equations you never get exactly one (the rank can't reach $n$), but with more equations than unknowns you can still get infinitely many, when the rank is low.
When there are infinitely many solutions, general = particular + homogeneous says what they look like: one solution plus every solution of $Ax = 0$, and the homogeneous part has $n - \rank(A)$ parameters.
Your turn: row reduce and count.
Loading practice engine...
For numbers, $5x = 10$ is solved by multiplying both sides by $\frac{1}{5}$, the number that undoes multiplying by $5$. The matrix version of $\frac{1}{5}$ has to undo $A$ from either side.
An $n \times n$ matrix $A$ is invertible if there is an $n \times n$ matrix $B$ with $AB = I$ and $BA = I$. Then $B$ is the inverse of $A$, written $A^{-1}$.
Only square matrices get inverses here, and a matrix can't have two: if $B$ and $C$ both work, then $B = BI = B(AC) = (BA)C = IC = C$. That's what lets us say the inverse.
Only square matrices get inverses here, and there's never more than one: an undo button can only send each landing spot back to where it came from. (The one-line algebra: if $B$ and $C$ both undo $A$, then $B = BI = B(AC) = (BA)C = IC = C$.)
An inverse plays a matrix backward. Here's $A = \begin{pmatrix} 2 & 1 \\ 1 & 1 \end{pmatrix}$ moving the plane, and $A^{-1} = \begin{pmatrix} 1 & -1 \\ -1 & 2 \end{pmatrix}$ bringing it back, then a matrix with no way back:
The second matrix, $\begin{pmatrix} 1 & 2 \\ 2 & 4 \end{pmatrix}$, flattens the whole plane onto the line $y = 2x$. Two different points land on the same spot, $(2, -1)$ and $(0, 0)$ both go to $(0, 0)$, so no matrix could know which one to send that spot back to. For $2 \times 2$ matrices there's a formula that decides it all at once.
The matrix $A = \begin{pmatrix} a & b \\ c & d \end{pmatrix}$ is invertible if and only if $ad - bc \ne 0$, and then $$A^{-1} = \frac{1}{ad - bc}\begin{pmatrix} d & -b \\ -c & a \end{pmatrix}.$$
Let $M = \begin{pmatrix} d & -b \\ -c & a \end{pmatrix}$. Multiplying out, $$AM = \begin{pmatrix} ad - bc & -ab + ba \\ cd - dc & -cb + da \end{pmatrix} = (ad - bc)I, \qquad MA = \begin{pmatrix} da - bc & db - bd \\ -ca + ac & -cb + ad \end{pmatrix} = (ad - bc)I.$$ If $ad - bc \ne 0$, divide by it: $\frac{1}{ad - bc}M$ undoes $A$ from both sides. If $ad - bc = 0$, then $AM = 0$. Were $A$ invertible, multiplying on the left by $A^{-1}$ would give $M = 0$, so $a = b = c = d = 0$ and $A = 0$, which isn't invertible ($0 \cdot B = 0 \ne I$). So $A$ has no inverse. $\blacksquare$
The formula you can check by multiplying: $\begin{pmatrix} a & b \\ c & d \end{pmatrix}\begin{pmatrix} d & -b \\ -c & a \end{pmatrix} = \begin{pmatrix} ad - bc & 0 \\ 0 & ad - bc \end{pmatrix}$, and the other order gives the same. Dividing by $ad - bc$ turns that into $I$. When $ad - bc = 0$, $ad = bc$ says the columns $(a, c)$ and $(b, d)$ point along one line through $0$, so $A$ flattens the plane onto that line, just like the matrix in the figure above, and nothing can unflatten it. The full proof is in the n version.
Remember it as: swap the diagonal, negate the other two, divide by $ad - bc$.
Invert $A = \begin{pmatrix} 3 & 1 \\ 5 & 2 \end{pmatrix}$ and $B = \begin{pmatrix} 2 & -3 \\ -4 & 6 \end{pmatrix}$, if you can.
For $A$, $ad - bc = 6 - 5 = 1$, so $$A^{-1} = \begin{pmatrix} 2 & -1 \\ -5 & 3 \end{pmatrix}.$$ Check: $AA^{-1} = \begin{pmatrix} 6 - 5 & -3 + 3 \\ 10 - 10 & -5 + 6 \end{pmatrix} = I$. For $B$, $ad - bc = 12 - 12 = 0$, so $B$ has no inverse. You can see why: row 2 is $-2$ times row 1, so $\rank(B) = 1$.
The number $ad - bc$ has a picture: it's the area of the parallelogram the unit square lands on (the Determinants page proves it). $A$ turns the square into a long, thin parallelogram, but its area is still $1$; $B$ squashes it flat:
Undoing a chain of moves works like taking off your shoes and socks: the last thing you put on comes off first.
Let $A$ and $B$ be invertible $n \times n$ matrices. Then:
(a) $AB$ is invertible, and $(AB)^{-1} = B^{-1}A^{-1}$;
(b) $A^{-1}$ is invertible, and $(A^{-1})^{-1} = A$;
(c) $A^t$ is invertible, and $(A^t)^{-1} = (A^{-1})^t$.
Each part is a check that a candidate multiplies to $I$ on both sides. (a) $(AB)(B^{-1}A^{-1}) = A(BB^{-1})A^{-1} = AIA^{-1} = AA^{-1} = I$, and $(B^{-1}A^{-1})(AB) = B^{-1}(A^{-1}A)B = B^{-1}B = I$. (b) $A^{-1}A = I$ and $AA^{-1} = I$ say exactly that $A$ undoes $A^{-1}$. (c) Transposing $AA^{-1} = I$ with the product rule $(XY)^t = Y^tX^t$ gives $(A^{-1})^tA^t = I^t = I$, and transposing $A^{-1}A = I$ gives $A^t(A^{-1})^t = I$. $\blacksquare$
This is a visual argument for (a), not a formal proof. $AB$ means "turn with $B$, then slide with $A$". To get back, you have to undo the slide first, while the F is still slid, and only then undo the turn:
So the undo is $B^{-1}A^{-1}$: $A^{-1}$ acts first (it's on the right, next to the arrow), then $B^{-1}$. Undoing in the original order, $A^{-1}B^{-1}$, would try to unturn an F that's still slid, and it lands somewhere else. Parts (b) and (c) are one-line checks: $A$ undoes $A^{-1}$, and transposing $AA^{-1} = I$ with rule (e) above gives $(A^{-1})^tA^t = I$. The full proof is in the n version.
Your turn: the formula, and what you can do with it.
Loading practice engine...
For $3 \times 3$ matrices and beyond, there's no formula worth memorizing. Instead, the inverse comes out of row reduction, once you see that every row operation is itself a matrix.
An $n \times n$ elementary matrix is the result of doing one elementary row operation to $I_n$.
In the plane, the three kinds are three familiar moves. A swap $R_1 \leftrightarrow R_2$ trades $x$ and $y$, which mirrors the plane across the line $y = x$. A scale $R_1 \to 2R_1$ stretches sideways. An add $R_1 \to R_1 + R_2$ slides each point sideways by its height. And each one has an undo:
In bigger sizes the recipe is the same: do the operation to $I$.
Write the $3 \times 3$ elementary matrices for $R_1 \leftrightarrow R_3$, for $R_2 \to 5R_2$, and for $R_3 \to R_3 - 2R_1$.
Do each operation to $I_3$: $$E_1 = \begin{pmatrix} 0 & 0 & 1 \\ 0 & 1 & 0 \\ 1 & 0 & 0 \end{pmatrix}, \quad E_2 = \begin{pmatrix} 1 & 0 & 0 \\ 0 & 5 & 0 \\ 0 & 0 & 1 \end{pmatrix}, \quad E_3 = \begin{pmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ -2 & 0 & 1 \end{pmatrix}.$$ In $E_3$, the $-2$ sits in row 3 (the row that changes) and column 1 (the row being added).
Here are all three acting on the unit cube: $E_1$ mirrors it across the slanted plane $x = z$, $E_2$ stretches it five times longer along $y$, and $E_3$ tips its top down along the $x$ direction without changing its volume:
Here's why anyone cares: doing a row operation to $A$ is the same as multiplying $A$ on the left by its elementary matrix.
Let $E$ be the $m \times m$ elementary matrix of a row operation. For every $m \times n$ matrix $A$, $EA$ is the result of doing that row operation to $A$.
By the entry rule, $(EA)_{ij} = \sum_k E_{ik}A_{kj}$, so row $i$ of $EA$ is $\sum_k E_{ik}\,(\text{row } k \text{ of } A)$: a combination of the rows of $A$, with row $i$ of $E$ as the weights. Row $i$ of $I$ has a $1$ in spot $i$ and $0$s elsewhere, so it picks out row $i$ of $A$ unchanged. The operation changes at most two rows of $I$. For a swap $R_i \leftrightarrow R_j$, row $i$ of $E$ is row $j$ of $I$ and the other way around, so $EA$ has rows $i$ and $j$ of $A$ swapped. For a scale $R_i \to kR_i$, row $i$ of $E$ is $k$ times row $i$ of $I$, so row $i$ of $EA$ is $k$ times row $i$ of $A$. For an add $R_i \to R_i + kR_j$, row $i$ of $E$ has a $1$ in spot $i$ and a $k$ in spot $j$, so row $i$ of $EA$ is row $i$ of $A$ plus $k$ times row $j$. Every other row of $EA$ is the same row of $A$. $\blacksquare$
This is a visual argument, not a formal proof. Row $i$ of $EA$ is row $i$ of $E$ times $A$, so each row of $E$ is a recipe: "take this many of $A$'s row 1, this many of row 2, this many of row 3". Here's $E_3 = \begin{pmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ -2 & 0 & 1 \end{pmatrix}$ from the example above cooking the rows of the rank example's $A$:
Rows 1 and 2 of $E_3$ are rows of $I$, recipes that say "keep your own row". Row 3 says "$-2$ of row 1, plus your own row", which is $R_3 \to R_3 - 2R_1$. The same reading works for a swap (each recipe takes the other row) and a scale (one recipe takes $k$ of its own row). The full proof is in the n version.
Since every row operation can be undone, so can every elementary matrix.
Every elementary matrix is invertible, and its inverse is the elementary matrix of the operation that undoes it.
Every row operation has an undo of the same kind: a swap undoes itself, $R_i \to kR_i$ is undone by $R_i \to \frac{1}{k}R_i$ (allowed since $k \ne 0$), and $R_i \to R_i + kR_j$ by $R_i \to R_i - kR_j$. Let $E$ be the elementary matrix of an operation and $F$ that of its undo. By the theorem above, $FE$ is the undo done to $E$, which is the undo done to the operation done to $I$, which is $I$. The same argument with the roles traded (the operation undoes its undo) gives $EF = I$. $\blacksquare$
The three moves in the plane figure above each come with their undo: mirror again, shrink back, slide back. Undoing an operation and then doing it (or the other way around) leaves $I$ alone, and by the recipe theorem that's what multiplying the two matrices does. The full proof is in the n version.
That makes their inverses the easiest ones on the page.
Inverting an elementary matrix is instant: undo the operation. For an add, flip the sign of the off-diagonal number. For a scale by $k$, put $\frac{1}{k}$ there instead. A swap is its own inverse. Don't reach for the $2 \times 2$ formula or a row reduction here.
Row reduction is a string of row operations, so it's a string of elementary matrices.
Write the reduction of $A = \begin{pmatrix} 1 & 0 & 1 \\ 2 & 1 & 3 \\ 1 & -1 & 0 \end{pmatrix}$ from the rank example as a product.
The operations were $R_2 \to R_2 - 2R_1$, $R_3 \to R_3 - R_1$, and $R_3 \to R_3 + R_2$, with elementary matrices $$E_1 = \begin{pmatrix} 1 & 0 & 0 \\ -2 & 1 & 0 \\ 0 & 0 & 1 \end{pmatrix}, \quad E_2 = \begin{pmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ -1 & 0 & 1 \end{pmatrix}, \quad E_3 = \begin{pmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 1 & 1 \end{pmatrix}.$$ The first operation touches $A$ first, so its matrix sits next to $A$: $$E_3E_2E_1A = \begin{pmatrix} 1 & 0 & 1 \\ 0 & 1 & 1 \\ 0 & 0 & 0 \end{pmatrix}.$$
Every one of those $E$'s has its multiplier below the diagonal. Undo them, $A = E_1^{-1}E_2^{-1}E_3^{-1}U$, and the three inverses multiply to one lower triangular matrix $L$, holding the multipliers $2$, $1$, $-1$ below its diagonal. That's $A = LU$, the subject of the next page, LU Factorization.
Your turn: an operation in, a matrix out.
Loading practice engine...
Suppose row operations with matrices $E_1, \ldots, E_k$ turn $A$ into $I$, so $E_k \cdots E_1A = I$. Then the product $E_k \cdots E_1$ undoes $A$, and it's exactly what you get by doing those same operations to $I$. To do both at once, glue $I$ onto $A$ and row reduce the pair.
Let $A$ be an $n \times n$ matrix. If row operations turn $(A|I)$ into $(I|B)$, then $A$ is invertible and $A^{-1} = B$.
Let $E_1, \ldots, E_k$ be the elementary matrices of the operations and $P = E_k \cdots E_1$. Each operation acts on whole rows of $(A|I)$, so by the recipe theorem the left half becomes $PA$ and the right half $PI = P$. So $PA = I$ and $B = P$. Each $E_i$ is invertible, so $P$ is invertible with $P^{-1} = E_1^{-1} \cdots E_k^{-1}$ (the product rule for inverses, used $k - 1$ times). Multiplying $PA = I$ on the left by $P^{-1}$ gives $A = P^{-1}$, so $AP = P^{-1}P = I$ as well. Hence $A$ is invertible and $A^{-1} = P = B$. $\blacksquare$
Each row operation is a move, and the left half records what the moves have done to $A$: when it reads $I$, the moves have exactly undone $A$. The right half started as $I$ and has had the same moves done to it, so it holds all of those moves rolled into one matrix: the undo. The first example below plays this out on the plane, and the full proof is in the n version.
Try it yourself: pick a row operation, and watch both halves change together.
Here is the recipe on a $2 \times 2$ you can check against the formula.
Find the inverse of $A = \begin{pmatrix} 1 & 1 \\ 2 & -1 \end{pmatrix}$ by row reduction.
Start with $(A|I)$, clear below the first pivot, scale the second row, and clear above it: $$\left(\begin{array}{rr|rr} 1 & 1 & 1 & 0 \\ 2 & -1 & 0 & 1 \end{array}\right) \xrightarrow{R_2 \to R_2 - 2R_1} \left(\begin{array}{rr|rr} 1 & 1 & 1 & 0 \\ 0 & -3 & -2 & 1 \end{array}\right) \xrightarrow{R_2 \to -\frac{1}{3}R_2} \left(\begin{array}{rr|rr} 1 & 1 & 1 & 0 \\ 0 & 1 & \frac{2}{3} & -\frac{1}{3} \end{array}\right) \xrightarrow{R_1 \to R_1 - R_2} \left(\begin{array}{rr|rr} 1 & 0 & \frac{1}{3} & \frac{1}{3} \\ 0 & 1 & \frac{2}{3} & -\frac{1}{3} \end{array}\right).$$ So $A^{-1} = \begin{pmatrix} \frac{1}{3} & \frac{1}{3} \\ \frac{2}{3} & -\frac{1}{3} \end{pmatrix}$. The formula agrees: $ad - bc = -1 - 2 = -3$, and $-\frac{1}{3}\begin{pmatrix} -1 & -1 \\ -2 & 1 \end{pmatrix}$ is the same matrix.
Here's what those three operations do to the picture. $A$ has bent the grid; each operation is a move of the plane, and the three of them bend it straight again, while the right half adds up the moves:
Now you do the straightening. A matrix has bent the grid; pick moves until the square grid is back, and the moves you used add up to its inverse.
The same recipe works in any size. Here is a $3 \times 3$.
Find the inverse of $A = \begin{pmatrix} 1 & 2 & 0 \\ 0 & 1 & 1 \\ 1 & 2 & 1 \end{pmatrix}$.
Start with $(A|I_3)$ and do $R_3 \to R_3 - R_1$: $$\left(\begin{array}{rrr|rrr} 1 & 2 & 0 & 1 & 0 & 0 \\ 0 & 1 & 1 & 0 & 1 & 0 \\ 0 & 0 & 1 & -1 & 0 & 1 \end{array}\right).$$ The left half is already upper triangular, so work upward: $R_2 \to R_2 - R_3$, then $R_1 \to R_1 - 2R_2$: $$\left(\begin{array}{rrr|rrr} 1 & 0 & 0 & -1 & -2 & 2 \\ 0 & 1 & 0 & 1 & 1 & -1 \\ 0 & 0 & 1 & -1 & 0 & 1 \end{array}\right), \quad\text{so}\quad A^{-1} = \begin{pmatrix} -1 & -2 & 2 \\ 1 & 1 & -1 \\ -1 & 0 & 1 \end{pmatrix}.$$ Check one entry of $AA^{-1}$: row 1 of $A$ times column 1 of $A^{-1}$ is $-1 + 2 + 0 = 1$. Good.
In space, $A$ turns the unit cube into a slanted box, and the three operations square it back up, one move at a time:
Sometimes the first pivot spot holds a $0$, and a swap has to go first.
Find the inverse of $A = \begin{pmatrix} 0 & 3 & -1 \\ 1 & 0 & 1 \\ 1 & -1 & 0 \end{pmatrix}$.
$R_1 \leftrightarrow R_2$ puts a pivot in the corner, and $R_3 \to R_3 - R_1$ clears below it. Then $R_2 \leftrightarrow R_3$ and $R_2 \to -R_2$ give a second pivot of $1$ with no fractions yet: $$\left(\begin{array}{rrr|rrr} 1 & 0 & 1 & 0 & 1 & 0 \\ 0 & 1 & 1 & 0 & 1 & -1 \\ 0 & 3 & -1 & 1 & 0 & 0 \end{array}\right).$$ $R_3 \to R_3 - 3R_2$ leaves $(0 \ 0 \ -4 \mid 1 \ -3 \ 3)$, and $R_3 \to -\frac{1}{4}R_3$ makes the last pivot $1$. Clearing above it ($R_2 \to R_2 - R_3$, $R_1 \to R_1 - R_3$): $$A^{-1} = \begin{pmatrix} \frac{1}{4} & \frac{1}{4} & \frac{3}{4} \\ \frac{1}{4} & \frac{1}{4} & -\frac{1}{4} \\ -\frac{1}{4} & \frac{3}{4} & -\frac{3}{4} \end{pmatrix}.$$ Check: row 1 of $A$ times column 3 of $A^{-1}$ is $0 - \frac{3}{4} + \frac{3}{4} = 0$, as an off-diagonal entry of $I$ should be.
Long reductions like that one leave plenty of room for slips.
Every operation goes on the whole augmented row. Doing it to the left half and forgetting the right half is the classic way to get a wrong inverse that looks perfectly reasonable. And it's row operations only: a column operation on $(A|I)$ mixes the two halves and breaks everything.
The recipe can also fail, and when it does, it tells you.
Try to invert $A = \begin{pmatrix} 1 & 1 \\ 2 & 2 \end{pmatrix}$.
One operation, $R_2 \to R_2 - 2R_1$, and the left half has a row of zeros: $$\left(\begin{array}{rr|rr} 1 & 1 & 1 & 0 \\ 0 & 0 & -2 & 1 \end{array}\right).$$ No row operation can put a pivot back in that row without wrecking the one above, so the left half never becomes $I$. $\rank(A) = 1$, and $A$ has no inverse.
$A$ squashes the whole plane onto the line $y = 2x$. Row operations are all moves you can undo, so they can never turn a flat line back into a plane:
That last example is the general story. For a square matrix, everything on this page is one fact told several ways.
For an $n \times n$ matrix $A$, these statements are all true or all false:
(a) $A$ is invertible;
(b) $\rank(A) = n$;
(c) the reduced row echelon form of $A$ is $I$;
(d) $Ax = b$ has exactly one solution for every $b$ in $\R^n$;
(e) $Ax = 0$ has only the solution $x = 0$;
(f) $A$ is a product of elementary matrices.
(a) $\implies$ (d): $x = A^{-1}b$ is a solution, since $A(A^{-1}b) = b$, and any solution satisfies $x = A^{-1}(Ax) = A^{-1}b$, so it's the only one. (d) $\implies$ (e): take $b = 0$. (e) $\implies$ (b): by the free-variable theorem, $Ax = 0$ has only $x = 0$ exactly when $\rank(A) = n$. (b) $\implies$ (c): the reduced form of an $n \times n$ matrix with $n$ pivots has a pivot in every row and every column, each a $1$ with $0$s around it, so it's $I$. (c) $\implies$ (f): if $E_k \cdots E_1A = I$, multiplying on the left by $E_1^{-1} \cdots E_k^{-1}$ gives $A = E_1^{-1} \cdots E_k^{-1}$, and each $E_i^{-1}$ is elementary. (f) $\implies$ (a): elementary matrices are invertible, and a product of invertible matrices is invertible. $\blacksquare$
For a $2 \times 2$ or $3 \times 3$ matrix $A$ (size $n = 2$ or $3$), these statements are all true or all false:
(a) $A$ is invertible;
(b) $\rank(A) = n$;
(c) $A$ row reduces to $I$;
(d) $Ax = b$ has exactly one solution for every $b$;
(e) $A$ flattens nothing: the only arrow it sends to $0$ is $0$ itself;
(f) $A$ is a chain of elementary moves.
This is a visual argument, not a formal proof. Here's the unit cube under three matrices, of rank 3, 2, and 1:
At rank 3 the cube lands on a real box with room inside: every direction survives, row reduction reaches $I$, and playing those moves backward builds $A$ out of elementary moves (that's (f)), each of which can be undone (that's (a)). At rank 2 the box is squashed flat onto a plane: a whole line of arrows lands on $0$, landing spots off the plane can't be reached at all, and no move can unflatten it. At rank 1 it's squashed onto a line. So "nothing gets flattened" and "the moves can be played backward" are the same thing. The full proof is in the n version.
One consequence saves work every time you check an inverse.
For square matrices, checking one side is enough: if $AB = I$, then $BA = I$ too. (If $Bx = 0$ then $x = ABx = 0$, so $B$ is invertible by (e), and $A = AB B^{-1} = B^{-1}$.) That's why you only check $AA^{-1}$ in the examples, not both orders. For non-square matrices this fails, which is one reason inverses are only for square ones.
For a $2 \times 2$ matrix, one number, $ad - bc$, decided whether the inverse exists. Every square matrix has a number like that, deciding everything in the theorem above at once: the determinant.
Your turn: a full $3 \times 3$ reduction.
Loading practice engine...
Why compute $A^{-1}$ at all, when row reducing $(A|b)$ solves one system just as fast? Because real problems rarely come with one $b$. The left side of a system usually describes a fixed machine, and the right side is this week's demand, so you solve the same $A$ over and over with new $b$'s. With $A^{-1}$ in hand, each new answer is one multiplication.
A workshop builds stools and benches. A stool takes $1$ hour on the saw and $1$ hour of sanding, and a bench takes $1$ hour on the saw and $2$ hours of sanding. How many of each should it build to use exactly the hours it has, for each of these weeks: $40$ saw hours and $60$ sanding hours; $45$ and $70$; $50$ and $65$?
With $x$ stools and $y$ benches, the hours are $$\begin{pmatrix} 1 & 1 \\ 1 & 2 \end{pmatrix}\begin{pmatrix} x \\ y \end{pmatrix} = \begin{pmatrix} \text{saw hours} \\ \text{sanding hours} \end{pmatrix}.$$ Here $ad - bc = 2 - 1 = 1$, so $A^{-1} = \begin{pmatrix} 2 & -1 \\ -1 & 1 \end{pmatrix}$, and every week is one multiplication: $$A^{-1}\begin{pmatrix} 40 \\ 60 \end{pmatrix} = \begin{pmatrix} 20 \\ 20 \end{pmatrix}, \qquad A^{-1}\begin{pmatrix} 45 \\ 70 \end{pmatrix} = \begin{pmatrix} 20 \\ 25 \end{pmatrix}, \qquad A^{-1}\begin{pmatrix} 50 \\ 65 \end{pmatrix} = \begin{pmatrix} 35 \\ 15 \end{pmatrix}.$$ Check the first week: $20 + 20 = 40$ saw hours and $20 + 40 = 60$ sanding hours.
The inverse also answers "what if" questions without solving anything. One extra saw hour changes $b$ by $(1, 0)$, so it changes the plan by $A^{-1}(1, 0)$, the first column of $A^{-1}$: two more stools and one fewer bench. That's not obvious from the story, but it's right: the extra hour is only usable if some sanding time moves from benches to stools. Engineers call this sensitivity analysis, and it's how you find out which numbers in a model have to be measured carefully. Here it is as two lines and their crossing point, with the saw hours sliding from $40$ to $45$:
The toolkit for this page: $AB$ is $A$ acting on each column of $B$, and the order matters. The rank is the number of pivots, and comparing $\rank(A)$ with $\rank(A|b)$ and with $n$ tells you how many solutions $Ax = b$ has. A square matrix is invertible exactly when its rank is full, and row reducing $(A|I)$ to $(I|A^{-1})$ finds the inverse, because every row operation is multiplication by an elementary matrix.
Every question so far came right after its idea, which is a hint a quiz never gives you. Down here the hint is gone. First, true or false: a claim from anywhere on this page, and you decide whether it holds.
Loading practice engine...
Next, a mixed set: every question type on this page, shuffled. Figuring out which idea a question needs is half the work. Type a matrix row by row, with commas between entries and a semicolon between rows, like 1, 0, 0; 0, 1, 0; -2, 0, 1.
Loading practice engine...
Made it this far? Time for something with a little heat. The coefficient matrix below has rank 2 no matter what, because its third row is built from the first two. The right-hand side has a $k$ in it, and exactly one value of $k$ keeps $\rank(A|b)$ at 2.
Loading practice engine...