A GPS fix, a sensor reading, a fitted curve: in each, you have a vector that doesn't lie where you want it, in some subspace $\W$ of $\R^n$ that holds the "allowed" answers, and you want the vector of $\W$ that comes closest. The answer is always the same move. Drop the vector straight down onto $\W$, at a right angle, and take the foot. That foot is the orthogonal projection, and this page builds the tools to compute it: orthogonal complements, the split of a vector into a piece in $\W$ and a piece square to it, and a matrix $P$ that does the dropping for you.
Hold a pencil above a desk under a lamp that shines straight down. Its shadow on the desk is the closest the desk can get to the pencil's tip, and the light ray from tip to shadow hits the desk at a right angle. That picture is this whole page. The desk becomes a plane (or a line) through $0$, the pencil becomes an arrow, and "straight down" means square to the plane, whichever way the plane is tilted. We'll find shadows by hand, build a matrix that casts them, and use the right angle to answer "how close?" questions.
You've already cast one kind of shadow: the projection of $v$ onto $u$, $\proj_u v = \frac{v \cdot u}{u \cdot u}u$, the shadow of $v$ on the line through $u$. This page trades the line for a plane, or for any subspace.
Two things to notice in that loop. The shadow $p$ lives in the plane, and the leftover $x - p$ is square to the plane, not just to one arrow in it. When the plane tilts, both change, but those two facts never do.
"Square to the plane" means square to every arrow in it. Collect all the vectors with that property and you get a subspace of its own, which gets a name.
Let $\W$ be a subspace of $\R^n$. The orthogonal complement of $\W$ is $$\W^\perp = \{x \in \R^n : x \cdot w = 0 \text{ for every } w \in \W\}.$$
Let $\W$ be a line or a plane through $0$ in $\R^2$ or $\R^3$. The orthogonal complement of $\W$, written $\W^\perp$ and read "W perp", is the set of all arrows $x$ with $x \cdot w = 0$ for every arrow $w$ in $\W$.
You never have to test "every $w$". If $x$ is square to each arrow of a spanning set, it is square to every combination of them, since $x \cdot (c_1u_1 + c_2u_2) = c_1(x \cdot u_1) + c_2(x \cdot u_2)$. So finding $\W^\perp$ is solving a few dot-product equations. Here is a plane, the example Hefferon uses.
Find the orthogonal complement of the plane $\W$: $3x + 2y - z = 0$ in $\R^3$.
Two arrows spanning $\W$: set $x = 1, y = 0$ to get $z = 3$, and $x = 0, y = 1$ to get $z = 2$. So $\W = \span\{(1, 0, 3), (0, 1, 2)\}$, and $v = (v_1, v_2, v_3)$ is in $\W^\perp$ exactly when $$v_1 + 3v_3 = 0, \qquad v_2 + 2v_3 = 0.$$ One free unknown: $v_3 = t$ gives $v = t(-3, -2, 1)$. So $\W^\perp$ is the line through $(-3, -2, 1)$, which is the line through the plane's normal $(3, 2, -1)$.
That's no coincidence: the equation $3x + 2y - z = 0$ says $(3, 2, -1) \cdot (x, y, z) = 0$, so the plane is exactly the arrows square to $(3, 2, -1)$. The complement of a plane through $0$ is its normal line.
In the figure, an arrow $w$ sweeps all the way around inside the plane, and its dot product with $(3, 2, -1)$ never budges from $0$.
A line works the same way in reverse: the complement of the line through $(3, 2, -1)$ is the plane $3x + 2y - z = 0$. Complements come in pairs, and one of the pair is always easy to spot wrong.
In $\R^3$, the orthogonal complement of the $xy$-plane is the $z$-axis, not the $yz$-plane. The $yz$-plane looks perpendicular to the floor, like a wall, but it contains $(0, 1, 0)$, which also lies in the floor and is definitely not square to itself. Two planes in $\R^3$ can never be each other's complements: there's only room for $2 + 1$.
That "room for $2 + 1$" is the main fact about complements. The dimensions always fill up the whole space, and taking the complement twice gets you back where you started.
Let $\W$ be a subspace of $\R^n$. Then $\W^\perp$ is a subspace of $\R^n$, the only vector in both $\W$ and $\W^\perp$ is $0$, $$\dim \W + \dim \W^\perp = n,$$ and $(\W^\perp)^\perp = \W$.
If $\W = \{0\}$, then $\W^\perp = \R^n$ and everything is immediate. Otherwise pick a basis $w_1, \ldots, w_k$ of $\W$ and let $A$ be the $k \times n$ matrix with those rows, so $\row(A) = \W$ and $\operatorname{rank}(A) = k$. Entry $i$ of $Ax$ is $w_i \cdot x$, so $Ax = 0$ says $x$ is square to every basis vector, hence (by the remark before the example) to all of $\W$. So $\W^\perp = \N(A)$, which is a subspace of dimension $n - k$.
A vector in both is square to itself: $x \cdot x = 0$, so $x = 0$. For the last claim, every $w \in \W$ is square to everything in $\W^\perp$, so $\W \subseteq (\W^\perp)^\perp$. Both are subspaces of dimension $n - (n - k) = k$ (apply the count to $\W^\perp$), and a subspace inside another of the same dimension is all of it. $\blacksquare$
Let $\W$ be $\{0\}$, a line or plane through $0$, or all of $\R^3$. Then $\W^\perp$ is one of those too, the only arrow in both $\W$ and $\W^\perp$ is $0$, $$\dim \W + \dim \W^\perp = 3,$$ and the complement of $\W^\perp$ is $\W$ again. (In $\R^2$, the dimensions add to $2$.)
A picture can check every case here, because ℝ³ has only four kinds of subspace: $\{0\}$, lines through $0$, planes through $0$, and all of $\R^3$. The figure pairs each with its complement. A line's complement is the plane it sticks straight out of, a plane's is its normal line, and the two extremes pair with each other. In every panel the dimensions add to $3$, and swapping the roles of $\W$ and $\W^\perp$ just reads the same panel the other way, which is $(\W^\perp)^\perp = \W$. (In $\R^2$ it's the same story with $2$: a line's complement is the line square to it.)
The full proof is in the n version.
Put that next to the Four Fundamental Subspaces page. There, the row space and null space of a matrix turned out to be square to each other with dimensions adding to $n$. Square to each other means $\N(A) \subseteq \row(A)^\perp$, and the dimensions say there's no room for more, so $\row(A)^\perp = \N(A)$. In the same way, $\col(A)^\perp = \N(A^t)$. The four subspaces are two complementary pairs.
Your turn. In free response, type the entries separated by commas; any nonzero vector on the right line counts.
Loading practice engine...
Now the opening picture as a theorem. Every vector splits into a piece in $\W$ and a piece in $\W^\perp$, and there's only one way to do it.
Let $\W$ be a subspace of $\R^n$a line or plane through $0$. Every vector $x$ can be written in exactly one way as $$x = p + z \quad \text{with } p \in \W \text{ and } z \in \W^\perp.$$
Existence. Take a basis $w_1, \ldots, w_k$ of $\W$ and a basis $z_1, \ldots, z_{n-k}$ of $\W^\perp$ (the dimensions are the previous theorem). Together that's $n$ vectors, and they are independent: if $a_1w_1 + \cdots + a_kw_k + b_1z_1 + \cdots + b_{n-k}z_{n-k} = 0$, then $a_1w_1 + \cdots + a_kw_k = -(b_1z_1 + \cdots)$ lies in both $\W$ and $\W^\perp$, so it is $0$, and then every $a_i$ and $b_j$ is $0$ because each list is independent. So the $n$ vectors are a basis of $\R^n$. Write $x$ in it and group the terms: the $w$ part is $p$, the $z$ part is $z$.
Uniqueness. If $x = p + z = p' + z'$, then $p - p' = z' - z$ lies in $\W$ and in $\W^\perp$, so it is $0$. $\blacksquare$
There's always a split, because you can always drop straight down: a line through $x$ in the direction of the normal hits the plane at exactly one point. The figure is about the "exactly one way" part. Any point $q$ of the plane splits $x$ as $q + (x - q)$, but for almost every $q$ the rest $x - q$ leans. Only the foot of the drop makes the rest stand square to the plane. And if two feet $p$ and $p'$ both worked, the arrow $p - p'$ would lie in the plane and be square to it at the same time, so it would be square to itself, which only $0$ is.
The full proof is in the n version.
The piece in $\W$ is the star of the page, so it gets a name and a symbol.
Let $\W$ be a subspace of $\R^n$ and write $x = p + z$ with $p \in \W$ and $z \in \W^\perp$. The orthogonal projection of $x$ onto $\W$ is $\proj_{\W} x = p$.
Let $\W$ be a line or a plane through $0$. The orthogonal projection of $x$ onto $\W$, written $\proj_{\W} x$, is the point $p$ of $\W$ for which $x - p$ is square to every arrow in $\W$.
When $\W$ is the line through $u$, this is the old shadow: $v - \proj_u v$ is square to $u$, so $\proj_{\W} v = \proj_u v$. For a plane in $\R^3$, the complement is a line, which gives a slick way in: find the part along the normal and throw it away. Hefferon projects this vector onto this plane with a matrix (we'll get there); here's the shortcut.
Find the orthogonal projection of $x = (1, -1, 1)$ onto the plane $\W$: $x + z = 0$.
The normal is $n = (1, 0, 1)$, and $\W^\perp$ is its line. The piece of $x$ in $\W^\perp$ is the shadow of $x$ on $n$: $$z = \frac{x \cdot n}{n \cdot n}n = \frac{1 + 0 + 1}{2}(1, 0, 1) = (1, 0, 1).$$ Whatever is left is the piece in $\W$: $$\proj_{\W} x = x - z = (1, -1, 1) - (1, 0, 1) = (0, -1, 0).$$ Check: $(0, -1, 0)$ satisfies $0 + 0 = 0$, so it's on the plane, and $x - p = (1, 0, 1)$ is the normal, so it's square to the plane.
Watch it happen: the shadow on the normal grows, then gets lifted off, and what stays behind is $p$.
"Shadow on the normal, then subtract" only works because the complement of a plane in $\R^3$ is a single line. For a subspace whose complement is bigger, we need a way to project onto $\W$ directly. The easy case is when $\W$ comes with arrows square to each other.
On the Inner Products page, coordinates in an orthogonal basis came out as one shadow each, with no system to solve. That was for a vector already in the subspace. The same formula, used on a vector that isn't, casts its shadow.
Let $u_1, \ldots, u_k$ be an orthogonal basis of a subspace $\W$ of $\R^n$. Then for every $x$ in $\R^n$, $$\proj_{\W} x = \frac{x \cdot u_1}{u_1 \cdot u_1}u_1 + \cdots + \frac{x \cdot u_k}{u_k \cdot u_k}u_k.$$
Let $\W$ be the plane through $0$ spanned by two arrows $u_1, u_2$ that are square to each other ($u_1 \cdot u_2 = 0$). Then for every arrow $x$, $$\proj_{\W} x = \frac{x \cdot u_1}{u_1 \cdot u_1}u_1 + \frac{x \cdot u_2}{u_2 \cdot u_2}u_2.$$ (For a line through $u_1$, keep just the first term.)
Call the right side $p$; it is a combination of the $u_i$, so $p \in \W$. For each $j$, every $u_i \cdot u_j$ with $i \ne j$ is $0$, so $$(x - p) \cdot u_j = x \cdot u_j - \frac{x \cdot u_j}{u_j \cdot u_j}(u_j \cdot u_j) = 0.$$ So $x - p$ is square to every basis vector, hence to all of $\W$: $x - p \in \W^\perp$. By the uniqueness in the decomposition theorem, $p = \proj_{\W} x$. $\blacksquare$
Each term is the shadow of $x$ on one arrow's line. In the figure, $u_1$ and $u_2$ lie in the plane and are square to each other, like the edges of a floor tile. The two shadows, laid tip to tail, land exactly on the foot of the drop. Then the second arrow is slanted, still in the plane but no longer square to the first, and the same recipe misses. So the picture shows both halves of the theorem: the shadows add up, and they only add up for square arrows.
The full proof is in the n version.
Each coefficient is one dot product and one division. Here it is on a plane spanned by two arrows that happen to be square to each other.
Let $\W = \span\{u_1, u_2\}$ with $u_1 = (1, 1, 0)$ and $u_2 = (1, -1, 1)$. Find $\proj_{\W} x$ for $x = (3, 1, 4)$.
First check: $u_1 \cdot u_2 = 1 - 1 + 0 = 0$, so the theorem applies. Then $$\frac{x \cdot u_1}{u_1 \cdot u_1} = \frac{3 + 1}{2} = 2, \qquad \frac{x \cdot u_2}{u_2 \cdot u_2} = \frac{3 - 1 + 4}{3} = 2,$$ so $\proj_{\W} x = 2(1, 1, 0) + 2(1, -1, 1) = (4, 0, 2)$. The leftover is $x - p = (-1, 1, 2)$, and indeed $(-1, 1, 2) \cdot u_1 = 0$ and $(-1, 1, 2) \cdot u_2 = -1 - 1 + 2 = 0$.
Here are the two shadows stacked tip to tail, landing on $(4, 0, 2)$, with the leftover standing straight up out of the plane.
Leaving out the check $u_1 \cdot u_2 = 0$ is the classic way to go wrong here.
The shadow sum needs an orthogonal basis. With the non-orthogonal basis $u_1 = (1, 0, 0)$, $u_2 = (1, 1, 0)$ of the $xy$-plane and $x = (0, 1, 5)$, the true shadow is $(0, 1, 0)$, but the recipe gives $0 \cdot u_1 + \frac{1}{2}u_2 = \left(\frac12, \frac12, 0\right)$. Nothing on the page warns you; the numbers just come out wrong. Check the dot products of the basis first.
Drag the second arrow and see how the shortcut behaves when the arrows aren't square. The green point $p$ is the true shadow, already dropped into the plane; you're looking straight down at the plane.
Square arrows rebuild $p$ exactly. Nearly parallel arrows count the same part of $p$ twice, and an arrow square to $p$ counts nothing at all.
So what if your basis isn't orthogonal? The next page, Gram–Schmidt & QR, straightens any basis into an orthogonal one by removing shadows, one arrow at a time. The tool it uses is this page's projection.
Try one. In free response, type the entries separated by commas; fractions like 5/11 are fine.
Loading practice engine...
Projecting is linear: the shadow of a sum is the sum of the shadows, and doubling $x$ doubles its shadow. So projecting onto $\W$ is multiplication by a matrix. For a line it's quick to find. Write $\proj_u x = \frac{u \cdot x}{u \cdot u}u = \frac{1}{u \cdot u}u(u^tx)$, treating $u$ as a column, so $$P = \frac{1}{u \cdot u}uu^t.$$ For the line through $u = (3, 1)$, one of Hefferon's exercises, $uu^t$ has entries $u_iu_j$ and $u \cdot u = 10$: $$P = \frac{1}{10}\begin{pmatrix} 9 & 3 \\ 3 & 1 \end{pmatrix}.$$ Try it on $(1, 2)$: $P(1, 2) = \frac{1}{10}(15, 5) = \left(\frac32, \frac12\right)$, which is $\frac12(3, 1)$, on the line.
Here's what that matrix does to the whole plane at once: every point slides straight onto the line, and then it's stuck there.
For a bigger subspace, put a basis in the columns of a matrix $A$. The shadow is some combination $Ac$ of the columns, and the leftover has to be square to every column. That's one equation per column, and the equations package up neatly.
Let $\W$ be a subspace of $\R^n$ with basis $a_1, \ldots, a_k$, and let $A$ be the $n \times k$ matrix with those columns. Then $A^tA$ is invertible, and for every $x$ in $\R^n$,Let $\W$ be the plane through $0$ spanned by two arrows $a_1, a_2$ that aren't parallel, and let $A$ be the $3 \times 2$ matrix with those columns. Then $A^tA$ is invertible, and for every arrow $x$, $$\proj_{\W} x = Ac, \quad \text{where } A^tAc = A^tx.$$ In other words, $\proj_{\W} x = Px$ for the projection matrix $P = A(A^tA)^{-1}A^t$.
$A^tA$ is invertible. It is $k \times k$; suppose $A^tAc = 0$. Then $0 = c \cdot A^tAc = (Ac) \cdot (Ac) = \|Ac\|^2$, so $Ac = 0$. That says $c_1a_1 + \cdots + c_ka_k = 0$, and the columns are independent, so $c = 0$.
The formula. $\proj_{\W} x$ lies in $\W$, so it is $Ac$ for some $c$. The leftover $x - Ac$ is in $\W^\perp$ exactly when it is square to every column, $a_j \cdot (x - Ac) = 0$ for each $j$, and those $k$ equations are the entries of $A^t(x - Ac) = 0$, which is $A^tAc = A^tx$. Since $A^tA$ is invertible, $c = (A^tA)^{-1}A^tx$, and $\proj_{\W} x = A(A^tA)^{-1}A^tx$. $\blacksquare$
Hunt for the foot by trial. A point $q = c_1a_1 + c_2a_2$ wanders over the plane, and two meters read $a_1 \cdot (x - q)$ and $a_2 \cdot (x - q)$. The leftover is square to the plane exactly when it's square to both spanning arrows, so we want both meters at $0$ at the same moment, and that happens at just one point. Writing "meter $1 = 0$, meter $2 = 0$" out in numbers gives two equations for $c_1, c_2$, and that pair of equations is $A^tAc = A^tx$: row $j$ of $A^t$ is $a_j$.
The full proof, including why the equations always have exactly one solution, is in the n version.
Notice what this buys: no orthogonal basis needed. Any basis goes in the columns. Here's the plane from the figure, with numbers.
Project $x = (1, 2, 3)$ onto $\W = \span\{(1, 0, 1), (1, 1, 0)\}$.
The spanning vectors aren't square to each other ($(1, 0, 1) \cdot (1, 1, 0) = 1$), so use the theorem. With $A = \begin{pmatrix} 1 & 1 \\ 0 & 1 \\ 1 & 0 \end{pmatrix}$, $$A^tA = \begin{pmatrix} 2 & 1 \\ 1 & 2 \end{pmatrix}, \qquad A^tx = \begin{pmatrix} 1 + 3 \\ 1 + 2 \end{pmatrix} = \begin{pmatrix} 4 \\ 3 \end{pmatrix}.$$ Solve $2c_1 + c_2 = 4$, $c_1 + 2c_2 = 3$: $c_1 = \frac53$, $c_2 = \frac23$. So $$\proj_{\W} x = \tfrac53(1, 0, 1) + \tfrac23(1, 1, 0) = \left(\tfrac73, \tfrac23, \tfrac53\right).$$ Check: $x - p = \left(-\frac43, \frac43, \frac43\right)$, and its dot products with $(1, 0, 1)$ and $(1, 1, 0)$ are both $0$.
We solved for $c$ without ever writing $(A^tA)^{-1}$, which is how you should do it by hand for one vector. If you'll project many vectors onto the same $\W$, it pays to build $P$ once. Hefferon does exactly that for the plane $x + z = 0$ from the earlier example.
Find the projection matrix onto the plane $\W$: $x + z = 0$, and use it on $x = (1, -1, 1)$.
A basis: $(0, 1, 0)$ and $(1, 0, -1)$, so $A = \begin{pmatrix} 0 & 1 \\ 1 & 0 \\ 0 & -1 \end{pmatrix}$ and $A^tA = \begin{pmatrix} 1 & 0 \\ 0 & 2 \end{pmatrix}$, with inverse $\begin{pmatrix} 1 & 0 \\ 0 & \frac12 \end{pmatrix}$. Then $$P = A(A^tA)^{-1}A^t = \begin{pmatrix} \frac12 & 0 & -\frac12 \\ 0 & 1 & 0 \\ -\frac12 & 0 & \frac12 \end{pmatrix}.$$ And $P(1, -1, 1) = \left(\frac12 - \frac12, -1, -\frac12 + \frac12\right) = (0, -1, 0)$, the same answer the normal shortcut gave.
A matrix that, when you multiply it by itself, just shrugs and gives you itself back. Unbothered. Doing it twice achieves nothing more than doing it once. Same, $P$. Same.
That $P$ is symmetric, and if you square it you get it back. Both are always true.
Let $P$ be the projection matrix onto a subspace of $\R^n$a line or plane through $0$ $\W$. Then $P^2 = P$ and $P^t = P$. Also, $I - P$ is the projection matrix onto $\W^\perp$.
With $P = A(A^tA)^{-1}A^t$: $P^2 = A(A^tA)^{-1}(A^tA)(A^tA)^{-1}A^t = A(A^tA)^{-1}A^t = P$. For the transpose, $(A^tA)^t = A^tA$, so its inverse is symmetric too, and $P^t = (A^t)^t\left((A^tA)^{-1}\right)^tA^t = A(A^tA)^{-1}A^t = P$.
Finally, $(I - P)x = x - \proj_{\W} x$ is the $\W^\perp$ piece of $x$. Since $(\W^\perp)^\perp = \W$, the decomposition of $x$ relative to $\W^\perp$ is the same split read the other way, so $(I - P)x = \proj_{\W^\perp} x$. $\blacksquare$
$P^2 = P$ says the shadow of a shadow is itself: once an arrow is lying in the plane, dropping it again does nothing. $P^t = P$ is sneakier. For any matrix, $P^t$ is the matrix with $Px \cdot y = x \cdot P^ty$, so $P^t = P$ says $Px \cdot y = x \cdot Py$. The figure swings two arrows around and keeps both numbers on screen: they always agree, because each one only measures the parts of $x$ and $y$ that lie in the plane (the parts square to the plane contribute $0$ either way). And $I - P$ sends $x$ to $x - Px$, the leftover, which is the shadow on $\W^\perp$.
The full proof is in the n version.
Projection onto an axis was one of the basic moves there, with matrix $\begin{pmatrix} 1 & 0 \\ 0 & 0 \end{pmatrix}$. It passes both tests: it squares to itself and it's symmetric. Now any line or plane through $0$ can be the axis.
Two questions. The first wants a whole matrix: in free response, type the rows separated by semicolons, like 9/10, 3/10; 3/10, 1/10. The second wants a shadow on a plane whose spanning vectors aren't square to each other.
Loading practice engine...
Loading practice engine...
Hefferon pictures the projection a second way: stand on the tip of $x$ holding a rope looped over the plane, and pull. The loop slides until the rope is tight, and a tight rope hangs at a right angle, right down to the foot. So the projection is also the point of $\W$ nearest to $x$.
Let $\W$ be a subspace of $\R^n$a line or plane through $0$ and $x$ any vector. Then $\proj_{\W} x$ is the point of $\W$ closest to $x$: for every $w \ne \proj_{\W} x$ in $\W$, $$\|x - \proj_{\W} x\| < \|x - w\|.$$
Write $p = \proj_{\W} x$ and $x - w = (x - p) + (p - w)$. Here $x - p \in \W^\perp$ and $p - w \in \W$, so the two pieces are orthogonal and $$\|x - w\|^2 = (x - p + p - w) \cdot (x - p + p - w) = \|x - p\|^2 + 2(x - p) \cdot (p - w) + \|p - w\|^2 = \|x - p\|^2 + \|p - w\|^2.$$ If $w \ne p$, the last term is positive, so $\|x - w\| > \|x - p\|$. $\blacksquare$
It's the Pythagorean theorem from geometry class. Pick any point $w$ of the plane. The points $x$, $p$, $w$ make a right triangle: the drop $x - p$ is square to the whole plane, so in particular to the side $p - w$, which lies in the plane. The rope $x - w$ is the hypotenuse, and a hypotenuse is longer than either leg. The figure lets $w$ wander and keeps the three squared lengths on screen; the rope only shrinks to the drop when $w$ lands on $p$.
The full proof is in the n version.
So the length of the drop is the honest distance from $x$ to $\W$: we write $\dist(x, \W) = \|x - \proj_{\W} x\|$. For a plane in $\R^3$, the drop is the shadow on the normal, which is the point-to-plane distance from Lines & Planes. For a line, you can skip the subtraction and use Pythagoras directly.
How far is $x = (2, 1, 3)$ from the line through $0$ and $u = (1, 1, 1)$?
The shadow is $\proj_u x = \frac{6}{3}(1, 1, 1) = (2, 2, 2)$, so the leftover is $x - (2, 2, 2) = (0, -1, 1)$ and the distance is $\|(0, -1, 1)\| = \sqrt{2}$.
Or by Pythagoras, without finding the shadow: $\|x\|^2 = 14$, and the shadow has squared length $\frac{(x \cdot u)^2}{u \cdot u} = \frac{36}{3} = 12$, so the leftover has squared length $14 - 12 = 2$. Same $\sqrt{2}$.
Now find the closest point yourself. Drag $w$ around on the plane; the rope from $x$ is the dashed orange line.
The closest point is the foot of the drop, whether $x$ is above the plane or below it. And all the points at one fixed distance form a circle around the foot, because only the leg $\|p - w\|$ can change.
Distance means the length of the leftover $x - \proj_{\W} x$, not of the shadow. The shadow's length is how far along $\W$ the point reaches, which is a different (and also useful) number. If your "distance" comes out bigger than $\|x\|$, you've measured something else: the drop is never longer than $x$ itself.
In free response, type a number; square roots are fine, like 2sqrt(3), or a decimal to two places.
Loading practice engine...
Hefferon closes his section on lines with a submarine, and it's a good one. A sub sits at the origin of a chart measured in miles. A ship is sailing along the line $y = 3x + 2$. The sub's torpedoes reach half a mile. If the sub stays put, does the ship ever come in range?
Find the ship's closest approach to the origin along $y = 3x + 2$, and decide whether it comes within $\frac12$ mile.
The course doesn't pass through $0$, so it isn't a subspace yet. Slide the whole chart down $2$ miles: the course becomes the line $y = 3x$, through $0$ in the direction $u = (1, 3)$, and the sub moves to $v = (0, -2)$. Now project: $$\proj_u v = \frac{v \cdot u}{u \cdot u}u = \frac{-6}{10}(1, 3) = \left(-\tfrac35, -\tfrac95\right).$$ The leftover is $v - \proj_u v = \left(\frac35, -\frac15\right)$, with length $\sqrt{\frac{9}{25} + \frac{1}{25}} = \frac{\sqrt{10}}{5} \approx 0.63$ miles. That's more than $\frac12$: the ship never comes in range. (Slide back up $2$ to find where the closest approach happens on the real chart: $\left(-\frac35, \frac15\right)$.)
Here's the chart in motion, unshifted. Watch the sight line from the sub as the ship goes by: it's shortest exactly when it meets the course at a right angle.
The trick of sliding the chart is worth keeping: a line or plane that misses $0$ isn't a subspace, but shifting by one of its points makes it one, and distances don't change when everything slides together.
Here's the big application. A system $Ax = b$ with no solution has $b$ sitting off the column space, the subspace of all outputs $Ax$. The best you can do is the output closest to $b$, which is $\proj_{\col(A)} b$. That's Least Squares & Regression, and it's how a line gets fitted to data.
Every question so far came right after its idea, which is a hint a quiz never gives you. Down here the hint is gone. First, true or false: a claim from anywhere on this page, and you decide whether it holds.
Loading practice engine...
Next, a mixed set: every question type on this page, shuffled. Type vectors as entries separated by commas.
Loading practice engine...
Made it this far? Time for something with a little heat: a mirror is a projection that went too far. In free response, type the entries separated by commas; fractions are fine.
Loading practice engine...