The dot product gave $\R^n$ lengths, angles, and right angles, and every one of those facts came from three of its properties: it splits over sums, it doesn't care about order, and a vector dotted with itself is positive. This page gives any product with those properties a name, an inner product, because engineers meet other ones all the time: a measurement in meters and one in seconds shouldn't count equally, so they get weights. Then it proves the two inequalities every length obeys, and finds the payoff of right angles: with a basis of mutually perpendicular vectors, coordinates come for free.
Every arrow casts a shadow on every line, and the dot product is that shadow measured and multiplied. This page squeezes two famous facts out of that one picture: a shadow is never longer than the arrow that casts it, and a detour is never shorter than the straight path. Then it finds the payoff of right angles. When three arrows meet each other squarely, finding the coordinates of a point takes three dot products and no system of equations at all.
On Dot & Cross Products, the dot product $u \cdot v$ came with three rulesthree rules, a length $\|u\| = \sqrt{u \cdot u}$, and the projection of one vector onto another: its shadow.
Here's the picture this whole page runs on. Hold $u$ still and turn $v$ all the way around. The shadow of $u$ on $v$'s line grows and shrinks, but it never gets longer than $u$.
The orange bar is $|u \cdot v|$ and the purple bar is $\|u\| \, \|v\|$. They touch only when $u$ and $v$ line up, and the orange one disappears when they meet squarely. Keep that picture in mind; it's about to become a theorem.
Length, angle, and "perpendicular" never used the formula $u_1v_1 + \cdots + u_nv_n$ directly. They used its rules. So any product that follows the same rules gets lengths and angles for free.
An inner product on $\R^n$ is a rule that assigns a real number $\langle x, y \rangle$ to every pair of vectors $x, y$ in $\R^n$, such that for all $x$, $y$, $z$ in $\R^n$ and every real number $c$: (a) $\langle x + z, y \rangle = \langle x, y \rangle + \langle z, y \rangle$; (b) $\langle cx, y \rangle = c\langle x, y \rangle$; (c) $\langle x, y \rangle = \langle y, x \rangle$; (d) $\langle x, x \rangle > 0$ whenever $x \neq 0$.
Rules (a) and (b) say the product is linear in its first slot, (c) says order doesn't matter, and (d) says every nonzero vector has a positive "length squared". The dot product passes, of course. That's where the rules came from.
Show that the dot product, $\langle x, y \rangle = x \cdot y = x_1y_1 + \cdots + x_ny_n$, is an inner product on $\R^n$.
(a) $\sum (x_i + z_i)y_i = \sum x_iy_i + \sum z_iy_i$. (b) $\sum (cx_i)y_i = c\sum x_iy_i$. (c) $\sum x_iy_i = \sum y_ix_i$, since multiplying numbers doesn't care about order. (d) $x \cdot x = x_1^2 + \cdots + x_n^2$ is a sum of squares, positive unless every $x_i = 0$. All four hold. This is the standard inner product, the one every page uses unless it says otherwise.
It isn't the only one. Give each coordinate a positive weight and every rule still holds. Such a weighted inner product changes which vectors count as perpendicular.
Check that $\langle x, y \rangle = 2x_1y_1 + 3x_2y_2$ is an inner product on $\R^2$, and compute $\langle x, y \rangle$ for $x = (1, 2)$ and $y = (3, -1)$.
(a), (b), and (c) hold exactly as for the dot product: each term is a weight times $x_iy_i$. For (d), $\langle x, x \rangle = 2x_1^2 + 3x_2^2$, which is positive unless $x_1 = x_2 = 0$, because both weights are positive. So it's an inner product.
Then $\langle x, y \rangle = 2(1)(3) + 3(2)(-1) = 6 - 6 = 0$. These two vectors are orthogonal for this inner product, even though their dot product is $3 - 2 = 1$.
So "perpendicular" depends on which inner product you pick. That sounds alarming, but there's a calm way to see it: a weighted inner product is just the dot product, measured after stretching the plane.
Stretching by $\sqrt{2}$ across and $\sqrt{3}$ up turns $2x_1y_1 + 3x_2y_2$ into an ordinary dot product of the stretched arrows, so the four rules come along for the ride. Negative weights can't be stretched away, though, and that's where things break.
Is $\langle x, y \rangle = x_1y_1 - x_2y_2$ an inner product on $\R^2$?
It passes (a), (b), and (c), just like the weighted one. But $\langle (0, 1), (0, 1) \rangle = 0 - 1 = -1$, which breaks (d). Not an inner product. (Even worse, $\langle (1, 1), (1, 1) \rangle = 0$ for a nonzero vector.)
Notice which rule did the damage. That's the pattern, and it's worth a warning.
Rule (d) is the one that fails most often, and the one people forget to check. Linearity and symmetry are usually obvious from the formula. Positivity means looking at $\langle x, x \rangle$ for every nonzero $x$, including the sneaky ones like $(0, 1)$ and $(1, -1)$. A cross term like $4x_1y_2 + 4x_2y_1$ can sink a formula whose weights look perfectly positive.
Rule (d) is the bouncer. Linear? Symmetric? Great, you're on the list. Now let's see your $\langle x, x \rangle$. Zero? Negative? Sorry, private party.
Your turn to be the bouncer. Exactly one formula below gets in.
Loading practice engine...
A few more rules come free with the four. Rule (a) is about the first slot; symmetry hands it to the second.
For any inner product on $\R^n$, all $x, y, z$ in $\R^n$, and every real $c$:
(a) $\langle x, y + z \rangle = \langle x, y \rangle + \langle x, z \rangle$; (b) $\langle x, cy \rangle = c\langle x, y \rangle$; (c) $\langle x, 0 \rangle = \langle 0, x \rangle = 0$; (d) if $\langle x, y \rangle = \langle x, z \rangle$ for every $x$, then $y = z$.
(a) $\langle x, y + z \rangle = \langle y + z, x \rangle = \langle y, x \rangle + \langle z, x \rangle = \langle x, y \rangle + \langle x, z \rangle$, using symmetry, then rule (a) of the definition, then symmetry again. (b) is the same with rule (b). (c) $\langle 0, x \rangle = \langle 0 \cdot 0, x \rangle = 0\langle 0, x \rangle = 0$, and symmetry gives the other. (d) If $\langle x, y \rangle = \langle x, z \rangle$ for every $x$, then $\langle x, y - z \rangle = 0$ for every $x$ by (a) and (b). Choose $x = y - z$: then $\langle y - z, y - z \rangle = 0$, and rule (d) of the definition forces $y - z = 0$. $\blacksquare$
Part (d) is worth a second look: one test vector $x$ is never enough to cancel, but every test vector is. Now try weights yourself. Slide them, drag $x$, and watch which arrows count as orthogonal to it.
With both weights positive, the purple curve is a closed ellipse and the red line of orthogonal vectors tilts as the ellipse squashes. Drag a weight to $0$ or below and the curve opens up: some nonzero vector now has $\langle z, z \rangle \le 0$, and the product is out.
In the plane and in space there's just one product to know, the dot product. Here's a way to read it straight off a picture: drop a line from the tip of $u$ squarely onto $v$'s line. The dot product is the shadow you get, times the length of $v$.
So $u \cdot v$ is the area of a rectangle with sides "shadow of $u$" and "length of $v$", counted as negative when the shadow points backward. That's the same as $\|u\| \, \|v\| \cos\theta$ from the angle formula, since the shadow is $\|u\| \cos\theta$ long.
Let $v = (4, 0)$. Read $u \cdot v$ off the shadow for (a) $u = (1, 3)$ and (b) $u = (-2, 2)$, then check with the formula.
(a) Dropping straight down from $(1, 3)$ lands at $(1, 0)$, so the shadow is $1$ long, pointing forward, and $u \cdot v = 1 \times 4 = 4$. Check: $1 \cdot 4 + 3 \cdot 0 = 4$.
(b) Dropping from $(-2, 2)$ lands at $(-2, 0)$: a shadow $2$ long, pointing backward. So $u \cdot v = -2 \times 4 = -8$. Check: $-2 \cdot 4 + 2 \cdot 0 = -8$.
Shadow times length. Hold on to that: both inequalities on this page are about how long a shadow can be.
Every inner product hands you a length, the same way the dot product did.
For an inner product on $\R^n$, the norm (or length) of $x$ is $\|x\| = \sqrt{\langle x, x \rangle}$.
Rule (d) is what makes the square root safe. Two facts follow straight from the rules: $\|cx\| = |c| \, \|x\|$ (pull $c$ out of both slots, then out of the root as $\sqrt{c^2} = |c|$), and $\|x\| = 0$ only for $x = 0$. The distance between $x$ and $y$ is $\|x - y\|$, the length of the arrow from one tip to the other. With the dot product these are the lengths and distances from Dot & Cross Products. With weights, they change.
With $\langle x, y \rangle = 2x_1y_1 + 3x_2y_2$, find $\|x\|$, $\|y\|$, and the distance between $x = (1, 2)$ and $y = (3, -1)$.
$\|x\|^2 = 2(1)^2 + 3(2)^2 = 14$ and $\|y\|^2 = 2(3)^2 + 3(-1)^2 = 21$, so $\|x\| = \sqrt{14}$ and $\|y\| = \sqrt{21}$.
$x - y = (-2, 3)$, so $\|x - y\|^2 = 2(-2)^2 + 3(3)^2 = 8 + 27 = 35$, and the distance is $\sqrt{35}$. Notice $35 = 14 + 21$: these two are orthogonal for this product, and Pythagoras holds for it. That's no accident, as the end of this page shows.
In free response, square roots are fine: type 2sqrt(3) or sqrt(14).
Loading practice engine...
Length and distance are the ones from Dot & Cross Products: $\|u\| = \sqrt{u \cdot u}$ is Pythagoras, and the distance between the tips of $u$ and $v$ is $\|u - v\|$, the length of the arrow from one tip to the other. Nothing new to learn here, just two questions to ask about them. How long can a shadow be? And how short can a trip be?
The opening picture said a shadow is never longer than its arrow. Multiply both sides by the length of $v$ and you get the most useful inequality in linear algebra.
For any inner product on $\R^n$ and all $x, y$ in $\R^n$, $$|\langle x, y \rangle| \le \|x\| \, \|y\|,$$ with equality exactly when one of $x$, $y$ is a multiple of the other.
If $y = 0$, both sides are $0$, and $0 = 0y$ is a multiple of $y$. So assume $y \neq 0$. For every real $c$, rule (d) and the rules above give $$0 \le \|x - cy\|^2 = \langle x - cy, x - cy \rangle = \|x\|^2 - 2c\langle x, y \rangle + c^2\|y\|^2.$$ Choose $c = \dfrac{\langle x, y \rangle}{\|y\|^2}$ (the shadow's coefficient). The right side becomes $\|x\|^2 - \dfrac{\langle x, y \rangle^2}{\|y\|^2}$, so $\langle x, y \rangle^2 \le \|x\|^2\|y\|^2$. Take square roots.
Equality means $\|x - cy\| = 0$ for that $c$, that is, $x = cy$. Conversely, if $x = cy$ then $|\langle x, y \rangle| = |c| \, \|y\|^2 = \|x\| \, \|y\|$; and if $y$ is a multiple of $x$, swap the roles. $\blacksquare$
For all vectors $u$, $v$ in the plane or in space, $$|u \cdot v| \le \|u\| \, \|v\|,$$ with equality exactly when $u$ and $v$ lie on one line through $0$ (one is a multiple of the other).
This is a visual argument, not a formal proof, but it's the real reason. Drop a line from the tip of $u$ squarely onto $v$'s line. That makes a right triangle, flat on one sheet of paper even in space: $u$ is the long side, and the legs are the shadow and the leftover piece.
Pythagoras says $\text{shadow}^2 + \text{leftover}^2 = \|u\|^2$, so the shadow can't be longer than $u$. And $u \cdot v$ is the shadow times $\|v\|$, so $|u \cdot v| \le \|u\| \, \|v\|$. The two sides are equal only when the leftover is $0$, which means $u$ already lies on $v$'s line. (If $v = 0$, both sides are $0$.) The full proof is in the n version.
Divide both sides by $\|u\| \, \|v\|$ and Cauchy–Schwarz says $\cos\theta$ always lands between $-1$ and $1$. In $\R^n$, with no protractor to check, that's exactly what makes "the angle between two vectors" a sensible definition, for any inner product.With a protractor that's no surprise, but the inequality says more: it bounds a dot product using nothing but two lengths. Here's a check with real numbers.
Check Cauchy–Schwarz for $u = (1, 2, 2)$ and $v = (3, 0, 4)$.
$u \cdot v = 3 + 0 + 8 = 11$, $\|u\| = \sqrt{1 + 4 + 4} = 3$, and $\|v\| = \sqrt{9 + 0 + 16} = 5$. So $|u \cdot v| = 11 \le 15 = \|u\| \, \|v\|$. Not equal, because $u$ isn't a multiple of $v$. The gap tells you the angle: $\cos\theta = \frac{11}{15}$.
The shadow of $u$ on $v$'s line is $\frac{11}{5} = 2.2$ long, shorter than $u$'s length $3$:
Now push the arrows around yourself. Each mission is one of the special cases of the inequality.
The top of the range is $\|u\| \, \|v\|$ (same direction), the bottom is $-\|u\| \, \|v\|$ (opposite directions), and $0$ sits in the middle (square). That top of the range is exactly what engineers want when they ask "how big can this get?"
What is the largest value of $2x - y + 2z$ over all points with $x^2 + y^2 + z^2 = 9$, and where does it happen?
Write $2x - y + 2z = a \cdot p$ with $a = (2, -1, 2)$ and $p = (x, y, z)$. The points are on a sphere of radius $3$, so $\|p\| = 3$, and $\|a\| = \sqrt{4 + 1 + 4} = 3$. Cauchy–Schwarz gives $a \cdot p \le 3 \times 3 = 9$.
Equality needs $p$ to be a positive multiple of $a$ with length $3$, and $a$ already has length $3$, so $p = (2, -1, 2)$. Check: $2(2) - (-1) + 2(2) = 9$. The largest value is $9$. (The smallest is $-9$, at $p = (-2, 1, -2)$.)
The point wanders over the sphere, and the value is $3$ times its shadow on $a$'s line:
No calculus, no Lagrange multipliers: one inequality and one length. Try a few. In free response, square roots are fine (3sqrt(6)).
Loading practice engine...
The second fact is about trips. Walk along $u$, then along $v$. You end up at the tip of $u + v$, and going straight there can't be longer than the detour.
For all $x, y$ in $\R^n$, with the norm of any inner product,vectors $x$, $y$ in the plane or in space, $$\|x + y\| \le \|x\| + \|y\|.$$
Expand and use Cauchy–Schwarz on the middle term: $$\begin{aligned} \|x + y\|^2 &= \langle x + y, x + y \rangle = \|x\|^2 + 2\langle x, y \rangle + \|y\|^2 \\ &\le \|x\|^2 + 2\|x\| \, \|y\| + \|y\|^2 = (\|x\| + \|y\|)^2. \end{aligned}$$ Both sides are nonnegative, so take square roots. $\blacksquare$
This is a visual argument, not a formal proof. Put $y$ at the tip of $x$. The arrows $x$, $y$, and $x + y$ are the three sides of a triangle (flat, even in space), and in a triangle each side is shorter than the other two together. That's the triangle inequality from geometry class. Now straighten the bend:
The detour is always $\|x\| + \|y\|$ long. The straight path grows as the bend straightens, and catches up only when there's no bend at all, when $x$ and $y$ point the same way. The full proof is in the n version.
The triangle inequality: going straight is never longer than taking a detour. Mathematicians proved it and called it a theorem. Every pigeon already knew.
Turned around, it gives a floor too: $\|x + y\| \ge \big|\|x\| - \|y\|\big|$, with the bottom reached when the two point opposite ways. So any trip of two legs ends up somewhere between "the legs cancel as much as they can" and "the legs add up".
A drone flies along $u = (2, 3, 6)$, then along $v = (1, -2, 2)$ (in kilometers). How far from the start does it end up? And if it had flown $v$'s distance in any other direction, what's the farthest and closest it could have ended up?
$\|u\| = \sqrt{4 + 9 + 36} = 7$ and $\|v\| = \sqrt{1 + 4 + 4} = 3$. It ends at $u + v = (3, 1, 8)$, which is $\sqrt{9 + 1 + 64} = \sqrt{74} \approx 8.6$ km from the start. That's less than $7 + 3 = 10$, as promised.
Flying $3$ km in any direction after the first leg, it ends between $7 - 3 = 4$ km (flying straight back toward the start) and $7 + 3 = 10$ km (flying straight on) from where it began.
The trip in space, then the second leg turning through every direction:
The two ends of that range are the two straight-line cases, and they're the ones people mix up.
Equality in the triangle inequality needs the two vectors to point the same way, not just to be parallel. Opposite arrows are parallel too, and they give the other end of the range: $\|(1, 0) + (-1, 0)\| = 0$, not $2$. And never assume a right angle: $\sqrt{a^2 + b^2}$ is the answer only when the legs are square.
Your turn. In free response, type one number; fractions like 7/2 are fine.
Loading practice engine...
For any inner product, $x$ and $y$ are orthogonal when $\langle x, y \rangle = 0$; with the dot product that's the right angle from Dot & Cross Products.Two arrows are orthogonal when they meet squarely, that is, when $u \cdot v = 0$. Right angles are the best thing that can happen between two vectors, so it pays to collect vectors that are all square to each other.
A set of vectors is an orthogonal set if any two different vectors in it are orthogonal: $\langle x, y \rangle = 0$$x \cdot y = 0$ whenever $x \neq y$ are both in the set.
Checking one means checking every pair, not just the neighbors in the list.
Is $S = \{(1, 1, 0), (1, -1, 1), (-1, 1, 2)\}$ an orthogonal set?
All three pairs: $(1)(1) + (1)(-1) + (0)(1) = 0$; $(1)(-1) + (1)(1) + (0)(2) = 0$; $(1)(-1) + (-1)(1) + (1)(2) = 0$. Every pair is orthogonal, so yes.
These three arrows are square to each other but have lengths $\sqrt{2}$, $\sqrt{3}$, and $\sqrt{6}$. Often you'd rather they all had length $1$.
A unit vector is a vector $x$ with $\|x\| = 1$.
Any nonzero vector becomes a unit vector pointing the same way if you divide it by its length: $\left\|\frac{x}{\|x\|}\right\| = \frac{\|x\|}{\|x\|} = 1$. That move is called normalizing. Do it to every vector of an orthogonal set and you get the nicest kind of set there is.
A set of vectors is orthonormal if it is an orthogonal set and every vector in it is a unit vector.
So the set $S$ above is one step away from orthonormal.
Normalize $S = \{(1, 1, 0), (1, -1, 1), (-1, 1, 2)\}$ to get an orthonormal set.
The lengths are $\sqrt{2}$, $\sqrt{3}$, and $\sqrt{6}$, so $$\left\{\tfrac{1}{\sqrt{2}}(1, 1, 0), \ \tfrac{1}{\sqrt{3}}(1, -1, 1), \ \tfrac{1}{\sqrt{6}}(-1, 1, 2)\right\}$$ is orthonormal. Dividing doesn't change any directions, so the pairs stay square, and now every length is $1$.
The three square corners, then the shrink onto the unit sphere:
Spot one yourself. Check every pair, and for "orthonormal", every length too.
Loading practice engine...
Orthogonal sets have a superpower: you never have to check them for linear independence. Right angles do it for you.
An orthogonal set of nonzero vectors is linearly independent.
Suppose $a_1v_1 + \cdots + a_kv_k = 0$ for different vectors $v_1, \ldots, v_k$ of the set. Take the inner product of both sides with $v_j$: $$a_1\langle v_1, v_j \rangle + \cdots + a_k\langle v_k, v_j \rangle = \langle 0, v_j \rangle = 0.$$ Every term with $i \neq j$ is $0$, since $\langle v_i, v_j \rangle = 0$, which leaves $a_j\|v_j\|^2 = 0$. As $v_j \neq 0$, $\|v_j\|^2 > 0$, so $a_j = 0$. This works for every $j$. $\blacksquare$
This is a visual argument, not a formal proof. Build any combination $a_1v_1 + a_2v_2 + a_3v_3$ head to tail and look at its shadow on $v_1$'s line. The pieces $a_2v_2$ and $a_3v_3$ run square to that line, so they cast no shadow on it. The shadow is exactly $a_1v_1$.
If the combination lands back at $0$, its shadow on every line is $0$. So $a_1v_1 = 0$, $a_2v_2 = 0$, $a_3v_3 = 0$, and since no $v_i$ is $0$, every coefficient is $0$. That's independence. The full proof is in the n version.
The word "nonzero" can't be dropped: the zero vector is orthogonal to everything, so $\{(1, 0, 0), 0\}$ is an orthogonal set, and no set containing $0$ is independent. A bonus: an orthogonal set of nonzero vectors in $\R^n$ has at most $n$ vectors, since an independent set can't have more.A bonus: in space you can find at most three nonzero vectors that are all square to each other, since four vectors in space are never independent.
An orthogonal set of nonzero vectors is independent, so if it has enough vectors, it's a basis. The best bases of all are orthonormal.
An orthonormal basis is an ordered basis that is an orthonormal set.
On Bases & Dimension, finding the coordinates of $y$ in a basis meant solving a system: $a_1v_1 + \cdots + a_kv_k = y$, row reduction and all.
The standard basis $e_1, \ldots, e_n$ is orthonormal, and coordinates in it are just the entries. Here's the surprise: every orthogonal basis is almost that easy. Each coefficient is one inner productdot product and one division.
Let $\{v_1, \ldots, v_k\}$ be an orthogonal set of nonzero vectors. If $y$ is in $\span\{v_1, \ldots, v_k\}$, then $$y = \frac{y \cdot v_1}{\|v_1\|^2}v_1 + \cdots + \frac{y \cdot v_k}{\|v_k\|^2}v_k.$$ (For another inner product, read $y \cdot v_i$ as $\langle y, v_i \rangle$.) If the set is orthonormal, every $\|v_i\|^2 = 1$ and the coefficients are just $y \cdot v_i$.
Since $y$ is in the span, $y = a_1v_1 + \cdots + a_kv_k$ for some numbers $a_i$. Take the inner product with $v_j$: $$\langle y, v_j \rangle = a_1\langle v_1, v_j \rangle + \cdots + a_k\langle v_k, v_j \rangle = a_j\langle v_j, v_j \rangle = a_j\|v_j\|^2,$$ because every other term is $0$. So $a_j = \dfrac{\langle y, v_j \rangle}{\|v_j\|^2}$. $\blacksquare$
This is a visual argument, not a formal proof; it's drawn in the plane, and space works the same way with one more piece. Write $y$ as its pieces head to tail, then drop squarely from $y$ onto each basis line.
The drop onto $v_1$'s line lands exactly at the end of the piece $a_1v_1$, because the other pieces run square to that line and add nothing to the shadow. So the shadow of $y$ on $v_1$'s line is $a_1v_1$. And Dot & Cross Products already measured that shadow: it's $\frac{y \cdot v_1}{v_1 \cdot v_1}v_1$. Match them up and $a_1 = \frac{y \cdot v_1}{\|v_1\|^2}$. The full proof is in the n version.
So each coefficient is the coefficient of a projection: $y$ is the sum of its shadows on the basis vectors. Let's cash that in.
Write $y = (3, 1, 4)$ in the orthogonal basis $S = \{(1, 1, 0), (1, -1, 1), (-1, 1, 2)\}$ from above.
No system needed. The squared lengths are $2$, $3$, $6$, and $$\frac{3 + 1 + 0}{2} = 2, \qquad \frac{3 - 1 + 4}{3} = 2, \qquad \frac{-3 + 1 + 8}{6} = 1.$$ So $y = 2(1, 1, 0) + 2(1, -1, 1) + 1(-1, 1, 2)$. Check: $(2 + 2 - 1, \ 2 - 2 + 1, \ 0 + 2 + 2) = (3, 1, 4)$.
Three shadows, then stacked head to tail:
With an orthonormal basis it's even cleaner: no dividing, every coordinate is a single dot product. Turn the basis below and drag $y$; each coordinate is the signed length of one shadow.
The widget's basis has length $1$, so each coordinate is a bare dot product. Back in an orthogonal basis that isn't unit length, there's one division to remember.
For an orthogonal (not orthonormal) basis, divide by $\|v_i\|^2$, the squared length, which is just $v_i \cdot v_i$: no square root anywhere. Dividing by $\|v_i\|$ instead is the classic slip, and it only gives the right answer when $\|v_i\| = 1$.
Your turn. In free response, type one number; fractions like -3/2 are fine.
Loading practice engine...
One more payoff. Lengths of square pieces combine the way Pythagoras said sides of a right triangle do.
If $\{v_1, \ldots, v_k\}$ is an orthogonal set, then $$\|v_1 + \cdots + v_k\|^2 = \|v_1\|^2 + \cdots + \|v_k\|^2.$$
Expand $\langle v_1 + \cdots + v_k, v_1 + \cdots + v_k \rangle$ with the rules: it's the sum of $\langle v_i, v_j \rangle$ over all pairs $i, j$. The terms with $i \neq j$ are $0$, leaving $\sum_i \langle v_i, v_i \rangle = \sum_i \|v_i\|^2$. $\blacksquare$
This is a visual argument, not a formal proof. Three pieces that are square to each other are the edges of a box (tilted, but with square corners everywhere), and their sum is the box's diagonal. Pythagoras on the floor, then Pythagoras up the wall:
The squared lengths just add. With two pieces it's the right triangle itself. The full proof is in the n version.
Check it on the last example: $y = 2v_1 + 2v_2 + 1v_3$, and the pieces have squared lengths $4 \cdot 2$, $4 \cdot 3$, and $1 \cdot 6$. They add to $8 + 12 + 6 = 26$, and sure enough $\|y\|^2 = 9 + 1 + 16 = 26$.
Three temperature readings, morning, noon, and evening, make a vector in $\R^3$. Three special patterns are square to each other: $(1, 1, 1)$ (flat), $(1, 0, -1)$ (a steady slope), and $(1, -2, 1)$ (a bend). Check the pairs: $1 + 0 - 1 = 0$, $1 - 2 + 1 = 0$, $1 + 0 - 1 = 0$. So any three readings split into an average, a trend, and a bend, each found with one dot product.
Split the readings $y = (14, 17, 26)$ into average, trend, and bend.
The squared lengths are $3$, $2$, $6$. The coefficients: $$\frac{14 + 17 + 26}{3} = 19, \qquad \frac{14 - 26}{2} = -6, \qquad \frac{14 - 34 + 26}{6} = 1.$$ So $y = 19(1, 1, 1) - 6(1, 0, -1) + 1(1, -2, 1)$: an average of $19^\circ$, a trend of $-6(1, 0, -1) = (-6, 0, 6)$ that rises $6^\circ$ from morning to noon and $6^\circ$ more by evening (the coefficient is negative because the pattern $(1, 0, -1)$ itself slopes down), and a bend of $1(1, -2, 1)$: the ends up $1^\circ$, noon down $2^\circ$.
Here are the three layers building up the readings, one dot product each:
Each coefficient ignores the other two patterns completely, because they're square to it. Drop the bend and you keep the best straight-line summary of the data, $19(1, 1, 1) - 6(1, 0, -1) = (13, 19, 25)$, without redoing anything. With many readings the same idea scales up: an orthogonal basis of smooth and wiggly patterns (cosines, in JPEG compression) splits a signal into pieces, and throwing away the small wiggly coefficients compresses it, one inner product per coefficient.Engineers do this with long lists of readings too, using sets of wiggly patterns that are all square to each other, and it's how photos get compressed: keep the big pieces, drop the tiny wiggles.
Dropping the bend kept the closest point to $y$ among all straight-line patterns. On Orthogonal Projection, that move gets its own name: projecting onto a subspaceplane through $0$, with the leftover square to the whole thing.
The toolkit for this page: an inner product is linear, symmetric, and positive, with $\|x\| = \sqrt{\langle x, x \rangle}$;a dot product is a shadow times a length; Cauchy–Schwarz caps $|u \cdot v|$ at $\|u\| \, \|v\|$; the triangle inequality caps $\|u + v\|$ at $\|u\| + \|v\|$; orthogonal sets of nonzero vectors are independent; and in an orthogonal basis each coefficient is $\frac{y \cdot v_i}{\|v_i\|^2}$.
Every question so far came right after its idea, which is a hint a quiz never gives you. Down here the hint is gone. First, true or false: a claim from anywhere on this page, and you decide whether it holds.
Loading practice engine...
Next, a mixed set: every question type on this page, shuffled. Type a number when it asks you to.
Loading practice engine...
Made it this far? Time for something with a little heat: three vectors you can't see, only their lengths and the fact that they're square to each other. That turns out to be enough. In free response, type a number; fractions and square roots are fine.
Loading practice engine...