On The Lᵖ Spaces we built a whole family of complete normed spaces, one for each $p$. This page is about the one member of the family that is special.
Every $L^p$ space has lengths: $\|f\|_p$ tells you how big $f$ is. But only $L^2$ has angles. In $L^2$ you can ask whether two functions are perpendicular, drop a perpendicular from a function onto a subspace, and find the closest point of a subspace by doing exactly what you'd do in the plane. All of that comes from one extra piece of structure, the inner product $$\langle f, g \rangle = \int f\,\overline{g}\,d\mu,$$ and from the fact that $L^2$ is complete. A complete space with an inner product is a Hilbert space, and Hilbert spaces are where infinite-dimensional geometry behaves like the geometry you already know.
The page builds up to two theorems. The projection theorem says every vector splits uniquely into a piece in a closed subspace and a piece perpendicular to it. The Riesz representation theorem says every bounded linear functional on a Hilbert space is "take the inner product with a fixed vector". Both rest on one identity about parallelograms.
We work over the complex numbers from the start, because $L^2$ is full of complex functions and the Fourier series pages will need them. The price is one conjugate, and it goes in a very particular place.
Let $H$ be a complex vector space. A sesquilinear form on $H$ is a map $\langle \cdot, \cdot \rangle\colon H \times H \to \mathbb{C}$ such that for all $x, y, z \in H$ and $\alpha \in \mathbb{C}$:
Rule (b) says the form is linear in the first slot. Rule (a) then decides what happens in the second slot: pull a scalar out of it and it comes out conjugated, $$\langle x, y + \alpha z \rangle = \overline{\langle y + \alpha z, x \rangle} = \overline{\langle y, x \rangle} + \overline{\alpha}\,\overline{\langle z, x \rangle} = \langle x, y \rangle + \overline{\alpha}\langle x, z \rangle.$$ Rule (a) with $y = x$ also says $\langle x, x \rangle = \overline{\langle x, x \rangle}$, so $\langle x, x \rangle$ is always real. Real, but not yet nonnegative; that takes a third rule.
A sesquilinear form on $H$ is positive semidefinite if (c) $\langle x, x \rangle \ge 0$ for every $x \in H$.
So a positive semidefinite form gives every vector a "squared length" $\langle x, x \rangle \ge 0$. It may still give some nonzero vectors length zero, and the smallest example already does.
On $\mathbb{C}^2$, let $\langle x, y \rangle = x_1\overline{y_1}$. Show it is a positive semidefinite sesquilinear form, and find a nonzero $x$ with $\langle x, x \rangle = 0$.
(a): $\langle y, x \rangle = y_1\overline{x_1} = \overline{x_1\overline{y_1}} = \overline{\langle x, y \rangle}$. (b): $(x_1 + \alpha z_1)\overline{y_1} = x_1\overline{y_1} + \alpha z_1\overline{y_1}$. (c): $\langle x, x \rangle = |x_1|^2 \ge 0$.
But $x = (0, 1)$ has $\langle x, x \rangle = 0$. The form only sees the first coordinate.
Ruling that out is the last condition.
Linear in the first slot, conjugate-linear in the second. Physicists picked the other slot. Nobody has spoken since.
So: the only vector with length zero should be zero.
A positive semidefinite sesquilinear form on $H$ with $\langle x, x \rangle = 0$ only for $x = 0$ is an inner product, and $H$ together with it is an inner product space.
The model is $\mathbb{C}^n$ with $\langle x, y \rangle = x_1\overline{y_1} + \cdots + x_n\overline{y_n}$. The conjugate on $y$ isn't decoration, and a small computation shows why.
In $\mathbb{C}^2$ with $\langle x, y \rangle = x_1\overline{y_1} + x_2\overline{y_2}$, let $x = (1 + i, 2)$ and $y = (i, 1 - i)$. Compute $\langle x, y \rangle$, $\langle y, x \rangle$, and $\langle x, x \rangle$. Then show that dropping the conjugate breaks rule (c).
Conjugate $y$'s entries first: $\overline{y} = (-i, 1 + i)$. Then $$\langle x, y \rangle = (1 + i)(-i) + 2(1 + i) = (1 - i) + (2 + 2i) = 3 + i.$$
By rule (a), $\langle y, x \rangle = \overline{3 + i} = 3 - i$. (Directly: $i(1 - i) + (1 - i)(2) = (1 + i) + (2 - 2i) = 3 - i$.)
$\langle x, x \rangle = |1 + i|^2 + |2|^2 = 2 + 4 = 6$, a nonnegative real number, as promised.
Without the conjugate, the "form" $x_1y_1 + x_2y_2$ gives the vector $(i, 0)$ the value $i \cdot i = -1$. A negative squared length: rule (c) fails.
So in a complex space, "$x$ dotted with itself" only makes sense with the conjugate in. If you've only seen the real dot product, nothing else changes: on a real space the conjugates do nothing. Other books don't all agree on where the conjugate goes, though.
Physics books, and some math books, make the inner product linear in the second slot and conjugate-linear in the first; the bra-ket notation of quantum mechanics, written with a bar and angle brackets, works that way. Older books write the inner product with round brackets, (x, y). This site follows Rudin: angle brackets, linear in the first slot. When you read another source, check which slot gets the conjugate before trusting any formula with a bar in it.
On the linear algebra page Inner Products & Norms, the inner product on $\mathbb{R}^n$ was the dot product, symmetric in its two slots. The complex version trades symmetry for rule (a), and on real vectors the two agree.
Before we insist on definiteness, it pays to see how far a merely semidefinite form gets. Here's the first surprise: a vector whose "length" is zero can't pair with anything.
Let $\langle \cdot, \cdot \rangle$ be a positive semidefinite sesquilinear form on $H$. If $\langle x, x \rangle = 0$, then $\langle x, y \rangle = 0$ for every $y \in H$.
Proof. Let $w = \langle x, y \rangle$. For any $s \in \mathbb{C}$, expanding with both slots' rules, $$0 \le \langle x + sy, x + sy \rangle = \langle x, x \rangle + \overline{s}\,w + s\,\overline{w} + |s|^2\langle y, y \rangle = 2\operatorname{Re}(\overline{s}\,w) + |s|^2\langle y, y \rangle.$$ Take $s = -tw$ with $t > 0$. Then $\overline{s}\,w = -t|w|^2$, so $$0 \le -2t|w|^2 + t^2|w|^2\langle y, y \rangle.$$ Divide by $t$ and let $t \to 0^+$: $0 \le -2|w|^2$. Hence $w = 0$. $\blacksquare$
The trick in that proof, "nudge by a small multiple and let the nudge go to zero", comes back twice on this page. For now, it tells us a semidefinite form still has a sensible length.
If $\langle \cdot, \cdot \rangle$ is a positive semidefinite sesquilinear form on $H$, the seminorm of $x \in H$ is $\|x\| = \sqrt{\langle x, x \rangle}$.
For an inner product, $\|x\| = 0$ only at $x = 0$, so the seminorm is a norm (we'll check the triangle inequality in a moment). Semidefinite forms aren't an abstract curiosity: one is hiding inside $L^2$.
On the space of all measurable $f\colon [0, 1] \to \mathbb{C}$ with $\int_0^1 |f|^2\,dm < \infty$ (actual functions, not classes), let $\langle f, g \rangle = \int_0^1 f\,\overline{g}\,dm$. Show this form is positive semidefinite but not an inner product, and check the proposition on $f = \chi_{\{0\}}$.
Rules (a) and (b) come from linearity of the integral and $\overline{f\overline{g}} = g\overline{f}$. Rule (c): $\langle f, f \rangle = \int_0^1 |f|^2\,dm \ge 0$.
But $f = \chi_{\{0\}}$ is not the zero function, and $\langle f, f \rangle = m(\{0\}) = 0$. So the form isn't definite.
The proposition says $f$ pairs to $0$ with everything, and it does: $\langle f, g \rangle = \int_{\{0\}} \overline{g}\,dm = 0$ because $\{0\}$ is a null set.
This is exactly why $L^2(\mu)$ is made of equivalence classes: identify functions that agree almost everywhere, and the vectors of seminorm zero collapse to the single class $0$. On the classes, the form is an inner product.
Time to check you can tell an inner product from a near miss. Each wrong choice fails exactly one rule.
Loading practice engine...
Four facts about the seminorm do almost all the work on this page. The first bounds an inner product by lengths. Here's the picture behind it, drawn in the real plane: drop a perpendicular from $x$ onto the line through $y$. The foot is $\lambda y$, the "shadow" of $x$, and the leftover $x - \lambda y$ is perpendicular to $y$.
A shadow is never longer than the thing casting it. In symbols, $\|\lambda y\| \le \|x\|$, and $\|\lambda y\| \cdot \|y\|$ is exactly $|\langle x, y \rangle|$. The proof just computes $\|x - \lambda y\|^2 \ge 0$.
If $\langle \cdot, \cdot \rangle$ is a positive semidefinite sesquilinear form on $H$, then for all $x, y \in H$, $$|\langle x, y \rangle| \le \|x\|\,\|y\|.$$
Proof. If $\|y\| = 0$, then $\langle y, y \rangle = 0$, so $\langle y, x \rangle = 0$ by the proposition above, and $\langle x, y \rangle = \overline{\langle y, x \rangle} = 0$. Both sides are $0$.
If $\|y\| \neq 0$, let $\lambda = \frac{\langle x, y \rangle}{\|y\|^2}$. Then $\overline{\lambda}\langle x, y \rangle = \frac{|\langle x, y \rangle|^2}{\|y\|^2}$ is real, and $$0 \le \|x - \lambda y\|^2 = \|x\|^2 - \overline{\lambda}\langle x, y \rangle - \lambda\langle y, x \rangle + |\lambda|^2\|y\|^2 = \|x\|^2 - 2\frac{|\langle x, y \rangle|^2}{\|y\|^2} + \frac{|\langle x, y \rangle|^2}{\|y\|^2} = \|x\|^2 - \frac{|\langle x, y \rangle|^2}{\|y\|^2}.$$ Multiply by $\|y\|^2$ and take square roots. $\blacksquare$
The case $\|y\| = 0$ is where the proposition earns its keep; a proof that only divides by $\|y\|^2$ has a hole in it for semidefinite forms.
On Convexity & Inequalities, Hölder's inequality with $p = q = 2$ said $\int |f\overline{g}|\,d\mu \le \|f\|_2\|g\|_2$. That's Cauchy–Schwarz in $L^2$, proved there with convexity and here with nothing but the algebra of the form.
Cauchy–Schwarz hands us the triangle inequality right away, and with it a metric.
If $\langle \cdot, \cdot \rangle$ is a positive semidefinite sesquilinear form on $H$, then $\|x + y\| \le \|x\| + \|y\|$ for all $x, y \in H$.
Proof. Since $\operatorname{Re}\langle x, y \rangle \le |\langle x, y \rangle| \le \|x\|\|y\|$ by Cauchy–Schwarz, $$\|x + y\|^2 = \|x\|^2 + 2\operatorname{Re}\langle x, y \rangle + \|y\|^2 \le \|x\|^2 + 2\|x\|\|y\| + \|y\|^2 = (\|x\| + \|y\|)^2.$$ Take square roots. $\blacksquare$
So on an inner product space, $\|\cdot\|$ is a norm and $d(x, y) = \|x - y\|$ is a metric; "closed", "Cauchy", and "converges" all refer to it from now on. Next, the identity that makes inner product spaces different from every other normed space. Add the squares of a parallelogram's two diagonals and you get the sum of the squares of its four sides.
The two bars never disagree, however the parallelogram leans. In a general space the proof is two lines of expansion.
If $\langle \cdot, \cdot \rangle$ is a positive semidefinite sesquilinear form on $H$, then for all $x, y \in H$, $$\|x + y\|^2 + \|x - y\|^2 = 2\|x\|^2 + 2\|y\|^2.$$
Proof. Expanding, $$\|x \pm y\|^2 = \langle x \pm y, x \pm y \rangle = \|x\|^2 \pm 2\operatorname{Re}\langle x, y \rangle + \|y\|^2.$$ Add the two identities; the middle terms cancel. $\blacksquare$
The parallelogram law is a test: a norm that fails it can't come from any inner product. Most norms fail.
On $\mathbb{C}^2$, let $\|x\|_1 = |x_1| + |x_2|$. Show this norm doesn't come from an inner product.
Take $x = (1, 0)$ and $y = (0, 1)$. Then $x + y = (1, 1)$ and $x - y = (1, -1)$, so $$\|x + y\|_1^2 + \|x - y\|_1^2 = 4 + 4 = 8, \qquad 2\|x\|_1^2 + 2\|y\|_1^2 = 2 + 2 = 4.$$
The law fails, so no inner product has $\langle x, x \rangle = \|x\|_1^2$.
That's one pair of vectors in one norm. Try more: lean the parallelogram anywhere you like, in three different norms, and see which one never misses.
Only $\ell^2$ balances every time. (The converse is a theorem of Jordan and von Neumann: a norm satisfying the parallelogram law always comes from an inner product. We won't need it.) The same test works on function spaces.
On $C([0, 1])$ with the sup norm $\|f\|_\infty = \sup_{t \in [0, 1]} |f(t)|$, show that no inner product has $\langle f, f \rangle = \|f\|_\infty^2$ for every $f$.
Proof. If such an inner product existed, its seminorm would be $\|\cdot\|_\infty$, and the parallelogram law would hold for it. Take $f = 1$ and $g(t) = t$. Then $f + g = 1 + t$ has sup $2$, $f - g = 1 - t$ has sup $1$, and $\|f\|_\infty = \|g\|_\infty = 1$. So $$\|f + g\|_\infty^2 + \|f - g\|_\infty^2 = 4 + 1 = 5 \neq 4 = 2\|f\|_\infty^2 + 2\|g\|_\infty^2,$$ a contradiction. $\blacksquare$
The parallelogram law says norms determine a lot. The last identity says they determine everything: if you know every length, you know every inner product.
If $\langle \cdot, \cdot \rangle$ is a sesquilinear form on a complex vector space $H$ and $\|x\|^2$ means $\langle x, x \rangle$, then for all $x, y \in H$, $$4\langle x, y \rangle = \|x + y\|^2 - \|x - y\|^2 + i\left(\|x + iy\|^2 - \|x - iy\|^2\right).$$ On a real vector space with a symmetric bilinear form, $4\langle x, y \rangle = \|x + y\|^2 - \|x - y\|^2$.
Proof. For any scalar $c$ with $|c| = 1$, $$\|x + cy\|^2 - \|x - cy\|^2 = 2\left(\overline{c}\langle x, y \rangle + c\langle y, x \rangle\right),$$ because the $\|x\|^2$ and $|c|^2\|y\|^2$ terms cancel. With $c = 1$ this is $2\langle x, y \rangle + 2\langle y, x \rangle$. With $c = i$ it is $-2i\langle x, y \rangle + 2i\langle y, x \rangle$, and multiplying by $i$ gives $2\langle x, y \rangle - 2\langle y, x \rangle$. Adding, $$\|x + y\|^2 - \|x - y\|^2 + i\left(\|x + iy\|^2 - \|x - iy\|^2\right) = 4\langle x, y \rangle.$$ In the real case, $\langle y, x \rangle = \langle x, y \rangle$ and the $c = 1$ line alone gives $4\langle x, y \rangle$. $\blacksquare$
The real part comes from the $\pm y$ pair and the imaginary part from the $\pm iy$ pair. Watch the sign on the imaginary part: swap the pair and you get $\langle y, x \rangle$ instead. Here it is run backward on the vectors from the first example.
With $x = (1 + i, 2)$ and $y = (i, 1 - i)$ in $\mathbb{C}^2$, compute the four squared norms and recover $\langle x, y \rangle$ from them.
$x + y = (1 + 2i, 3 - i)$, so $\|x + y\|^2 = 5 + 10 = 15$. $x - y = (1, 1 + i)$, so $\|x - y\|^2 = 1 + 2 = 3$.
$iy = (-1, 1 + i)$, so $x + iy = (i, 3 + i)$ with $\|x + iy\|^2 = 1 + 10 = 11$, and $x - iy = (2 + i, 1 - i)$ with $\|x - iy\|^2 = 5 + 2 = 7$.
So $4\langle x, y \rangle = (15 - 3) + i(11 - 7) = 12 + 4i$, and $\langle x, y \rangle = 3 + i$, matching the direct computation.
Now be the one asking the questions. The box below only answers "how long is this vector?", and your job is to get an inner product out of it.
With the $\ell^1$ box, the identity still spits out a number, but the number isn't linear in $x$. Polarization only recovers an inner product when there was one to recover.
Polarization identity: the inner product's way of saying it can be recovered from norms alone, thank you very much.
Your turn. Four squared norms in, one complex number out.
Loading practice engine...
Geometry needs one more ingredient: limits. A minimizing sequence should converge to something, and that something should still be in the space.
An inner product space $H$ that is complete with respect to the metric $d(x, y) = \|x - y\|$ is a Hilbert space.
The star of the show was built two pages ago.
Let $\mu$ be any positive measure on $X$. Show that $L^2(\mu)$ with $\langle f, g \rangle = \int_X f\,\overline{g}\,d\mu$ is a Hilbert space, and identify $\ell^2$ as a special case.
The integral exists: $|f\overline{g}| \le \frac{1}{2}(|f|^2 + |g|^2)$, which is integrable. Rules (a), (b), and (c) come from linearity of the integral, and $\langle f, f \rangle = \int |f|^2\,d\mu = \|f\|_2^2$. As in the semidefinite example, $\langle f, f \rangle = 0$ means $f = 0$ almost everywhere, which is the zero class. So $L^2(\mu)$ is an inner product space whose norm is $\|\cdot\|_2$, and $L^2(\mu)$ is complete in that norm.
With $X = \mathbb{N}$ and counting measure, $L^2$ is $\ell^2$, the square-summable sequences, with $\langle a, b \rangle = \sum_n a_n\overline{b_n}$.
On The Lᵖ Spaces we proved $L^p(\mu)$ is complete for every $1 \le p \le \infty$, using fast Cauchy subsequences. With $p = 2$, that theorem is the whole reason $L^2$ is a Hilbert space.
From here on, $H$ is a Hilbert space unless we say otherwise. Completeness is a real hypothesis, though, and the most natural inner product space of functions doesn't have it.
Show that $C([0, 1])$ with $\langle f, g \rangle = \int_0^1 f\,\overline{g}\,dt$ is an inner product space but not a Hilbert space.
It's an inner product space: a continuous $f$ with $\int_0^1 |f|^2\,dt = 0$ vanishes everywhere, because $|f|^2$ would be positive on a whole interval around any point where $f \neq 0$.
Let $f_n$ be $0$ on $[0, \frac{1}{2}]$, climb linearly to $1$ on $[\frac{1}{2}, \frac{1}{2} + \frac{1}{n}]$, and equal $1$ after. Let $\chi = \chi_{[1/2, 1]}$. Then $f_n$ and $\chi$ differ only on the ramp, so $$\|f_n - \chi\|_2^2 = \int_{1/2}^{1/2 + 1/n} \left(1 - n(t - \tfrac{1}{2})\right)^2 dt = \frac{1}{3n} \to 0.$$ So $(f_n)$ is Cauchy (it converges in $L^2$). If it had a limit $f$ in $C([0, 1])$, then $f = \chi$ almost everywhere. But then $f = 0$ on $[0, \frac{1}{2})$ and $f = 1$ on $(\frac{1}{2}, 1]$ almost everywhere, hence everywhere there by continuity, and no continuous function does that. So the Cauchy sequence has no limit in $C([0, 1])$.
The inner product itself is continuous, which is what lets us pass limits through it. The notes say the norm's continuity follows from this; it actually follows from the triangle inequality, so the proof below does both.
Let $H$ be an inner product space and $g \in H$. The maps $f \mapsto \langle f, g \rangle$, $f \mapsto \langle g, f \rangle$, and $f \mapsto \|f\|$ are uniformly continuous on $H$. The first is a linear functional.
Proof. Linearity is rule (b). By Cauchy–Schwarz, $$|\langle f, g \rangle - \langle h, g \rangle| = |\langle f - h, g \rangle| \le \|f - h\|\,\|g\|,$$ so given $\varepsilon > 0$, $\delta = \varepsilon / (\|g\| + 1)$ works for every pair $f, h$ at once. The second map is the complex conjugate of the first, so it is uniformly continuous too.
For the norm, the triangle inequality gives $\|f\| \le \|f - h\| + \|h\|$ and $\|h\| \le \|h - f\| + \|f\|$, so $\big|\|f\| - \|h\|\big| \le \|f - h\|$, and $\delta = \varepsilon$ works. $\blacksquare$
The subspaces we can project onto are the closed ones.
A closed subspace of $H$ is a subspace of $H$ that is a closed set in the metric $d(x, y) = \|x - y\|$.
Not every subspace is closed, and the ramps from the last example already show one that isn't.
Show that $C([0, 1])$ (continuous functions, each identified with its class) is a subspace of $L^2([0, 1])$ that is not closed.
Sums and scalar multiples of continuous functions are continuous, so it's a subspace. The ramps $f_n$ above are in $C([0, 1])$ and converge in $L^2$ to $\chi_{[1/2, 1]}$, which is not equal almost everywhere to any continuous function. A closed set contains the limits of its convergent sequences, so $C([0, 1])$ is not closed.
Closing up a subspace fixes that, and the closure is still a subspace.
Let $M$ be a subspace of $H$. Show that its closure $\overline{M}$ is a closed subspace of $H$.
Proof. $\overline{M}$ is closed, and it contains $0 \in M$. Let $x, y \in \overline{M}$ and $\alpha \in \mathbb{C}$. In a metric space, points of the closure are limits of sequences from the set, so there are $x_n, y_n \in M$ with $x_n \to x$ and $y_n \to y$. Then $\alpha x_n + y_n \in M$, and $$\|(\alpha x_n + y_n) - (\alpha x + y)\| \le |\alpha|\,\|x_n - x\| + \|y_n - y\| \to 0,$$ so $\alpha x + y$ is a limit of points of $M$, that is, $\alpha x + y \in \overline{M}$. $\blacksquare$
Projections onto subspaces are the goal, but the nearest-point argument works on a wider class of sets, and it's no harder there.
A set $E$ in a complex vector space is convex if $(1 - t)x + ty \in E$ for all $x, y \in E$ and all $t \in (0, 1)$.
That is, the segment between any two points of $E$ stays in $E$. Every subspace is convex, and every translate $E + x$ of a convex set is convex. Balls are convex too, and an annulus isn't; both take one line to check.
Show that the closed ball $B = \{y \in H : \|y - x\| \le r\}$ is convex, and that the annulus $A = \{y \in \mathbb{R}^2 : 1 \le \|y\| \le 2\}$ is not.
For $y, z \in B$ and $t \in (0, 1)$, write $(1 - t)y + tz - x = (1 - t)(y - x) + t(z - x)$. By the triangle inequality, $$\|(1 - t)y + tz - x\| \le (1 - t)\|y - x\| + t\|z - x\| \le (1 - t)r + tr = r,$$ so the point is in $B$.
For the annulus, $(1, 0)$ and $(-1, 0)$ are in $A$, but the midpoint of the segment between them is $(0, 0)$, with norm $0 < 1$. So $A$ is not convex.
Keep the annulus in mind; it's about to be the set where nearest points go wrong.
With an inner product, "perpendicular" means "pairs to zero".
Vectors $x, y \in H$ are orthogonal, written $x \perp y$, if $\langle x, y \rangle = 0$. For $S \subseteq H$, the orthogonal complement of $S$ is $S^\perp = \{x \in H : \langle x, y \rangle = 0 \text{ for all } y \in S\}$.
Since $\langle y, x \rangle = \overline{\langle x, y \rangle}$, orthogonality is symmetric. And orthogonal vectors obey Pythagoras: if $x \perp y$, the cross terms in $\|x + y\|^2$ vanish and $\|x + y\|^2 = \|x\|^2 + \|y\|^2$. Here's a complement you can write down.
In $L^2([0, 1])$, find $\{1\}^\perp$, where $1$ is the constant function.
$\langle f, 1 \rangle = \int_0^1 f\,dt$, so $\{1\}^\perp = \{f \in L^2([0, 1]) : \int_0^1 f\,dt = 0\}$, the functions with average zero. For instance $t - \frac{1}{2} \perp 1$.
That complement is the kernel of the continuous linear functional $f \mapsto \int_0^1 f\,dt$, so it's a closed subspace. Every complement works the same way.
For every $S \subseteq H$, $S^\perp$ is a closed subspace of $H$.
Proof. For $y \in S$, the map $\lambda_y(x) = \langle x, y \rangle$ is a continuous linear functional (the proposition above), so $\{y\}^\perp = \lambda_y^{-1}(\{0\})$ is a subspace (by linearity) and closed (as the preimage of the closed set $\{0\}$). Then $$S^\perp = \bigcap_{y \in S} \{y\}^\perp,$$ and an intersection of closed subspaces is a closed subspace. $\blacksquare$
Notice that $S$ itself didn't have to be closed, or a subspace, or anything. The complement is always as nice as possible.
Here's the question every projection answers: given a set $K$ and a point $h$, is there a point of $K$ closest to $h$, and only one? Let $\delta = \inf\{\|k - h\| : k \in K\}$. There's always a sequence in $K$ whose distances to $h$ tend to $\delta$. The worry is that the sequence wanders around without converging. The parallelogram law, applied to $k_i - h$ and $k_j - h$, says it can't, as long as $K$ contains the midpoints.
Each midpoint is in $K$, so it's at least $\delta$ from $h$. That one fact pins two far-apart points of a minimizing sequence to within a shrinking distance of each other. The notes prove this for closed subspaces; the same proof works for any closed convex set, and gives uniqueness for free.
Let $K$ be a nonempty closed convex subset of a Hilbert space $H$, and let $h \in H$. Then there is exactly one $k_0 \in K$ with $$\|k_0 - h\| = \delta = \inf\{\|k - h\| : k \in K\}.$$
Proof. The estimate. For $k_1, k_2 \in K$, apply the parallelogram law to $x = k_1 - h$ and $y = k_2 - h$; since $x + y = 2(\frac{k_1 + k_2}{2} - h)$ and $x - y = k_1 - k_2$, $$\|k_1 - k_2\|^2 = 2\|k_1 - h\|^2 + 2\|k_2 - h\|^2 - 4\left\|\tfrac{k_1 + k_2}{2} - h\right\|^2.$$ By convexity, $\frac{k_1 + k_2}{2} \in K$, so $\|\frac{k_1 + k_2}{2} - h\| \ge \delta$, and $$\|k_1 - k_2\|^2 \le 2\|k_1 - h\|^2 + 2\|k_2 - h\|^2 - 4\delta^2. \qquad (*)$$
Existence. Choose $k_n \in K$ with $\|k_n - h\| \to \delta$. By $(*)$, $\|k_n - k_m\|^2 \le 2\|k_n - h\|^2 + 2\|k_m - h\|^2 - 4\delta^2$, and the right side tends to $2\delta^2 + 2\delta^2 - 4\delta^2 = 0$ as $n, m \to \infty$. So $(k_n)$ is Cauchy. $H$ is complete, so $k_n \to k_0$ for some $k_0 \in H$; $K$ is closed, so $k_0 \in K$; and the norm is continuous, so $\|k_0 - h\| = \lim \|k_n - h\| = \delta$.
Uniqueness. If $k_1, k_2 \in K$ both have $\|k_i - h\| = \delta$, then $(*)$ gives $\|k_1 - k_2\|^2 \le 2\delta^2 + 2\delta^2 - 4\delta^2 = 0$, so $k_1 = k_2$. $\blacksquare$
Every hypothesis did a job: convexity put the midpoint in $K$, completeness gave the limit, closedness kept it in $K$. Drop one and the theorem breaks. Try it.
Off convex sets, a point can have two nearest neighbors, or a whole circle of them. Off closed sets, it can have none. And off Hilbert spaces, convexity isn't enough, because the parallelogram law was doing the real work.
In $\mathbb{R}^2$ with the sup norm $\|x\|_\infty = \max\{|x_1|, |x_2|\}$, let $K$ be the line $x_2 = 1$ and $h = 0$. Find every point of $K$ nearest to $h$.
For $k = (t, 1)$, $\|k - h\|_\infty = \max\{|t|, 1\} \ge 1$, with equality exactly when $|t| \le 1$. So $\delta = 1$, and every point $(t, 1)$ with $-1 \le t \le 1$ is nearest: a whole segment of them, though $K$ is closed and convex.
In a Hilbert space, every point has exactly one closest friend in each closed convex set. In a Banach space it might have a whole crowd, or nobody at all.
The estimate $(*)$ is worth practicing on its own: it's the parallelogram law with one term traded for $\delta$. This question asks for one of the law's four quantities, or for that estimate.
Loading practice engine...
Now let $K$ be a closed subspace $M$. The nearest point $m$ to $h$ exists, and the leftover $n = h - m$ should be perpendicular to $M$, just as in the plane. Why? If $n$ leaned at all, sliding $m$ a little along $M$ would bring it closer to $h$.
That's the whole proof, written with the "small nudge" trick from the first proposition.
Let $M$ be a closed subspace of a Hilbert space $H$. Every $h \in H$ can be written uniquely as $h = m + n$ with $m \in M$ and $n \in M^\perp$. Moreover, $\|h\|^2 = \|m\|^2 + \|n\|^2$, $m$ is the point of $M$ nearest to $h$, and the maps $h \mapsto m$ and $h \mapsto n$ are linear.
Proof. Existence. $M$ is nonempty, closed, and convex, so by the nearest point theorem there is $m \in M$ nearest to $h$. Let $n = h - m$. For $x \in M$ and $\alpha \in \mathbb{C}$, $m + \alpha x \in M$, so $$\|n - \alpha x\|^2 = \|h - (m + \alpha x)\|^2 \ge \|n\|^2.$$ Expanding the left side, $-2\operatorname{Re}(\overline{\alpha}\langle n, x \rangle) + |\alpha|^2\|x\|^2 \ge 0$. Take $\alpha = t\langle n, x \rangle$ with $t > 0$: then $-2t|\langle n, x \rangle|^2 + t^2|\langle n, x \rangle|^2\|x\|^2 \ge 0$. Divide by $t$ and let $t \to 0^+$ to get $\langle n, x \rangle = 0$. So $n \in M^\perp$.
Pythagoras. $m \perp n$, so $\|h\|^2 = \|m + n\|^2 = \|m\|^2 + \|n\|^2$.
Uniqueness. If $h = m + n = m' + n'$ with $m, m' \in M$ and $n, n' \in M^\perp$, then $m - m' = n' - n$ lies in $M \cap M^\perp$. A vector $x$ in both has $x \perp x$, so $\|x\|^2 = 0$ and $x = 0$. Hence $m = m'$ and $n = n'$. In particular any decomposition has $m$ equal to the nearest point.
Linearity. If $h_1 = m_1 + n_1$ and $h_2 = m_2 + n_2$, then $h_1 + \alpha h_2 = (m_1 + \alpha m_2) + (n_1 + \alpha n_2)$ with the first piece in $M$ and the second in $M^\perp$. By uniqueness, this is the decomposition of $h_1 + \alpha h_2$. $\blacksquare$
The two pieces get names.
Let $M$ be a closed subspace of $H$ and write $h = m + n$ with $m \in M$, $n \in M^\perp$. The maps $P\colon h \mapsto m$ and $Q\colon h \mapsto n$ are the orthogonal projections of $H$ onto $M$ and onto $M^\perp$; we write $m = Ph$ and $n = Qh$.
So $h = Ph + Qh$ always, with the two pieces at a right angle. Here's $\mathbb{R}^3$ with $M$ a plane: $Ph$ is the shadow and $Qh$ is the drop.
Now move $h$ and the plane yourself. Each mission is a special case of the theorem.
In $L^2$ the same picture computes best approximations. Projecting onto a space of polynomials gives the polynomial closest to $f$ in the $L^2$ sense, which is usually not the Taylor polynomial.
In $L^2([0, 1])$, let $M$ be the polynomials of degree at most $1$. Find $P(t^2)$ and the distance from $t^2$ to $M$.
$M = \operatorname{span}\{1, t - \frac{1}{2}\}$, and these two are orthogonal: $\int_0^1 (t - \frac{1}{2})\,dt = 0$. $M$ is finite-dimensional, hence closed. Try $m = a + b(t - \frac{1}{2})$ and ask for $t^2 - m \perp 1$ and $t^2 - m \perp t - \frac{1}{2}$: $$\langle t^2 - m, 1 \rangle = \tfrac{1}{3} - a = 0, \qquad \langle t^2 - m, t - \tfrac{1}{2} \rangle = \tfrac{1}{12} - \tfrac{b}{12} = 0,$$ using $\int_0^1 t^2(t - \frac{1}{2})\,dt = \frac{1}{4} - \frac{1}{6} = \frac{1}{12}$ and $\|t - \frac{1}{2}\|^2 = \frac{1}{12}$. So $a = \frac{1}{3}$, $b = 1$, and $t^2 - m$ is orthogonal to both spanning vectors, hence to all of $M$. By uniqueness in the theorem, $$P(t^2) = \tfrac{1}{3} + (t - \tfrac{1}{2}) = t - \tfrac{1}{6}.$$
The distance is $\|Q(t^2)\| = \|t^2 - t + \frac{1}{6}\|$. By Pythagoras, $\|Q(t^2)\|^2 = \|t^2\|^2 - \|P(t^2)\|^2 = \frac{1}{5} - \left(\frac{1}{9} + \frac{1}{12}\right) = \frac{1}{180}$, so the distance is $\frac{1}{\sqrt{180}} = \frac{1}{6\sqrt{5}}$.
Notice the trick: guess the form of $Ph$, impose orthogonality, and let uniqueness certify the answer. Projections have a few structural properties worth proving once.
Let $P$ and $Q$ be the orthogonal projections onto a closed subspace $M$ and onto $M^\perp$. Show that $P^2 = P$, $\|Ph\| \le \|h\|$, $\langle Px, y \rangle = \langle x, Py \rangle$ for all $x, y \in H$, and $\ker P = M^\perp$.
Proof. If $m \in M$, then $m = m + 0$ is its decomposition, so $Pm = m$. Since $Ph \in M$, $P(Ph) = Ph$: $P^2 = P$.
By Pythagoras, $\|h\|^2 = \|Ph\|^2 + \|Qh\|^2 \ge \|Ph\|^2$.
Write $x = Px + Qx$ and $y = Py + Qy$. Since $Px \perp Qy$ and $Qx \perp Py$, $$\langle Px, y \rangle = \langle Px, Py \rangle + \langle Px, Qy \rangle = \langle Px, Py \rangle = \langle Px, Py \rangle + \langle Qx, Py \rangle = \langle x, Py \rangle.$$
If $h \in M^\perp$, then $h = 0 + h$ is its decomposition, so $Ph = 0$. Conversely, if $Ph = 0$, then $h = Qh \in M^\perp$. So $\ker P = M^\perp$. $\blacksquare$
Now compute some projections. In $\mathbb{C}^n$ and in $L^2$, projecting onto a line $\operatorname{span}\{u\}$ is one formula: $Ph = c\,u$ with $c = \langle h, u \rangle / \|u\|^2$, because that $c$ makes $h - cu \perp u$: $$\langle h - cu, u \rangle = \langle h, u \rangle - c\|u\|^2 = 0.$$
The order inside $\langle h, u \rangle$ matters. $\langle u, h \rangle$ is its complex conjugate, so writing it gives $\overline{c}$, a wrong answer that looks completely reasonable. And the denominator is $\|u\|^2$, not $\|u\|$: if $u$ isn't a unit vector, forgetting the square is the other classic slip.
Loading practice engine...
The distance from $h$ to $M$ is $\|Qh\|$, and Pythagoras computes it without ever writing $Qh$ down: $\|Qh\|^2 = \|h\|^2 - \|Ph\|^2$.
Loading practice engine...
One consequence of the theorem gets used constantly: a closed subspace that isn't everything leaves room for a nonzero perpendicular vector.
If $M$ is a closed subspace of $H$ and $M \neq H$, then $M^\perp \neq \{0\}$.
Proof. Pick $h \in H \setminus M$. Then $Qh = h - Ph \in M^\perp$, and $Qh \neq 0$ because $Ph \in M$ while $h \notin M$. $\blacksquare$
"Closed" matters here. $C([0, 1])$ is a proper subspace of $L^2([0, 1])$, but continuous functions are dense in $L^2$, so anything orthogonal to all of them is orthogonal to everything, and $C([0, 1])^\perp = \{0\}$. Taking the complement twice shows the same effect.
If $M$ is a closed subspace of $H$, prove that $M = (M^\perp)^\perp$. Is there a similar true statement for subspaces $M$ that are not necessarily closed?
Proof. If $x \in M$, then $\langle x, y \rangle = 0$ for every $y \in M^\perp$, so $x \in (M^\perp)^\perp$. Conversely, let $x \in (M^\perp)^\perp$ and write $x = Px + Qx$ with $Px \in M$ and $Qx \in M^\perp$. Then $$0 = \langle x, Qx \rangle = \langle Px, Qx \rangle + \langle Qx, Qx \rangle = \|Qx\|^2,$$ so $Qx = 0$ and $x = Px \in M$. Hence $M = (M^\perp)^\perp$.
For any subspace $M$, $(M^\perp)^\perp = \overline{M}$. First, $M^\perp = \overline{M}^\perp$: since $M \subseteq \overline{M}$, $\overline{M}^\perp \subseteq M^\perp$; and if $y \in M^\perp$ and $x_n \to x$ with $x_n \in M$, then $\langle x, y \rangle = \lim \langle x_n, y \rangle = 0$ by continuity, so $y \in \overline{M}^\perp$. Then, since $\overline{M}$ is a closed subspace (the Exercise above), $(M^\perp)^\perp = (\overline{M}^\perp)^\perp = \overline{M}$. So the closed case is the only one where $(M^\perp)^\perp = M$. $\blacksquare$
Fix $y \in H$. The map $x \mapsto \langle x, y \rangle$ is a linear functional, and Cauchy–Schwarz bounds it: $|\langle x, y \rangle| \le \|y\|\,\|x\|$. The question is whether every well-behaved linear functional looks like this. "Well-behaved" needs a definition.
A linear functional $\Lambda$ on a normed space $X$ is bounded if there is a constant $C$ with $|\Lambda x| \le C\|x\|$ for every $x \in X$. Its norm is $\|\Lambda\| = \sup\{|\Lambda x| : x \in X, \ \|x\| \le 1\}$.
So $\|\Lambda\|$ is the smallest $C$ that works, and $|\Lambda x| \le \|\Lambda\|\,\|x\|$. The word "bounded" here trips people up.
A bounded linear functional is not a bounded function on $X$: unless $\Lambda = 0$, $|\Lambda(tx)| = |t|\,|\Lambda x|$ grows without limit as $t$ does. "Bounded" means bounded on the unit ball, or equivalently $|\Lambda x| \le C\|x\|$.
For linear functionals, bounded is the same as continuous, which is why the notes say "continuous (hence bounded)".
A linear functional $\Lambda$ on a normed space $X$ is continuous if and only if it is bounded.
Proof. If $|\Lambda x| \le C\|x\|$, then $|\Lambda x - \Lambda z| = |\Lambda(x - z)| \le C\|x - z\|$, so $\Lambda$ is (uniformly) continuous.
If $\Lambda$ is continuous at $0$, there is $\delta > 0$ with $|\Lambda x| \le 1$ whenever $\|x\| \le \delta$. For $x \neq 0$, the vector $\frac{\delta}{\|x\|}x$ has norm $\delta$, so $\frac{\delta}{\|x\|}|\Lambda x| \le 1$, that is, $|\Lambda x| \le \frac{1}{\delta}\|x\|$. $\blacksquare$
The hypothesis has teeth. A functional that looks perfectly reasonable can fail it, if the norm can't see what the functional is measuring.
On $C([0, 1])$ with the norm $\|f\|_2 = (\int_0^1 |f|^2\,dt)^{1/2}$, show that $\Lambda f = f(0)$ is a linear functional that is not bounded.
Linearity is clear. Let $f_n(t) = \max\{0, 1 - nt\}$, a tent of height $1$ and width $\frac{1}{n}$. Then $\Lambda f_n = 1$, while $$\|f_n\|_2^2 = \int_0^{1/n} (1 - nt)^2\,dt = \frac{1}{3n}.$$ A bound $|\Lambda f| \le C\|f\|_2$ would give $1 \le \frac{C}{\sqrt{3n}}$ for every $n$, which fails for large $n$. So $\Lambda$ is unbounded, and no $y$ represents it.
The $L^2$ norm can't see a single point, so evaluation at a point is invisible to it. Functionals the norm can see are all inner products, and the proof goes through the kernel. A nonzero functional sends everything to $\mathbb{C}$, a one-dimensional space, so its kernel should leave exactly one dimension perpendicular to it. Prove that first.
Let $L \not\equiv 0$ be a bounded linear functional on $H$ and $M = \{x : Lx = 0\}$. Prove that $M^\perp$ is a vector space of dimension $1$. (If $L \equiv 0$, then $M = H$ and $M^\perp = \{0\}$, so the hypothesis is needed.)
Proof. $L$ is continuous, so $M = L^{-1}(\{0\})$ is a closed subspace, and $M \neq H$ because $L \not\equiv 0$. By the corollary, there is $z \in M^\perp$ with $z \neq 0$. Then $Lz \neq 0$, since otherwise $z \in M \cap M^\perp = \{0\}$.
Let $w \in M^\perp$. The vector $v = w - \frac{Lw}{Lz}z$ is in $M^\perp$ (a subspace), and $Lv = Lw - \frac{Lw}{Lz}Lz = 0$, so $v \in M$ too. Hence $v \in M \cap M^\perp = \{0\}$, and $w = \frac{Lw}{Lz}z$. So $M^\perp = \mathbb{C}z$, which has dimension $1$. $\blacksquare$
With $M^\perp = \mathbb{C}z$ in hand, every $x$ splits as $Px + cz$, and $\Lambda$ only sees the $cz$ part. Matching that with an inner product pins $y$ down.
Let $\Lambda$ be a bounded (equivalently, continuous) linear functional on a Hilbert space $H$. Then there is a unique $y \in H$ such that $$\Lambda x = \langle x, y \rangle \quad \text{for every } x \in H.$$ Moreover, $\|\Lambda\| = \|y\|$.
Proof. If $\Lambda \equiv 0$, take $y = 0$. Otherwise let $M = \ker \Lambda$. $M$ is closed because $\Lambda$ is continuous and $M = \Lambda^{-1}(\{0\})$, and $M \neq H$. By the Exercise above, $M^\perp = \mathbb{C}z$ for some $z \neq 0$ with $\Lambda z \neq 0$. Let $$y = \frac{\overline{\Lambda z}}{\|z\|^2}\,z.$$
It represents $\Lambda$. Given $x \in H$, write $x = Px + Qx$ with $Px \in M$ and $Qx = cz \in M^\perp$. Then $\Lambda x = \Lambda(Px) + c\Lambda z = c\Lambda z$. On the other side, $y \in M^\perp$, so $\langle Px, y \rangle = 0$, and pulling the scalar out of the second slot conjugates it: $$\langle x, y \rangle = \langle cz, y \rangle = c\,\frac{\Lambda z}{\|z\|^2}\langle z, z \rangle = c\Lambda z = \Lambda x.$$
Uniqueness. If $\langle x, y \rangle = \langle x, y' \rangle$ for every $x$, then $\langle x, y - y' \rangle = 0$ for every $x$; take $x = y - y'$ to get $\|y - y'\|^2 = 0$.
The norm. Cauchy–Schwarz gives $|\Lambda x| = |\langle x, y \rangle| \le \|x\|\|y\|$, so $\|\Lambda\| \le \|y\|$. If $y \neq 0$, then $x = y/\|y\|$ has norm $1$ and $\Lambda x = \langle y, y \rangle / \|y\| = \|y\|$, so $\|\Lambda\| \ge \|y\|$. $\blacksquare$
So a bounded functional on a Hilbert space is the same thing as a vector, and its norm is that vector's length. Geometrically, $y$ is the normal to the kernel, scaled so that the level sets $\{\Lambda = c\}$ are spaced $1/\|y\|$ apart. Here it is in $\mathbb{R}^2$, where the kernel is a line.
Read the picture backward and you can find $y$ from the level lines alone: its direction is perpendicular to them, and its length is one over their spacing.
In $\mathbb{C}^n$ and $L^2$, you find $y$ by reading it off the formula, conjugated.
On $L^2([0, 1])$, let $\Lambda f = \int_0^1 f(t)(1 + it)\,dt$. Find the $y$ with $\Lambda f = \langle f, y \rangle$, and compute $\|\Lambda\|$.
We need $\int_0^1 f\,\overline{y}\,dt = \int_0^1 f\,(1 + it)\,dt$ for all $f$, so $\overline{y(t)} = 1 + it$ and $y(t) = 1 - it$. (Uniqueness says this is the only one.)
Then $\|\Lambda\| = \|y\|_2$, and $|1 - it|^2 = 1 + t^2$, so $$\|\Lambda\|^2 = \int_0^1 (1 + t^2)\,dt = \frac{4}{3}, \qquad \|\Lambda\| = \frac{2}{\sqrt{3}} = \frac{2\sqrt{3}}{3}.$$
On The Riesz Representation Theorem, a positive linear functional on $C_c(X)$ turned out to be integration against a measure. Same Riesz, same idea (a functional is secretly a concrete object), different space. On a Hilbert space the concrete object is just a vector.
Two quick ones to lock it in. First the vector, minding the conjugate.
Loading practice engine...
Then the norm. It's the $\ell^2$ length of $y$, not the sum of the coefficients and not the biggest one.
Loading practice engine...
Projecting onto a line was one formula. The next page, Orthonormal Bases, projects onto infinitely many perpendicular lines at once, and $L^2$ of the circle turns into Fourier series.
That's the whole toolkit: an inner product gives lengths and angles, the parallelogram law turns minimizing sequences into convergent ones, projections split every vector into two perpendicular pieces, and Riesz says functionals are vectors.
Every question so far came right after its idea, which is a hint. Down here the hint is gone. First, true or false: a claim about anything on this page, some of them the kind that sounds right and isn't. Decide, then read why.
Loading practice engine...
Next, the proofs. Each one is a line from a proof on this page with a hole in it; pick what fills the hole.
Loading practice engine...
Then the mixed set. Every question type on this page except true or false goes into one hat, and each click draws a new one at random. Flip the switch to type your answer; the types that need their answer choices will say so.
Loading practice engine...
Last, the Spicy Bonus. A bounded functional on a subspace that isn't even closed, extended to the whole space without growing its norm. In a general normed space this needs the Hahn–Banach theorem, and the extension isn't unique. In a Hilbert space, Riesz does it in a few lines, and uniqueness comes from Pythagoras.
Let $f$ be a bounded linear functional on a subspace $M$ of a Hilbert space $H$ (not necessarily closed), with $\|f\| = \sup\{|f(x)| : x \in M, \ \|x\| \le 1\}$. Prove that $f$ has a unique norm-preserving extension to a bounded linear functional $F$ on $H$ (that is, $F = f$ on $M$ and $\|F\| = \|f\|$), and that this extension vanishes on $M^\perp$.
Proof. Step 1: extend to $\overline{M}$. Let $x \in \overline{M}$ and pick $x_n \in M$ with $x_n \to x$. Then $|f(x_n) - f(x_m)| \le \|f\|\,\|x_n - x_m\|$, so $(f(x_n))$ is Cauchy in $\mathbb{C}$ and converges. Its limit doesn't depend on the sequence: if also $x_n' \to x$, then $|f(x_n) - f(x_n')| \le \|f\|\,\|x_n - x_n'\| \to 0$. Define $F_0(x) = \lim f(x_n)$. Taking sums and scalar multiples of approximating sequences shows $F_0$ is linear, $F_0 = f$ on $M$ (use the constant sequence), and $|F_0(x)| = \lim |f(x_n)| \le \lim \|f\|\,\|x_n\| = \|f\|\,\|x\|$. So $\|F_0\| \le \|f\|$, and $\|F_0\| \ge \|f\|$ because $F_0$ extends $f$.
Step 2: Riesz on $\overline{M}$. $\overline{M}$ is a closed subspace of a complete space, so it is a Hilbert space. By the Riesz representation theorem there is $y_0 \in \overline{M}$ with $F_0(x) = \langle x, y_0 \rangle$ for $x \in \overline{M}$, and $\|y_0\| = \|F_0\| = \|f\|$.
Step 3: the extension. Define $F(x) = \langle x, y_0 \rangle$ for all $x \in H$. It is a bounded linear functional on $H$, it agrees with $F_0$ (hence with $f$) on $M$, and $\|F\| = \|y_0\| = \|f\|$ by the norm part of Riesz. If $x \in M^\perp$, then $x \in \overline{M}^\perp$ (continuity of the inner product, as in the $(M^\perp)^\perp$ Exercise), and $y_0 \in \overline{M}$, so $F(x) = \langle x, y_0 \rangle = 0$.
Step 4: uniqueness. Let $G$ be any bounded linear functional on $H$ with $G = f$ on $M$ and $\|G\| = \|f\|$. By Riesz, $G(x) = \langle x, z \rangle$ with $\|z\| = \|f\|$. For $x \in M$, $\langle x, z \rangle = f(x) = \langle x, y_0 \rangle$, so $\langle x, z - y_0 \rangle = 0$: $z - y_0 \in M^\perp = \overline{M}^\perp$. Write $w = z - y_0$. Since $y_0 \in \overline{M}$ and $w \in \overline{M}^\perp$, Pythagoras gives $$\|f\|^2 = \|z\|^2 = \|y_0 + w\|^2 = \|y_0\|^2 + \|w\|^2 = \|f\|^2 + \|w\|^2.$$ So $w = 0$, $z = y_0$, and $G = F$. $\blacksquare$