On Inner Products & Projections we set up the three facts this page leans on: a Hilbert space is a complex inner product space that is complete in the norm $\|x\| = \sqrt{\langle x, x \rangle}$; orthogonal vectors obey Pythagoras, $\|x + y\|^2 = \|x\|^2 + \|y\|^2$ when $\langle x, y \rangle = 0$; and the norm determines the inner product, by polarization.
In $\mathbb{R}^3$ you find the coordinates of a vector by dotting it with $(1, 0, 0)$, $(0, 1, 0)$, and $(0, 0, 1)$. Three numbers, and they rebuild the vector exactly. This page asks how far that idea goes. Can you take a function in $L^2$, dot it against a list of perpendicular unit vectors, and get numbers that rebuild the function?
Yes, and the answer is one of the great payoffs of the whole course. The perpendicular unit vectors for functions on a circle are the waves $e^{int}$, and the numbers you get are Fourier coefficients. Along the way we'll have to sum over sets too big to list, decide what "basis" even means when no finite combination is enough, and meet an operator that keeps every length but still misses vectors. At the very end, a sawtooth wave will hand us $\sum \frac{1}{n^2} = \frac{\pi^2}{6}$.
Throughout, $H$ is a Hilbert space with inner product $\langle x, y \rangle$, linear in $x$ and conjugate-linear in $y$.
The coordinate vectors of $\mathbb{R}^3$ have two properties that make dotting work: each has length $1$, and any two are perpendicular. That's the whole definition. The only change is that we allow any number of vectors, indexed by any set $A$, countable or not.
A family $\{u_\alpha\}_{\alpha \in A}$ in $H$ is orthonormal if $\langle u_\alpha, u_\beta \rangle = 0$ for all $\alpha \neq \beta$ in $A$ and $\|u_\alpha\| = 1$ for every $\alpha \in A$.
Two examples run through this whole page: the point masses in a sequence space, and the waves on a circle. For the waves we use the normalized inner product on $T = [-\pi, \pi]$ (the circle, with its ends identified, so functions are $2\pi$-periodic): $$\langle f, g \rangle = \frac{1}{2\pi} \int_{-\pi}^{\pi} f(t) \overline{g(t)} \, dt.$$
(a) In $\ell^2(\mathbb{N})$, let $\chi_{\{k\}}$ be the sequence with a $1$ in place $k$ and $0$ elsewhere. Show $\{\chi_{\{k\}} : k \ge 1\}$ is orthonormal. (b) In $L^2(T)$, let $e_n(t) = e^{int}$. Show $\{e_n : n \in \mathbb{Z}\}$ is orthonormal. (c) What goes wrong in (b) without the $\frac{1}{2\pi}$?
(a) $\langle \chi_{\{j\}}, \chi_{\{k\}} \rangle = \sum_m \chi_{\{j\}}(m) \overline{\chi_{\{k\}}(m)}$, and the only possibly nonzero term is $m = j = k$. So the inner product is $1$ when $j = k$ and $0$ otherwise.
(b) $e_n \overline{e_m} = e^{i(n - m)t}$. If $n = m$ this is $1$, and $\frac{1}{2\pi} \int_{-\pi}^{\pi} 1 \, dt = 1$. If $n \neq m$, with $k = n - m \neq 0$, $$\frac{1}{2\pi} \int_{-\pi}^{\pi} e^{ikt} \, dt = \frac{1}{2\pi} \left[ \frac{e^{ikt}}{ik} \right]_{-\pi}^{\pi} = \frac{e^{ik\pi} - e^{-ik\pi}}{2\pi i k} = \frac{\sin(k\pi)}{\pi k} = 0.$$
(c) With $\int_{-\pi}^{\pi} f \overline{g} \, dt$ instead, the $e_n$ are still orthogonal, but $\|e_n\|^2 = 2\pi$, so they aren't unit vectors. That's the reason for the normalization: it makes $\{e_n\}$ orthonormal with no extra constants floating around.
Perpendicular vectors can't conspire to cancel each other out. Here's the first thing that buys.
Show that every finite orthonormal set $\{u_1, \ldots, u_n\}$ is linearly independent.
Proof. Suppose $\sum_{k=1}^{n} c_k u_k = 0$. Take the inner product of both sides with $u_j$. By linearity in the first slot and orthonormality, $$0 = \left\langle \sum_{k=1}^{n} c_k u_k, u_j \right\rangle = \sum_{k=1}^{n} c_k \langle u_k, u_j \rangle = c_j.$$ This holds for every $j$, so all the $c_j$ are $0$. $\blacksquare$
Now the dotting itself. In $\mathbb{R}^3$, the dot product with a coordinate vector reads off a coordinate. In general, the inner product with $u_\alpha$ reads off the "$u_\alpha$-coordinate", and these numbers get a famous name.
Let $\{u_\alpha\}_{\alpha \in A}$ be orthonormal in $H$ and $x \in H$. The complex numbers $\langle x, u_\alpha \rangle$, $\alpha \in A$, are the Fourier coefficients of $x$ relative to $\{u_\alpha\}$.
On $T$, relative to $\{e_n\}$, they're written $\hat{f}(n) = \langle f, e_n \rangle = \frac{1}{2\pi} \int_{-\pi}^{\pi} f(t) e^{-int} \, dt$. Notice there's no extra constant in front: the normalized inner product already did that job. When a vector is a finite combination of the $u_\alpha$, the Fourier coefficients are exactly the weights in the combination, and its length is Pythagoras with as many legs as you like.
Let $\{u_\alpha\}_{\alpha \in A}$ be orthonormal, $F \subseteq A$ finite, and $c_\alpha \in \mathbb{C}$ for $\alpha \in F$. Then $y = \sum_{\alpha \in F} c_\alpha u_\alpha$ has $$\langle y, u_\beta \rangle = \begin{cases} c_\beta, & \beta \in F, \\ 0, & \beta \notin F, \end{cases} \qquad \text{and} \qquad \|y\|^2 = \sum_{\alpha \in F} |c_\alpha|^2.$$
Proof. By linearity, $\langle y, u_\beta \rangle = \sum_{\alpha \in F} c_\alpha \langle u_\alpha, u_\beta \rangle$. Every term with $\alpha \neq \beta$ is $0$, and the term $\alpha = \beta$ (present only if $\beta \in F$) is $c_\beta$. For the norm, expand using linearity in the first slot and conjugate-linearity in the second: $$\|y\|^2 = \left\langle \sum_{\alpha \in F} c_\alpha u_\alpha, \sum_{\beta \in F} c_\beta u_\beta \right\rangle = \sum_{\alpha, \beta \in F} c_\alpha \overline{c_\beta} \langle u_\alpha, u_\beta \rangle = \sum_{\alpha \in F} c_\alpha \overline{c_\alpha} = \sum_{\alpha \in F} |c_\alpha|^2. \ \blacksquare$$
So reading off a coefficient is easy once you write $f$ in terms of the $e_n$. For sines and cosines, that's Euler's formula, run backward.
Find every Fourier coefficient $\hat{f}(n)$ of $f(t) = 3 + 4\cos 2t - 2\sin t$ on $T$.
Use $\cos \theta = \frac{e^{i\theta} + e^{-i\theta}}{2}$ and $\sin \theta = \frac{e^{i\theta} - e^{-i\theta}}{2i}$: $$f = 3e_0 + 2(e_2 + e_{-2}) - \frac{2}{2i}(e_1 - e_{-1}) = 3e_0 + 2e_2 + 2e_{-2} + i e_1 - i e_{-1},$$ since $-\frac{1}{i} = i$. By the proposition, the coefficients are the weights: $$\hat{f}(0) = 3, \quad \hat{f}(\pm 2) = 2, \quad \hat{f}(1) = i, \quad \hat{f}(-1) = -i,$$ and $\hat{f}(n) = 0$ for every other $n$. As a check, $\|f\|^2 = 9 + 4 + 4 + 1 + 1 = 19$, which matches $\frac{1}{2\pi} \int_{-\pi}^{\pi} |f|^2 \, dt$.
For a function that isn't a finite combination, like a square wave, you compute the integral. Here's a way to see what that integral measures: $\hat{f}(n)$ is the average of $f$ against one wave. Where $f$ and the wave agree in sign the product is positive, where they disagree it's negative, and the coefficient is what's left after the cancelling. Guess a few before you look.
Your turn. Some of these are finite combinations in disguise, and some need the integral.
Loading practice engine...
Now suppose $x$ is not a combination of finitely many $u_\alpha$. You can still form the sum of its first few coordinates, $s(x) = \sum_{\alpha \in F} \langle x, u_\alpha \rangle u_\alpha$. It isn't $x$. But it's the best you can do with those vectors, and the reason is a right angle.
The picture says $x - s(x)$ is perpendicular to the whole floor. Then any other point $s$ on the floor is the far corner of a right triangle whose hypotenuse $x - s$ beats the leg $x - s(x)$.
Let $\{u_\alpha\}_{\alpha \in A}$ be orthonormal, $F \subseteq A$ finite, $M_F = \operatorname{span}\{u_\alpha : \alpha \in F\}$, and $x \in H$. Put $s(x) = \sum_{\alpha \in F} \langle x, u_\alpha \rangle u_\alpha$. Then $$\|x - s(x)\| < \|x - s\| \quad \text{for every } s \in M_F \text{ with } s \neq s(x).$$ So $s(x)$ is the unique point of $M_F$ closest to $x$.
Proof. By the proposition above, $\langle s(x), u_\beta \rangle = \langle x, u_\beta \rangle$ for every $\beta \in F$, so $\langle x - s(x), u_\beta \rangle = 0$. The $u_\beta$ with $\beta \in F$ span $M_F$, so $x - s(x)$ is orthogonal to every vector of $M_F$.
Let $s \in M_F$. Then $s(x) - s \in M_F$, so it is orthogonal to $x - s(x)$, and Pythagoras gives $$\|x - s\|^2 = \|(x - s(x)) + (s(x) - s)\|^2 = \|x - s(x)\|^2 + \|s(x) - s\|^2.$$ If $s \neq s(x)$, the last term is positive, so $\|x - s\|^2 > \|x - s(x)\|^2$. $\blacksquare$
The shortcut, "take inner products and you've got the best weights", depends on the vectors being orthonormal. Try it with two pairs of directions, one orthonormal and one slanted, and see where it breaks.
The proof also hands us a Pythagorean split of $x$ itself, with $s = 0$. It's small, but the rest of the page runs on it.
With $F$, $M_F$, and $s(x)$ as in the theorem, $$\|x\|^2 = \|x - s(x)\|^2 + \sum_{\alpha \in F} |\langle x, u_\alpha \rangle|^2. \quad \text{In particular,} \quad \sum_{\alpha \in F} |\langle x, u_\alpha \rangle|^2 \le \|x\|^2.$$
Proof. In the proof of the theorem, $x - s(x)$ is orthogonal to $M_F$, and $s(x) \in M_F$. So Pythagoras applies to $x = (x - s(x)) + s(x)$: $$\|x\|^2 = \|x - s(x)\|^2 + \|s(x)\|^2,$$ and $\|s(x)\|^2 = \sum_{\alpha \in F} |\langle x, u_\alpha \rangle|^2$ by the proposition. The inequality drops the first term, which is at least $0$. $\blacksquare$
That's the distance formula you'll use most: the squared distance from $x$ to $M_F$ is $\|x\|^2$ minus the sum of the squared coefficients. Here it is with numbers.
In $\mathbb{R}^3$ with the dot product, let $x = (1, 2, 3)$, $u_1 = \frac{1}{\sqrt{2}}(1, 1, 0)$, and $u_2 = (0, 0, 1)$. Find the closest point to $x$ in $M = \operatorname{span}\{u_1, u_2\}$, and the squared distance.
The pair is orthonormal: each has length $1$ and $u_1 \cdot u_2 = 0$. The coefficients are $\langle x, u_1 \rangle = \frac{1 + 2}{\sqrt{2}} = \frac{3}{\sqrt{2}}$ and $\langle x, u_2 \rangle = 3$, so $$s(x) = \frac{3}{\sqrt{2}} \cdot \frac{1}{\sqrt{2}}(1, 1, 0) + 3(0, 0, 1) = \left(\tfrac{3}{2}, \tfrac{3}{2}, 3\right).$$ Then $x - s(x) = (-\frac{1}{2}, \frac{1}{2}, 0)$, with squared length $\frac{1}{2}$. The corollary agrees: $\|x\|^2 - \frac{9}{2} - 9 = 14 - \frac{27}{2} = \frac{1}{2}$.
Orthonormal sets are so convenient that you'd want to manufacture them. Any linearly independent list can be straightened out, one vector at a time, by subtracting its best approximation from the ones already built.
Let $x_1, x_2, \ldots$ be linearly independent in $H$. Put $u_1 = \frac{x_1}{\|x_1\|}$ and, having $u_1, \ldots, u_{n-1}$, $$v_n = x_n - \sum_{i=1}^{n-1} \langle x_n, u_i \rangle u_i, \qquad u_n = \frac{v_n}{\|v_n\|}.$$ Show that $(u_n)$ is orthonormal and that $\operatorname{span}\{x_1, \ldots, x_N\} = \operatorname{span}\{u_1, \ldots, u_N\}$ for every $N$. (This is Gram–Schmidt.)
Proof. By induction on $N$ we show: $u_1, \ldots, u_N$ are well defined and orthonormal, and they span the same space $S_N$ as $x_1, \ldots, x_N$. For $N = 1$, $x_1 \neq 0$ (independent vectors are nonzero), so $u_1$ is a unit vector with the same span.
Suppose the claim holds for $N - 1$. Then $v_N = x_N - s(x_N)$, where $s(x_N)$ is the best approximation to $x_N$ in $S_{N-1} = \operatorname{span}\{u_1, \ldots, u_{N-1}\}$. If $v_N = 0$, then $x_N \in S_{N-1} = \operatorname{span}\{x_1, \ldots, x_{N-1}\}$, contradicting independence. So $v_N \neq 0$, and $u_N$ is a unit vector. By the proof of the best approximation theorem, $v_N$ is orthogonal to $S_{N-1}$, so $u_N \perp u_i$ for $i < N$.
For the spans: $u_N$ is a combination of $x_N$ and vectors of $S_{N-1}$, so $u_N \in \operatorname{span}\{x_1, \ldots, x_N\}$. Conversely $x_N = \|v_N\| u_N + \sum_{i < N} \langle x_N, u_i \rangle u_i \in \operatorname{span}\{u_1, \ldots, u_N\}$. Together with the induction hypothesis, the two spans agree. $\blacksquare$
Gram–Schmidt gives an orthonormal set with the same span at every stage, but it doesn't say the set is "big enough". That question, when an orthonormal set captures everything, is the rest of the page. First, compute some distances.
Loading practice engine...
The corollary says every finite batch of squared coefficients fits under $\|x\|^2$. We'd like to say the same about all of them at once. The trouble is that $A$ might be uncountable, and then "add them all up" doesn't even have an order to add them in. So we define the sum without one.
Let $A$ be any set and $\varphi$ a function on $A$ with values in $[0, \infty]$. The sum of $\varphi$ over $A$ is $$\sum_{\alpha \in A} \varphi(\alpha) = \sup\left\{ \sum_{\alpha \in F} \varphi(\alpha) : F \subseteq A \text{ finite} \right\}.$$
A quick example shows how it behaves, including when it's infinite.
Compute $\sum_{x \in \mathbb{R}} \varphi(x)$ for (a) $\varphi(x) = \frac{1}{n^2}$ if $x = \frac{1}{n}$ for some $n \ge 1$, and $\varphi(x) = 0$ otherwise; (b) $\varphi(x) = 1$ for $x \in [0, 1]$ and $0$ otherwise.
(a) A finite $F \subseteq \mathbb{R}$ only picks up terms at points $\frac{1}{n}$, and those form a finite set of $n$'s, so every finite sum is at most $\sum_{n=1}^{\infty} \frac{1}{n^2}$. Taking $F = \{1, \frac{1}{2}, \ldots, \frac{1}{N}\}$ gets within any margin of it. So the sum is $\sum_{n=1}^{\infty} \frac{1}{n^2}$, which we'll show is $\frac{\pi^2}{6}$ later on this page.
(b) A finite $F \subseteq [0, 1]$ with $N$ points has sum $N$, and $N$ can be as large as you like. So the sum is $\infty$.
Two sanity checks. If $A = \mathbb{N}$, this is the ordinary series $\sum_{n=1}^{\infty} \varphi(n)$, since every finite set sits inside some $\{1, \ldots, N\}$ and the partial sums only grow. And in general it's the integral of $\varphi$ against counting measure on $A$, the supremum of the integrals of finite step functions under $\varphi$. Once the terms can be complex, we only allow square-summable ones.
For a set $A$, $\ell^2(A)$ is the set of complex-valued functions $\varphi$ on $A$ with $\sum_{\alpha \in A} |\varphi(\alpha)|^2 < \infty$, with the inner product $$\langle \varphi, \psi \rangle = \sum_{\alpha \in A} \varphi(\alpha) \overline{\psi(\alpha)}.$$
$\ell^2(A)$ is just $L^2(\mu)$ for counting measure $\mu$ on $A$, so completeness of $L^p$ makes it a Hilbert space for free. With $A = \mathbb{N}$ it's the familiar $\ell^2$.
(The sum defining $\langle \varphi, \psi \rangle$ converges absolutely, because $|\varphi \overline{\psi}| \le \frac{1}{2}(|\varphi|^2 + |\psi|^2)$, and it's the $L^2$ inner product $\int_A \varphi \overline{\psi} \, d\mu$.) Even when $A$ is uncountable, a square-summable function can only be nonzero on a countable set. The reason: only a few terms can be big.
If $\varphi \in \ell^2(A)$, then $\{\alpha \in A : \varphi(\alpha) \neq 0\}$ is countable. More precisely, for each $n \ge 1$ the set $A_n = \{\alpha : |\varphi(\alpha)| > \frac{1}{n}\}$ has fewer than $n^2 \sum_{\alpha \in A} |\varphi(\alpha)|^2$ elements (or none).
Proof. Let $S = \sum_{\alpha \in A} |\varphi(\alpha)|^2 < \infty$, and let $F \subseteq A_n$ be finite and nonempty. Each $\alpha \in F$ has $|\varphi(\alpha)|^2 > \frac{1}{n^2}$, so $$\frac{|F|}{n^2} < \sum_{\alpha \in F} |\varphi(\alpha)|^2 \le S, \quad \text{that is,} \quad |F| < n^2 S.$$ Every finite subset of $A_n$ has fewer than $n^2 S$ elements, so $A_n$ itself is finite, with fewer than $n^2 S$ elements. Every $\alpha$ with $\varphi(\alpha) \neq 0$ has $|\varphi(\alpha)| > \frac{1}{n}$ for some $n$, so $\{\varphi \neq 0\} = \bigcup_{n} A_n$, a countable union of finite sets. $\blacksquare$
Watch the bar come down. At each height only finitely many coefficients stand above it, and every nonzero coefficient is above the bar eventually.
Uncountably many basis vectors, and every single vector only bothers talking to countably many of them. Introverts.
So a sum $\sum_{\alpha \in A} |\varphi(\alpha)|^2$ is really a countable series in disguise: list the nonzero terms in any order and add. With that settled, the finite corollary goes infinite in one line.
Let $\{u_\alpha\}_{\alpha \in A}$ be orthonormal in $H$. Then for every $h \in H$, $$\sum_{\alpha \in A} |\langle h, u_\alpha \rangle|^2 \le \|h\|^2.$$ In particular, $\alpha \mapsto \langle h, u_\alpha \rangle$ is in $\ell^2(A)$, and only countably many $\langle h, u_\alpha \rangle$ are nonzero.
Proof. For every finite $F \subseteq A$, the corollary to the best approximation theorem gives $\sum_{\alpha \in F} |\langle h, u_\alpha \rangle|^2 \le \|h\|^2$. The sum over $A$ is the supremum of these finite sums, so it is at most $\|h\|^2$ too. That makes the coefficients square-summable, and the last claim is the theorem above. $\blacksquare$
Bessel is an inequality, and it can be strict. That happens exactly when the orthonormal set is missing a direction that $h$ points along.
In $L^2(T)$, take the orthonormal set $\{e_n : n \ge 0\}$ and $h = e_{-1} + e_1$. Compare both sides of Bessel's inequality.
By orthonormality $\langle h, e_n \rangle = 1$ for $n = 1$ and $0$ for every other $n \ge 0$ (the $e_{-1}$ part is orthogonal to all of them). So the left side is $1$. By the proposition, $\|h\|^2 = 1 + 1 = 2$. Bessel holds, $1 < 2$, and the missing $1$ is exactly $\|e_{-1}\|^2$, the part of $h$ the set can't see.
Here's that gap as a picture. Stack the squared coefficients of $h(t) = t$ into a column as tall as $\|h\|^2$. One set fills it. The other, missing every odd frequency, stalls.
Now build your own orthonormal sets out of waves and see how much of a function each one catches.
Uncountable orthonormal sets aren't exotic, by the way. You can write one down in a line.
Find an uncountable orthonormal set in a Hilbert space, and describe the Fourier coefficients of a vector relative to it.
In $\ell^2(\mathbb{R})$, let $\chi_{\{x\}}$ be the function that is $1$ at $x$ and $0$ elsewhere. Exactly as in $\ell^2(\mathbb{N})$, $\langle \chi_{\{x\}}, \chi_{\{y\}} \rangle$ is $1$ if $x = y$ and $0$ otherwise, so $\{\chi_{\{x\}} : x \in \mathbb{R}\}$ is orthonormal, with one vector for each real number. For $\varphi \in \ell^2(\mathbb{R})$, the Fourier coefficients are $\langle \varphi, \chi_{\{x\}} \rangle = \varphi(x)$, and by the theorem above only countably many are nonzero.
Bessel gives a quick count: if $k$ coefficients each have size at least $c$, then $k c^2 \le \|h\|^2$. Try it.
Loading practice engine...
The example and the picture point to the right definition. An orthonormal set is "big enough" when Bessel's inequality never has a gap, for any vector at all.
On Inner Products & Norms in linear algebra, an orthonormal basis of $\mathbb{R}^n$ was a basis: every vector was a finite combination. In infinite dimensions that's hopeless, so "basis" will mean something weaker and better.
An orthonormal set $\{u_\alpha\}_{\alpha \in A}$ in $H$ is complete, or an orthonormal basis, if $$\|h\|^2 = \sum_{\alpha \in A} |\langle h, u_\alpha \rangle|^2 \quad \text{for every } h \in H.$$
There's a second natural meaning of "big enough": you can't add another vector and keep the set orthonormal.
An orthonormal set $U$ in $H$ is a maximal orthonormal set if the only orthonormal set $V$ with $U \subseteq V$ is $V = U$.
Maximality is easy to test on our two examples.
(a) Is $\{e_n : n \ge 0\}$ a maximal orthonormal set in $L^2(T)$? (b) Is $\{\chi_{\{k\}} : k \ge 1\}$ a maximal orthonormal set in $\ell^2(\mathbb{N})$?
(a) No. $e_{-1}$ is a unit vector orthogonal to every $e_n$ with $n \ge 0$, so $\{e_n : n \ge 0\} \cup \{e_{-1}\}$ is a strictly bigger orthonormal set.
(b) Yes. If $V \supseteq \{\chi_{\{k\}}\}$ is orthonormal and $x \in V$ is not one of the $\chi_{\{k\}}$, then $x \perp \chi_{\{k\}}$ for every $k$, so every entry $x_k = \langle x, \chi_{\{k\}} \rangle$ is $0$. Then $x = 0$, which isn't a unit vector. So no such $x$ exists, and $V = \{\chi_{\{k\}}\}$.
And there's a third: you can get as close as you like to anything using finite combinations. The big theorem of this page says all three are the same.
Let $\{u_\alpha\}_{\alpha \in A}$ be orthonormal in $H$, and let $P$ be the set of all finite linear combinations of the $u_\alpha$. The following are equivalent.
Proof. (1) ⇒ (2). Let $h \in H$ and $\varepsilon > 0$. By (1), the supremum of the finite sums $\sum_{\alpha \in B} |\langle h, u_\alpha \rangle|^2$ is $\|h\|^2$, so some finite $B$ has $\sum_{\alpha \in B} |\langle h, u_\alpha \rangle|^2 > \|h\|^2 - \varepsilon^2$. With $g_B = \sum_{\alpha \in B} \langle h, u_\alpha \rangle u_\alpha \in P$, the corollary to the best approximation theorem gives $$\|h - g_B\|^2 = \|h\|^2 - \sum_{\alpha \in B} |\langle h, u_\alpha \rangle|^2 < \varepsilon^2.$$
(2) ⇒ (3). Suppose (3) fails: there is $u$ with $\|u\| = 1$ and $u \perp u_\alpha$ for every $\alpha$. Let $p = \sum_{\alpha \in B} c_\alpha u_\alpha \in P$. Then $p \perp u$, so by Pythagoras $$\|u - p\|^2 = \|u\|^2 + \|p\|^2 \ge 1.$$ Every point of $P$ is at distance at least $1$ from $u$, so $P$ is not dense.
(3) ⇒ (1). Suppose (1) fails: some $h$ has $\sum_{\alpha} |\langle h, u_\alpha \rangle|^2 < \|h\|^2$ (Bessel rules out $>$). By Bessel, only countably many coefficients are nonzero; list those indices as $\alpha_1, \alpha_2, \ldots$ (if there are finitely many, the sums below are finite and there's nothing to take a limit of). Let $g_N = \sum_{k=1}^{N} \langle h, u_{\alpha_k} \rangle u_{\alpha_k}$. For $M > N$, by the proposition, $$\|g_M - g_N\|^2 = \sum_{k = N+1}^{M} |\langle h, u_{\alpha_k} \rangle|^2,$$ which tends to $0$ because the full series converges (it is at most $\|h\|^2$). So $(g_N)$ is Cauchy, and since $H$ is complete, $g_N \to g$ for some $g \in H$.
The inner product is continuous, so $\langle g, u_\beta \rangle = \lim_N \langle g_N, u_\beta \rangle$. If $\beta = \alpha_k$, this is $\langle h, u_\beta \rangle$ (true for every $N \ge k$). If $\beta$ is not on the list, every $\langle g_N, u_\beta \rangle = 0 = \langle h, u_\beta \rangle$. Either way $\langle h - g, u_\beta \rangle = 0$: $h - g$ is orthogonal to every $u_\beta$. Finally, by the corollary, $$\|h - g\|^2 = \lim_{N} \|h - g_N\|^2 = \|h\|^2 - \sum_{k=1}^{\infty} |\langle h, u_{\alpha_k} \rangle|^2 > 0.$$ So $u = \frac{h - g}{\|h - g\|}$ is a unit vector orthogonal to every $u_\alpha$, and $\{u_\alpha\} \cup \{u\}$ is a strictly bigger orthonormal set. Then $\{u_\alpha\}$ is not maximal. $\blacksquare$
The last step of that proof is the one worth seeing. When Bessel has a gap, the gap itself is a vector, perpendicular to everything in the set, and it becomes the new member.
There's a fourth equivalent form, and it's often the quickest to check in practice.
Show that an orthonormal set $\{u_\alpha\}$ in $H$ is an orthonormal basis if and only if the only vector orthogonal to every $u_\alpha$ is $0$.
Proof. (⇒) If $h \perp u_\alpha$ for every $\alpha$, then $\|h\|^2 = \sum_\alpha |\langle h, u_\alpha \rangle|^2 = 0$, so $h = 0$.
(⇐) If $\{u_\alpha\}$ were not maximal, some orthonormal $V \supsetneq \{u_\alpha\}$ would contain a unit vector $u$ outside the set, and $u \perp u_\alpha$ for every $\alpha$, with $u \neq 0$. That contradicts the hypothesis, so the set is maximal, and by the theorem ((3) ⇒ (1)) it is an orthonormal basis. $\blacksquare$
Use it on the point masses, which are the model for every orthonormal basis (you'll see why at the end of the page).
Show that $\{\chi_{\{\alpha\}} : \alpha \in A\}$ is an orthonormal basis of $\ell^2(A)$.
Proof. As in $\ell^2(\mathbb{N})$, $\langle \chi_{\{\alpha\}}, \chi_{\{\beta\}} \rangle = \sum_{\gamma \in A} \chi_{\{\alpha\}}(\gamma) \overline{\chi_{\{\beta\}}(\gamma)}$ is $1$ if $\alpha = \beta$ and $0$ otherwise, so the set is orthonormal. For $\varphi \in \ell^2(A)$, $\langle \varphi, \chi_{\{\alpha\}} \rangle = \varphi(\alpha)$, so $$\sum_{\alpha \in A} |\langle \varphi, \chi_{\{\alpha\}} \rangle|^2 = \sum_{\alpha \in A} |\varphi(\alpha)|^2 = \|\varphi\|^2,$$ which is the definition of an orthonormal basis. $\blacksquare$
Telling a basis from a near miss means checking three things in order: unit length, orthogonality (with the conjugate in the second slot), and nothing left out. Sort these.
The near misses are the instructive ones: a set can be perfectly orthonormal and still leave out one direction. Try a few more.
Loading practice engine...
One word of warning before we go on, because the name invites a mistake.
An orthonormal basis is usually not a basis in the linear-algebra sense. In $\ell^2(\mathbb{N})$, the vector $(1, \frac{1}{2}, \frac{1}{3}, \ldots)$ is not a finite combination of the $\chi_{\{k\}}$; it's only a limit of them. Books call the linear-algebra kind a Hamel basis, and some call an orthonormal basis a "complete orthonormal system" or a "Hilbert basis".
Does every Hilbert space have an orthonormal basis? For the ones we care about, we can write one down. For an arbitrary $H$ there's no recipe, and we need Zorn's lemma: if every chain in a nonempty partially ordered set has an upper bound, there is a maximal element.
Every Hilbert space $H \neq \{0\}$ has an orthonormal basis.
Proof. Let $\mathcal{S}$ be the collection of orthonormal subsets of $H$, ordered by inclusion. It is nonempty: pick $x \neq 0$, and $\{x / \|x\|\}$ is in $\mathcal{S}$. The empty chain has an upper bound (any member of $\mathcal{S}$). Let $\mathcal{C}$ be a nonempty chain and $V = \bigcup \mathcal{C}$.
$V$ is orthonormal: each element lies in some member of $\mathcal{C}$, so it has norm $1$. If $x_1 \neq x_2$ are in $V$, say $x_1 \in U_1$ and $x_2 \in U_2$ with $U_1, U_2 \in \mathcal{C}$, then because $\mathcal{C}$ is a chain, one contains the other, say $U_1 \subseteq U_2$. Both points lie in the orthonormal set $U_2$, so $\langle x_1, x_2 \rangle = 0$. Thus $V \in \mathcal{S}$, and it is an upper bound for $\mathcal{C}$.
By Zorn's lemma $\mathcal{S}$ has a maximal element $W$, a maximal orthonormal set. By the theorem ((3) ⇒ (1)), $W$ is an orthonormal basis. $\blacksquare$
Zorn's lemma: for when you need a maximal thing and refuse to say where it came from.
When $H$ has a countable dense set, you don't need Zorn: Gram–Schmidt on that set builds a basis by hand. And it turns out those are exactly the spaces whose bases are countable.
Show that $H$ is separable (has a countable dense subset) if and only if $H$ has an orthonormal basis that is at most countable. (Then every orthonormal set in $H$ is at most countable.)
Proof. (⇐) Let $\{u_n\}$ be a countable orthonormal basis. The finite combinations $\sum_{n \le N} c_n u_n$ with every $c_n \in \mathbb{Q} + i\mathbb{Q}$ form a countable set $D$. Given $h$ and $\varepsilon > 0$, the theorem ((1) ⇒ (2)) gives $p = \sum_{n \le N} c_n u_n$ with $\|h - p\| < \frac{\varepsilon}{2}$. Choose $q_n \in \mathbb{Q} + i\mathbb{Q}$ with $|c_n - q_n| < \frac{\varepsilon}{2N}$; then $q = \sum_{n \le N} q_n u_n \in D$ and $\|p - q\| \le \sum_{n \le N} |c_n - q_n| < \frac{\varepsilon}{2}$, so $\|h - q\| < \varepsilon$. So $D$ is dense. (A finite basis works the same way.)
(⇒) Let $\{x_m\}$ be dense, and let $\{u_\alpha\}_{\alpha \in A}$ be any orthonormal set. For $\alpha \neq \beta$, Pythagoras gives $\|u_\alpha - u_\beta\|^2 = 2$. So the open balls $B(u_\alpha, \frac{\sqrt{2}}{2})$ are pairwise disjoint (a point in two of them would put $u_\alpha$ and $u_\beta$ less than $\sqrt{2}$ apart). Each ball contains some $x_m$, by density, and different balls contain different $x_m$. Choosing one such $m$ for each $\alpha$ gives a one-to-one map $A \to \mathbb{N}$, so $A$ is at most countable. In particular an orthonormal basis, which exists by the theorem above, is at most countable. $\blacksquare$
The definition of a basis is about lengths. Now we cash it in: a basis lets you rebuild every vector from its coefficients, and compute every inner product from them. Sums over $A$ of vectors need a meaning first, and the countable support tells us what it should be.
Let $\{u_\alpha\}_{\alpha \in A}$ be an orthonormal basis of $H$, and $h \in H$.
Proof. (a) Let $c_k = \langle h, u_{\alpha_k} \rangle$ and $g_N = \sum_{k=1}^{N} c_k u_{\alpha_k}$. Exactly as in the proof of (3) ⇒ (1), $(g_N)$ is Cauchy, because $\|g_M - g_N\|^2 = \sum_{k=N+1}^{M} |c_k|^2$ and $\sum_k |c_k|^2 \le \|h\|^2$. So $g_N \to g$. By the corollary to the best approximation theorem and the definition of a basis, $$\|h - g_N\|^2 = \|h\|^2 - \sum_{k=1}^{N} |c_k|^2 \longrightarrow \|h\|^2 - \sum_{\alpha \in A} |\langle h, u_\alpha \rangle|^2 = 0.$$ (The countable series in any order adds up to the sum over $A$, since the terms are $\ge 0$ and vanish off the list.) So $g = h$, whatever order we listed the indices in. If there are finitely many nonzero coefficients, the same computation shows $h$ equals the finite sum.
(b) For complex numbers $a, b$, $|a \overline{b}| \le \frac{1}{2}(|a|^2 + |b|^2)$, so the sum converges absolutely by Bessel. For the identity, use polarization, which holds in any complex inner product space (in $H$, and in $\mathbb{C}$ with $\langle a, b \rangle = a\overline{b}$): $$\langle x, y \rangle = \frac{1}{4} \sum_{j=0}^{3} i^j \|x + i^j y\|^2, \qquad a \overline{b} = \frac{1}{4} \sum_{j=0}^{3} i^j |a + i^j b|^2.$$ By the definition of a basis applied to $x + i^j y$, whose coefficients are $\langle x, u_\alpha \rangle + i^j \langle y, u_\alpha \rangle$, $$\begin{aligned} \langle x, y \rangle &= \frac{1}{4} \sum_{j=0}^{3} i^j \sum_{\alpha \in A} \left|\langle x, u_\alpha \rangle + i^j \langle y, u_\alpha \rangle\right|^2 \\ &= \sum_{\alpha \in A} \frac{1}{4} \sum_{j=0}^{3} i^j \left|\langle x, u_\alpha \rangle + i^j \langle y, u_\alpha \rangle\right|^2 = \sum_{\alpha \in A} \langle x, u_\alpha \rangle \overline{\langle y, u_\alpha \rangle}. \end{aligned}$$ Swapping the finite sum over $j$ with the sum over $\alpha$ is fine: each inner sum is a convergent series over the same countable set of indices. $\blacksquare$
Part (b) turns integrals into sums. Here's a check on two small functions, done both ways.
In $L^2(T)$, compute $\langle p, q \rangle$ for $p(t) = 1 + 2\cos t$ and $q(t) = \cos t + \sin t$, first from the Fourier coefficients and then from the integral.
Coefficients: $p = e_0 + e_1 + e_{-1}$, so $\hat{p}(0) = \hat{p}(\pm 1) = 1$. And $q = \frac{e_1 + e_{-1}}{2} + \frac{e_1 - e_{-1}}{2i}$, so $\hat{q}(1) = \frac{1}{2} - \frac{i}{2}$ and $\hat{q}(-1) = \frac{1}{2} + \frac{i}{2}$. Then $$\langle p, q \rangle = \sum_n \hat{p}(n) \overline{\hat{q}(n)} = 1 \cdot \left(\tfrac{1}{2} + \tfrac{i}{2}\right) + 1 \cdot \left(\tfrac{1}{2} - \tfrac{i}{2}\right) = 1.$$
Integral: $(1 + 2\cos t)(\cos t + \sin t) = \cos t + \sin t + 2\cos^2 t + 2\cos t \sin t$. Over a full period, $\cos t$, $\sin t$, and $2 \cos t \sin t = \sin 2t$ average to $0$, and $2\cos^2 t = 1 + \cos 2t$ averages to $1$. So $\langle p, q \rangle = 1$, as promised.
Parseval goes from a vector to its coefficients. The converse question: given any square-summable list of numbers, is there a vector with exactly those coefficients? Completeness says yes.
Let $\{u_\alpha\}_{\alpha \in A}$ be an orthonormal basis of $H$, and let $c \in \ell^2(A)$. Then there is exactly one $x \in H$ with $\langle x, u_\alpha \rangle = c(\alpha)$ for every $\alpha \in A$, namely $x = \sum_{\alpha \in A} c(\alpha) u_\alpha$ (summed over the countably many $\alpha$ with $c(\alpha) \ne 0$, in any order), and $\|x\|^2 = \sum_{\alpha \in A} |c(\alpha)|^2$.
Proof. List the $\alpha$ with $c(\alpha) \neq 0$ as $\alpha_1, \alpha_2, \ldots$ (countably many, by the countable support theorem). Let $x_N = \sum_{k=1}^{N} c(\alpha_k) u_{\alpha_k}$. For $M > N$, $\|x_M - x_N\|^2 = \sum_{k=N+1}^{M} |c(\alpha_k)|^2 \to 0$, since $\sum_k |c(\alpha_k)|^2 < \infty$. So $(x_N)$ is Cauchy, and $H$ is complete, so $x_N \to x$.
By continuity of the inner product, $\langle x, u_\beta \rangle = \lim_N \langle x_N, u_\beta \rangle$, which is $c(\beta)$ by the proposition on finite combinations (for $\beta$ on the list, once $N$ passes it; for $\beta$ off the list, $0 = c(\beta)$). Then $\|x\|^2 = \sum_\alpha |c(\alpha)|^2$ by the definition of a basis. If $x'$ has the same coefficients, then $x - x'$ is orthogonal to every $u_\alpha$, so $x = x'$ (the fourth form of a basis, from the exercise). $\blacksquare$
Put Parseval and Riesz–Fischer side by side: vectors in $H$ and square-summable coefficient lists are the same thing, with the same lengths and the same inner products. We'll make that precise at the end of the page. First, the example that started it all.
We know $\{e_n : n \in \mathbb{Z}\}$ is orthonormal in $L^2(T)$. To use everything above, we need it to be a basis, and the cleanest route is form (2): finite combinations of the $e_n$, called trigonometric polynomials, are dense. We'll borrow one theorem to get there.
The Stone–Weierstrass theorem (Rudin's Theorem 4.25, not on this site yet) gives the fact we need: every continuous $2\pi$-periodic function is a uniform limit of trigonometric polynomials. Fourier Series, a later page in this course, proves this fact again directly, with Fejér's kernel.
With $\langle f, g \rangle = \frac{1}{2\pi} \int_{-\pi}^{\pi} f \overline{g} \, dt$, the set $\{e_n : n \in \mathbb{Z}\}$, $e_n(t) = e^{int}$, is an orthonormal basis of $L^2(T)$.
Proof. Orthonormality is part (b) of the first example on this page. By the theorem on equivalent forms, it is enough to show the trigonometric polynomials are dense. Write $\|f\|_2 = \langle f, f \rangle^{1/2}$, and let $f \in L^2(T)$ and $\varepsilon > 0$.
Step 1: a continuous periodic function. By density of $C_c$ in $L^p$, applied on the open interval $(-\pi, \pi)$ with Lebesgue measure, there is a continuous $g$ with compact support inside $(-\pi, \pi)$ and $\frac{1}{2\pi}\int_{-\pi}^{\pi} |f - g|^2 \, dt < \varepsilon^2$. (The endpoints $\pm\pi$ form a null set, so it doesn't matter whether we work on $(-\pi, \pi)$ or $[-\pi, \pi]$.) Since $g$ vanishes near $\pm \pi$, $g(-\pi) = g(\pi) = 0$, and $g$ extends to a continuous $2\pi$-periodic function. So $\|f - g\|_2 < \varepsilon$.
Step 2: a trigonometric polynomial. By Stone–Weierstrass there is a trigonometric polynomial $p$ with $|g(t) - p(t)| < \varepsilon$ for all $t$. Then $$\|g - p\|_2^2 = \frac{1}{2\pi} \int_{-\pi}^{\pi} |g - p|^2 \, dt < \frac{1}{2\pi} \cdot 2\pi \varepsilon^2 = \varepsilon^2.$$
So $\|f - p\|_2 \le \|f - g\|_2 + \|g - p\|_2 < 2\varepsilon$, and the trigonometric polynomials are dense. $\blacksquare$
Now Parseval and Riesz–Fischer apply to functions on the circle, and they say something you'd never guess from calculus alone.
If $f \in L^2(T)$, then the symmetric partial sums $S_N f = \sum_{|n| \le N} \hat{f}(n) e_n$ converge to $f$ in $L^2(T)$, and $$\frac{1}{2\pi} \int_{-\pi}^{\pi} |f(t)|^2 \, dt = \sum_{n \in \mathbb{Z}} |\hat{f}(n)|^2.$$ Conversely, if $(c_n)_{n \in \mathbb{Z}}$ has $\sum_{n \in \mathbb{Z}} |c_n|^2 < \infty$, there is exactly one $f \in L^2(T)$ with $\hat{f}(n) = c_n$ for every $n$, and $\sum_{|n| \le N} c_n e_n \to f$ in $L^2(T)$.
Proof. $\{e_n\}$ is an orthonormal basis, so the identity is the definition of a basis. For the partial sums, the corollary to the best approximation theorem gives $\|f - S_N f\|_2^2 = \|f\|_2^2 - \sum_{|n| \le N} |\hat{f}(n)|^2$, which tends to $0$ because the full sum is $\|f\|_2^2$. (Parseval's part (a) says the same for any listing of the indices; this is the listing $0, 1, -1, 2, -2, \ldots$.) The converse is the Riesz–Fischer theorem above with $A = \mathbb{Z}$, and the same subtraction shows the symmetric partial sums of $\sum c_n e_n$ converge to $f$. $\blacksquare$
"Converges in $L^2$" is weaker than it sounds. Watch the square wave: the energy of the error drains away, but near the jump the partial sums keep overshooting by the same amount.
That overshoot (the Gibbs phenomenon) lives on shrinking intervals, so it carries less and less energy. Compare the two error readouts for yourself, on functions with and without jumps.
Now the payoff. Pick a function whose integral and coefficients you can both compute, and Parseval turns them into a series.
Use $f(t) = t$ on $(-\pi, \pi)$ to show that $\sum_{n=1}^{\infty} \frac{1}{n^2} = \frac{\pi^2}{6}$.
The coefficients: $\hat{f}(0) = \frac{1}{2\pi} \int_{-\pi}^{\pi} t \, dt = 0$. For $n \neq 0$, integrate by parts with $u = t$ and $dv = e^{-int} dt$, so $v = \frac{e^{-int}}{-in}$: $$\int_{-\pi}^{\pi} t e^{-int} \, dt = \left[ \frac{t e^{-int}}{-in} \right]_{-\pi}^{\pi} + \frac{1}{in} \int_{-\pi}^{\pi} e^{-int} \, dt = \frac{\pi e^{-in\pi} + \pi e^{in\pi}}{-in} + 0 = \frac{2\pi (-1)^n}{-in} = \frac{2\pi i (-1)^n}{n}.$$ So $\hat{f}(n) = \frac{i(-1)^n}{n}$, and $|\hat{f}(n)|^2 = \frac{1}{n^2}$.
The norm: $\|f\|_2^2 = \frac{1}{2\pi} \int_{-\pi}^{\pi} t^2 \, dt = \frac{1}{2\pi} \cdot \frac{2\pi^3}{3} = \frac{\pi^2}{3}$.
Parseval: $\frac{\pi^2}{3} = \sum_{n \neq 0} \frac{1}{n^2} = 2 \sum_{n=1}^{\infty} \frac{1}{n^2}$, since $n$ and $-n$ give the same term. Divide by $2$: $\sum_{n=1}^{\infty} \frac{1}{n^2} = \frac{\pi^2}{6}$.
A sawtooth wave coughing up $\pi^2/6$ is the most dramatic plot twist in all of calculus.
The two classic slips are forgetting that $n$ and $-n$ both contribute and forgetting $\hat{f}(0)$ when it isn't $0$. Keep both in mind on these.
Loading practice engine...
Fourier Series, a page that isn't written yet, asks the harder question: does $S_N f(t) \to f(t)$ at individual points? For continuous $f$, not always. For $f \in L^2(T)$, it holds almost everywhere, but that's Carleson's theorem, one of the deepest results in analysis. $L^2$ convergence is the easy, clean statement, and it's what this page proves.
Parseval said the coefficient map $x \mapsto (\langle x, u_\alpha \rangle)_\alpha$ keeps lengths and inner products. Maps like that deserve a name, and a sharper one for when they're also onto.
Let $H$ and $K$ be Hilbert spaces. A linear map $\Lambda\colon H \to K$ is an isometry if $\langle \Lambda h, \Lambda g \rangle_K = \langle h, g \rangle_H$ for all $h, g \in H$.
An isometry is automatically one-to-one: if $\Lambda h = 0$, then $\|h\|^2 = \|\Lambda h\|^2 = 0$. What it doesn't have to be is onto.
A linear map $\Lambda\colon H \to K$ is unitary if it is an isometry and it is onto $K$.
The simplest operators already show that the two conditions are independent.
On $\mathbb{C}^2$, decide whether each map is an isometry and whether it is unitary: (a) $\Lambda(x_1, x_2) = (x_2, x_1)$; (b) $\Lambda(x_1, x_2) = (i x_1, x_2)$; (c) $\Lambda x = 2x$.
(a) $\langle \Lambda x, \Lambda y \rangle = x_2 \overline{y_2} + x_1 \overline{y_1} = \langle x, y \rangle$, and $\Lambda$ is its own inverse, so it's onto: unitary.
(b) $\langle \Lambda x, \Lambda y \rangle = (i x_1)\overline{(i y_1)} + x_2 \overline{y_2} = x_1 \overline{y_1} + x_2 \overline{y_2}$, since $i \cdot \overline{i} = 1$. It's onto, with inverse $(x_1, x_2) \mapsto (-i x_1, x_2)$: unitary.
(c) Onto (every $y$ is $\Lambda(\frac{y}{2})$), but $\langle \Lambda x, \Lambda y \rangle = 4\langle x, y \rangle$: not an isometry, so not unitary.
A unitary map is a perfect dictionary between two Hilbert spaces: a linear bijection that keeps every inner product. So we name the relationship.
Hilbert spaces $H$ and $K$ are isomorphic if there is a unitary map $\Lambda\colon H \to K$.
Checking $\langle \Lambda h, \Lambda g \rangle = \langle h, g \rangle$ for every pair sounds like work. It's enough to check lengths, because lengths determine inner products.
A linear map $\Lambda\colon H \to K$ between Hilbert spaces is an isometry if and only if $\|\Lambda h\| = \|h\|$ for every $h \in H$.
Proof. (⇒) Take $g = h$: $\|\Lambda h\|^2 = \langle \Lambda h, \Lambda h \rangle = \langle h, h \rangle = \|h\|^2$.
(⇐) By polarization in $K$, then linearity of $\Lambda$, then the hypothesis, then polarization in $H$: $$\begin{aligned} \langle \Lambda h, \Lambda g \rangle &= \frac{1}{4} \sum_{j=0}^{3} i^j \|\Lambda h + i^j \Lambda g\|^2 = \frac{1}{4} \sum_{j=0}^{3} i^j \|\Lambda(h + i^j g)\|^2 \\ &= \frac{1}{4} \sum_{j=0}^{3} i^j \|h + i^j g\|^2 = \langle h, g \rangle. \ \blacksquare \end{aligned}$$
In finite dimensions an isometry of a space into itself is always onto. In infinite dimensions it isn't, and the classic example is the simplest operator you could write down: slide everything over.
The right shift on $\ell^2(\mathbb{N})$ is $R(x_1, x_2, x_3, \ldots) = (0, x_1, x_2, \ldots)$. Show that $R$ is an isometry but not unitary.
$R$ is linear, and $\|Rx\|^2 = 0 + |x_1|^2 + |x_2|^2 + \cdots = \|x\|^2$, so $R$ is an isometry by the proposition. But every $Rx$ has first entry $0$, so $\chi_{\{1\}} = (1, 0, 0, \ldots)$ is not $Rx$ for any $x$. So $R$ is not onto, and not unitary.
The left shift $L(x_1, x_2, \ldots) = (x_2, x_3, \ldots)$ undoes it, $LRx = x$, but $RL x = (0, x_2, x_3, \ldots)$ loses $x_1$. So $L$ is onto but not an isometry ($L\chi_{\{1\}} = 0$).
Here's the shift in motion, along with the vector it can never reach.
The right shift is an isometry with commitment issues: it never quite becomes unitary.
The shift needed room to move: an infinite row with nothing to fall off. Without that room it can't happen.
Let $V$ be a finite-dimensional inner product space and $\Lambda\colon V \to V$ a linear isometry. Show that $\Lambda$ is unitary.
Proof. An isometry is one-to-one (if $\Lambda v = 0$ then $\|v\| = \|\Lambda v\| = 0$), so $\ker \Lambda = \{0\}$. By the rank–nullity theorem, $\dim \Lambda(V) = \dim V - \dim \ker \Lambda = \dim V$, so the subspace $\Lambda(V)$ is all of $V$. So $\Lambda$ is onto, hence unitary. $\blacksquare$
Here's a shift that is unitary, hiding inside $L^2(T)$.
On $L^2(T)$, let $(\Lambda f)(t) = e^{it} f(t)$. Show that $\Lambda$ is unitary, and describe what it does to Fourier coefficients.
$|e^{it} f(t)| = |f(t)|$, so $\|\Lambda f\|_2 = \|f\|_2$, and $\Lambda$ is an isometry. Every $g$ is $\Lambda(e^{-it} g)$, so $\Lambda$ is onto: unitary.
On the basis, $\Lambda e_n = e^{it} e^{int} = e_{n+1}$. So $\widehat{\Lambda f}(n + 1) = \hat{f}(n)$: the coefficient list slides one place to the right. It's the shift again, but on $\mathbb{Z}$ instead of $\mathbb{N}$, where there's no first place to leave empty. That's why this one is onto and the right shift isn't.
That example compared an operator on functions with an operator on coefficient lists. In fact any Hilbert space with an orthonormal basis indexed by $A$ is $\ell^2(A)$, in disguise.
Let $\{u_\alpha\}_{\alpha \in A}$ be an orthonormal basis of $H$. Then $U\colon H \to \ell^2(A)$, $(Ux)(\alpha) = \langle x, u_\alpha \rangle$, is unitary, and $U u_\alpha = \chi_{\{\alpha\}}$. So $H$ is isomorphic to $\ell^2(A)$.
Proof. $Ux$ is in $\ell^2(A)$ by Bessel, and $U$ is linear because the inner product is linear in its first slot. By the definition of a basis, $\|Ux\|^2 = \sum_\alpha |\langle x, u_\alpha \rangle|^2 = \|x\|^2$, so $U$ is an isometry by the proposition. It is onto by the Riesz–Fischer theorem: every $c \in \ell^2(A)$ is $Ux$ for $x = \sum_\alpha c(\alpha) u_\alpha$. Finally $(U u_\alpha)(\beta) = \langle u_\alpha, u_\beta \rangle$ is $1$ for $\beta = \alpha$ and $0$ otherwise, so $U u_\alpha = \chi_{\{\alpha\}}$. $\blacksquare$
So up to isomorphism, a Hilbert space is determined by the size of its orthonormal bases. The source proves this theorem a different way: define $U$ on finite combinations, then extend it by continuity. That extension argument is worth having on its own.
Let $H$ and $K$ be Hilbert spaces, $D \subseteq H$ a dense subspace, and $W\colon D \to K$ linear with $\|Wp\| = \|p\|$ for all $p \in D$. Show there is exactly one continuous linear $U\colon H \to K$ with $U = W$ on $D$, and that $U$ is an isometry. Show also that $U(H)$ is closed in $K$.
Proof. Definition. Let $h \in H$ and pick $p_n \in D$ with $p_n \to h$. Then $(p_n)$ is Cauchy, and $\|Wp_n - Wp_m\| = \|W(p_n - p_m)\| = \|p_n - p_m\|$, so $(Wp_n)$ is Cauchy in $K$. Since $K$ is complete, $Wp_n \to g$ for some $g$. Set $Uh = g$.
Well defined. If also $p_n' \to h$ with $Wp_n' \to g'$, then $\|Wp_n - Wp_n'\| = \|p_n - p_n'\| \le \|p_n - h\| + \|h - p_n'\| \to 0$, so $g = g'$. Taking $p_n = p$ for $p \in D$ shows $U = W$ on $D$.
Linear and isometric. If $p_n \to h$ and $q_n \to k$ in $D$, then $a p_n + b q_n \to ah + bk$ and $W(a p_n + b q_n) = a Wp_n + b Wq_n \to a Uh + b Uk$, so $U(ah + bk) = aUh + bUk$. And $\|Uh\| = \lim \|Wp_n\| = \lim \|p_n\| = \|h\|$, so $U$ is an isometry by the proposition, and in particular continuous.
Unique. Two continuous maps that agree on the dense set $D$ agree on $H$: if $p_n \to h$, both send $h$ to $\lim Wp_n$.
Closed range. If $Uh_n \to y$ in $K$, then $\|h_n - h_m\| = \|Uh_n - Uh_m\|$, so $(h_n)$ is Cauchy, $h_n \to h$, and by continuity $y = Uh \in U(H)$. $\blacksquare$
Now put operators on the bench. For each one, test lengths, then test whether a target vector is ever hit.
Two tests, four outcomes. Classify these.
Loading practice engine...
You've now seen every proof on the page. Check that you can put them back together.
Loading practice engine...
Plancherel's Theorem, the last page of this course, does all of this for functions on the whole line: the Fourier transform turns out to be a unitary map of $L^2(\mathbb{R})$ onto itself, with no basis in sight.
Every question so far came right after its idea, which is a hint. Down here the hint is gone. First, true or false: a claim about anything on this page, some of them the kind that sounds right and isn't. Decide, then read why.
Loading practice engine...
Next, the mixed set. Every question type on this page except true or false goes into one hat, and each click draws a new one at random. Flip the switch to type your answer; the types that need their answer choices will say so.
Loading practice engine...
Last, the Spicy Bonus. Orthonormal bases are about $L^2$ approximation, which can't see what happens at individual points. This problem is about values at points, so it needs the uniform approximation from Stone–Weierstrass instead, with the waves $e^{ikt}$ doing the computing. First, what it says.
Let $f$ be a continuous $2\pi$-periodic function on $\mathbb{R}$, and let $\alpha$ be irrational. Show that $$\lim_{N \to \infty} \frac{1}{N} \sum_{n=1}^{N} f(2\pi n \alpha) = \frac{1}{2\pi} \int_{-\pi}^{\pi} f(t) \, dt.$$ (Hint: first take $f(t) = e^{ikt}$ with $k \in \mathbb{Z}$.)
Proof. For a $2\pi$-periodic $g$, write $A_N(g) = \frac{1}{N} \sum_{n=1}^{N} g(2\pi n \alpha)$ and $I(g) = \frac{1}{2\pi} \int_{-\pi}^{\pi} g(t) \, dt$. Both are linear in $g$, and both satisfy $|A_N(g)| \le \sup |g|$ and $|I(g)| \le \sup |g|$.
Step 1: one wave. Let $g = e_k$. If $k = 0$, then $A_N(g) = 1 = I(g)$. If $k \neq 0$, then $I(e_k) = 0$ (the first example on this page), and with $z = e^{2\pi i k \alpha}$, $$A_N(e_k) = \frac{1}{N} \sum_{n=1}^{N} z^n.$$ Here $z \neq 1$: $z = 1$ would mean $k\alpha \in \mathbb{Z}$, impossible since $\alpha$ is irrational and $k \neq 0$. So the geometric sum gives $$|A_N(e_k)| = \frac{1}{N} \left| \frac{z(1 - z^N)}{1 - z} \right| \le \frac{2}{N|1 - z|} \longrightarrow 0 = I(e_k).$$
Step 2: trigonometric polynomials. By linearity, $A_N(p) \to I(p)$ for every trigonometric polynomial $p = \sum_{|k| \le m} c_k e_k$.
Step 3: continuous $f$. Let $\varepsilon > 0$. By Stone–Weierstrass there is a trigonometric polynomial $p$ with $\sup |f - p| < \frac{\varepsilon}{3}$. (An orthonormal basis alone would only give $\|f - p\|_2$ small, which says nothing about the values of $f$ at the countably many points $2\pi n \alpha$, a null set.) Then $$|A_N(f) - I(f)| \le |A_N(f - p)| + |A_N(p) - I(p)| + |I(p - f)| < \frac{\varepsilon}{3} + |A_N(p) - I(p)| + \frac{\varepsilon}{3},$$ and by Step 2 the middle term is less than $\frac{\varepsilon}{3}$ for all large $N$. So $A_N(f) \to I(f)$. $\blacksquare$
For a rational $\alpha = \frac{p}{q}$ the conclusion fails: the points $2\pi n \alpha$ only visit $q$ spots on the circle, so a continuous $f$ that is $0$ at those spots but has positive integral breaks it. In Step 1, that's $z = 1$ for $k = q$.