On The Lᵖ Spaces we proved that $L^p(\mu)$ is complete: every Cauchy sequence converges. That proof used integrals, Fatou, and almost-everywhere limits.
The Lᵖ spaces gave us two things at once: a way to measure the size of a function, $\|f\|_p$, and the promise that Cauchy sequences don't fall into holes. This page throws away the integrals and keeps only those two things. What's left is a vector space with a length function that behaves, and that is complete. That's a Banach space, and an astonishing amount of analysis happens inside one.
Then we look at the maps between them. A linear map between finite-dimensional spaces can't do anything wild. In infinite dimensions it can stretch some vectors by an unlimited amount, and that turns out to be exactly the same as failing to be continuous. So the page builds to one number per linear map, its operator norm, and one theorem: bounded means continuous.
Start with the length function. We want three things from "the size of $x$": sizes add at most the way they should (the triangle inequality), stretching a vector stretches its size by the same factor, and only the zero vector has size zero.
Let $X$ be a complex vector space. A norm on $X$ is a function $\|\cdot\|\colon X \to [0, \infty)$ such that for all $x, y \in X$ and all $\alpha \in \mathbb{C}$:
We'll call these the triangle inequality, scaling, and definiteness. A function with (i) and (ii) but maybe not (iii) is a seminorm: it can't tell some nonzero vectors from $0$. A vector space together with a norm gets its own name.
A normed vector space $(X, \|\cdot\|)$ is a complex vector space $X$ with a norm $\|\cdot\|$ on it. It is a metric space with the metric $d(x, y) = \|x - y\|$.
So everything from metric spaces comes for free: open balls, convergence, Cauchy sequences, continuity. Here are some norms to keep in your pocket, and one impostor.
Check which of these are norms: (a) $\|x\|_1 = |x_1| + |x_2|$ on $\mathbb{C}^2$; (b) $\|f\|_\infty = \sup_{[0, 1]} |f|$ on $C[0, 1]$; (c) $\|f\|_1 = \int_0^1 |f(t)| \, dt$ on $C[0, 1]$; (d) $N(f) = \sup_{[0, 1]} |f'|$ on $C^1[0, 1]$.
(a) Triangle: $|x_i + y_i| \le |x_i| + |y_i|$ for each coordinate; add. Scaling: $|\alpha x_i| = |\alpha| |x_i|$. Definiteness: $|x_1| + |x_2| = 0$ forces both to be $0$. A norm.
(b) For each $t$, $|f(t) + g(t)| \le |f(t)| + |g(t)| \le \|f\|_\infty + \|g\|_\infty$; take the sup over $t$. Scaling pulls $|\alpha|$ out of the sup. If $\sup |f| = 0$, then $f = 0$. A norm.
(c) Triangle and scaling come from the same facts for the integral. Definiteness is where continuity matters: if $f$ is continuous and $f(t_0) \ne 0$, then $|f| > \frac{1}{2}|f(t_0)|$ on some interval of length $\ell > 0$ around $t_0$, so $\int_0^1 |f| \ge \frac{1}{2}|f(t_0)| \, \ell > 0$. A norm.
(d) Triangle and scaling hold, but $f = 1$ is not $0$ and has $f' = 0$. So $N$ is only a seminorm.
Part (c) is worth a second look. On $L^1[0, 1]$, $\int |f| = 0$ only says $f = 0$ almost everywhere, which is why $L^1$ had to be a space of equivalence classes. On continuous functions, almost everywhere zero means zero, so no classes are needed. Every axiom can fail on its own, and spotting which one is the whole skill. Hunt them down.
The middle impostor, $\sqrt{\|x\|_2}$, has a perfectly round, convex unit ball and still isn't a norm, because it fails scaling. The first one fails only the triangle inequality, and its ball caves in. That's not a coincidence. When scaling holds, the triangle inequality is exactly convexity of the unit ball. Here is one direction, which uses nothing but the axioms.
Let $X$ be a normed vector space. Prove that the open unit ball $\{x : \|x\| < 1\}$ and the closed unit ball $\{x : \|x\| \le 1\}$ are convex: if $x, y$ are in the ball and $0 \le t \le 1$, then $tx + (1 - t)y$ is in the ball.
Proof. By the triangle inequality and then scaling (with $t \ge 0$ and $1 - t \ge 0$), $$\|tx + (1 - t)y\| \le \|tx\| + \|(1 - t)y\| = t\|x\| + (1 - t)\|y\|.$$ If $\|x\|, \|y\| \le 1$, the right side is at most $t + (1 - t) = 1$, so $tx + (1 - t)y$ is in the closed ball. If $\|x\|, \|y\| < 1$, then $t\|x\| + (1 - t)\|y\| < 1$: if $t > 0$, the first term is strictly less than $t$ and the second is at most $1 - t$, and if $t = 0$, the sum is $\|y\| < 1$. So the open ball is convex too. $\blacksquare$
Now sort a whole pile. Every impostor in it breaks exactly one axiom, so each card has one right bin.
Your turn. Test each candidate against all three axioms; the one that survives is the norm (or, when the question flips, the one that doesn't is the impostor).
Loading practice engine...
A norm lets us add up infinitely many vectors. Same definition as in calculus, with the norm doing the measuring.
Let $(x_n)$ be a sequence in a normed vector space $X$. The series $\sum_{n=1}^{\infty} x_n$ converges to $x \in X$ if its partial sums converge to $x$: $\left\|x - \sum_{n=1}^{N} x_n\right\| \to 0$ as $N \to \infty$.
In calculus, the fastest test for convergence was to throw away the signs: if $\sum |a_n| < \infty$, then $\sum a_n$ converges. The norm version of "throw away the signs" is to add up the lengths.
A series $\sum_{n=1}^{\infty} x_n$ in a normed vector space is absolutely convergent if $\sum_{n=1}^{\infty} \|x_n\| < \infty$.
Here's the surprise: in a general normed space, absolutely convergent series don't have to converge. Take $c_{00}$, the space of sequences with only finitely many nonzero terms, with the sup norm $\|x\|_\infty = \sup_n |x_n|$. Add up $2^{-n}e_n$, where $e_n$ is the sequence with a $1$ in slot $n$ and $0$ elsewhere.
Every partial sum lives in $c_{00}$, and the lengths add up to $1$. But the place the partial sums are heading has infinitely many nonzero terms. Let's make sure nothing else in $c_{00}$ could be the limit.
In $c_{00}$ with $\|x\|_\infty$, show that $\sum_n 2^{-n} e_n$ is absolutely convergent but does not converge.
Absolutely convergent: $\|2^{-n} e_n\|_\infty = 2^{-n}$, and $\sum_n 2^{-n} = 1 < \infty$.
Not convergent: suppose the partial sums $s_k = \sum_{n \le k} 2^{-n} e_n$ converged to some $z \in c_{00}$. Since $z$ has finitely many nonzero terms, there is $M$ with $z_n = 0$ for all $n > M$. Each coordinate is controlled by the sup norm: $|(s_k)_{M+1} - z_{M+1}| \le \|s_k - z\|_\infty \to 0$. But $(s_k)_{M+1} = 2^{-(M+1)}$ for every $k \ge M + 1$, so $z_{M+1} = 2^{-(M+1)} \ne 0$. Contradiction.
So $c_{00}$ has a hole exactly where $(2^{-n})$ should be. A Cauchy sequence that doesn't converge, and an absolutely convergent series that doesn't converge, are the same kind of failure. The next theorem says they are literally the same failure.
A normed vector space $X$ is complete if and only if every absolutely convergent series in $X$ converges.
Proof. ($\implies$) Suppose $X$ is complete and $\sum_n \|x_n\| < \infty$. Let $s_k = x_1 + \cdots + x_k$. Given $\varepsilon > 0$, choose $N$ with $\sum_{n > N} \|x_n\| < \varepsilon$. For $k > m \ge N$, the triangle inequality gives $$\|s_k - s_m\| = \left\|\sum_{n=m+1}^{k} x_n\right\| \le \sum_{n=m+1}^{k} \|x_n\| \le \sum_{n > N} \|x_n\| < \varepsilon.$$ So $(s_k)$ is Cauchy. Since $X$ is complete, $s_k \to x$ for some $x \in X$, which says the series converges.
($\impliedby$) Suppose every absolutely convergent series converges, and let $(x_n)$ be Cauchy. For each $k$ there is $N_k$ with $\|x_i - x_j\| < 2^{-k}$ whenever $i, j \ge N_k$. Choose indices $n_1 < n_2 < \cdots$ with $n_k \ge N_k$ (for instance $n_k = \max\{N_1, \ldots, N_k\} + k$). Put $$y_1 = x_{n_1}, \qquad y_{k+1} = x_{n_{k+1}} - x_{n_k} \quad (k \ge 1).$$ Both $n_{k+1}$ and $n_k$ are at least $N_k$, so $\|y_{k+1}\| < 2^{-k}$, and $\sum_k \|y_k\| < \|y_1\| + \sum_k 2^{-k} < \infty$. By hypothesis $\sum_k y_k$ converges to some $x \in X$. Its partial sums telescope: $y_1 + \cdots + y_k = x_{n_k}$, so $x_{n_k} \to x$.
Finally, a Cauchy sequence with a convergent subsequence converges to the same limit: given $\varepsilon > 0$, pick $N$ with $\|x_i - x_j\| < \frac{\varepsilon}{2}$ for $i, j \ge N$ and then $k$ with $n_k \ge N$ and $\|x_{n_k} - x\| < \frac{\varepsilon}{2}$. For $i \ge N$, $\|x_i - x\| \le \|x_i - x_{n_k}\| + \|x_{n_k} - x\| < \varepsilon$. So $x_n \to x$, and $X$ is complete. $\blacksquare$
The "if" direction is a trick worth owning: a Cauchy sequence, sampled fast enough, is an absolutely convergent series in disguise. Watch the disguise come off.
The proof that $L^p$ is complete started the same way, with a subsequence whose jumps are at most $2^{-k}$. There the series $\sum \|f_{n_{k+1}} - f_{n_k}\|_p$ was fed to Minkowski and the monotone convergence theorem. Here the hypothesis does that work for us.
The theorem turns "is this space complete?" into a hunt for an escaping series. Go build some, in the polynomials with the sup norm on $[0, 1]$. Since $\sup_{[0, 1]} |t^n| = 1$, the length of the term $a_n t^n$ is just $|a_n|$.
Both $\sum t^n/n!$ and $\sum 2^{-n}t^n$ converge absolutely, and both land outside the polynomials, at $e^t$ and $\frac{2}{2 - t}$. So the polynomials with the sup norm aren't complete either. The harmonic coefficients $\frac{1}{n}$ are a different story: that series isn't absolutely convergent, so the theorem says nothing about it.
The spaces with no holes get the name.
A Banach space is a normed vector space that is complete in the metric $d(x, y) = \|x - y\|$.
We already own a pile of them. Every $L^p(\mu)$ with $1 \le p \le \infty$ is a Banach space (that was the Lᵖ completeness theorem), and so is every $\ell^p$, which is $L^p$ of counting measure. A cheap way to get more is to take closed pieces of the ones we have.
A closed subspace $M$ of a Banach space $X$, with the norm of $X$, is a Banach space.
Proof. The restriction of a norm to a subspace is a norm. Let $(x_n)$ be Cauchy in $M$. It is Cauchy in $X$, so $x_n \to x$ for some $x \in X$. Since $M$ is closed and every $x_n \in M$, the limit $x$ is in $M$. So $(x_n)$ converges in $M$. $\blacksquare$
So a Banach space's closed subspaces are Banach, and its non-closed ones, like $c_{00}$ inside $\ell^\infty$, have holes. The most important example is the space between them: $c_0$, the sequences that tend to $0$.
Let $c_0 = \{x \in \ell^\infty : x_n \to 0\}$, with the sup norm. Show that $c_0$ is a Banach space.
It is a subspace: if $x_n \to 0$ and $y_n \to 0$, then $\alpha x_n + y_n \to 0$. By the theorem above it's enough to show $c_0$ is closed in $\ell^\infty$.
Let $u_k \in c_0$ converge to $x \in \ell^\infty$, and write $u_{k,n}$ for the $n$th term of $u_k$, and let $\varepsilon > 0$. Pick $k$ with $\|u_k - x\|_\infty < \frac{\varepsilon}{2}$, then $N$ with $|u_{k,n}| < \frac{\varepsilon}{2}$ for $n \ge N$. For $n \ge N$, $$|x_n| \le |x_n - u_{k,n}| + |u_{k,n}| < \tfrac{\varepsilon}{2} + \tfrac{\varepsilon}{2} = \varepsilon.$$ So $x_n \to 0$, and $x \in c_0$. And $c_{00} \subseteq c_0$, with $(2^{-n})$ in $c_0$: the hole in $c_{00}$ is filled.
On Inner Products & Projections, a Hilbert space was an inner product space that is complete in the norm $\|x\| = \langle x, x \rangle^{1/2}$. So every Hilbert space is a Banach space, $L^2(\mu)$ first among them.
The continuous functions $C[0, 1]$ with the sup norm are a Banach space too: a Cauchy sequence in the sup norm is uniformly Cauchy, so it converges uniformly, and a uniform limit of continuous functions is continuous. The same functions with a weaker norm are a different story.
Show that $C[0, 1]$ with $\|f\|_1 = \int_0^1 |f(t)| \, dt$ is not a Banach space.
For $n \ge 2$, let $f_n$ be $0$ on $[0, \frac{1}{2}]$, rise linearly to $1$ on $[\frac{1}{2}, \frac{1}{2} + \frac{1}{n}]$, and equal $1$ after that. Two of these differ only on a ramp, and the region between two ramps is a triangle of height $1$ and base $|\frac{1}{n} - \frac{1}{m}|$, so $\|f_n - f_m\|_1 = \frac{1}{2}|\frac{1}{n} - \frac{1}{m}| \to 0$. The sequence is Cauchy.
Suppose $f_n \to f$ in $\|\cdot\|_1$ for some continuous $f$. On $[0, \frac{1}{2}]$ every $f_n$ is $0$, so $\int_0^{1/2} |f| \le \|f - f_n\|_1 \to 0$, and $f = 0$ there (it's continuous). For $\delta > 0$ and $n > \frac{1}{\delta}$, $f_n = 1$ on $[\frac{1}{2} + \delta, 1]$, so likewise $f = 1$ there. Hence $f = 0$ on $[0, \frac{1}{2}]$ and $f = 1$ on $(\frac{1}{2}, 1]$: not continuous at $\frac{1}{2}$. No continuous limit exists.
Same functions, different norm, different answer. Completeness is a property of the pair $(X, \|\cdot\|)$, never of $X$ alone. Here's a space where you get to build the norm and check completeness from scratch.
For $0 < \alpha \le 1$, call $f\colon [a, b] \to \mathbb{C}$ Hölder continuous of order α if its Hölder constant $$[f]_\alpha = \sup_{s \ne t} \frac{|f(s) - f(t)|}{|s - t|^\alpha}$$ is finite, and let $\mathrm{Lip}\,\alpha$ be the space of all such $f$. (For $\alpha = 1$ these are the Lipschitz functions.)
Prove that $\mathrm{Lip}\,\alpha$ is a Banach space with the norm $\|f\| = |f(a)| + [f]_\alpha$, and also with the norm $\|f\|' = [f]_\alpha + \sup_{[a, b]} |f|$.
Proof. A vector space, and $[\cdot]_\alpha$ is a seminorm. For $s \ne t$, $$\frac{|(f + g)(s) - (f + g)(t)|}{|s - t|^\alpha} \le \frac{|f(s) - f(t)|}{|s - t|^\alpha} + \frac{|g(s) - g(t)|}{|s - t|^\alpha} \le [f]_\alpha + [g]_\alpha,$$ so $[f + g]_\alpha \le [f]_\alpha + [g]_\alpha$, and clearly $[\beta f]_\alpha = |\beta| [f]_\alpha$. So $\mathrm{Lip}\,\alpha$ is a vector space.
Both are norms. Each is a sum of seminorms ($f \mapsto |f(a)|$ and $f \mapsto \sup |f|$ are seminorms too), so the triangle inequality and scaling hold. Definiteness: if $[f]_\alpha = 0$, then $f(s) = f(t)$ for all $s, t$, so $f$ is constant; with $f(a) = 0$ (or $\sup |f| = 0$) it is $0$.
The key bound. For $t \in [a, b]$, $|f(t)| \le |f(a)| + |f(t) - f(a)| \le |f(a)| + [f]_\alpha (b - a)^\alpha$. So $$\sup |f| \le C\|f\| \quad \text{with } C = \max\{1, (b - a)^\alpha\}.$$
Completeness for $\|\cdot\|$. Let $(f_n)$ be Cauchy in $\|\cdot\|$. By the key bound it is uniformly Cauchy, so it converges uniformly to some function $f$ on $[a, b]$. Given $\varepsilon > 0$, pick $N$ with $\|f_n - f_m\| < \varepsilon$ for $n, m \ge N$. Fix $n \ge N$ and $s \ne t$. For every $m \ge N$, $$|(f_n - f_m)(s) - (f_n - f_m)(t)| \le [f_n - f_m]_\alpha |s - t|^\alpha < \varepsilon |s - t|^\alpha,$$ and letting $m \to \infty$ gives $|(f_n - f)(s) - (f_n - f)(t)| \le \varepsilon |s - t|^\alpha$. So $[f_n - f]_\alpha \le \varepsilon$. In particular $f_n - f \in \mathrm{Lip}\,\alpha$, so $f = f_n - (f_n - f) \in \mathrm{Lip}\,\alpha$. Also $|f_n(a) - f(a)| \to 0$. Hence $\|f_n - f\| \le |f_n(a) - f(a)| + \varepsilon$ for all $n \ge N$, so $\limsup_n \|f_n - f\| \le \varepsilon$ for every $\varepsilon$, and $f_n \to f$.
Completeness for $\|\cdot\|'$. The two norms are comparable: $\|f\| \le \|f\|'$ because $|f(a)| \le \sup |f|$, and by the key bound $\|f\|' \le [f]_\alpha + C\|f\| \le (1 + C)\|f\|$. So a sequence is Cauchy for one norm exactly when it is Cauchy for the other, and converges for one exactly when it converges for the other, to the same limit. Completeness for $\|\cdot\|$ gives it for $\|\cdot\|'$. $\blacksquare$
The proof's shape is the standard one for "this space is Banach": a Cauchy sequence has some limit in a bigger, known-complete space (here $C[a, b]$), then show the limit belongs to the smaller space and that the convergence happens in the smaller space's norm. Now tell the Banach spaces from the leaky ones.
Loading practice engine...
Now the maps. A linear map $\Lambda\colon X \to Y$ between normed spaces can shrink some vectors and stretch others. The question that matters is the worst case: how much can it stretch a vector of length at most $1$?
Let $(X, \|\cdot\|_X)$ and $(Y, \|\cdot\|_Y)$ be normed vector spaces and $\Lambda\colon X \to Y$ linear. The operator norm of $\Lambda$ is $$\|\Lambda\| = \sup\{\|\Lambda x\|_Y : \|x\|_X \le 1\}.$$
That sup can be $\infty$. The maps where it isn't are the ones this page is about.
A linear map $\Lambda\colon X \to Y$ between normed vector spaces is bounded if $\|\Lambda\| < \infty$.
Two quick ones before any theory, to see the sup get computed and attained.
Find the operator norms of (a) $\Lambda f = f(\frac{1}{2})$ on $C[0, 1]$ with the sup norm, and (b) the right shift $S(x_1, x_2, \ldots) = (0, x_1, x_2, \ldots)$ on $\ell^p$, $1 \le p \le \infty$.
(a) If $\sup |f| \le 1$, then $|f(\frac{1}{2})| \le 1$, so $\|\Lambda\| \le 1$. The constant function $f = 1$ has $\sup |f| = 1$ and $\Lambda f = 1$. So $\|\Lambda\| = 1$.
(b) Shifting doesn't change which numbers appear, so $\|Sx\|_p = \|x\|_p$ for every $x$ (the extra $0$ adds nothing to the sum or the sup). So $\|Sx\|_p \le 1$ whenever $\|x\|_p \le 1$, with equality at $x = e_1$. So $\|S\| = 1$.
In both, the sup was attained. It doesn't have to be (we'll see a diagonal operator where it isn't), but the definition never needs it to be. Three ways to read the number $\|\Lambda\|$ come straight out of linearity, and we'll use all three constantly.
Let $\Lambda\colon X \to Y$ be linear, with $X \ne \{0\}$.
Proof. (a) Let $S$ be the sup over the sphere. The sphere is inside the ball, so $S \le \|\Lambda\|$. Conversely, let $\|x\| \le 1$. If $x = 0$, then $\|\Lambda x\| = 0 \le S$ ($S \ge 0$ because $X \ne \{0\}$ has unit vectors). If $x \ne 0$, then $u = x/\|x\|$ has $\|u\| = 1$ and $\|\Lambda x\| = \|x\| \, \|\Lambda u\| \le 1 \cdot S$. So $\|\Lambda\| \le S$.
(b) The inequality is trivial if $\|\Lambda\| = \infty$ or $x = 0$. Otherwise $x/\|x\|$ is in the closed unit ball, so $\|\Lambda x\| = \|x\| \, \|\Lambda(x/\|x\|)\| \le \|x\| \, \|\Lambda\|$. If $C$ is any constant with $\|\Lambda x\| \le C\|x\|$ for all $x$, then for $\|x\| \le 1$, $\|\Lambda x\| \le C$, so $\|\Lambda\| \le C$. So $\|\Lambda\|$ is the smallest such constant.
(c) If $\|x\| \le 1$, then by (b), $\|\Lambda x\| \le \|\Lambda\|$. $\blacksquare$
Part (c) is the picture to carry around. For a $2 \times 2$ matrix on $\mathbb{R}^2$ with the Euclidean norm, the unit ball goes to an ellipse, and $\|A\|$ is how far the ellipse reaches.
Change the norm on $\mathbb{C}^n$ and the operator norm of the same matrix changes. For the norms $\|\cdot\|_1$ and $\|\cdot\|_\infty$ there are formulas you can read off the matrix, and they're the main computational tool on this page.
Let $A = (a_{ij})$ be an $m \times n$ complex matrix, viewed as a map $\mathbb{C}^n \to \mathbb{C}^m$.
Proof. (a) Let $c = \max_j \sum_i |a_{ij}|$. Then $$\|Ax\|_1 = \sum_i \Big|\sum_j a_{ij} x_j\Big| \le \sum_j |x_j| \sum_i |a_{ij}| \le c \sum_j |x_j| = c\|x\|_1,$$ so $\|A\| \le c$. If column $j$ attains $c$, then $\|e_j\|_1 = 1$ and $\|Ae_j\|_1 = \sum_i |a_{ij}| = c$, so $\|A\| \ge c$.
(b) Let $r = \max_i \sum_j |a_{ij}|$. For each $i$, $|(Ax)_i| \le \sum_j |a_{ij}| \, |x_j| \le r\|x\|_\infty$, so $\|A\| \le r$. If row $i$ attains $r$, choose $x_j$ with $|x_j| = 1$ and $a_{ij}x_j = |a_{ij}|$ (take $x_j = \overline{a_{ij}}/|a_{ij}|$, or $x_j = 1$ when $a_{ij} = 0$). Then $\|x\|_\infty = 1$ and $(Ax)_i = \sum_j |a_{ij}| = r$, so $\|A\| \ge r$.
(c) By the spectral theorem, $A = QDQ^t$ with $Q$ real orthogonal and $D$ diagonal, holding the eigenvalues $\lambda_1, \ldots, \lambda_n$. A real orthogonal matrix preserves $\|\cdot\|_2$ on $\mathbb{C}^n$ too, since $\|Qz\|_2^2 = \overline{z}^t Q^t Q z = \|z\|_2^2$. Let $\lambda_{\max} = \max_k |\lambda_k|$ and $w = Q^t x$. Then $$\|Ax\|_2 = \|D w\|_2 = \Big(\sum_k |\lambda_k|^2 |w_k|^2\Big)^{1/2} \le \lambda_{\max} \|w\|_2 = \lambda_{\max} \|x\|_2,$$ and a unit eigenvector $v$ for an eigenvalue with $|\lambda| = \lambda_{\max}$ has $\|Av\|_2 = \lambda_{\max}$.
(d) Let $s = \sup_n |d_n|$. Then $|d_n x_n| \le s |x_n|$ in every coordinate, so $\|Dx\|_p \le s\|x\|_p$ (raise to the $p$th power and add, or take the sup when $p = \infty$). And $\|e_n\|_p = 1$ with $\|De_n\|_p = |d_n|$, so $\|D\| \ge |d_n|$ for every $n$, hence $\|D\| \ge s$. $\blacksquare$
The trap is mixing up (a) and (b): columns for $\ell^1$, rows for $\ell^\infty$. A way to remember it is the proof itself. On $\ell^1$ the worst input is a single $e_j$, which picks out a column. On $\ell^\infty$ the worst input is a vector of signs, which pairs up with a row.
Let $A = \begin{pmatrix} 1 & -2 \\ 3 & 4 \end{pmatrix}$. Find $\|A\|$ on $(\mathbb{C}^2, \|\cdot\|_1)$ and on $(\mathbb{C}^2, \|\cdot\|_\infty)$, and a vector that attains each.
Columns: $|1| + |3| = 4$ and $|-2| + |4| = 6$. So on $\ell^1$, $\|A\| = 6$, attained at $e_2 = (0, 1)$: $Ae_2 = (-2, 4)$, with $\|Ae_2\|_1 = 6$.
Rows: $|1| + |-2| = 3$ and $|3| + |4| = 7$. So on $\ell^\infty$, $\|A\| = 7$, attained at the signs of the second row, $x = (1, 1)$: $Ax = (-1, 7)$, with $\|Ax\|_\infty = 7$.
Six and seven: the same matrix, two norms, two answers. Drag the columns of a matrix around and watch all three norms respond.
On $\ell^1$ the image of the diamond is the parallelogram spanned by $\pm Ae_1$ and $\pm Ae_2$, so the farthest point is always one of the columns. On $\ell^\infty$ the image of the square has its corners at $A(\pm 1, \pm 1)$. Now play the other side: the matrix is fixed, and you're hunting the input.
Corners when the ball has corners, the long axis when it doesn't. Compute a few.
Loading practice engine...
Diagonal operators work on function spaces too. On $L^2(\mu)$, multiplying by a bounded function $f$ is the continuous version of multiplying coordinates by $d_n$, and you'd guess its norm is the essential sup of $|f|$. The guess is right, but only for measures that aren't too strange. A measure $\mu$ is semifinite if every measurable set $E$ with $\mu(E) = \infty$ contains a measurable $F \subseteq E$ with $0 < \mu(F) < \infty$. (Every σ-finite measure is semifinite.)
Let $\mu$ be a positive measure, and for $f \in L^\infty(\mu)$ define the multiplication operator $M_f\colon L^2(\mu) \to L^2(\mu)$ by $M_f g = fg$.
Proof. (a) $|f| \le \|f\|_\infty$ a.e., so $|fg|^2 \le \|f\|_\infty^2 |g|^2$ a.e. Integrating, $\|fg\|_2 \le \|f\|_\infty \|g\|_2$. So $M_f g \in L^2(\mu)$, $M_f$ is linear, and $\|M_f\| \le \|f\|_\infty$.
(b) Semifinite gives equality. If $\|f\|_\infty = 0$, both sides are $0$. Otherwise let $0 < \varepsilon < \|f\|_\infty$. By the definition of the essential sup, $E = \{|f| > \|f\|_\infty - \varepsilon\}$ has $\mu(E) > 0$, so it contains $F$ with $0 < \mu(F) < \infty$ (take $F = E$ if $\mu(E) < \infty$, and use semifiniteness otherwise). Then $g = \chi_F \in L^2(\mu)$ with $\|g\|_2^2 = \mu(F) > 0$, and $$\|M_f g\|_2^2 = \int_F |f|^2 \, d\mu \ge (\|f\|_\infty - \varepsilon)^2 \mu(F) = (\|f\|_\infty - \varepsilon)^2 \|g\|_2^2.$$ So $\|M_f\| \ge \|f\|_\infty - \varepsilon$ for every such $\varepsilon$, and with (a), $\|M_f\| = \|f\|_\infty$.
Not semifinite breaks it. Suppose some $E$ has $\mu(E) = \infty$ while every measurable subset of $E$ has measure $0$ or $\infty$. Let $f = \chi_E$, so $\|f\|_\infty = 1$. For $g \in L^2(\mu)$ and $n \ge 1$, $\mu(\{|g| > \frac{1}{n}\}) \le n^2 \|g\|_2^2 < \infty$ (Markov's inequality applied to $|g|^2$), so the subset $E \cap \{|g| > \frac{1}{n}\}$ of $E$ has finite measure, hence measure $0$. Taking the union over $n$, $g = 0$ a.e. on $E$, so $fg = 0$ a.e. Hence $M_f = 0$ and $\|M_f\| = 0 \ne 1 = \|f\|_\infty$.
(c) ($\impliedby$) If $|f| \ge \delta$ a.e., then $1/f$ is defined a.e., measurable, and $|1/f| \le 1/\delta$. For $g \in L^2(\mu)$, $h = g/f$ has $|h| \le |g|/\delta$, so $h \in L^2(\mu)$ and $M_f h = g$.
($\implies$) Suppose no such $\delta$ exists; we find $g \in L^2(\mu)$ outside the range. Case 1: $Z = \{f = 0\}$ has $\mu(Z) > 0$. Take $F \subseteq Z$ with $0 < \mu(F) < \infty$ as in (b). If $fh = \chi_F$ a.e., then on $F$ we get $0 = 1$ a.e., impossible since $\mu(F) > 0$. Case 2: $\mu(Z) = 0$. For $k \ge 0$ let $A_k = \{2^{-k-1} \le |f| < 2^{-k}\}$. For every $K$, $\mu(\{|f| < 2^{-K}\}) > 0$ (no $\delta$ works), and up to the null set $Z$ this set is $\bigcup_{k \ge K} A_k$. So infinitely many $A_k$ have positive measure: say $A_{k_1}, A_{k_2}, \ldots$ with $k_1 < k_2 < \cdots$, so $k_j \ge j - 1$. Pick $F_j \subseteq A_{k_j}$ with $0 < \mu(F_j) < \infty$. The $F_j$ are disjoint. Let $$g = \sum_{j=1}^{\infty} \frac{2^{-j}}{\sqrt{\mu(F_j)}} \chi_{F_j}, \qquad \|g\|_2^2 = \sum_{j} 4^{-j} < \infty.$$ If $fh = g$ a.e., then on $F_j$, $|h| = |g|/|f| > 2^{k_j} \cdot \frac{2^{-j}}{\sqrt{\mu(F_j)}} \ge \frac{1}{2\sqrt{\mu(F_j)}}$, so $\int_{F_j} |h|^2 \, d\mu \ge \frac{1}{4}$ for every $j$, and $\|h\|_2 = \infty$. So $g$ is not in the range of $M_f$. $\blacksquare$
Part (c) really needs semifinite. Take $E$ as in the second half of (b) and $f = \chi_{E^c}$. Every $g \in L^2(\mu)$ vanishes a.e. on $E$, so $fg = g$: $M_f$ is the identity, which is certainly onto, but $|f| = 0$ on the set $E$ of infinite measure. Prelim answers that skip this hypothesis are wrong as stated.
Why care whether $\|\Lambda\|$ is finite? Because for linear maps, "bounded" is just a quantitative way of saying "continuous", and continuity at a single point is already enough.
Let $(X, \|\cdot\|_X)$ and $(Y, \|\cdot\|_Y)$ be normed vector spaces and $\Lambda\colon X \to Y$ linear. The following are equivalent:
Proof. This is the proposition on Inner Products & Projections with $|\Lambda x|$ replaced by $\|\Lambda x\|_Y$, and its proof works word for word: its first line shows a bounded map is uniformly continuous, so (1) $\implies$ (2), and (2) $\implies$ (3) is immediate. Its second half needs continuity at $0$ only, and continuity at $x_0$ gives that, because linearity moves the ball around $x_0$ to the origin: $\|\Lambda y\|_Y = \|\Lambda(x_0 + y) - \Lambda x_0\|_Y$. So (3) $\implies$ (1). $\blacksquare$
The last step is the one to understand: continuity at $x_0$ is a statement about a tiny ball, linearity drags that ball to the origin, and scaling blows it up to the unit ball. Here it is with $\varepsilon = 0.6$ and $\delta = 0.4$.
A bounded operator: continuous, Lipschitz, and frankly a little boring at parties.
The first step of the proof says more than "uniformly continuous": a bounded map is Lipschitz with constant $\|\Lambda\|$, and $\|\Lambda\|$ is the best constant. Because the bound only sees $x - y$, it holds everywhere in the space at once.
On Compact Spaces, uniform continuity came from compactness: a continuous map on a compact metric space is uniformly continuous. A normed space is never compact (unless it's $\{0\}$), so here the uniformity comes from linearity instead.
In $\mathbb{C}^n$ every linear map is bounded: with the norm $\|\cdot\|_1$, $\|\Lambda x\| = \|\sum_i x_i \Lambda e_i\| \le (\max_i \|\Lambda e_i\|) \|x\|_1$. In infinite dimensions that argument has infinitely many $e_i$, and the max becomes a sup that can be infinite. Here is the most famous unbounded map.
Let $P$ be the space of polynomials on $[0, 1]$ with $\|p\| = \sup_{[0, 1]} |p|$, and $D p = p'$. Show that $D\colon P \to P$ is linear but unbounded, and find a sequence $p_n \to 0$ in $P$ with $Dp_n \not\to 0$.
Linear: $(\alpha p + q)' = \alpha p' + q'$.
Unbounded: $p_n(t) = t^n$ has $\|p_n\| = 1$, while $Dp_n = nt^{n-1}$ has $\|Dp_n\| = n$. So $\|D\| \ge n$ for every $n$.
Not continuous at $0$: $q_n = \frac{t^n}{n}$ has $\|q_n\| = \frac{1}{n} \to 0$, but $\|Dq_n\| = \|t^{n-1}\| = 1$ for every $n$. By the theorem, an unbounded linear map can't be continuous anywhere, and this is the witness at $0$.
Polynomials are steep without being tall. Trigonometric functions make the same point even more vividly: $\frac{\sin(nt)}{n}$ is barely there, and its derivative is a full-size wave.
Differentiation: linear, defined on every polynomial, and still unbounded. Some maps just can't be held accountable.
The proof of unboundedness is always a witness: a sequence with norm $1$ (or norm going to $0$) whose images refuse to cooperate. Finding the witness is a skill, and the wrong sequence proves nothing. Find one for each map.
Notice that every one of those maps becomes bounded if you change the norm on its domain to one that "pays" for the bad behavior: $\sup |p| + \sup |p'|$ for the derivative, $\|x\|_1$ for the sum, the sup norm for evaluation at $0$. That's the most common way to misread the word.
"Bounded" for a linear map does not mean its range is a bounded set. The only linear map with bounded range is $0$, since $\|\Lambda(tx)\| = |t| \, \|\Lambda x\|$. Bounded means the unit ball goes to a bounded set. And boundedness depends on the norms: $f \mapsto f(0)$ is bounded on $C[0, 1]$ with the sup norm (norm $1$) and unbounded with $\int_0^1 |f|$ (thin tents).
Bounded or not? Look for a bound first, and if none comes, hunt for the witness.
Loading practice engine...
That's every proof on the page so far. Check that you can put them back together.
Loading practice engine...
One more payoff of continuity, and it's a big one. A bounded linear functional is determined by what it does to the unit vectors $e_n$, as long as every $x$ is a limit of finite combinations of them. On $c_0$ that's true, and it pins down every bounded linear functional there is. The test vectors in the proof match the signs of $y$, one more term at a time.
The animation is the second half of the next exercise. The first half is where continuity earns its keep.
Let $\Lambda\colon c_0 \to \mathbb{C}$ be a bounded linear functional, and let $y_n = \Lambda e_n$. Prove that $y = (y_n) \in \ell^1$, that $$\Lambda x = \sum_{n=1}^{\infty} y_n x_n \quad \text{for every } x \in c_0,$$ and that $\|\Lambda\| = \|y\|_1$. Conversely, show that every $y \in \ell^1$ defines a bounded linear functional on $c_0$ this way.
Proof. The formula. Let $x \in c_0$ and $s_N = \sum_{n \le N} x_n e_n$. Then $\|x - s_N\|_\infty = \sup_{n > N} |x_n| \to 0$, because $x_n \to 0$. A bounded linear map is continuous, so $\Lambda s_N \to \Lambda x$. By linearity $\Lambda s_N = \sum_{n \le N} x_n y_n$, so the series $\sum_n y_n x_n$ converges to $\Lambda x$. (Linearity alone only handles the finite sums; continuity is what passes to the limit.)
$y \in \ell^1$ and $\|y\|_1 \le \|\Lambda\|$. For each $n$ choose $\theta_n$ with $|\theta_n| = 1$ and $\theta_n y_n = |y_n|$ (take $\theta_n = \overline{y_n}/|y_n|$, or $\theta_n = 1$ if $y_n = 0$). The vector $u_N = \sum_{n \le N} \theta_n e_n$ is in $c_0$ with $\|u_N\|_\infty \le 1$, so $$\sum_{n \le N} |y_n| = \sum_{n \le N} \theta_n y_n = \Lambda u_N \le |\Lambda u_N| \le \|\Lambda\| \, \|u_N\|_\infty \le \|\Lambda\|.$$ This holds for every $N$, so $\|y\|_1 \le \|\Lambda\| < \infty$.
$\|\Lambda\| \le \|y\|_1$. For $x \in c_0$, $|\Lambda x| = |\sum_n y_n x_n| \le \sum_n |y_n| \, |x_n| \le \|y\|_1 \|x\|_\infty$. Together, $\|\Lambda\| = \|y\|_1$.
Conversely, if $y \in \ell^1$, then for $x \in c_0$ the series $\sum_n y_n x_n$ converges absolutely (its terms are at most $|y_n| \, \|x\|_\infty$), the map $x \mapsto \sum_n y_n x_n$ is linear, and the same estimate shows it is bounded with norm at most $\|y\|_1$. $\blacksquare$
So the bounded linear functionals on $c_0$ are exactly the sequences in $\ell^1$, with the norms matching. The same three moves (bound with the right inequality, then test on the right vectors) compute $\|\Lambda\|$ on other sequence spaces too.
Let $\Lambda x = 2x_1 - 3x_2 + 6x_3$. Find $\|\Lambda\|$ on $c_0$ (sup norm), on $\ell^1$, and on $\ell^2$.
Here $y = (2, -3, 6, 0, 0, \ldots)$.
On $c_0$: by the exercise, $\|\Lambda\| = \|y\|_1 = 2 + 3 + 6 = 11$, attained at the signs $x = (1, -1, 1, 0, \ldots)$. (The same bound and the same test vector work on $\ell^\infty$.)
On $\ell^1$: $|\Lambda x| \le 6(|x_1| + |x_2| + |x_3|) \le 6\|x\|_1$, and $x = e_3$ gives $|\Lambda e_3| = 6$. So $\|\Lambda\| = \|y\|_\infty = 6$.
On $\ell^2$: Cauchy–Schwarz gives $|\Lambda x| \le \|y\|_2 \|x\|_2$ with $\|y\|_2 = \sqrt{4 + 9 + 36} = 7$, and $x = y/7$ gives $\Lambda x = \frac{49}{7} = 7$. So $\|\Lambda\| = 7$.
The pattern: the sup norm pairs with $\|y\|_1$, the $\ell^1$ norm pairs with $\|y\|_\infty$, and $\ell^2$ pairs with itself. Pair them up.
Loading practice engine...
On Baire Category & Its Consequences, completeness pays off three times: the Banach–Steinhaus theorem, the open mapping theorem, and the closed graph theorem. All three are about bounded operators between Banach spaces, and none of them survives a hole in the space.
Every question so far came right after its idea, which is a hint. Down here the hint is gone. First, true or false: a claim about anything on this page, some of them the kind that sounds right and isn't. Decide, then read why.
Loading practice engine...
Next, the mixed set. Every question type on this page except true or false goes into one hat, and each click draws a new one at random. Flip the switch to type your answer; the types that need their answer choices will say so.
Loading practice engine...
Last, the Spicy Bonus: the sequence spaces' other dual, and why $\ell^\infty$ is too big to be separable. The first part runs the $c_0$ exercise with the roles of the norms swapped.
Proof. (a) $y$ is bounded. Let $y_n = \Lambda e_n$. Since $\|e_n\|_1 = 1$, $|y_n| \le \|\Lambda\|$, so $y \in \ell^\infty$ with $\|y\|_\infty \le \|\Lambda\|$.
The formula. For $x \in \ell^1$, $\|x - \sum_{n \le N} x_n e_n\|_1 = \sum_{n > N} |x_n| \to 0$, the tail of a convergent series. By continuity and linearity, $\Lambda x = \lim_N \sum_{n \le N} x_n y_n = \sum_n y_n x_n$.
The norm. $|\Lambda x| \le \sum_n |y_n| \, |x_n| \le \|y\|_\infty \|x\|_1$, so $\|\Lambda\| \le \|y\|_\infty$. With the first step, $\|\Lambda\| = \|y\|_\infty$.
Uniqueness and every $y$. If $\Lambda x = \sum_n z_n x_n$ for all $x$, then $z_n = \Lambda e_n = y_n$. And for any $y \in \ell^\infty$, the series $\sum_n y_n x_n$ converges absolutely for $x \in \ell^1$, defines a linear map, and the estimate above bounds it by $\|y\|_\infty$; testing on $e_n$ gives norm at least $|y_n|$ for every $n$, so exactly $\|y\|_\infty$. So $y \mapsto \Lambda_y$ is a linear bijection from $\ell^\infty$ onto the bounded linear functionals on $\ell^1$ that preserves norms.
(b) $c_0$ and $\ell^1$. Let $D$ be the set of finitely supported sequences whose entries have rational real and imaginary parts. $D$ is countable (a countable union, over $N$, of copies of the countable set $(\mathbb{Q} + i\mathbb{Q})^N$). For $x \in c_0$ and $\varepsilon > 0$, pick $N$ with $|x_n| < \varepsilon$ for $n > N$, and $q_n \in \mathbb{Q} + i\mathbb{Q}$ with $|q_n - x_n| < \varepsilon$ for $n \le N$; then $q = (q_1, \ldots, q_N, 0, \ldots) \in D$ has $\|x - q\|_\infty < \varepsilon$. For $x \in \ell^1$, pick $N$ with $\sum_{n > N} |x_n| < \frac{\varepsilon}{2}$ and $q_n$ with $|q_n - x_n| < \frac{\varepsilon}{2N}$ for $n \le N$; then $\|x - q\|_1 < \frac{\varepsilon}{2} + \frac{\varepsilon}{2} = \varepsilon$. So $D$ is dense in both.
$\ell^\infty$. For each $S \subseteq \mathbb{N}$, the indicator $\chi_S$ (with $1$ in the slots of $S$ and $0$ elsewhere) is in $\ell^\infty$, and $\|\chi_S - \chi_T\|_\infty = 1$ whenever $S \ne T$. Suppose $E$ were a countable dense subset. Each open ball $B(\chi_S, \frac{1}{2})$ would contain a point $e_S \in E$. If $e_S = e_T$, then $\|\chi_S - \chi_T\|_\infty < \frac{1}{2} + \frac{1}{2} = 1$, so $S = T$. So $S \mapsto e_S$ is one-to-one from the uncountable set $\mathcal{P}(\mathbb{N})$ into the countable set $E$, a contradiction. $\blacksquare$