Josh Blake Flynn II · University of Houston · April 2024
I wrote this paper in April 2024, when I was still fairly new to higher math. It walks through a method from Gidea and Katz (2018): take the four big U.S. stock indices, slide a window along their history, and ask what shape the data in each window has. The shape gets measured with persistent homology, the shape gets turned into a single number, and that number did something interesting before the crashes of 2000 and 2008.
This page is the paper rebuilt: the same story, the math cleaned up, and pictures you can play with. Every persistence diagram and landscape on this page is computed live in your browser from the points you see.
Start with the data. Stock prices come one day at a time, in order, and that kind of data has a name.
A time series is a sequence of data points indexed in time order. A financial time series is a time series of financial data, such as the daily price of a stock.
If you want to watch the whole market rather than one company, you watch stock market indices: numbers that track the stocks in some slice of the economy. In the United States the four best known are the S&P 500, the Dow Jones Industrial Average (DJIA), the NASDAQ, and the Russell 2000. Together they give a pretty complete picture of how the U.S. economy is doing.
So it's natural to bundle them: each trading day gives one number per index, which makes one vector in $\mathbb{R}^4$. That vector-valued time series is the object this whole page studies. A day is a point, and a stretch of days is a cloud of points in $\mathbb{R}^4$.
Predicting market crashes is a famous unsolved problem. Plenty of variables have been correlated with crashes, but none has held up as a causal explanation. Nothing here claims persistent homology can predict a crash. The question is narrower: what does persistent homology see when you point it at public market data?
A cloud of points has no shape on its own: it's just dots. To talk about loops and holes we first connect the dots, and the standard way to connect dots is a simplicial complex.
Let $V$ be a finite set. A simplicial complex on $V$ is a collection $K$ of nonempty subsets of $V$ such that
An element $\sigma \in K$ with $k + 1$ vertices is a $k$-simplex (a vertex, an edge, a triangle, a tetrahedron, ...), and the nonempty subsets of $\sigma$ are its faces.
Rule 2 is what keeps the shape honest: you can't have a filled triangle without its three edges, and you can't have an edge without its two endpoints. Smaller complexes sitting inside a bigger one get their own name.
Let $K$ be a simplicial complex. A subcollection $L \subseteq K$ is a subcomplex of $K$ if for each simplex $\tau \in L$, every face of $\tau$ also belongs to $L$.
The trick of persistent homology is not to build one complex but a whole growing family of them, each sitting inside the next.
Let $K$ be a simplicial complex. A filtration of $K$ of length $n$ is a nested sequence of subcomplexes
$$F_1 K \subseteq F_2 K \subseteq \cdots \subseteq F_{n-1} K \subseteq F_n K = K.$$For point clouds there's one filtration everybody uses. Pick a scale $\varepsilon$, and fill in every simplex whose points are all within $\varepsilon$ of each other.
Let $(X, d)$ be a finite metric space and $\varepsilon \geq 0$. The Vietoris–Rips complex $R(X, \varepsilon)$ is the simplicial complex on $X$ whose simplices are the nonempty subsets $\{x_0, x_1, \dots, x_k\} \subseteq X$ with
$$d(x_i, x_j) \leq \varepsilon \quad \text{for all } i, j.$$If $\varepsilon \leq \varepsilon'$ then $R(X, \varepsilon) \subseteq R(X, \varepsilon')$, so the complexes form an increasing family, the Vietoris–Rips filtration of $X$.
A finite cloud has only finitely many distances, so the complex only changes at finitely many values $\varepsilon_1 < \varepsilon_2 < \cdots < \varepsilon_n$, and listing $R(X, \varepsilon_1) \subseteq \cdots \subseteq R(X, \varepsilon_n)$ gives a filtration in the sense above.
Pictures of the Rips complex usually draw a ball around every point. On this page $\varepsilon$ is a distance, so the balls have radius $\varepsilon/2$: two balls touch exactly when their points are within $\varepsilon$, which is exactly when the edge appears. Some books instead let $\varepsilon$ be the radius, which doubles every number. Also, the Rips complex only checks pairs: three balls that touch pairwise get a filled triangle even if no single spot lies in all three.
Here's the toy example from the paper, small enough to see everything.
Two companies, A and B, have stocks priced at $\$5$ and $\$3$ per share on day 0. Here are their prices over the next 16 days.
What does the pair of stocks look like as a point cloud?
Each day gives the point $(A(t), B(t)) \in \mathbb{R}^2$, so the cloud is
$$X = \{(A(0), B(0)),\ (A(1), B(1)),\ \dots,\ (A(16), B(16))\}.$$For days 0 to 8 the points march around a circle of radius 1 centered at $(4, 3)$, one eighth of a lap per day. For days 8 to 16 they march around a circle of radius 2 centered at $(3, 3)$. The point $(5, 3)$ shows up on days 0, 8, and 16, so $X$ has 15 distinct points: two octagons that share a corner. And the small octagon passes through $(3, 3)$, the big one's center. Keep an eye on that point.
Now grow $\varepsilon$ and watch the complex fill in. The dots on the right are the persistence diagram, which the next section defines; for now just watch them climb.
The widget kept count of two things: connected components and loops. Homology is the machine that counts them, in every dimension at once. It runs on linear algebra.
Let $\mathbb{F}$ be a field. A chain complex $(C_\bullet, d_\bullet)$ over $\mathbb{F}$ is a sequence of $\mathbb{F}$-vector spaces $C_k$ and linear maps $d_k \colon C_k \to C_{k-1}$ with $d_k \circ d_{k+1} = 0$ for every $k$. For each $k \geq 0$, the $k$-th homology group is the quotient vector space
$$H_k(C_\bullet, d_\bullet) = \ker d_k \,/\, \operatorname{im} d_{k+1}.$$A simplicial complex $K$ gives a chain complex: $C_k$ is the vector space with one basis vector for each $k$-simplex, and $d_k$ sends a simplex to the sum of its faces with one fewer vertex (over $\mathbb{F} = \mathbb{Z}/2$ no signs are needed). Write $H_k(K)$ for its homology. Then:
Now feed in a whole filtration. Each inclusion $R(X, \varepsilon) \subseteq R(X, \varepsilon')$ sends cycles to cycles, so it induces a linear map $H_k(R(X, \varepsilon)) \to H_k(R(X, \varepsilon'))$. Lining up those maps gives the structure persistent homology is built on.
An $\mathbb{N}$-indexed persistence module over $\mathbb{F}$ is a sequence $(V_\bullet, a_\bullet)$ of $\mathbb{F}$-vector spaces $V_k$ and linear maps $a_k$ for $k \geq 0$ that fit into a diagram
$$V_0 \xrightarrow{\ a_0\ } V_1 \xrightarrow{\ a_1\ } \cdots \xrightarrow{\ a_{k-1}\ } V_k \xrightarrow{\ a_k\ } V_{k+1} \xrightarrow{\ a_{k+1}\ } \cdots$$For a point cloud, take $V_i = H_k(R(X, \varepsilon_i))$ at the scales where the complex changes, and let the $a_i$ be the induced maps.
The induced maps are not inclusions, and they don't have to be one-to-one. When a triangle fills in a loop, the loop's class gets sent to $0$. That's not a bug: it's exactly how a feature "dies." So $H_k(R(X, \varepsilon))$ is not a subspace of $H_k(R(X, \varepsilon'))$; there's only a map between them.
Following one class along those maps gives it a lifespan.
A homology class $\alpha$ is born at $b_\alpha$ if it appears in $H_k(R(X, b_\alpha))$ and is not the image of any class from an earlier scale. It dies at $d_\alpha$ if $d_\alpha$ is the first scale where its image becomes $0$, or merges with the image of a class born before it. The persistence of $\alpha$ is $d_\alpha - b_\alpha$.
The persistence diagram $P_k$ is the multiset of points $(b_\alpha, d_\alpha)$, one for each $k$-dimensional feature, drawn with birth on the horizontal axis and death on the vertical axis.
Every point lands above the diagonal, since nothing dies before it's born. Points far above the diagonal lived a long time; points hugging it were born and died almost at once. Long life usually means real structure, and short life usually means noise, and the nice part is that nobody has to pick a cutoff between the two: every feature stays in the diagram, weighted by how long it lasted.
Find the $H_1$ persistence diagram of the two-stock point cloud.
There are two loops, one per octagon.
So $P_1 = \{(2\sin(\pi/8),\ 2\sin(3\pi/8)),\ (4\sin(\pi/8),\ 2)\} \approx \{(0.77, 1.85),\ (1.53, 2)\}$. These match the dots the widget froze at. The hollow dots on its death axis are $P_0$: every component is born at $0$ and dies when it merges into another.
Diagrams can be compared: two diagrams are close when you can pair up their points so that every point moves only a little. Points are allowed to pair with the diagonal, which is how a diagram with one short-lived extra feature stays close to a diagram without it.
For $p \geq 1$, the degree-$p$ Wasserstein distance between persistence diagrams $P$ and $\tilde{P}$ (each including every point of the diagonal, with infinite multiplicity) is
$$W_p(P, \tilde{P}) = \inf_{\phi} \Big( \sum_{x \in P} \lVert x - \phi(x) \rVert_\infty^{\,p} \Big)^{1/p},$$where the infimum runs over all bijections $\phi \colon P \to \tilde{P}$ and $\lVert \cdot \rVert_\infty$ is the sup norm on $\mathbb{R}^2$. The case $p = \infty$, $W_\infty(P, \tilde{P}) = \inf_\phi \sup_{x \in P} \lVert x - \phi(x) \rVert_\infty$, is the bottleneck distance.
This distance is what makes persistent homology trustworthy on noisy data. Wiggle the points a little, and the diagram moves only a little.
Let $X$ and $Y$ be finite point clouds in $\mathbb{R}^d$ whose Hausdorff distance is $\delta$: every point of each is within $\delta$ of some point of the other. Then for every $k$, the bottleneck distance between the persistence diagrams of their Vietoris–Rips filtrations is at most $2\delta$.
Chazal, de Silva, and Oudot (2014) prove it in this form, building on Cohen-Steiner, Edelsbrunner, and Harer (2007).
So persistent homology is stable under small perturbations of the data, not invariant: the diagram does move, just by a controlled amount. That's why the long-lived points are trustworthy and the short ones near the diagonal are where noise lives.
The space of diagrams with $W_p$ is a perfectly good metric space, so why go any further? Because statistics needs more than distances. To average a hundred diagrams, or compute a variance, you need to add diagrams and scale them, and diagrams don't add in any useful way. The fix is to turn each diagram into a function, where adding and averaging make sense.
Take the diagram, tip it $45^\circ$ clockwise so the diagonal lies flat, and stand a triangle on each point. That's the whole idea; here it is carefully.
For a birth-death pair $(b, d)$, the tent function $f_{(b, d)} \colon \mathbb{R} \to \mathbb{R}$ is
$$f_{(b, d)}(x) = \begin{cases} x - b & \text{if } b < x \leq \frac{b + d}{2}, \\[2pt] d - x & \text{if } \frac{b + d}{2} < x < d, \\[2pt] 0 & \text{otherwise.} \end{cases}$$Its graph is an isosceles triangle over the interval $[b, d]$ with its peak at $\left( \frac{b + d}{2},\ \frac{d - b}{2} \right)$.
The peak sits at the midpoint of the lifespan, with height half the persistence. One diagram gives a pile of overlapping tents, like a mountain range, and the landscape reads that range layer by layer.
Let $P_k$ be a persistence diagram with finitely many points off the diagonal. Its persistence landscape is the sequence of functions $\Lambda = (\lambda_j)_{j \geq 1}$, $\lambda_j \colon \mathbb{R} \to [0, \infty)$, where
$$\lambda_j(x) = \text{the } j\text{-th largest value of } \{ f_{(b, d)}(x) : (b, d) \in P_k \},$$and $\lambda_j(x) = 0$ if there are fewer than $j$ points.
So $\lambda_1$ runs along the top of the mountain range, $\lambda_2$ along the top of what's left once you ignore the highest layer, and so on. Each $\lambda_j$ is continuous and piecewise linear, and $\lambda_1 \geq \lambda_2 \geq \cdots \geq 0$. Drag the points on the diagram and watch.
Landscapes live in a vector space: add two of them, or scale one, entry by entry. With a norm on top, that space is a Banach space, and it's where the statistics happens.
For $1 \leq p < \infty$, the space $L^p(\mathbb{N} \times \mathbb{R})$ consists of sequences $\eta = (\eta_j)_{j \geq 1}$ of functions $\eta_j \colon \mathbb{R} \to \mathbb{R}$ with finite $p$-norm
$$\lVert \eta \rVert_p = \Big( \sum_{j=1}^{\infty} \lVert \eta_j \rVert_p^{\,p} \Big)^{1/p}, \qquad \lVert \eta_j \rVert_p = \Big( \int_{\mathbb{R}} |\eta_j(x)|^p \, dx \Big)^{1/p},$$with addition $(\eta + \eta')_j = \eta_j + \eta'_j$ and scaling $(c\,\eta)_j = c\,\eta_j$. It is a Banach space, and every persistence landscape lies in it.
For landscapes the norms have a neat shortcut. At each $x$, the values $\lambda_1(x), \lambda_2(x), \dots$ are just the tent values $f_{(b, d)}(x)$ sorted, so summing over $j$ is the same as summing over tents.
If $P$ has finitely many points off the diagonal, then
$$\lVert \Lambda \rVert_1 = \sum_{(b, d) \in P} \frac{(d - b)^2}{4} \qquad \text{and} \qquad \lVert \Lambda \rVert_2^{\,2} = \sum_{(b, d) \in P} \frac{(d - b)^3}{12}.$$Fix $x$. The numbers $\lambda_1(x), \lambda_2(x), \dots$ (ignoring zeros) are the tent values $f_{(b, d)}(x)$ listed in decreasing order, so $\sum_j \lambda_j(x)^p = \sum_{(b, d)} f_{(b, d)}(x)^p$. Integrating over $x$ and swapping the finite sums with the integral gives $\lVert \Lambda \rVert_p^{\,p} = \sum_{(b, d)} \lVert f_{(b, d)} \rVert_p^{\,p}$. A tent with half-width $h = \frac{d - b}{2}$ has $\int f = h^2 = \frac{(d - b)^2}{4}$ and $\int f^2 = 2 \int_0^h t^2 \, dt = \frac{2h^3}{3} = \frac{(d - b)^3}{12}$. $\blacksquare$
So the norm of a landscape is mostly a measure of how much persistence there is, with long-lived features counting a lot more than short ones.
Find $\lVert \Lambda \rVert_1$ for the $H_1$ diagram of the two-stock example.
The persistences are $d_1 - b_1 = 2\sin(3\pi/8) - 2\sin(\pi/8) \approx 1.082$ and $d_2 - b_2 = 2 - 4\sin(\pi/8) \approx 0.469$. By the identity,
$$\lVert \Lambda \rVert_1 \approx \frac{1.082^2 + 0.469^2}{4} \approx \frac{1.172 + 0.220}{4} \approx 0.348.$$The small octagon's loop lived more than twice as long, so it contributes more than five times as much.
Now the actual method. One point cloud is a snapshot; a market is a movie. So cut the movie into overlapping clips.
Let $\{x^1_n\}, \dots, \{x^d_n\}$ be $d$ time series, so each time $t_n$ gives a point $x(t_n) = (x^1_n, \dots, x^d_n) \in \mathbb{R}^d$. For a window size $w$, the sliding-window point clouds are
$$X_n = \{ x(t_n),\ x(t_{n+1}),\ \dots,\ x(t_{n+w-1}) \}, \qquad n = 1, 2, \dots,$$each a cloud of $w$ points in $\mathbb{R}^d$. Stored as data, $X_n$ is a $d \times w$ matrix: one row per series, one column per day of the window.
For each window, the method computes three things in a row:
That turns the whole history into two ordinary time series of norms, one number per window, and those are what you watch. The older approach to 1-dimensional persistence in multidimensional data uses time-delay embeddings with two parameters; this method has just one, $w$.
Before trusting it on markets, test it on data where you know the answer. Economic swings are often compared to chaos, so the test case is a classic chaotic system.
For real parameters $a$ and $b$, the Hénon map is the sequence
$$x_{n+1} = 1 - a x_n^2 + b y_n, \qquad y_{n+1} = x_n.$$It's a two-dimensional cousin of the logistic map. For some values of $a$ and $b$, every starting point in the basin of attraction settles onto the same invariant set; for others, the same starting points behave chaotically. The test lets $a$ creep upward over time and adds noise:
$$x_{n+1} = 1 - a_n x_n^2 + b y_n + \sigma W_n \sqrt{\Delta t}, \qquad y_{n+1} = x_n + \sigma W_n \sqrt{\Delta t}, \qquad a_{n+1} = a_n + \Delta t,$$where each $W_n$ is a standard normal random variable, $\Delta t > 0$ is a small step, and $\sigma > 0$ is the noise intensity. Run it four times with $b = 0.27, 0.28, 0.29, 0.30$, use the four $x$-series as the four coordinates, and slide a window of $w = 50$. Press play.
The top plot is the familiar bifurcation picture of the Hénon map: one fixed value, then two, then four, then a smear. The norms ignore the regular regimes entirely and jump right where the dynamics turn chaotic. Passing from regular to chaotic motion changes the topology of the attractor, and the norms pick that up cleanly. (Hit "New noise" a few times: the jump lands at the same place every time.)
Gidea and Katz also tested two models that are standard in quantitative finance.
So the method passed its tests on data that's believed to behave like stocks. On to the real thing.
The real data set is the S&P 500, DJIA, NASDAQ, and Russell 2000 over 7301 trading days. Prices themselves aren't used; each day's coordinate is the index's log-return
$$r_{i, j} = \ln \frac{P_{i, j}}{P_{i-1, j}},$$where $P_{i, j}$ is the adjusted closing value of the $j$-th index on day $i$. So each day is a point in $\mathbb{R}^4$, and the windows have $w = 50$ or $w = 100$ trading days, sliding one day at a time.
The animation below is the method in miniature. It runs on made-up data shaped like the real thing, but its persistence is computed for real. The paper's figures for the dot-com crash look like this story: a window shortly before the crash has a persistence diagram with points far from the diagonal and a landscape with tall tents; a calm window earlier has only noise near the diagonal. (The paper's landscape plots use the rotated coordinates $x = \frac{b + d}{2}$, $y = \frac{d - b}{2}$, so a tent's peak sits at a feature's midpoint and half-life.)
In the real data, the same thing happened. As the market moved toward a crash, loops in the point clouds became far more prominent, and the $L^1$ and $L^2$ norms of the landscapes rose in the run-up to the dot-com crash of 2000. Changing the window from 50 to 100 days hardly mattered: the spikes flattened a little and the main peak shifted slightly later.
The natural worry is that the norms are just measuring volatility in disguise, since in the synthetic tests, more variance meant bigger norms. The comparison with the VIX, the standard volatility index, says otherwise: the two behave quite independently. Before the 2000 crash the VIX actually showed declining variability, while before the 2008 crash it trended upward for about 250 days. Meanwhile the variability of the $L^p$-norm time series showed a strong rising trend about 250 trading days before both the dot-com crash and the Lehman Brothers bankruptcy in 2008.
Read that carefully: it's the variability of the norm series that rose before both crashes, and that's a pattern found looking backward at two events. It's an interesting early-warning signal, not a crash predictor.
The shape of financial time series depends strongly on the state of the market. When the market moves from a stable period into a volatile one, loops in the point clouds persist longer, and the landscape norms pick that up. The method works on any multidimensional time series, and for the major index funds it gives a view of the market that's fairly independent of the measures already in use, like the VIX. It's a new kind of econometric: one that looks at the shape of the data rather than its size.