Rolling Correlations and Regime Instability

Why Correlation Should Be Examined Over Time

A single correlation calculated from a long history describes average linear association over that history. It can conceal changes in the association between two risks: their relationship may be stronger in some periods and weaker in others. A rolling correlation recalculates the sample correlation repeatedly using a fixed-length window of consecutive observations. It is a useful diagnostic for time-ordered risk data, but a changing estimate is not, by itself, proof that the underlying relationship has changed.

The key distinction is between an observed rolling estimate and an underlying correlation that may or may not vary over time. Each estimate is calculated from a finite set of observations. It can move because the relationship has changed, because the particular observations in the window differ by chance, or because the analyst chose different window endpoints or lengths.

Definition: Let $(X_t,Y_t)$, for $t=1,\ldots,T$, be paired observations recorded in time order. For a window length $m$, the rolling sample correlation ending at time $t$, where $m\leq t\leq T$, is the sample Pearson correlation calculated from observations $t-m+1,\ldots,t$. It measures linear association within that window; it does not establish causation or guarantee that the relationship persists outside the window.

Calculating a Rolling Sample Correlation

For a particular window $W$ containing $m$ paired observations, let $\bar X_W$ and $\bar Y_W$ be the respective sample means. Define the centered sums of squares and cross-products by $S_{XX,W}=\sum_{j\in W}(X_j-\bar X_W)^2$, $S_{YY,W}=\sum_{j\in W}(Y_j-\bar Y_W)^2$, and $S_{XY,W}=\sum_{j\in W}(X_j-\bar X_W)(Y_j-\bar Y_W)$.

If both variables vary within the window, so that $S_{XX,W}>0$ and $S_{YY,W}>0$, its sample correlation is

r_W=\frac{S_{XY,W}}{\sqrt{S_{XX,W}S_{YY,W}}}.

A rolling series reports this value at successive endpoints, usually keeping $m$ fixed. Adjacent windows then share $m-1$ observations. The estimates are therefore related to each other; they should not be treated as independent estimates merely because they have different endpoint labels.

Why the formula works: In a window, sample covariance is $S_{XY,W}/(m-1)$, and the two sample variances are $S_{XX,W}/(m-1)$ and $S_{YY,W}/(m-1)$. Pearson correlation divides the covariance by the product of the sample standard deviations. Substituting these expressions cancels the common factor $1/(m-1)$: $$ \frac{S_{XY,W}/(m-1)} {\sqrt{[S_{XX,W}/(m-1)][S_{YY,W}/(m-1)]}} = \frac{S_{XY,W}}{\sqrt{S_{XX,W}S_{YY,W}}}. $$ Thus the rolling calculation is the usual sample Pearson correlation applied separately to each selected block of consecutive observations.

Worked Example: A Large Change Between Two Overlapping Windows

A hypothetical insurer tracks two weekly risk indicators, $X$ and $Y$. The four recorded pairs are $(1,1),(2,2),(3,3),(4,0)$. Calculate the rolling correlations using windows of three consecutive weeks and explain what the change does and does not show.

Method. Calculate the sample correlation in each window from its means, centered cross-product, and centered sums of squares. The two windows share weeks 2 and 3.

For weeks 1–3, $X=(1,2,3)$ and $Y=(1,2,3)$. Both means are $2$. The centered values for both variables are $(-1,0,1)$, so $S_{XY}=2$, $S_{XX}=2$, and $S_{YY}=2$. Therefore $r=2/\sqrt{2\cdot2}=1.0000$.

For weeks 2–4, $X=(2,3,4)$, with mean $3$, and $Y=(2,3,0)$, with mean $5/3$. The centered values are $X-\bar X=(-1,0,1)$ and $Y-\bar Y=(1/3,4/3,-5/3)$. Thus $S_{XY}=(-1)(1/3)+(0)(4/3)+(1)(-5/3)=-2$, $S_{XX}=(-1)^2+0^2+1^2=2$, and $S_{YY}=(1+16+25)/9=14/3$. The second correlation is $r=-2/\sqrt{2(14/3)}=-2/\sqrt{28/3}=-0.6547$, rounded to four decimal places.

The estimates move from $1.0000$ to $-0.6547$. This is a substantial change in these two particular windows, but each uses only three observations and the windows overlap. It is not enough to conclude that the underlying association underwent a persistent regime shift. The common mistake is to read a dramatic change in a small rolling estimate as conclusive evidence of a changed risk relationship; the correction is to assess its uncertainty and sensitivity to window choice.

Window Length and Endpoint Choices

A shorter window reacts quickly to recent observations, but typically gives a noisier estimate. A longer window usually uses more information, but may blur a genuine change by combining observations from different periods. Neither choice is universally best. The window should be selected for the purpose of the analysis, and conclusions should be checked against reasonable alternatives.

Endpoints matter as well. Moving the start or end of a window by one observation can change the estimate, especially when the window is short or contains an influential observation. A rolling plot is therefore a descriptive tool for identifying periods to investigate, not a method that automatically locates or verifies a change point.

Worked Example: How Window Length Changes the Story

For six consecutive hypothetical periods, suppose a risk analyst observes $X=(-1,0,1,-1,0,1)$ and $Y=(-1,0,1,1,0,-1)$. Calculate the correlations for the first three periods, the last three periods, and all six periods. Then interpret why the answers differ.

Method. Use the sample correlation formula for each selected set of periods. The first and last three-period windows are disjoint; the six-period calculation combines both.

For periods 1–3, $X=Y=(-1,0,1)$. Both means are $0$, and $S_{XX}=S_{YY}=S_{XY}=2$. Hence $r=2/\sqrt{2\cdot2}=1$.

For periods 4–6, $X=(-1,0,1)$ and $Y=(1,0,-1)$. Both means are $0$; $S_{XX}=S_{YY}=2$, while $S_{XY}=-1+0-1=-2$. Hence $r=-2/\sqrt{2\cdot2}=-1$.

Across all six periods, both means are $0$. Each variable has sum of squares $4$. The cross-product sum is $(-1)(-1)+(0)(0)+(1)(1)+(-1)(1)+(0)(0)+(1)(-1)=0$. Thus the full-period correlation is $0/\sqrt{4\cdot4}=0$.

The full-period estimate of zero summarizes two distinct patterns in these data: perfect positive association in the first three periods and perfect negative association in the last three. A long-window result can hide such local differences, while very short-window results can also be unstable. The calculation describes this dataset; by itself, it does not establish why the pattern changed or whether it will recur.

Assessing Whether a Change Exceeds Sampling Noise

A change in rolling estimates should be evaluated relative to the uncertainty in those estimates. One approximate method for comparing correlations uses Fisher’s transformation. For independent, identically distributed paired observations from a bivariate normal distribution, the transformed estimate $z=\operatorname{atanh}(r)$ is approximately normal with standard error $1/\sqrt{n-3}$, where $n$ is the number of pairs. This approximation is for a sample correlation under those assumptions; it is not an automatic uncertainty formula for serially dependent data.

For two independent periods with sample sizes $n_1,n_2$ and correlations $r_1,r_2$, a common approximate comparison is

Z_{\mathrm{diff}}= \frac{\operatorname{atanh}(r_1)-\operatorname{atanh}(r_2)} {\sqrt{1/(n_1-3)+1/(n_2-3)}}.

Under the stated approximation, the magnitude of $Z_{\mathrm{diff}}$ can be compared with standard normal reference values. A value near zero means the observed difference is small relative to its estimated sampling uncertainty; a large magnitude is evidence against equal correlations under the model. This is not a universal test for regime shifts.

Worked Example: Comparing Correlations From Two Independent Periods

Two non-overlapping periods of a hypothetical risk-monitoring study each contain 12 paired observations. Assume the pairs within each period are independent and identically distributed with bivariate normal distributions, and that the periods are independent. The centered summary statistics are $S_{XX}=100$ and $S_{YY}=100$ in each period. The first period has $S_{XY}=60$; the second has $S_{XY}=20$. Compare the sample correlations using the approximate Fisher method at the 5% two-sided level.

Method. First calculate each sample correlation from the centered sums, then transform the estimates and standardize their difference.

In period 1, $r_1=60/\sqrt{100\cdot100}=0.6$. In period 2, $r_2=20/\sqrt{100\cdot100}=0.2$. Their transformed estimates are $\operatorname{atanh}(0.6)=0.6931$ and $\operatorname{atanh}(0.2)=0.2027$, to four decimal places.

Each period has $n-3=9$, so the standard error of the difference is $\sqrt{1/9+1/9}=\sqrt{2/9}=0.4714$. The standardized difference is $(0.6931-0.2027)/0.4714=1.040$, rounded to three decimals.

Since $1.040<1.96$, this comparison does not reject equal correlations at the approximate 5% two-sided level. The point estimates differ, but with these sample sizes the difference is not large relative to the approximate sampling uncertainty. The common mistake is to compare $0.6$ and $0.2$ by eye and declare a shift without considering uncertainty. The correction is to quantify the uncertainty under explicit assumptions. This conclusion does not rule out a change; it says the specified comparison does not provide strong evidence for one.

Overlapping Windows, Serial Dependence, and Interpretation

The independent-period comparison above should not be applied mechanically to adjacent rolling windows. Overlapping windows reuse observations, so their estimated correlations are dependent even if the observations themselves are independent. Serial dependence in the underlying risk data can add further dependence and reduce the effective information in a window. In either case, standard errors based on independent pairs may be too small or otherwise inappropriate.

A disciplined interpretation separates three questions:

What changed descriptively?
Report the direction, size, and timing of the estimated movement, including the window length and endpoints.

How sensitive is the pattern?
Check whether a similar pattern appears under reasonable window lengths and endpoint choices. Treat this as a sensitivity check, not independent confirmation when windows overlap.

What supports an inferential claim?
Use an uncertainty method appropriate to the dependence and sampling structure. State its assumptions and distinguish evidence of a changed association from a descriptive fluctuation.

For risk management, correlation estimates can inform how analysts understand relationships among risk measures over time. A historical rolling estimate is not a guarantee about future association and should not be treated as a complete description of joint risk. Record the data period, window rule, and uncertainty approach so that another analyst can reproduce and challenge the conclusion.

Common Traps

Calling every movement a regime shift. A rolling estimate varies because its observations vary. A claim of persistent change needs evidence beyond a visually irregular line.
Ignoring the window length. A short window may be responsive but noisy; a long window may conceal local variation. State the chosen length and examine sensitivity.
Treating overlapping estimates as independent. Adjacent windows share data. A comparison formula for independent periods does not automatically apply to them.
Using an uncertainty approximation without its assumptions. The Fisher method described here is approximate and relies on independent bivariate normal observations within independent periods. Serial dependence or overlapping periods requires additional care.
Interpreting correlation as a complete dependence description. Pearson correlation measures linear association. Its value alone does not establish causation or describe every aspect of joint risk.

Key takeaway: Rolling correlation is a window-specific sample estimate. Interpret its movement alongside the window length, endpoint choices, and sampling uncertainty; a changing plot is a prompt for investigation, not proof of a regime shift.

Check Your Understanding

For each problem, calculate or interpret the rolling-correlation result as requested.

A four-period window has $X=(1,2,3,4)$ and $Y=(4,3,2,1)$. Calculate its sample correlation using centered sums. (Answer: $r=-1.0000$.)
In a three-observation window, $X=(1,2,3)$ and $Y=(1,3,2)$. Calculate the sample correlation. (Answer: $r=0.5000$.)
Two independent periods each have $n=10$, with correlations $r_1=0.5$ and $r_2=-0.1$. Under the approximate Fisher comparison, calculate the standardized difference and state whether it exceeds 1.96 in magnitude. Use $\operatorname{atanh}(0.5)=0.5493$ and $\operatorname{atanh}(-0.1)=-0.1003$. (Answer: $Z_{\mathrm{diff}}\approx1.215$; it does not exceed 1.96.)
Two adjacent rolling windows each contain 20 observations, and their endpoint correlations differ substantially. Why is it not valid to treat them automatically as independent estimates? (Answer: They share 19 observations, so the estimates are dependent.)
A short-window rolling correlation changes sign, while a longer-window correlation stays near zero. Give one reason the results can differ and one check that would help interpret them. (Answer: The short window is more sensitive to local observations, while the longer window averages over more periods; compare reasonable window lengths and endpoints and assess uncertainty under an appropriate dependence model.)

What You'll Learn