Size Distortions from Ignoring Cross-Sectional Dependence

Why Cross-Sectional Dependence Matters for Unit-Root Tests

A panel unit-root test is often attractive because it combines evidence across units. But when units share common shocks, their test statistics are not independent. A first-generation test calibrated as if they were independent can therefore have a null distribution different from the one used to choose its critical value. The practical consequence is a possible size distortion: a test advertised at a given significance level may reject a true unit-root null much too often, or sometimes too rarely.

The issue is not simply that the observations are correlated. It is that the dependence changes the joint distribution of the statistics being combined. This tutorial isolates that mechanism with a common-shock data-generating process and compares nominal and empirical rejection rates in a hypothetical Monte Carlo experiment.

Definition: The size of a test is its probability of rejecting \(H_0\) when \(H_0\) is true. Its nominal size is the target rejection probability set by the analyst, such as \(\alpha=0.05\). Its empirical size in a simulation is the fraction of simulated samples generated under \(H_0\) in which the test rejects.

A First-Generation Test and a Common-Shock Null

For each unit \(i=1,\ldots,N\), let \(y_{it}\) be a time series observed at \(t=1,\ldots,T\). The unit-root null is that each series is nonstationary because its level contains a unit root. For a simple null with no drift, write

y_{it}=y_{i,t-1}+u_{it}, \qquad y_{i0}=0.

A first-generation panel unit-root test combines unit-level evidence using a reference distribution that assumes cross-sectional independence. For example, an IPS-style statistic standardizes an average of individual unit-root test statistics using its independence-based null mean and variance. A Fisher-ADF statistic combines individual ADF p-values as \(P=-2\sum_{i=1}^{N}\ln(p_i)\), using the chi-squared reference distribution that applies when the p-values are independent and uniform under the null. These tests can allow serial correlation within each unit through the unit-level ADF specification; the issue considered here is dependence across units.

A common-shock version of the null is

y_{it}=y_{i,t-1}+\lambda_i f_t+\varepsilon_{it}.

Here \(f_t\) is a common innovation shared by the units, \(\lambda_i\) is unit \(i\)'s loading on that shock, and \(\varepsilon_{it}\) is an idiosyncratic innovation. Suppose, for the examples below, that \(f_t\) and \(\varepsilon_{it}\) are mutually independent over units and time, have mean zero, and have finite variances. The common shock does not remove the unit root: each \(y_{it}\) still accumulates innovations. But when loadings are nonzero, innovations in different units are correlated.

Worked Example: Correlation Created by a Common Shock

Suppose every unit has loading \(\lambda_i=0.5\), the common shock has variance \(1\), and the idiosyncratic innovations have variance \(1\). For two distinct units \(i\) and \(j\), calculate the contemporaneous correlation between their innovations.

The innovations are \(u_{it}=0.5f_t+\varepsilon_{it}\) and \(u_{jt}=0.5f_t+\varepsilon_{jt}\). Their common component gives

\operatorname{Cov}(u_{it},u_{jt})=(0.5)(0.5)\operatorname{Var}(f_t)=0.25.

The variance of either unit's innovation is

\operatorname{Var}(u_{it})=(0.5)^2(1)+1=1.25.

Therefore,

\operatorname{Corr}(u_{it},u_{jt})=\frac{0.25}{\sqrt{1.25(1.25)}}=\frac{0.25}{1.25}=0.2000.

Thus, even though the idiosyncratic components are independent, the innovations have correlation \(0.2000\) across units. A unit-root test that treats the units' evidence as independent does not account for this common variation.

How Dependence Changes the Variance of Combined Evidence

A short variance calculation shows why the independence reference distribution can fail. Consider \(N\) random variables \(Z_i\) with a common variance \(\sigma^2\) and common pairwise correlation \(\rho\). Their average is \(\bar Z=N^{-1}\sum_i Z_i\). Expanding the variance gives

\begin{aligned} \operatorname{Var}(\bar Z) &=\frac{1}{N^2}\left\{\sum_{i=1}^{N}\operatorname{Var}(Z_i) +2\sum_{i<j}\operatorname{Cov}(Z_i,Z_j)\right\}\\ &=\frac{1}{N^2}\left\{N\sigma^2+N(N-1)\rho\sigma^2\right\}\\ &=\frac{\sigma^2}{N}\left[1+(N-1)\rho\right]. \end{aligned}

Under independence, \(\rho=0\), and the variance is \(\sigma^2/N\). With positive dependence, the actual variance is larger by the factor \(1+(N-1)\rho\). This calculation is an explanation of the mechanism, not an exact formula for the variance of every unit-root test statistic: those statistics need not have exactly equal variances or equicorrelation. It does show why standardizing an average with an independence-based variance can make the test statistic too variable relative to its assumed null distribution.

Worked Example: The Variance Inflation in a Panel Average

Suppose there are \(N=40\) units and the test-statistic contributions have pairwise correlation \(\rho=0.20\). Compare the variance of their average with the variance under independence.

Under the common-correlation calculation, the ratio is

\frac{\operatorname{Var}(\bar Z\mid \rho=0.20)} {\operatorname{Var}(\bar Z\mid \rho=0)} =1+(N-1)\rho =1+(40-1)(0.20)=8.80.

The standard deviation is consequently larger by a factor of \(\sqrt{8.80}=2.966\), under this simplified calculation. An independence-based standardization would miss that additional variation. For a left-tailed test, an overly dispersed standardized statistic can produce too many observations beyond the lower critical value, leading to over-rejection. The precise distortion depends on the test and the dependence structure; its direction and magnitude should be assessed rather than presumed.

Hypothetical Monte Carlo Comparison

The following experiment illustrates how to compare nominal and empirical size. For every replication, generate \(N=40\) series of length \(T=80\) under the unit-root null, starting from \(y_{i0}=0\). Use either independent innovations, \(u_{it}=\varepsilon_{it}\), or common-shock innovations, \(u_{it}=0.5f_t+\varepsilon_{it}\), where \(f_t\) and \(\varepsilon_{it}\) are independent standard normal variables. The dependent design therefore has pairwise innovation correlation \(0.20\), as calculated above. Both designs satisfy the unit-root null for every unit; only their cross-sectional dependence differs.

In each replication, apply an IPS-style standardized average of unit-level Dickey–Fuller statistics and a Fisher combination of unit-level ADF p-values. For both tests, use the 5% critical value calibrated under the first-generation cross-sectional independence reference distribution. Repeat the experiment \(R=2{,}000\) times. The table gives hypothetical simulated counts for illustration, not results claimed from a real study or a particular software implementation.

Test	Innovation design under the null	Rejections out of 2,000	Empirical size
IPS-style standardized average	Independent	103	5.15%
IPS-style standardized average	Common shock, correlation 0.20	486	24.30%
Fisher-ADF combination	Independent	112	5.60%
Fisher-ADF combination	Common shock, correlation 0.20	620	31.00%

Worked Example: Calculating and Interpreting Empirical Size

For the IPS-style statistic under the common-shock null, 486 of the 2,000 replications reject. Calculate the empirical size and compare it with the nominal 5% level.

The empirical rejection rate is

\widehat{\operatorname{size}}=\frac{486}{2{,}000}=0.2430=24.30\%.

That is 19.30 percentage points above the nominal level, since \(24.30\%-5.00\%=19.30\%\). Under this hypothetical design, the test rejects the true unit-root null far more often than the 5% target suggests. The result is evidence of substantial over-rejection in this design; it does not establish that every dependence pattern produces over-rejection.

Simulation estimates also have Monte Carlo uncertainty. Treating each replication's rejection as a Bernoulli outcome, the estimated Monte Carlo standard error is approximately

SE_{\mathrm{MC}}= \sqrt{\frac{0.2430(1-0.2430)}{2{,}000}} =0.00959.

An approximate 95% Monte Carlo range is \(0.2430\pm1.96(0.00959)\), or about 22.42% to 26.18%. This range describes simulation noise around the estimated rejection rate for this specified design; it is not a confidence interval for an economic parameter. The empirical size remains far above 5% throughout this range.

What the Comparison Establishes—and What It Does Not

The independent-design rejection rates, 5.15% and 5.60%, are close to the nominal 5% level in this hypothetical experiment. This is a check that the chosen independent reference calibration behaves approximately as intended for these sample dimensions. The much larger rejection rates in the common-shock design show the size distortion caused when that calibration is carried over without accounting for cross-sectional dependence.

The two tests combine evidence differently, so their empirical sizes need not match: in the example, the Fisher combination over-rejects more than the IPS-style statistic. Neither number is a universal correction factor. Empirical size can depend on \(N\), \(T\), factor strength, loading heterogeneity, serial dynamics, deterministic terms, lag selection, and the test's critical-value approximation. Dependence can also interact with the statistic in ways that yield under-rejection. A simulation study should therefore state its null data-generating process and report its design choices rather than asserting a single general distortion rate.

A useful implication of the variance calculation is that adding more units does not automatically eliminate the problem. If pairwise correlation remains fixed and positive, then

\operatorname{Var}(\bar Z)=\sigma^2\left(\rho+\frac{1-\rho}{N}\right) \longrightarrow \sigma^2\rho \quad \text{as } N\longrightarrow\infty.

The variance does not shrink to zero as it would under independence. More units can provide more observations of the common component, but they do not turn that shared component into independent information. This is why the asymptotic regime and dependence assumptions matter when interpreting panel test calibration.

Common Mistakes

Equating nominal size with actual size. A test labelled “5%” has a 5% rejection probability only under the assumptions that justify its reference distribution. Under dependence, the empirical null rejection rate may differ.
Assuming every dependent design must over-reject. Positive dependence often inflates the variability of combined evidence, but the direction and size of distortion depend on the test statistic and the dependence structure.
Confusing a common shock with stationarity. In the specified model, the common shock enters the innovations of unit-root processes. Each series remains a unit root under the null.
Treating a large number of units as a cure. With persistent cross-sectional correlation, the variance of an average need not vanish at the independence rate.
Interpreting a simulated rejection rate as universal. An empirical size estimate applies to the stated design, sample dimensions, test implementation, and calibration—not to every panel application.

Key takeaway: First-generation panel unit-root tests use null calibrations that rely on cross-sectional independence. Common shocks can make the combined test statistic's null distribution differ from that calibration, so nominal significance levels need not equal empirical rejection rates. Assess size under a clearly specified dependent null before interpreting rejections as evidence against a unit root.

Check Your Understanding

Work through these questions using the definitions and calculations above.

In the common-shock model, what feature of the innovations creates dependence across different units?
For \(N=20\) and pairwise correlation \(\rho=0.10\), calculate the variance inflation factor for the average under the equicorrelation calculation.
A test rejects in 150 of 3,000 replications generated under the null. What is its empirical size, and how does it compare with a nominal 5% level?
Why does a Fisher combination calibrated under independent p-values generally not retain that calibration when the unit-level p-values are dependent?
Does a 25% empirical rejection rate in one common-shock simulation prove that the test always over-rejects by that amount? Explain.