Finite-Sample Choices For Unit-Root And Cointegration Tests

Why finite-sample choices matter

Unit-root and cointegration tests are often introduced with nominal significance levels and asymptotic critical values. In an application, however, the sample is finite, and a test’s rejection rate can depend on its deterministic terms, lag order, bandwidth, and assumptions about structural change. A nominal 5% test does not necessarily reject exactly 5% of true null hypotheses in samples of the size and type being analyzed.

Simulation provides a direct way to investigate that gap. Specify a data-generating process (DGP), generate many data sets from it, apply a fully specified test to each, and record the fraction of rejections. The exercise does not establish that one specification is universally best: its conclusion is conditional on the DGPs, sample lengths, and test procedures chosen.

Definition: The empirical size of a test in a simulation is its rejection frequency when data are generated under the null hypothesis. The empirical power is its rejection frequency under a specified alternative. With \(R\) simulated data sets and \(K\) rejections, the estimated rejection frequency is \(\hat{\pi}=K/R\). For a test with nominal significance level \(\alpha\), size near \(\alpha\) is the target under the simulated null; power is assessed under each stated alternative.

These are estimates, not exact probabilities. If replications are independent and each test result is recorded as rejection or nonrejection, the rejection count under a fixed DGP has a binomial distribution. Its estimated Monte Carlo standard error is

SE_{\mathrm{MC}}(\hat{\pi})= \sqrt{\frac{\hat{\pi}(1-\hat{\pi})}{R}}.

This standard error measures simulation noise, not uncertainty in an empirical data set or uncertainty about whether the chosen DGP represents reality. Increasing \(R\) reduces simulation noise, while changing the DGP changes the question being answered.

Make the comparison interpretable

A useful simulation states the null and alternative processes, sample length, deterministic terms, test statistic, lag or bandwidth rule, critical values, and number of replications. All procedures being compared should face the same simulated data or equivalent independent replications from the same DGP. In particular, critical values must match the test statistic and deterministic specification. Comparing a statistic with critical values derived for a different specification can create apparent size distortion that is only a calibration mismatch.

The null DGP should satisfy the test’s null hypothesis, including its assumptions about deterministic components and breaks. For power, specify a particular alternative rather than referring to “the alternative” in general. A result such as “power is 60%” is incomplete unless it identifies the alternative and design under which that frequency was measured.

Worked Example: Sample Length and Deterministic Terms

Consider a hypothetical size simulation for an augmented Dickey–Fuller (ADF) test at nominal \(\alpha=0.05\). Generate 2,000 independent series for each sample length from the unit-root process \(y_t=y_{t-1}+\varepsilon_t\), where \(y_0=0\) and \(\varepsilon_t\) are independent \(N(0,1)\) innovations. Compare tests with no deterministic terms, an intercept, and an intercept plus linear trend. For each version, use its corresponding critical values and a fixed lag order selected in advance.

Suppose the simulated rejection counts for \(n=60\) are 96, 142, and 151, respectively. The empirical sizes are \(96/2000=0.0480\), \(142/2000=0.0710\), and \(151/2000=0.0755\). For \(n=240\), suppose the counts are 101, 116, and 109, giving \(0.0505\), \(0.0580\), and \(0.0545\).

The short-sample results suggest over-rejection for the intercept and intercept-plus-trend versions in this particular simulation, while the no-deterministic-term version is close to 5%. At \(n=240\), all three frequencies are closer to the nominal level. For example, the Monte Carlo standard error for the \(n=60\), intercept result is \( \sqrt{0.0710(1-0.0710)/2000}=0.00574 \), or about 0.57 percentage points.

This comparison does not show that omitting an intercept is a generally preferable procedure. The simulated process has no drift or trend; in an application with a nonzero drift or deterministic trend, omitting an appropriate term would change the maintained model. The conclusion is limited to size under the stated DGP. Power would require separately simulating specified stationary alternatives.

Lag order: residual dependence versus lost power

In an ADF regression, lagged differences are included to account for serial dependence in the innovations. Too few lags can leave residual dependence that undermines the test’s intended calibration. Too many lags use degrees of freedom and can make it harder to detect a stationary alternative, particularly in a short sample. Lag selection is therefore a finite-sample trade-off, not simply a contest to maximize or minimize the lag order.

The comparison should assess both size under a unit-root null and power under an explicit stationary alternative. A lag order that has attractive power but substantially inflated size can produce too many false rejections. Conversely, a specification with controlled size may have limited power against a particular alternative.

Worked Example: ADF Lag Order and the Size–Power Trade-Off

Suppose a hypothetical simulation uses \(n=100\), 2,000 replications, and a nominal 5% ADF test with an intercept. Under the null, generate a unit-root series whose difference is serially correlated: \(\Delta y_t=u_t\), with \(u_t=0.6u_{t-1}+\varepsilon_t\) and independent \(N(0,1)\) innovations. Under the alternative, use \(y_t=0.8y_{t-1}+u_t\) with the same serially correlated \(u_t\). Assume critical values are calibrated for the stated deterministic specification.

Suppose tests with 0, 2, and 8 lagged differences reject 228, 122, and 96 times under the null. Their empirical sizes are \(228/2000=0.114\), \(122/2000=0.0610\), and \(96/2000=0.0480\). Under the stated alternative, suppose the corresponding counts are 1,260, 1,142, and 858, giving estimated powers of \(0.630\), \(0.571\), and \(0.429\).

In this simulation, zero lagged differences gives the greatest power but also a size of 11.4%, more than twice the nominal level. Eight lags bring size close to 5% but lower power to 42.9%. Two lags provide an intermediate result. The simulation therefore identifies a trade-off for these particular processes; it does not establish that two lags are appropriate in another data set. Residual diagnostics and a stated lag-selection procedure remain relevant.

Bandwidth choices in Phillips–Perron tests

Phillips–Perron (PP) tests use a long-run variance correction rather than adding lagged differences in the ADF regression. Their finite-sample behavior can depend on the bandwidth used to estimate that correction. A bandwidth that is too small may not adequately capture dependence at longer lags; one that is too large can add estimation variability. The direction and size of the resulting distortion depend on the DGP, sample length, kernel, and bandwidth rule.

A fair bandwidth comparison holds the DGP, deterministic specification, kernel, and critical-value calibration fixed. It records size under the null and power under clearly stated alternatives. If an automatic bandwidth rule is itself being evaluated, it should be applied anew to each replication, just as it would be to observed data.

Worked Example: PP Bandwidth Sensitivity

Suppose a hypothetical PP simulation uses \(n=80\), 2,000 replications, an intercept, and the same nominal 5% level for each bandwidth. Under a unit-root null with dependent innovations, bandwidths of 2, 6, and 12 produce 146, 108, and 82 rejections. The empirical sizes are \(146/2000=0.0730\), \(108/2000=0.0540\), and \(82/2000=0.0410\).

Under a specified stationary alternative, suppose the rejection counts are 1,070, 970, and 820. The estimated powers are \(0.535\), \(0.485\), and \(0.410\). The smallest bandwidth has the greatest power here, but also the largest size distortion; the largest bandwidth is conservative in this particular null simulation and has the lowest power under this alternative.

These figures cannot identify a universally optimal bandwidth. They show why a reported PP conclusion should be checked against plausible bandwidth choices, especially if rejection changes across those choices. The bandwidths should be interpreted alongside the dependence structure and sample size, not selected after observing which one gives the desired conclusion.

Structural breaks and cointegration conclusions

A structural change can alter the finite-sample behavior of unit-root and residual-based cointegration tests. For cointegration, a relation that is stable except for a change in its intercept or slope may be difficult for a procedure that assumes one unchanging relation to detect. A break-aware procedure can instead include a specified break structure or search over possible break dates. That added flexibility must be reflected in the null distribution and critical values; using critical values for a no-break test after searching for a break is not a matched comparison.

Simulation should distinguish at least two questions. Under a no-cointegration null, does each procedure maintain its intended size for the chosen DGP? Under a cointegrated alternative with a stated break, how often does each procedure reject? A break-aware method may gain power for that alternative, but the result depends on the break type, magnitude, timing, and treatment of the break date.

Worked Example: A Break in a Cointegrating Intercept

Consider a hypothetical comparison with \(n=200\), 2,000 replications, and a break at observation 120. Under the alternative, generate \(x_t\) as a random walk and set \(y_t=1.4x_t+\mu+\delta I(t>120)+e_t\), where \(e_t\) is stationary. Thus the long-run slope is constant but the intercept shifts after the break. Compare a residual-based cointegration test that assumes no break with a procedure that allows the stated intercept shift. Give each procedure critical values calibrated for its own specification and break treatment.

Suppose that under this alternative the no-break test rejects 620 times and the break-aware test rejects 1,380 times. Their estimated powers are \(620/2000=0.310\) and \(1380/2000=0.690\). In this design, allowing for the intercept change is associated with a 38.0 percentage-point increase in rejection frequency.

Now also simulate a no-cointegration null, with independent stochastic trends and the same testing procedures. Suppose the matched null simulation yields 104 rejections for the no-break test and 112 for the break-aware procedure. Their empirical sizes are \(0.0520\) and \(0.0560\), respectively. These null results indicate that the power comparison is not accompanied by a large size distortion in this particular design. They do not guarantee correct size under every break process. If the break date is estimated by searching the data, that search must be included in the simulation and in the critical-value calibration.

A short justification for simulation precision

For one fixed DGP and test, let \(Z_r=1\) if replication \(r\) rejects and \(Z_r=0\) otherwise. If the replications are independent, then \(E(Z_r)=\pi\) and \(\operatorname{Var}(Z_r)=\pi(1-\pi)\), where \(\pi\) is the true rejection probability under that DGP. Since \(\hat{\pi}=R^{-1}\sum_{r=1}^{R}Z_r\), independence gives

E(\hat{\pi})=\pi,\qquad \operatorname{Var}(\hat{\pi})=\frac{\pi(1-\pi)}{R}.

Replacing the unknown \(\pi\) with the observed \(\hat{\pi}\) gives the Monte Carlo standard error used above. For example, at \(\hat{\pi}=0.05\) and \(R=2{,}000\), it is approximately \( \sqrt{0.05(0.95)/2000}=0.00487 \), or 0.49 percentage points. Small differences between rejection frequencies should therefore not automatically be treated as meaningful. More replications improve precision, but they cannot repair a poorly chosen DGP or mismatched critical values.

Diagnosing an empirical conclusion

A simulation is most useful when it informs a sensitivity assessment rather than a search for a preferred answer. If a unit-root or cointegration conclusion changes across plausible specifications, report that dependence directly. Check whether the alternatives use the right deterministic terms, a defensible lag or bandwidth rule, and an appropriate treatment of breaks. Interpret rejection frequencies relative to their simulated null sizes as well as their powers.

Common pitfall: “The specification with the highest simulated power is best.” This reasoning ignores size: a test that rejects too often under the null can appear powerful simply because it over-rejects. Compare size and power together under matched critical values, and describe the DGPs to which each result applies.

A disciplined comparison also keeps several distinctions clear:

Nominal level versus empirical size: \(\alpha\) is the chosen target; empirical size is the rejection frequency under a particular simulated null.
Simulation precision versus model uncertainty: Monte Carlo standard errors quantify replication noise, not uncertainty about the true process generating observed data.
Robustness versus proof: a conclusion that survives several plausible specifications is less sensitive to those choices, but simulation cannot prove that the specifications cover reality.
Power under one alternative versus general ability: power belongs to a specific alternative, sample length, and design.

Check Your Understanding

Use the definitions and comparisons above to answer these questions.

In 1,500 null replications, a test rejects 105 times. What is its empirical size, and how does it compare with a nominal 5% level?
Why should a simulation comparing deterministic specifications use critical values matched to each specification?
An ADF test with more lagged differences has lower power but empirical size closer to 5% in a simulation. What trade-off does this suggest, and what does it not establish?
How can a PP bandwidth affect both empirical size and power? Why is neither effect guaranteed to have the same direction in every DGP?
Why must a break-date search be reproduced in the simulation used to assess a break-aware test?
What additional information is needed before interpreting a reported simulated power of 60%?