Covariate Continuity Checks in RD

Why Check Covariates at the Cutoff?

A regression discontinuity design relies on outcomes that would otherwise change smoothly with the running variable at the threshold. If units just above and just below the cutoff are locally comparable, a sharp change in the outcome at the threshold can be attributed to the treatment assignment under the design’s assumptions. Predetermined characteristics—such as age, prior achievement, or baseline health—offer a practical diagnostic: they should not themselves jump at the cutoff in a well-behaved design.

A covariate continuity check estimates whether a predetermined variable has a discontinuity at the same cutoff used for the primary analysis. A similar check can use a placebo outcome: a measure recorded before treatment, and therefore not caused by the treatment. A detected jump is a reason to investigate local comparability, measurement, or the assignment process. It is not, by itself, proof that the RD design is invalid. Conversely, failing to detect a jump does not prove that the design’s assumptions hold.

Estimating and Testing a Discontinuity

A standard local approach fits a separate linear relationship with the running variable on each side of the cutoff, using observations near \(c\). Write \(Z_i=1(X_i\ge c)\), where \(Z_i\) indicates which side of the threshold unit \(i\) lies on. Within a chosen bandwidth \(h\), a local linear specification for a covariate is

The intercept on the left at the cutoff is \(\alpha\); the intercept on the right is \(\alpha+\delta\). Thus \(\delta\) estimates the right-minus-left discontinuity. The slopes are allowed to differ, so the estimate compares fitted limits at the threshold rather than raw averages across two bands. In practice, local polynomial estimation commonly weights observations by their distance from the cutoff, and inference should use an appropriate standard error for the design and sampling structure.

For a single check, the usual null hypothesis is \(H_0:\Delta_W(c)=0\), against the two-sided alternative \(H_a:\Delta_W(c)\ne 0\). With estimate \(\hat{\delta}\) and standard error \(SE(\hat{\delta})\), a large-sample test statistic is

Under the null and the applicable large-sample approximation, this statistic is approximately standard normal. A confidence interval is \(\hat{\delta}\pm 1.96SE(\hat{\delta})\) for an approximate 95% interval. The estimate and interval are often more informative than a significance label alone: they show the direction, size, and precision of the observed difference.

What a Discontinuity Does—and Does Not—Tell You

The diagnostic is most compelling when the variable is genuinely predetermined: its value was fixed before the treatment could affect it, and it could not have been manipulated in response to assignment. If such a variable jumps at the cutoff, the finding is unexpected under local comparability. It may indicate sorting, a separate eligibility rule, a change in who is observed, a measurement problem, or chance variation. It calls for investigation of the design and data; it does not identify which explanation is correct.

A smooth set of observed covariates is reassuring but cannot verify continuity of unobserved determinants of the outcome. Nor can it show that every other RD assumption holds. Covariates may be weakly measured, the sample may be too small to detect meaningful differences, or the chosen bandwidth and functional form may obscure a pattern. A non-significant test is not evidence that the true discontinuity is exactly zero.

The logic of the check can be stated directly. If \(W_i\) is fixed before treatment and its conditional mean is continuous at \(c\), then its right and left limits agree, so \(\Delta_W(c)=0\). A statistically detectable estimate away from zero is evidence against that particular continuity implication, subject to the model and sampling uncertainty. But continuity of several measured covariates does not imply continuity of all potential outcomes or all relevant characteristics. The diagnostic tests implications of local comparability; it does not prove the design as a whole.

Placebo Outcomes as Predetermined Checks

A placebo outcome for this purpose is an outcome recorded before treatment assignment or exposure, chosen because the treatment cannot plausibly have caused it. For example, a study of a new service might check prior-year service use at the eligibility cutoff. The estimate and null test are constructed just as for a baseline covariate. The key distinction is temporal and causal: a post-treatment outcome is not a valid negative-control check merely because it seems unrelated to treatment. If treatment could affect it, a discontinuity might be a real treatment effect rather than a design warning.

The placebo should also be substantively interpretable. A variable that is nearly constant, measured with substantial error, or observed only for a selected subset may provide little diagnostic value. Report its scale and units, the estimated difference, uncertainty, and how it was measured. The check is about a discontinuity at the actual assignment threshold; comparisons at alternative thresholds and broader placebo designs are separate topics.

Planning and Reporting a Set of Checks

Analysts often examine several predetermined covariates. A disciplined analysis defines the variables before inspecting results, uses a defensible bandwidth and estimation method, and reports estimates with uncertainty rather than listing only p-values. The same local specification and inferential approach should be applied transparently across checks, while acknowledging that different variable scales and measurement quality affect their precision. A pattern across related measures can be more informative than one isolated result, but it still requires substantive investigation.

Testing many covariates also raises a multiple-testing issue. Even if every covariate is truly continuous, the chance of at least one small p-value increases as the number of tests grows. Thus, a single marginal result among many checks may occur by chance. Conversely, adjusting for multiple comparisons does not make a large, substantively important imbalance irrelevant. Report how many checks were examined, which were planned, and whether multiplicity adjustments were used; interpret the whole pattern alongside effect sizes and intervals.

A useful report states what was measured and when, the cutoff and bandwidth, the local estimation approach, each estimated right-minus-left difference in context, and its standard error or confidence interval. If a check shows a discontinuity, explore plausible design and data explanations and discuss how it affects confidence in local comparability. Do not simply add the covariate to an outcome regression and assume the concern is resolved; covariate adjustment and its role in RD estimation require separate justification.

Common Mistakes

• Calling a non-significant result proof of balance. A large p-value does not establish a zero discontinuity. Inspect the interval to see which differences remain compatible with the data.
• Treating any significant covariate check as conclusive invalidation. A jump challenges a particular continuity implication and merits investigation, but it may reflect sampling variation, measurement, selection, or an assignment feature. It does not alone diagnose the cause.
• Checking a variable that treatment could affect. A post-treatment outcome may jump because the treatment worked. Use genuinely predetermined measures for this diagnostic interpretation.
• Reporting only whether \(p<0.05\). Include the estimated discontinuity, units, uncertainty, and analysis choices. Statistical significance alone does not convey practical size or precision.
• Assuming many smooth covariates prove RD validity. Observed checks cannot establish continuity of unmeasured determinants or verify every identifying assumption.