Why Check Predetermined Covariates?
A regression discontinuity (RD) design compares units just below and just above an assignment cutoff in a running variable. Its causal interpretation depends on the units near the cutoff being comparable in relevant ways, apart from the change in treatment assignment. Predetermined covariates—characteristics measured before assignment and not affected by it—can help assess that comparability.
For example, if age, a pre-program test score, or prior health status changes sharply at the cutoff, the difference may indicate a change in who appears on either side, a problem with measurement, or another feature of the design that deserves investigation. If these variables show no detectable jumps, that is reassuring evidence, but it does not establish that the design is valid: unmeasured characteristics may still change, and statistical tests may have limited precision.
The focus here is the discontinuity in predetermined covariates, not the treatment effect. The earlier material on testing for sorting at the threshold addresses a related but distinct question: whether the distribution of the running variable itself changes near the cutoff.
Population Jumps and Estimated Jumps
Let \(R_i\) be unit \(i\)'s running variable, \(c\) the cutoff, and \(W_i\) a predetermined covariate. Define the population jump in the conditional mean of \(W_i\) at the cutoff as the right-hand limit minus the left-hand limit:
Here \(r\downarrow c\) denotes approaching \(c\) from above and \(r\uparrow c\) denotes approaching from below. Continuity of the covariate's conditional mean at the cutoff implies \(\Delta_W=0\). A sample analysis produces an estimate \(\hat\Delta_W\), which generally will not be exactly zero, even if the population jump is zero. The estimate is subject to sampling variability.
A common approach is to fit local linear regressions on each side of the cutoff within a chosen neighborhood. Let \(h\) denote the neighborhood width and \(K(\cdot)\) a kernel weight function. In conceptual form, the two side-specific fits are:
The local regressions use observations near the cutoff, with weights determined by distance from it. Since \(R_i-c=0\) at the cutoff, the fitted intercepts represent the estimated conditional means at the cutoff from each side. Their difference estimates the jump:
This estimate should be accompanied by a standard error and, typically, a confidence interval. The regression specification, neighborhood, weighting, and inference method should be appropriate for the RD analysis. In particular, use inference that accounts for the potential smoothing bias of local polynomial estimates; a small conventional standard error does not by itself guarantee accurate inference.
What Continuity Checks Assume
A covariate check is most interpretable when the covariate is genuinely predetermined: it was measured before treatment assignment and could not have been changed by assignment or anticipation of treatment. A post-treatment measure is not a clean balance diagnostic, because treatment itself may affect it. The running variable and cutoff must also be defined consistently, and there must be observations sufficiently close to the cutoff on both sides to estimate the local limits.
The diagnostic asks whether a covariate's conditional mean appears continuous at \(c\), in the same local sense used by RD. A zero population jump is a useful implication of the local comparability story for baseline characteristics. However, continuity of a short list of observed covariates is not the full identifying assumption for the outcome effect. It cannot verify continuity in unobserved determinants of the outcome, and a covariate may be continuous even when some other design assumption fails.
For a binary covariate, the conditional mean is a probability. Its jump is therefore measured in percentage points when reported as a proportion difference. For a quantitative covariate, the jump is in the covariate's units. Always state the units and the direction of subtraction.
Worked Example: A Continuous Baseline Age Measure
In a hypothetical RD study with cutoff \(c=0\), age was measured in years before assignment. Local linear fits within the prespecified neighborhood estimate a mean age of 41.28 years just above the cutoff and 41.10 years just below it. The estimated standard error of the difference is 0.12 years.
The estimated jump, defined as above minus below, is:
For a two-sided test of \(H_0:\Delta_{\text{age}}=0\) against \(H_a:\Delta_{\text{age}}\ne 0\), the standardized statistic is:
Using a standard normal reference distribution gives a two-sided \(p\)-value of approximately 0.134. A corresponding 95% confidence interval is:
At the 5% level, this analysis does not reject a zero population jump. The interval is compatible with a small negative jump or with a positive jump as large as about 0.42 years. The result is not proof that age is continuous, and it does not imply that the finite-sample estimate should equal zero.
Interpreting One or Many Checks
For a single covariate, report the estimated jump, its standard error or confidence interval, the units, and the local specification used. A \(p\)-value describes how unusual an estimate at least this extreme would be under the stated null and reference model; it is not the probability that the null is true. A narrow interval concentrated near zero provides more evidence against substantively large jumps than a wide interval that includes them.
When several covariates are examined, reading only separate \(p\)-values can be misleading. Even if every population jump is zero, a large enough collection of tests can produce a small \(p\)-value by chance. Conversely, separate tests may miss a pattern of modest jumps. A joint Wald test evaluates a vector of jumps together. If \(\hat{\boldsymbol{\Delta}}\) is the vector of estimated jumps and \(\widehat V\) its estimated covariance matrix, the statistic is:
Under the joint null that every population jump is zero, and under the usual large-sample conditions for the estimator, \(Q\) is compared with a chi-square distribution with degrees of freedom equal to the number of covariates tested. The covariance matrix matters: estimates for covariates measured on the same units are often correlated. Do not calculate a joint statistic by simply adding squared individual \(z\)-statistics unless the relevant covariance structure justifies that simplification.
Worked Example: Checking a Binary Prior-Enrollment Measure
In a hypothetical program evaluation, prior enrollment is a binary covariate recorded before assignment. Local regressions estimate proportions of 0.51 just above the cutoff and 0.42 just below it. The standard error of the estimated difference is 0.035.
The estimated jump is 0.09, or 9 percentage points:
Testing \(H_0:\Delta_W=0\) gives:
The two-sided normal-reference \(p\)-value is approximately 0.010. The 95% confidence interval is:
The estimated prior-enrollment proportion is higher above the cutoff, and the interval is approximately 2.1 to 15.9 percentage points above zero. This is evidence of a discontinuity in this observed baseline characteristic under the analysis assumptions. It motivates investigation of the design and data; it does not, by itself, establish the cause of the imbalance or quantify bias in the treatment effect.
Worked Example: A Joint Check of Two Covariates
Suppose a hypothetical analysis estimates jumps of 0.08 points in a pre-treatment test score and -0.06 years in age. For illustration, assume each estimate has standard error 0.10 in its own units and their estimated covariance is zero. The joint null is \(H_0:\Delta_1=\Delta_2=0\). With the diagonal covariance matrix shown below, the Wald statistic is:
There are two restrictions, so the reference distribution is chi-square with \(df=2\). The upper-tail probability is \(P(\chi^2_2\ge1.00)\approx0.607\). At the 5% level, the joint null is not rejected. That conclusion is limited by the uncertainty: for example, the test-score interval is \(0.08\pm1.96(0.10)=(-0.116,\ 0.276)\) points, which includes nontrivial positive and negative jumps. In real analyses, use the estimated covariance matrix rather than assume zero covariance.
Diagnostics, Alternatives, and Common Pitfalls
A useful assessment considers estimates and uncertainty, not only whether a test crosses a significance threshold. Inspect local data and fitted covariate relationships on each side, check that the covariate is measured consistently, and examine whether the conclusion is sensitive to reasonable analysis choices. The running-variable sorting diagnostic provides complementary evidence about the distribution of units around the cutoff; a covariate continuity check asks whether particular observed characteristics change there. Neither substitutes for the other.
Covariate adjustment is another related tool, but it serves a different purpose: including baseline covariates in an outcome analysis can improve precision under suitable conditions. Adjustment does not make a discontinuous covariate harmless, nor does it repair a compromised assignment process automatically. Covariate balance should not be used as a mechanical rule for selecting which covariates to adjust for; such selection can make the outcome analysis depend on noisy diagnostic tests.
A common mistake is to reason, “The covariate test is not significant, so the covariate is continuous at the cutoff and the RD design is valid.” The correction is that failing to reject \(H_0:\Delta_W=0\) means the data do not provide sufficient evidence against that null at the chosen level, given the model and precision. It neither proves continuity nor validates unobserved characteristics or other identifying assumptions. Similarly, a statistically detectable jump is a warning worth investigating, not automatic proof that the treatment-effect estimate is biased by a particular amount.
For multiple covariates, report the individual estimates and a joint assessment when appropriate, while acknowledging multiplicity and limited power. Give substantive context: a small jump may be precisely estimated yet practically minor, whereas a wide interval may conceal an important imbalance. These checks matter because they make observed evidence about local comparability explicit, while keeping the distinction between diagnostics and causal identification clear.
Confirm that each covariate was measured before assignment and could not be affected by treatment or anticipation.
Use a clearly stated right-minus-left convention and an RD specification appropriate to the cutoff analysis.
Report standard errors or intervals, units, and whether the estimated jump is substantively important.
Consider individual and joint patterns alongside sorting and other design information; do not treat balance tests as proof of validity.
Check Your Understanding
Answer each question, then compare with the brief solutions.
- If the population conditional mean of a baseline covariate is continuous at the cutoff, what does this imply about \(\Delta_W\), and must \(\hat\Delta_W\) equal zero?
- A binary baseline covariate has estimated proportions 0.37 above and 0.32 below the cutoff. What is the estimated jump, in percentage points?
- Why is a post-treatment measure generally unsuitable as a predetermined-covariate continuity check?
- What does a nonsignificant covariate-jump test establish, and what does it not establish?
- Why should a joint test of several covariate jumps generally use their covariance matrix?
Solutions. (1) Continuity implies \(\Delta_W=0\) in the population; sampling variability means \(\hat\Delta_W\) need not be zero. (2) The estimate is \(0.37-0.32=0.05\), or 5 percentage points. (3) Treatment may affect a post-treatment measure, so a discontinuity could reflect the treatment rather than a pre-existing imbalance. (4) It means the data do not provide sufficient evidence against a zero jump at the chosen level; it does not prove continuity or validate the whole RD design. (5) The estimates can be correlated because they are measured on the same units, and the covariance affects the joint statistic's uncertainty.