Covariate Balance and Placebo Diagnostics in Regression Discontinuity

Why Examine Covariates and Placebo Outcomes?

A regression-discontinuity (RD) design compares observations just below and just above an assignment threshold. Its causal interpretation depends on how outcomes would behave near that threshold in the absence of the treatment. Diagnostic checks examine variables that should not themselves jump there, looking for evidence that the comparison may be less credible than the design assumes.

Two useful checks are continuity in predetermined covariates and continuity in placebo outcomes. Predetermined covariates are measured before treatment and cannot be caused by it. Placebo outcomes are outcomes that the treatment cannot plausibly affect, often because they were measured before treatment. A third check repeats a discontinuity analysis at false cutoffs, where the assignment rule does not actually change. These are diagnostic exercises: they can expose warning signs, but passing them does not prove that a causal effect is identified.

What a Balance Check Tests

For each predetermined covariate \(W_i\), the balance null is \(H_0:\tau_W=0\). Examples include age, baseline test scores, or a pre-treatment health measure. A check is usually performed by estimating the covariate’s left- and right-hand limits using the same general local-polynomial approach used for the outcome, and then comparing their difference with its standard error. The bandwidth, polynomial specification, and inference procedure should be chosen and reported transparently; using the same analysis choices as the main RD estimate can make the diagnostic easier to compare.

The key reason for expecting continuity is that a predetermined covariate cannot respond to treatment assigned at the cutoff. If its conditional mean is otherwise smooth through the threshold, its two limits should agree. A visible or precisely estimated jump can therefore signal that the groups immediately on either side differ in ways that deserve investigation. But a zero estimated jump does not establish that every relevant characteristic is balanced: the check addresses only the measured covariate, and its precision depends on the data and specification.

The null \(H_0:\tau_W=0\) can be assessed with a confidence interval and a test statistic. With an approximately normal large-sample estimate \(\widehat{\tau}_W\) and standard error \(SE(\widehat{\tau}_W)\), the test statistic is \(z=\widehat{\tau}_W/SE(\widehat{\tau}_W)\). The p-value is calculated under the null; it is not the probability that the null is true. A confidence interval that contains zero is not evidence that the covariate has been proved balanced. Its width also matters: a wide interval may be compatible with substantively important imbalance.

Placebo Outcomes

A placebo outcome is a measure for which the treatment cannot plausibly cause a discontinuity. A pre-treatment value of the main outcome is a common choice, provided it was recorded before treatment assignment could affect behavior or measurement. The logic resembles a covariate balance check, but the interpretation is often especially direct: a jump in a pre-treatment outcome suggests that units near the threshold may already have differed before treatment.

The same local-limit definition applies, with \(V_i\) equal to the placebo outcome. Under continuity of the relevant untreated outcome process, its conditional mean should not jump at the assignment threshold. A placebo outcome must be chosen with care: if assignment can affect anticipation, reporting, or the timing of measurement, the proposed placebo may not actually be unaffected. In that case a jump cannot be interpreted simply as evidence of a failed RD comparison.

False Cutoffs as Placebo Thresholds

A false-cutoff diagnostic estimates apparent jumps at values \(c^\ast\) where no treatment assignment rule changes. The running variable should be recentered at each candidate value, and the local comparison should be constructed in a way that is meaningful there. If the outcome-generating process is smooth at a false cutoff, a large apparent jump is unexpected and may indicate misspecification, other institutional thresholds, or chance variation.

A false cutoff is not automatically a valid placebo. It should not coincide with another policy rule or a genuine outcome discontinuity, and there must be adequate observations on both sides. Nor should a collection of false-cutoff tests be treated as independent confirmations: estimates may be correlated, and searching across many cutoffs makes it more likely that at least one looks unusual by chance. Report how cutoffs were selected, show the pattern rather than only the most striking result, and account for multiple testing when formal claims rely on many tests.

Why a Clean Diagnostic Does Not Identify the Effect

The distinction between evidence and identification can be seen directly from the sharp RD argument. Let \(Y_i(1)\) and \(Y_i(0)\) denote the potential outcomes under treatment and no treatment. Under sharp assignment, the observed outcome is \(Y_i=D_iY_i(1)+(1-D_i)Y_i(0)\). If the conditional means of both potential outcomes are continuous at \(X_i=0\), then $$ \begin{aligned} \lim_{x\downarrow 0}E[Y_i\mid X_i=x] &=\lim_{x\downarrow 0}E[Y_i(1)\mid X_i=x],\\ \lim_{x\uparrow 0}E[Y_i\mid X_i=x] &=\lim_{x\uparrow 0}E[Y_i(0)\mid X_i=x]. \end{aligned} $$

Subtracting the second limit from the first gives the local average treatment effect at the cutoff, under those continuity conditions. The equality depends on assumptions about the potential outcomes, not on the results of covariate or placebo tests. A smooth baseline covariate is reassuring because it is consistent with one implication of a credible local comparison. It does not verify continuity of unobserved determinants of outcomes, eliminate every alternative threshold mechanism, or prove that the functional form and inference procedure are adequate.

The earlier material on continuity-based identification in sharp RD gives the identifying conditions, while manipulation tests address a different concern: sorting or precise control of the running variable near the threshold. Balance and placebo diagnostics complement those checks, but neither substitutes for the identifying assumptions. Likewise, a significant diagnostic can arise by chance, from model choice, or from a real design problem; it should prompt investigation rather than mechanical acceptance or rejection of the causal analysis.

Common Mistake: Treating a Nonsignificant Test as Proof

A common error is to report, “The covariate test is nonsignificant, so the groups are balanced and the RD assumptions hold.” A p-value above the chosen significance level only means that the test did not reject its null at that level; it does not show that the true discontinuity is exactly zero. With limited precision, meaningful discontinuities may remain compatible with the interval. Further, even exact balance on the measured covariates would not establish continuity of unmeasured potential outcomes.

A better report gives the estimated jump, its standard error or confidence interval, the covariate’s units, and the substantive scale of plausible differences. It then describes the diagnostic as one piece of evidence and separately states the assumptions needed for causal identification. Significant placebo results also deserve careful attention, but they are not automatically proof of a causal design failure without checking specification, data quality, and the possibility of chance findings.