Placebo Cutoffs and Outcomes in RD

Why Use Placebo Tests in Regression Discontinuity?

A regression-discontinuity (RD) design attributes a discontinuity in an outcome at a known assignment threshold to the treatment induced by crossing that threshold, subject to the design’s continuity assumptions. Placebo tests probe whether the data show discontinuities where the design says none should occur. They can reveal warning signs, but they do not establish that the causal assumptions hold.

Two common checks use different comparisons. A placebo-cutoff test asks whether an outcome appears to jump at a running-variable value where no assignment rule changes. An unaffected-outcome test asks whether an outcome that treatment cannot affect jumps at the actual cutoff. The first probes the location-specific pattern; the second probes for a discontinuity in a measure that should not respond causally to treatment.

Definition: Let \(R\) be the running variable, \(c\) the actual RD cutoff, \(Z\) the treatment-assignment indicator, \(D\) the treatment received, and \(Y\) the outcome. A placebo cutoff \(c^*\) is a value distinct from \(c\) at which the assignment mechanism is not supposed to change. A placebo outcome (or unaffected outcome) is an outcome that the treatment cannot causally affect, so a discontinuity in it at the actual cutoff is not attributable to a treatment effect.

A placebo test typically estimates a local difference between the right- and left-hand limits of an outcome’s conditional mean. For an outcome \(W\) and candidate threshold \(a\), write this discontinuity as

\tau_W(a)=\lim_{r\to a^+}E[W\mid R=r]-\lim_{r\to a^-}E[W\mid R=r].

In practice, the limits are estimated from observations near \(a\), often using local linear regression and an appropriate bandwidth and standard error. The estimate tests for a local jump, not for equality of the entire outcome distributions on both sides. The same basic diagnostic logic applies whether the primary design is sharp or fuzzy, but the assignment and treatment processes must be understood correctly.

When Should a False Cutoff Have No Jump?

At the true cutoff, a sharp RD changes treatment assignment according to \(Z=1[R\ge c]\). A false cutoff placed away from \(c\) does not create that assignment change. In a sharp design, treatment status is therefore constant in a sufficiently small neighborhood of \(c^*\), provided that neighborhood does not include \(c\). A zero-jump prediction then requires continuity of the potential outcome corresponding to that constant treatment status.

Let \(Y(0)\) and \(Y(1)\) denote potential outcomes under no treatment and treatment, respectively, and let \(m_d(r)=E[Y(d)\mid R=r]\). If treatment status is constantly \(d^*\) near \(c^*\), the observed outcome there is \(Y(d^*)\). If \(m_{d^*}(r)\) has equal left- and right-hand limits at \(c^*\), the expected observed outcome has no jump there. The continuity condition must concern the potential outcome actually observed in that neighborhood.

\lim_{r\to c^{*+}}m_{d^*}(r) = \lim_{r\to c^{*-}}m_{d^*}(r) \quad\Longrightarrow\quad \tau_Y(c^*)=0.

This qualification matters. If the neighborhood is constantly treated, continuity of \(m_0(r)\) alone is not enough: observed outcomes follow \(Y(1)\), not \(Y(0)\). One must instead assume continuity of \(m_1(r)\), or choose a neighborhood that is untreated and invoke continuity of \(m_0(r)\).

In a fuzzy RD, treatment receipt \(D\) need not be constant on either side, even where assignment \(Z\) is constant. A false cutoff should not induce a jump in assignment, but a zero outcome jump also depends on the local treatment and outcome processes being continuous there. For example, a discontinuous change in take-up or in the composition of treated units could produce an outcome jump even without a new assignment threshold. Thus, a placebo-cutoff test is informative only when the chosen location and the relevant continuity conditions justify the no-jump prediction.

Worked Example: Testing a False Cutoff in an Untreated Region

Suppose a hypothetical sharp RD assigns treatment at \(c=70\), and investigators examine \(c^*=60\), a threshold in a region where everyone is untreated. They use a narrow symmetric window around 60 and, for illustration, a local-constant comparison. Assume the untreated conditional mean is continuous at 60 and approximately flat within this window. The mean outcome on the left is 50.2 points for 100 observations, with standard deviation 8.0 points; on the right it is 51.0 points for 100 observations, with standard deviation 7.0 points. Treat the observations in the two groups as independent for this illustrative calculation.

The estimated placebo jump is right minus left:

\hat{\tau}_Y(60)=51.0-50.2=0.800\text{ points}.

Using the standard error for a difference in independent sample means,

SE(\hat{\tau}_Y) = \sqrt{\frac{8.0^2}{100}+\frac{7.0^2}{100}} = \sqrt{1.13} = 1.063\text{ points}.

The test statistic for a zero jump is \(z=0.800/1.063\approx0.753\), giving a two-sided normal-approximation p-value of about 0.452. A 95% confidence interval is

0.800\pm 1.96(1.063) = [-1.283,\ 2.883]\text{ points}.

This illustrative test does not detect a discontinuity at 60. The interval says that, under the calculation’s assumptions, values from about 1.28 points lower to 2.88 points higher are compatible with the estimated jump at the 95% confidence level. It does not establish that the jump is exactly zero. In an applied RD analysis, local linear estimation and inference appropriate to the design would generally replace this local-constant calculation.

Testing an Outcome Treatment Cannot Affect

An unaffected outcome is measured or defined so that changing treatment status cannot change its value. A pre-treatment measurement can sometimes serve this role, but timing alone is not sufficient: the outcome must not be affected by anticipation or by another intervention triggered at the cutoff. Other suitable outcomes require a substantive argument that treatment cannot cause them to change.

For a sharp RD, suppose the potential values of an unaffected outcome \(Y^{NC}\) are identical under treatment and no treatment: \(Y^{NC}(1)=Y^{NC}(0)\). If its conditional mean is continuous at \(c\), the actual assignment switch cannot create a discontinuity in its expected value. For a fuzzy RD, the same idea applies because changes in treatment receipt cannot change an outcome that is genuinely unaffected; continuity of the outcome process at the cutoff is still needed. A jump may signal another cutoff-linked change, a measurement problem, chance variation, or a mistaken claim that the outcome is unaffected.

Worked Example: A Discontinuity in a Pre-Treatment Measure

In a hypothetical sharp RD, an eligibility threshold at \(c=50\) offers a training program. A recorded test score from before the offer is used as an unaffected outcome, with the justification that the offer cannot change a score already measured and there was no anticipatory response to the threshold. In a narrow window, the left-side mean is 72 points (standard deviation 10, \(n=100\)); the right-side mean is 76 points (standard deviation 12, \(n=100\)). Suppose the conditional mean of this pre-treatment measure would otherwise be continuous at 50, and use independent-group standard errors for this illustration.

The estimated discontinuity is

\hat{\tau}_{Y^{NC}}(50)=76-72=4.00\text{ points}.

Its standard error is

SE(\hat{\tau}_{Y^{NC}}) = \sqrt{\frac{10^2}{100}+\frac{12^2}{100}} = \sqrt{2.44} = 1.562\text{ points}.

Thus \(z=4.00/1.562=2.561\), with a two-sided normal-approximation p-value of about 0.0104. The 95% confidence interval is

4.00\pm1.96(1.562) \approx [0.938,\ 7.062]\text{ points}.

Under the stated assumptions, this is evidence of a discontinuity in a measure that should not respond to the offer. It raises concern about the design—for example, another process may change at the threshold—but does not identify which assumption or mechanism is responsible. Nor does it, by itself, prove that the treatment-effect estimate for the primary outcome is biased by a particular amount.

Diagnostic Power: A Quiet Test Can Be Uninformative

A falsification test is useful when it has a clear zero-jump prediction and enough precision to detect departures that would matter. A large p-value alone is not evidence that the design is sound. If the estimate is imprecise, the data may be consistent both with no jump and with a consequential one. The confidence interval, the size of a substantively important jump, and the quality of the negative-control argument all matter.

Worked Example: A Nonsignificant Test With Little Precision

Consider a hypothetical unaffected outcome measured near the actual cutoff. The estimated left and right means are 100.0 and 100.5 units. Each group has a standard deviation of 20 units and contains 25 observations. Assume independent groups for this calculation and use a two-sided normal approximation.

The estimated jump is \(100.5-100.0=0.500\) units, and its standard error is

SE(\hat{\tau}) = \sqrt{\frac{20^2}{25}+\frac{20^2}{25}} = \sqrt{32} = 5.657\text{ units}.

The test statistic is \(z=0.500/5.657=0.0884\), giving a two-sided p-value of about 0.930. But the 95% confidence interval is

0.500\pm1.96(5.657) = [-10.588,\ 11.588]\text{ units}.

The test does not detect a jump, but the interval also includes discontinuities as large as roughly 11.6 units in either direction. If a jump of 5 units would be concerning, these data do not rule it out. The appropriate conclusion is that the test is inconclusive about jumps of that size, not that continuity has been demonstrated.

How to Design and Report a Placebo Test

Choose the placebo location or outcome based on the design’s assumptions, not because it happens to produce a reassuring estimate. For false cutoffs, explain why assignment does not change there and why the observed outcome process should be continuous there. For unaffected outcomes, explain why treatment cannot affect the measure and why other threshold-linked processes are not expected to change it. A test with no defensible zero-jump prediction is not a meaningful falsification test.

State the falsification claim.
Name the false cutoff or unaffected outcome, and say why the expected discontinuity is zero.

Check the relevant process.
At a false cutoff, check the assignment and treatment process and identify the potential outcome observed locally. For an unaffected outcome, justify that treatment cannot change it.

Use a comparable local analysis.
Estimate the jump with a stated window or bandwidth, functional form, and standard-error method. Avoid selecting specifications solely because they yield a null result.

Report magnitude and uncertainty.
Give the estimated jump in context, its standard error or confidence interval, and the test result. Explain whether the interval excludes jumps that would matter substantively.

Interpret the diagnostic modestly.
A detected jump is a warning requiring investigation; a small, precise estimate can be reassuring for that particular check, but neither result proves or disproves all RD assumptions.

Multiple placebo tests can be useful because each probes a different implication, but testing many locations and outcomes also makes chance rejections more likely. Report the set of checks rather than highlighting only the most favorable one. Interpret patterns, uncertainty, and design knowledge together.

Common Mistake

Wrong reasoning: “The placebo test has \(p>0.05\), so there is no discontinuity and the RD assumptions are verified.” This treats failure to reject a point null as proof of equality and ignores the test’s precision. Another error is to claim a zero jump at a false cutoff merely because untreated potential outcomes are continuous, even when the neighborhood is constantly treated.

Correction: State the assumptions that imply no observed-outcome jump at the chosen placebo. In a locally treated neighborhood, that includes continuity of the treated potential outcome, not only the untreated one. Then interpret the estimate and confidence interval: a narrow interval around zero can rule out some meaningful jumps, while a wide interval cannot. Falsification tests examine selected implications; they cannot verify every continuity or assignment assumption in an RD design.

Key takeaway: A placebo cutoff is informative only when the assignment and observed outcome processes justify a zero jump there. An unaffected outcome is informative only when treatment truly cannot affect it and other discontinuities are not expected. In either case, the estimate’s uncertainty determines how much a quiet test can tell you.

Check Your Understanding

Use the assumptions behind each test, not only its p-value, to answer these questions.

At a false cutoff, treatment status is constantly 1 in the neighborhood. Which potential-outcome conditional mean must be continuous to justify a zero observed-outcome jump?
Why does a false cutoff in a fuzzy RD require attention to treatment receipt as well as the absence of an assignment-threshold change?
A pre-treatment outcome jumps at the actual cutoff. Give two possible explanations other than a causal effect of treatment on that already-recorded outcome.
A placebo test has a large p-value but a confidence interval that includes substantively large jumps. What is the appropriate conclusion about its diagnostic power?
What additional justification is needed before calling a pre-treatment measurement an unaffected outcome?