Manipulation Tests at the Regression Discontinuity Threshold

Why Examine the Running Variable’s Density?

Regression discontinuity (RD) uses a cutoff in a running variable to assign treatment. The identifying argument depends on units just below and just above the cutoff being comparable in the relevant ways, apart from their treatment assignment. That argument is less credible if units can precisely adjust their running-variable values to get onto a preferred side of the threshold. A density diagnostic asks whether the number of observed units changes abruptly at the cutoff in a way consistent with such sorting.

For example, suppose eligibility for a hypothetical program begins at a score of 50. If applicants can influence their scores, some may try to move from just below 50 to just above it. Such sorting could create an unusually high concentration of observations on one side and a shortage on the other. A density test can detect evidence of a discontinuity in the distribution of scores. It cannot, by itself, establish why that discontinuity occurred or whether every other RD assumption is satisfied.

Definition: Let \(X\) be the running variable and \(c\) the treatment cutoff. Write \(f(c^-)\) and \(f(c^+)\) for the limiting densities of \(X\) as the cutoff is approached from the left and right. A density-continuity test evaluates the null hypothesis \(H_0: f(c^-)=f(c^+)\), against a discontinuity such as \(H_a: f(c^-)\ne f(c^+)\). This diagnostic is most naturally applied when the running variable is continuously measured and has positive density near the cutoff.

Density continuity is an implication of a no-precise-sorting argument under suitable regularity conditions: if units cannot precisely manipulate which side of the cutoff they occupy, their distribution should not jump there. A discontinuity is therefore a warning sign. It is not a direct test of whether any particular unit manipulated a score, nor does the absence of a discontinuity prove that manipulation did not occur.

The Log Density Jump

A common formulation estimates the size of the discontinuity on a logarithmic scale. The log scale is useful because exponentiating the estimate gives a ratio of densities, which is straightforward to interpret.

Definition: The population log density jump at cutoff \(c\) is \(\theta=\log f(c^+)-\log f(c^-)\), assuming both limiting densities are positive. Its exponent is the right-to-left density ratio: \(\exp(\theta)=f(c^+)/f(c^-)\). Density continuity is equivalent to \(\theta=0\), or equivalently to a ratio of 1.

\theta=\log\!\left(\frac{f(c^+)}{f(c^-)}\right), \qquad \frac{f(c^+)}{f(c^-)}=\exp(\theta).

If \(\theta\) is positive, the estimated density immediately to the right is higher than the density immediately to the left. If it is negative, the estimated right-side density is lower. The percentage difference, expressed relative to the left-side density, is \(100[\exp(\theta)-1]\%\). For a negative estimate, this is a negative percentage; its absolute value describes how much lower the right-side density is.

In practice, the densities at the cutoff are not known. A standard diagnostic, commonly called the McCrary density test, estimates the density on each side using observations near the cutoff and tests the estimated log jump. Implementations use smoothing and account for uncertainty from estimating the density. The reported estimate \(\hat{\theta}\) and standard error \(SE(\hat{\theta})\) summarize that procedure. The calculation below describes the usual large-sample, two-sided test; an applied analysis should use a method appropriate to the data and report its implementation choices.

z=\frac{\hat{\theta}}{SE(\hat{\theta})}, \qquad p=2\{1-\Phi(|z|)\},

Here \(\Phi\) is the cumulative distribution function of a standard normal random variable. Under the null and the method’s regularity conditions, the statistic is approximately standard normal. A two-sided test compares the p-value with the chosen significance level \(\alpha\). A confidence interval on the log-jump scale is \(\hat{\theta}\pm z_{1-\alpha/2}SE(\hat{\theta})\); exponentiating both endpoints gives an interval for the density ratio.

How the Diagnostic Is Constructed

A histogram around the cutoff is a useful first look, but its appearance depends on bin width and bin placement. A formal density test instead estimates the left and right limits using observations near the threshold. The exact estimation details vary across implementations, but the essential comparison is the same: estimate each side’s density close to \(c\), take their log difference, and quantify uncertainty in that difference.

Specify the cutoff and running variable.
State the assignment threshold, the units of the running variable, and the sample used for the diagnostic. Verify that the recorded values and treatment rule correspond to the intended RD design.

Estimate the density on both sides.
Use a density-discontinuity procedure that smooths locally around the cutoff. The observations farther from the cutoff should not dominate an estimate intended to describe the limiting densities at the threshold.

Test the log jump and report its scale.
Calculate the test statistic and p-value, then exponentiate \(\hat{\theta}\) to report the estimated right-to-left density ratio. Include uncertainty and explain the ratio in the units and context of the design.

Interpret the diagnostic cautiously.
Assess whether the result is sensitive to reasonable estimation choices and consider institutional knowledge about score assignment. A density result is one diagnostic, not a complete identification argument.

Worked Interpretations

Worked Example: A Significant Estimated Density Increase

In a hypothetical eligibility program, the cutoff is 50 points. A density procedure applied to the running-variable observations reports \(\hat{\theta}=0.182\) and \(SE(\hat{\theta})=0.075\). Test density continuity at \(\alpha=0.05\), and interpret the estimated jump.

The null is \(H_0:\theta=0\), meaning equal limiting densities. The test statistic is \[ z=\frac{0.182}{0.075}=2.4267. \] For a two-sided test, the p-value is approximately \(2[1-\Phi(2.4267)]=0.0152\). Since \(0.0152<0.05\), reject density continuity at the 5% significance level. This is evidence of a density discontinuity under the test’s assumptions; it is not proof that applicants manipulated their scores.

The estimated right-to-left density ratio is \[ \exp(\hat{\theta})=\exp(0.182)\approx1.200. \] Thus, the estimated density immediately to the right is about 1.20 times the density immediately to the left, or approximately 20.0% higher. This ratio describes the estimated density contrast at the cutoff, not the percentage of applicants who manipulated their scores.

Worked Example: A Non-Significant Result Is Not Proof of No Sorting

For a hypothetical second program with the same cutoff of 50 points, the reported estimates are \(\hat{\theta}=-0.040\) and \(SE(\hat{\theta})=0.090\). Calculate the test result and a 95% interval for the density ratio.

The statistic is \[ z=\frac{-0.040}{0.090}=-0.4444, \] giving a two-sided p-value of approximately \(0.657\). At \(\alpha=0.05\), this result does not reject \(H_0:\theta=0\). It does not show that the densities are exactly equal; the estimate is too uncertain to distinguish this small observed jump from zero using this test.

The 95% interval on the log scale is \[ -0.040\pm1.96(0.090)=-0.040\pm0.1764=[-0.2164,\ 0.1364]. \] Exponentiating the endpoints gives \[ [\exp(-0.2164),\exp(0.1364)]\approx[0.805,\ 1.146]. \] The interval is compatible with a right-side density about 19.5% lower than the left-side density, equal densities, or a right-side density about 14.6% higher. The point estimate’s ratio is \(\exp(-0.040)\approx0.961\), about 3.92% lower, but the interval shows substantial uncertainty around that estimate.

Worked Example: Checking Sensitivity to the Estimation Window

A hypothetical analysis of the same assignment-score data reports two estimates under two reasonable local estimation windows. With a narrower window, \(\hat{\theta}=0.214\) and \(SE(\hat{\theta})=0.095\). With a wider window, \(\hat{\theta}=0.121\) and \(SE(\hat{\theta})=0.090\). Compare the results.

For the narrower window, \[ z=\frac{0.214}{0.095}=2.2526, \] with a two-sided p-value of approximately \(0.0243\). The estimated ratio is \(\exp(0.214)\approx1.239\), or about 23.9% higher density to the right.

For the wider window, \[ z=\frac{0.121}{0.090}=1.3444, \] with a two-sided p-value of approximately \(0.179\). The estimated ratio is \(\exp(0.121)\approx1.129\), or about 12.9% higher density to the right. At the 5% level, the narrower-window result rejects continuity and the wider-window result does not.

This difference is a reason to examine the estimation choices and the data near the cutoff, not a reason to select whichever window gives the preferred conclusion. The estimates describe different local smoothing choices and have different uncertainty. An analysis should justify its primary procedure and report meaningful sensitivity checks transparently.

What Density Continuity Can—and Cannot—Establish

The density test addresses one specific feature: whether the marginal distribution of the running variable has a jump at the cutoff. A detected jump raises concern because strategic sorting could make units just above and below the threshold systematically different. Institutional knowledge matters: if participants, administrators, or evaluators can influence the running variable with knowledge of the cutoff, the concern is more plausible. Conversely, a density discontinuity may arise for reasons other than deliberate manipulation, such as a change in how scores are recorded or generated.

A non-significant result is also limited. It may reflect genuinely continuous density, but it may instead reflect low power, few observations near the cutoff, noisy measurement, or a form of sorting that does not produce a detectable net density jump. In particular, a density test does not examine whether potential outcomes would otherwise evolve smoothly through the cutoff, whether the assignment rule is implemented as described, or whether all other design assumptions are credible. Density continuity is not the same as full RD validity.

The standard approach also relies on a sufficiently well-behaved running variable near the cutoff and enough data to estimate both sides. If values are discrete, heavily heaped, or otherwise poorly represented by a continuous density, the usual test may be inappropriate or misleading; specialized diagnostics are needed. This is not resolved simply by changing a bandwidth.

Common Pitfalls

Wrong: “The p-value exceeds 0.05, so there was no manipulation.” Correction: A non-significant test means the data did not provide sufficient evidence against density continuity at the chosen level. It does not prove the null or rule out every kind of sorting.
Wrong: “A significant density jump proves that people manipulated the score.” Correction: The test finds a discontinuity in the running-variable density. Manipulation is one possible explanation, and the institutional process should be examined.
Wrong: “A smooth density validates the RD design.” Correction: The diagnostic concerns the distribution of the running variable. It cannot establish the other conditions needed for a causal RD interpretation.
Wrong: “The log jump is a percentage change in density.” Correction: Exponentiate the log jump to obtain the density ratio; calculate the percentage difference as \(100[\exp(\hat{\theta})-1]\%\).
Wrong: “A histogram that looks smooth settles the question.” Correction: A visual histogram is affected by binning choices. Use a formal procedure and report its uncertainty and relevant sensitivity checks.

Key takeaway: A density-continuity test is a focused diagnostic for possible sorting at an RD threshold. Report the log jump, its uncertainty, and the exponentiated density ratio; interpret rejection as evidence of a discontinuity, not proof of its cause, and interpret non-rejection as inconclusive rather than as validation of the design.

Check Your Understanding

Use the definitions and calculations above to answer the following questions.

If \(\hat{\theta}=0.10\), is the estimated density higher immediately to the right or to the left? What ratio gives its size?
A test has p-value 0.12 at \(\alpha=0.05\). What conclusion is justified, and what stronger conclusion is not justified?
Why does a significant density discontinuity raise concern without proving that units manipulated the running variable?
If a 95% interval for the log density jump includes zero, what can be said about the corresponding interval for the density ratio?
Name one important aspect of RD validity that a density-continuity test does not establish.