Testing for Sorting at the RD Cutoff

Why Examine the Running-Variable Distribution?

A regression discontinuity (RD) design compares observations on opposite sides of a threshold in a running variable. The design is more credible when individuals cannot precisely arrange which side of the threshold they occupy. One diagnostic is therefore to ask whether the distribution of the running variable itself changes abruptly at the cutoff.

If the density is noticeably higher on one side immediately at the threshold, that pattern may be consistent with sorting: for example, individuals or an institution may have been able to influence recorded scores so that more observations fall just above an eligibility rule. But a density test does not observe anyone’s intentions or prove that manipulation occurred. It tests a specific feature of the running-variable distribution and must be interpreted alongside the design and how the score was generated.

Definition: Let $R$ be a continuously measured running variable with cutoff $c$, and let $f_R(r)$ denote its density near $c$. A density discontinuity at the cutoff is present when the one-sided limits differ: $$ f_R(c+) \ne f_R(c-). $$ The usual no-discontinuity null hypothesis is $H_0:f_R(c+)=f_R(c-)$. A common equivalent parameterization is the log density ratio $$ \theta=\log\!\left(\frac{f_R(c+)}{f_R(c-)}\right), $$ for which the null is $H_0:\theta=0$. The alternative for a two-sided test is $H_a:\theta\ne 0$.

The density need not be flat near the cutoff. It may increase or decrease smoothly as $R$ changes. The question is whether its limiting level has a jump at $c$, not whether the counts to the left and right are equal across broad intervals. A valid diagnostic must allow for smooth trends on each side and estimate the two sides’ density levels at the cutoff.

How a Density Discontinuity Test Works

A widely used approach first divides the running-variable axis into narrow bins and estimates the density in each bin as its observation count divided by the total sample size and bin width. It then fits separate smooth local trends to the log estimated densities on either side of the cutoff and extrapolates each trend to the cutoff. The difference between the right-side and left-side fitted log densities estimates $\theta$.

In a local-linear version, the log density estimates in bins near the cutoff are regressed separately on their distance from the cutoff. The fitted intercept on each side is that side’s estimated log density at the cutoff. Local fitting matters: comparing raw counts in wide windows can confuse a smooth density trend with a discontinuity. The choice of bin width and fitting bandwidth affects the estimate and its uncertainty, so the analysis should report these choices and examine whether conclusions are sensitive to reasonable alternatives.

For inference, a typical large-sample Wald statistic is the estimated log jump divided by its standard error. Under the null and the estimator’s regularity conditions, this statistic is approximately standard normal. The two-sided p-value measures how unusual a statistic at least this far from zero would be if the null hypothesis and the approximation were correct.

\widehat{\theta} = \widehat{\log f_R(c+)} - \widehat{\log f_R(c-)}, \qquad z=\frac{\widehat{\theta}}{SE(\widehat{\theta})}.

The corresponding estimated density ratio is $\exp(\widehat{\theta})$. A ratio of 1 corresponds to equal density limits; a ratio above 1 indicates a higher estimated limiting density to the right, and a ratio below 1 indicates a lower one. The standard error for the difference must account for the uncertainty in the two estimated limits (and any covariance included by the estimator).

Assumptions and What the Test Establishes

The test is meaningful when the running variable is measured on a sufficiently fine, effectively continuous scale and there are enough observations close to the cutoff to estimate the two density limits. The local density should be positive and well behaved on both sides. The estimator also relies on its specified smoothing and asymptotic conditions; for example, the bandwidth must localize the fit while retaining enough data for useful precision.

The null hypothesis is a statement about the density at the cutoff. It is not a null hypothesis that no person manipulated a score, and rejection does not identify who sorted, how sorting occurred, or whether it was intentional. A discontinuity can arise for other reasons, including changes in measurement or data recording at the threshold, institutional rules that alter which values are observed, or chance variation. Conversely, strategic actions may occur without producing a detectable density jump, particularly when the sample is small or manipulation is limited.

The test is also not a test of the RD treatment effect. It does not estimate an outcome discontinuity or establish that the causal assumptions for an RD design hold. It provides one diagnostic signal about the distribution of the assignment variable. Failure to reject the null means that the data do not provide sufficiently strong evidence of a density jump under the chosen procedure; it is not proof that sorting is absent.

Worked Example: A Statistically Detectable Density Increase

Suppose a hypothetical eligibility program uses a continuously measured score with cutoff $c=70$. In a sample of 4,800 applicants, a binned local-linear density analysis, using the same prespecified bandwidth on both sides, estimates a log density jump of $0.182$, with standard error $0.071$. Test for a density discontinuity at the 5% significance level.

The hypotheses are $H_0:\theta=0$ and $H_a:\theta\ne 0$. The Wald statistic is $$ z=\frac{0.182}{0.071}=2.563. $$

The two-sided standard normal p-value is approximately $0.0104$. Since $0.0104<0.05$, reject the no-discontinuity null at the 5% level. The estimated ratio of the limiting densities is $$ \exp(0.182)=1.200. $$

In this hypothetical sample, the estimated density just to the right of the cutoff is about 1.20 times the density just to the left—approximately 20% higher. The test provides evidence of a density discontinuity under the chosen estimator and bandwidth. It does not, by itself, establish that applicants deliberately manipulated their scores.

A Short Justification for the Test Statistic

The log ratio expresses a comparison of two positive density limits on an additive scale. If the null is true, the ratio is 1 and its logarithm is 0. If an estimator of the log ratio is approximately normal in large samples, centering it at the null value and dividing by its standard error gives an approximately standard normal statistic.

\widehat{\theta}\ \dot{\sim}\ N\!\left(\theta,SE(\widehat{\theta})^2\right) \quad\Longrightarrow\quad \left. \frac{\widehat{\theta}-0}{SE(\widehat{\theta})} \right|_{H_0} \ \dot{\sim}\ N(0,1).

This is the justification for comparing the observed $z$ statistic with the standard normal distribution. The approximation is not exact in a finite sample. It can be poor when there are few observations near the cutoff, when bins contain very sparse counts, or when the running variable has many repeated values. The reported standard error and p-value should therefore be understood as relying on the estimator’s assumptions and approximation, not as guarantees independent of the data structure.

Worked Example: A Nonsignificant Test Is Not Proof of No Sorting

In a second hypothetical analysis of a running variable measured in points, the estimated log density jump at the cutoff is $-0.064$, with standard error $0.090$. Use a two-sided test at $\alpha=0.05$.

The test statistic is $$ z=\frac{-0.064}{0.090}=-0.711. $$

The two-sided p-value is approximately $0.477$, which is greater than $0.05$. The analysis therefore does not reject $H_0:\theta=0$. The estimated density ratio is $$ \exp(-0.064)=0.938. $$

The point estimate corresponds to a right-side limiting density about 6.2% lower than the left-side limit, but the estimate is imprecise: the test does not distinguish this pattern from a zero jump at conventional significance levels. It would be incorrect to conclude that the density is continuous or that no sorting occurred. The correct conclusion is that this analysis did not find statistically significant evidence of a density discontinuity.

Practical Limits and Interpretation

A density test’s result can depend on the neighborhood and binning used. A narrow bandwidth focuses closely on the cutoff but may leave little information; a wider bandwidth provides more observations but can make smooth curvature harder to distinguish from a jump. Binning choices also affect the intermediate histogram. These are reasons to explain the analysis and assess whether the qualitative conclusion changes under defensible choices, rather than selecting a specification because it gives the smallest p-value.

The issue is especially important when the running variable is discrete or heavily rounded. If values occur only at a finite set of mass points, a continuous density at every point may not be an appropriate description. The density-limit interpretation and usual continuous-running-variable test may then fail. With a discrete running variable, do not treat a standard histogram-based density test as automatically valid: the support structure must be considered, and the test’s assumptions may not be met. A lack of a valid density test in that setting is not evidence for or against sorting.

Worked Example: Bandwidth Sensitivity

A third hypothetical analysis uses the same continuous running variable and sample but reports two defensible bandwidth choices. With a narrower bandwidth, the estimated log jump is $0.150$ with standard error $0.080$. With a wider bandwidth, it is $0.060$ with standard error $0.050$. Calculate and compare the two tests.

For the narrower bandwidth, $$ z_{\text{narrow}}=\frac{0.150}{0.080}=1.875, $$ with two-sided p-value approximately $0.0608$. Its estimated density ratio is $\exp(0.150)=1.162$.

For the wider bandwidth, $$ z_{\text{wide}}=\frac{0.060}{0.050}=1.200, $$ with two-sided p-value approximately $0.230$. Its estimated density ratio is $\exp(0.060)=1.062$.

Neither result rejects the null at the 5% level. The estimated jump is larger under the narrower bandwidth, but both estimates are uncertain and the evidence is not robustly significant across these choices. This pattern should be reported as bandwidth-sensitive evidence, not as proof of a jump under one specification or proof of continuity under the other.

Common Mistakes

Equating a rejection with proven manipulation. A rejection is evidence that the estimated density limits differ under the model and procedure. Other features of measurement, recording, or the data-generating process can also produce a jump.
Equating non-rejection with no sorting. A large p-value does not establish the null. Limited sample size or a small discontinuity can make the test uninformative.
Comparing broad counts as if they tested the density limit. Counts over wide intervals can differ because of a smooth trend. The diagnostic estimates and compares local density levels at the cutoff.
Ignoring discreteness or heaping. A test based on a continuous density approximation may not be appropriate when the running variable has many repeated or rounded values.
Reading the density test as a treatment-effect test. The test concerns the running-variable distribution, not the size or causal interpretation of an outcome discontinuity.

Key takeaway: A density discontinuity test asks whether the running-variable density has different one-sided limits at the RD cutoff. Report the estimated log jump or density ratio, its uncertainty, and the test result; interpret rejection as diagnostic evidence, not proof of manipulation, and interpret non-rejection as lack of sufficient evidence, not proof of continuity.

Check Your Understanding

Answer each question using the definitions and interpretation rules above.

What null hypothesis is tested when the parameter is the log ratio of right-side to left-side density at the cutoff?
A test reports an estimated log jump of $0.10$ and a standard error of $0.05$. Calculate the Wald statistic and explain what the sign of the estimate indicates.
Why can comparing counts in broad intervals on either side of a cutoff confuse a smooth density trend with a discontinuity?
A density test has a p-value of $0.42$. What is an appropriate conclusion, and what conclusion would go too far?
Why can a standard continuous-density test be inappropriate for a running variable with only a few distinct values?