Randomization Inference For Local Regression Discontinuity

Why the Assignment Mechanism Matters

A local-randomization analysis treats assignment to either side of a regression discontinuity cutoff as randomized within a selected window around that cutoff. This is an assumption about the assignment process within that window, not a claim that every unit in an ordinary regression discontinuity design was literally randomized. The assumption is useful only if it justifies a specific set of possible treatment assignments and their probabilities.

Randomization inference uses that assignment mechanism to ask how unusual the observed statistic would be if a specified null hypothesis were true. The essential operation is not to shuffle treatment labels however we please. It is to reproduce the assignments that could have occurred under the assumed mechanism, with the correct probabilities. The introductory material on local randomization near a threshold motivates the window; here the focus is inference conditional on that window and its assignment process.

Definition: For units \(i=1,\ldots,n\) in a local window, let \(Z_i=1\) indicate assignment to the treatment side of the cutoff and \(Z_i=0\) indicate assignment to the control side. A randomization mechanism is a probability distribution over the allowed assignment vectors \(Z=(Z_1,\ldots,Z_n)\). A permutation test is valid for that mechanism when its reference distribution uses those allowed assignments and their actual probabilities, under the stated null hypothesis.

Assumptions and the Randomization Test

Let \(Y_i(1)\) and \(Y_i(0)\) denote unit \(i\)'s potential outcomes under treatment and control. The observed outcome is \(Y_i=Z_iY_i(1)+(1-Z_i)Y_i(0)\). A common test statistic is the difference in observed means, \(T(Z,Y)=\bar{Y}_{Z=1}-\bar{Y}_{Z=0}\), where each mean is calculated among the units assigned to that arm. The statistic should be defined for every assignment allowed by the mechanism being tested.

For an exact Fisher randomization test, the central null is the sharp null that treatment changes no unit's outcome: \(H_0:Y_i(1)=Y_i(0)\) for every unit in the window. Under this null, the observed outcome for each unit is known under every hypothetical assignment: it is the same fixed value regardless of \(Z_i\). The assignment mechanism can therefore be applied to the fixed outcomes to obtain a reference distribution for \(T\).

This procedure requires more than a sharp null. The local window and assignment process must be specified so that the allowed assignments are credible. Potential outcomes must be well-defined, with no interference between units for the treatment being tested. The test also presumes that the outcome and statistic are recorded consistently under the possible assignments. If assignment probabilities depend on strata, blocks, a fixed treatment count, or unequal probabilities, the reference distribution must preserve that feature.

Definition: For an observed statistic \(T_{\mathrm{obs}}\), a two-sided randomization p-value is the probability, under the assignment mechanism and the null, of obtaining a statistic at least as far from zero as observed: \(p=P(|T(Z,Y)|\geq |T_{\mathrm{obs}}|\mid H_0)\). For a finite assignment set, include all ties in the tail probability. For equally likely assignments, this is the number of assignments in the tail divided by the total number of allowed assignments.

The exactness follows directly from the mechanism. Under the sharp null, the outcomes are fixed, so the only source of variation in the statistic is the assignment vector. If the observed assignment is drawn from the specified mechanism, its statistic has exactly the reference distribution generated by that same mechanism. Thus a tail probability calculated from that distribution controls the test's rejection probability at the chosen level, up to discreteness and the chosen treatment of ties.

A sharp null is stronger than the claim that the average treatment effect is zero. If effects vary across units, a zero average does not reveal every unit's missing potential outcome, so simply reassigning labels while holding observed outcomes fixed need not give an exact test of that weaker null. A test or interval for a constant additive effect can instead be obtained by testing candidate values \(\tau\) under \(Y_i(1)=Y_i(0)+\tau\), adjusting outcomes under each candidate and inverting the tests. That conclusion depends on the constant-effect null being tested.

Worked Examples

Worked Example: One Treated Unit in Each Pair

Suppose four hypothetical units lie in a justified local window. The assignment mechanism requires exactly one treated unit in each of two pre-specified pairs. Their outcomes under the sharp null are fixed at 1 and 4 in the first pair, and 7 and 2 in the second pair. Suppose the observed treated units have outcomes 1 and 2. Compute the two-sided p-value for the difference in means.

There are four allowed assignments: choose one of the two units in each pair. Under the observed assignment, the treated mean is \((1+2)/2=1.5\), and the control mean is \((4+7)/2=5.5\). Thus \(T_{\mathrm{obs}}=1.5-5.5=-4\).

Enumerating all allowed assignments gives the following statistics. Treating outcomes 1 and 7 gives \(4-3=1\), because the treated mean is \((1+7)/2=4\) and the control mean is \((4+2)/2=3\). Treating 1 and 2 gives \(1.5-5.5=-4\). Treating 4 and 7 gives \(5.5-1.5=4\). Treating 4 and 2 gives \(3-4=-1\), since the treated mean is \((4+2)/2=3\) and the control mean is \((1+7)/2=4\).

Only \(-4\) and \(4\) have absolute value at least \(|T_{\mathrm{obs}}|=4\). The assignments are equally likely, so \(p=2/4=0.50\). This is not evidence against the sharp null at conventional significance levels. The calculation depends on the pairwise assignment rule: unrestricted reassignment of two treatment labels among all four units would use a different reference distribution.

Worked Example: Complete Random Assignment With a Fixed Count

In another hypothetical local window, the assignment mechanism selects exactly two of four units for treatment, with every such assignment equally likely. Their outcomes under the sharp null are 2, 4, 5, and 8. The observed treated outcomes are 5 and 8. Find the two-sided randomization p-value using the difference in means.

The observed statistic is \((5+8)/2-(2+4)/2=6.5-3=3.5\). There are \(\binom{4}{2}=6\) allowed assignments. Treating \(\{2,4\}\) gives \(3-6.5=-3.5\); \(\{2,5\}\) gives \(3.5-6=-2.5\); and \(\{2,8\}\) gives \(5-4.5=0.5\). Treating \(\{4,5\}\) gives \(4.5-5=-0.5\); \(\{4,8\}\) gives \(6-3.5=2.5\); and \(\{5,8\}\) gives the observed statistic \(3.5\).

The statistics at least as extreme in absolute value as 3.5 are \(-3.5\) and \(3.5\). Therefore \(p=2/6=0.3333\). This test conditions on the fixed count of two treated units. A procedure that allowed assignments with one, three, or four treated units would not represent the stated mechanism.

Worked Example: Unequal Assignment Probabilities

Suppose three hypothetical units have outcomes 1, 3, and 6 under the sharp null. Before conditioning, their treatment assignments are independent, with probabilities 0.8, 0.5, and 0.25, respectively. The observed assignment treats only the first unit. For a test conditional on observing exactly one treated unit, calculate the two-sided p-value.

Conditional on one treatment, the probability that unit \(i\) is selected is proportional to its assignment odds \(p_i/(1-p_i)\). The three odds are \(4\), \(1\), and \(1/3\), with sum \(16/3\). Thus the conditional probabilities of selecting units 1, 2, and 3 are \(3/4=0.75\), \(3/16=0.1875\), and \(1/16=0.0625\), respectively.

If unit 1 is treated, the statistic is \(1-(3+6)/2=-3.5\), which is the observed statistic. If unit 2 is treated, it is \(3-(1+6)/2=-0.5\). If unit 3 is treated, it is \(6-(1+3)/2=4\). The two-sided tail includes units 1 and 3 because \(3.5\geq3.5\) and \(4\geq3.5\). Therefore \(p=0.75+0.0625=0.8125\).

The three assignments are not equally likely after conditioning, so counting two of three assignments would give the wrong p-value. The calculation uses the conditional probabilities induced by the original independent assignment mechanism.

Practical Workflow and Diagnostics

A defensible analysis makes the connection between the window and the assignment mechanism explicit. For example, a fixed-count mechanism permits only assignments with the observed treatment count; a blocked mechanism permits only assignments meeting each block's treatment count. Unequal-probability assignment requires probability weighting. These restrictions are part of the method, not optional refinements.

Specify the window and null.
Identify the units included and state the sharp null being tested. Explain why the local-randomization model is credible for this window.

Write down the assignment rule.
State which assignments are allowed and their probabilities, including any fixed counts, blocks, strata, or conditioning.

Choose and calculate a statistic.
Compute the observed statistic and ensure it is defined for every allowed assignment. A mean difference is common, but other prespecified statistics can be used.

Generate the reference distribution.
Enumerate all assignments when feasible, or sample from the specified mechanism when enumeration is too large. Preserve the assignment probabilities.

Report the tail probability and limitations.
State the null, statistic, window, assignment rule, sidedness, and whether the p-value is exact or Monte Carlo approximated.

Diagnostics should ask whether the design assumptions and computation match. Check that the window was selected using a defensible rule, that the stated treatment counts and strata match the observed design, and that every generated assignment is permitted. Predetermined covariate balance can help assess the plausibility of local randomization, but good balance does not prove that assignment was random. Conversely, severe imbalance may signal that the window or assignment model is not credible.

Window selection deserves particular care. If the window is chosen after examining outcomes or test results, reporting the ordinary randomization p-value as if the window had been fixed can understate uncertainty from that selection. If the window is selected adaptively using assignment or covariate information, the inferential procedure must account for that selection or condition on it correctly. A narrower window can improve plausibility of local exchangeability but often leaves fewer units and fewer distinct assignments, making inference coarse and less powerful.

Common Mistakes and Alternatives

Permuting labels freely. Randomly reshuffling \(Z_i\) across all units is invalid when the design fixed counts within pairs or strata. Permute only among assignments allowed by the assumed mechanism.
Assuming a sharp-null test tests a zero average effect. The sharp null fixes every unit's outcome under both assignments. A zero average effect does not. State which null the p-value addresses.
Equating a small p-value with proof of local randomization. The test's validity relies on the assignment model; the outcome test cannot establish that model.
Ignoring ties or unequal probabilities. Include tail ties, and weight assignments according to their probabilities rather than treating them as equally likely.

Local randomization inference is an alternative to regression-based inference for a regression discontinuity design, not a universal replacement. It can be attractive when a credible window supports a transparent assignment mechanism and the assignment space is small enough to enumerate. Regression-based methods instead model outcome patterns as a function of the running variable and rely on their own assumptions. Neither approach repairs an unjustified window or an implausible design model. With very few allowable assignments, exact p-values can be highly discrete: a test may be unable to attain the nominal significance level even when the observed statistic is extreme.

Key takeaway: A permutation is valid because the assignment mechanism permits it, not because it is easy to generate. State the mechanism, test a clearly defined null, and calculate the reference distribution using exactly the assignments and probabilities that mechanism specifies.

Check Your Understanding

Answer each question, then compare with the brief solutions below.

Under a sharp null, why can outcomes be held fixed while treatment assignments are varied?
A design assigns exactly three treated units in a window. Why is it generally invalid to include permutations with two or four treated units?
In a test with equally likely assignments, 3 of 20 statistics are at least as extreme as the observed statistic. What is the two-sided p-value if those 3 already count both tails and ties?
Why does a sharp-null randomization test not automatically provide an exact test of a zero average treatment effect?

Solutions. (1) The sharp null states \(Y_i(1)=Y_i(0)\) for every unit, so each unit's outcome is unchanged under any assignment. (2) Those assignments have probability zero under the fixed-count mechanism and must not enter its reference distribution. (3) \(p=3/20=0.15\). (4) A zero average effect allows individual effects to differ, so the missing potential outcomes are not all determined by the observed outcomes; the sharp-null permutation distribution is therefore not generally exact for that weaker null.