Why Diagnose the First Stage?
In a fuzzy regression discontinuity (RD) design, crossing the cutoff changes the probability of receiving treatment, but does not determine treatment perfectly. Before interpreting the fuzzy RD estimate, examine whether treatment receipt actually changes at the threshold, how precisely that change is estimated, and whether it is sensitive to reasonable choices of bandwidth. A clear, precisely estimated treatment jump supports the relevance of the cutoff as a local source of treatment variation. A small, imprecise, or unstable jump makes the resulting fuzzy RD estimate difficult to interpret.
This diagnostic concerns treatment receipt, not the outcome. A discontinuity in the outcome cannot substitute for a treatment discontinuity: the fuzzy RD estimate uses the outcome jump relative to the treatment jump. If the denominator is close to zero or very uncertain, the ratio can be unstable even when the outcome jump appears clear.
For binary \(D_i\), each conditional mean is a treatment proportion. For example, \(\Delta_D=0.20\) means that the treatment probability rises by 20 percentage points at the cutoff. It does not mean that treatment receipt rises by 20 percent relative to its value below the cutoff.
Estimating the Treatment Jump
A standard local linear estimate fits separate lines on the two sides of the cutoff using observations within bandwidth \(h\). Center the running variable at the cutoff, so the horizontal coordinate is \(R_i-c\), and weight observations by a kernel that gives more weight to observations nearer the cutoff. One convenient specification is
Here \(\tau_D\) is the estimated discontinuity: at \(R_i=c\), the fitted value just below is \(\alpha\), and the fitted value just above is \(\alpha+\tau_D\). The slopes may differ on the two sides. With kernel weights \(K((R_i-c)/h)\), the estimate is a weighted local linear regression; its standard error should account for the design and estimation procedure. The earlier material on bandwidth choice, bias correction, and robust bias-corrected inference provides the relevant tools for selecting \(h\) and reporting inference.
The model is a local approximation, not a claim that treatment receipt follows straight lines across the entire running-variable range. The diagnostic targets the fitted difference at the cutoff. A large bandwidth includes more observations and can improve precision, but may make the local linear approximation less credible. A very small bandwidth can reduce approximation bias while leaving too few observations for useful precision.
Worked Example: Estimating a First-Stage Jump
Suppose a hypothetical program becomes available at \(c=50\), and treatment receipt is recorded as \(D_i=1\) for participants. In a local linear analysis using a specified bandwidth and kernel, the fitted treatment proportion at the cutoff is 0.31 just below and 0.49 just above. The estimated jump is
Thus, estimated treatment receipt increases by 18 percentage points at the cutoff. Suppose the reported standard error is 0.06. A normal-approximation 95% interval is
In this hypothetical analysis, the interval ranges from an increase of about 6.2 to 29.8 percentage points. It excludes zero under the stated approximation, so the data provide evidence of a positive treatment-receipt discontinuity at this bandwidth. This interval describes uncertainty about the estimated first-stage jump; by itself it is not an interval for the causal effect of treatment on the outcome. In practice, use the inference method appropriate to the RD estimator rather than assuming this simple normal interval is always adequate.
Plotting the First Stage
A useful first-stage plot puts the running variable on the horizontal axis and treatment receipt on the vertical axis. Because \(D_i\) is binary, individual points can overlap heavily. Binned means show, for each running-variable bin, the proportion treated; separate local fits on each side can make the estimated cutoff jump visible. The plot should show the cutoff, identify the bandwidth used for the displayed fit, and avoid giving the impression that distant observations determine a local estimate.
The plot is a diagnostic, not a replacement for estimating the discontinuity and its uncertainty. Bins can conceal within-bin patterns, and the apparent jump can change with bin boundaries or scale. A smooth-looking graph does not establish continuity of potential outcomes, and a visually noticeable gap does not establish that the gap is estimated precisely. Inspect the underlying support near the cutoff as well: sparse observations on one side, gaps, or a few influential observations can make a fitted jump unreliable.
Worked Example: Reading the Plot Alongside the Estimate
Consider a hypothetical plot with treatment proportions in successive bins around a cutoff of 100. The bins just below the cutoff show proportions near 0.20 and 0.22; the bins just above show proportions near 0.39 and 0.41. A local linear fit estimates values of 0.21 below and 0.40 above at the cutoff, with standard error 0.05 for the difference.
The estimated jump is \(0.40-0.21=0.19\), or 19 percentage points. Its simple normal-approximation interval is \(0.19\pm1.96(0.05)=[0.092,\ 0.288]\). The plotted proportions and fitted jump tell a broadly consistent story: treatment receipt is higher above the cutoff, and the interval excludes zero under this approximation.
Now suppose the first bin above the cutoff contains only three units, while the displayed proportion is based on one treated unit. That bin's proportion, \(1/3\), can move substantially if a single unit's treatment status differs. The plot therefore calls for checking bin counts and local support, not for treating the apparent gap as conclusive evidence. The local regression estimate and its uncertainty, together with the support information, are more informative than the visual gap alone.
Size, Precision, and Bandwidth Stability
Report the estimated jump in percentage points, its standard error or interval, the bandwidth, and the number of observations contributing on each side. A statistically distinguishable jump may still be modest in size. Conversely, a seemingly large estimate with a wide interval may be too imprecise to establish a useful first stage. Statistical significance is not a measure of practical strength, and a non-significant estimate does not prove that the true jump is zero.
Re-estimate the first stage over a range of defensible bandwidths. Use the same running-variable definition, treatment definition, kernel and local polynomial order so that the comparison is interpretable. Record how the point estimate, uncertainty, and sample support change. A jump that remains similar across nearby bandwidths is reassuring about this particular specification choice, but does not prove the identifying assumptions. Large changes can indicate curvature, sparse support, influential observations, measurement problems, or sensitivity to the local approximation.
Worked Example: Comparing Bandwidths
A hypothetical study estimates the treatment jump using the same local linear specification and kernel at three bandwidths. The estimates are as follows.
| Bandwidth \(h\) | Observations below | Observations above | Estimated jump | Standard error |
|---|---|---|---|---|
| 5 | 82 | 76 | 0.16 | 0.07 |
| 8 | 131 | 126 | 0.18 | 0.05 |
| 12 | 190 | 184 | 0.11 | 0.04 |
The estimates at \(h=5\) and \(h=8\) are 16 and 18 percentage points, respectively, while the estimate at \(h=12\) is 11 percentage points. The larger bandwidth has more observations and a smaller standard error, but its estimate is also lower. For illustration, the normal-approximation interval at \(h=8\) is \(0.18\pm1.96(0.05)=[0.082,\ 0.278]\); at \(h=12\) it is \(0.11\pm1.96(0.04)=[0.0316,\ 0.188]\).
These results suggest a positive jump across the displayed specifications, but its estimated magnitude is not identical. The larger bandwidth may include observations farther from the cutoff whose treatment pattern differs from the local pattern; the smaller estimates also have less precision. The evidence should be reported as bandwidth-sensitive in magnitude rather than summarized as perfectly stable. Neither the smallest standard error nor the largest estimate should be selected just because it is more favorable.
What the First-Stage Diagnostic Can and Cannot Establish
The first-stage jump matters because the fuzzy RD Wald estimate divides the outcome discontinuity by the treatment discontinuity. In general terms, a small denominator magnifies uncertainty and can make the ratio highly sensitive to sampling variation. Thus, inspect the first-stage estimate and its uncertainty before interpreting that ratio. A positive jump is not, on its own, proof that every assumption needed for a causal fuzzy RD interpretation holds.
The relevant assumptions include a nonzero change in treatment probability at the cutoff and the continuity conditions required for the RD design: absent the cutoff-induced treatment change, the relevant potential outcomes and treatment behavior would not themselves jump at the threshold. The usual fuzzy RD causal interpretation also relies on the design's treatment-assignment and monotonicity conditions. First-stage plots and estimates address the size and precision of the treatment discontinuity; they cannot verify all of these assumptions. Sorting tests, predetermined covariate checks, and support diagnostics provide complementary evidence, but no single diagnostic proves identification.
A useful benchmark for uncertainty is the ratio of the estimated jump to its standard error, \(\widehat{\Delta}_D/SE(\widehat{\Delta}_D)\), interpreted with the chosen inference procedure. A small ratio signals that zero is not well distinguished from plausible values by that estimate and procedure. It is not a universal pass/fail rule: the adequacy of the first stage depends on the design, sample size, estimator, and uncertainty in the final ratio. Avoid importing a single threshold from a different instrumental-variables setting as a definitive RD diagnostic.
Common Pitfalls and Alternatives
- Confusing a large estimate with a precise one. A 20-percentage-point estimate with a 15-point standard error is not as informative as the same estimate with a 3-point standard error. Report both magnitude and uncertainty.
- Calling a non-significant jump zero. A wide interval may include both zero and substantively important jumps. Describe the range of uncertainty rather than accepting a zero-jump claim.
- Choosing the bandwidth that gives the preferred result. This makes the diagnostic outcome-driven. Use a defensible bandwidth strategy and show sensitivity over reasonable alternatives.
- Reading the binned graph as the estimate. Bin choices affect the display. Estimate the discontinuity with a stated method and report uncertainty and support.
- Treating stability as proof of validity. Similar first-stage estimates across bandwidths do not establish potential-outcome continuity or eliminate sorting and measurement concerns.
The sharp RD approach is appropriate when cutoff assignment determines treatment receipt without noncompliance; it does not describe a fuzzy first stage. In fuzzy RD, the first-stage estimate is paired with the outcome discontinuity in the Wald ratio, so its uncertainty is consequential. A global polynomial regression is another possible modeling choice, but fitting across a broad range can impose a different functional form and does not replace checking the local treatment jump. For this diagnostic, local estimates and transparent bandwidth comparisons keep attention on the behavior near the cutoff.
Show treatment proportions around the cutoff, bin counts, and the local fitting region. Look for sparse data, gaps, and influential bins.
Fit the stated local model and report the treatment-probability difference, uncertainty, bandwidth, and observations on each side.
Compare nearby defensible bandwidths using a consistent specification, and explain meaningful changes in magnitude or precision.
Use the first-stage evidence to assess relevance and uncertainty, while recognizing that it does not establish the other conditions for causal identification.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- At the cutoff, estimated treatment proportions are 0.27 below and 0.44 above. What is the first-stage jump, in percentage points?
- An estimated jump is 0.08 with standard error 0.07. Calculate a simple normal-approximation 95% interval. What does it imply?
- Why is a treatment-receipt plot not enough to establish a precise first-stage estimate?
- Across bandwidths, an estimate changes from 0.17 to 0.06 while its standard error decreases. What should be reported or investigated?
- Does a clearly positive first-stage jump by itself establish the causal interpretation of a fuzzy RD estimate? Explain.
Solutions. (1) \(0.44-0.27=0.17\), a 17-percentage-point increase. (2) \(0.08\pm1.96(0.07)=0.08\pm0.1372=[-0.0572,\ 0.217]\). The interval includes zero and is wide, so the estimate does not precisely distinguish the jump from zero under this approximation; it does not prove the jump is zero. (3) Binning and plotting choices can affect the display, and the graph does not itself quantify sampling uncertainty; report a stated estimator and its uncertainty. (4) Report both estimates, standard errors, bandwidths, and side-specific support. Investigate whether the larger bandwidth changes the local approximation or incorporates observations with a different treatment pattern; do not select a bandwidth solely for a preferred result. (5) No. The first stage supports relevance, but causal interpretation also requires the other RD identification conditions, which this diagnostic cannot establish by itself.