Why Bandwidth Sensitivity Needs Support Diagnostics
A regression discontinuity (RD) estimate uses observations near a cutoff to compare the outcome just below it with the outcome just above it. The bandwidth \(h\) determines how near: observations farther than \(h\) from the cutoff are excluded. Changing \(h\) can change both the fitted lines and which observations influence them. A sensitivity analysis therefore asks not only whether estimates look similar across bandwidths, but also whether each estimate is supported by enough observations on both sides.
The aim is not to find a bandwidth that produces a preferred answer. Instead, compare estimates across plausible choices and inspect which observations contribute to each fit. If a result changes substantially, support diagnostics can help show whether the change reflects a different local approximation, a handful of observations, or a gap in the running variable.
What Is Being Compared?
For a sharp RD design, assignment changes from untreated below the cutoff to treated at or above it. The target is the outcome discontinuity at \(X_i=0\). A local linear analysis fits a line separately on each side using observations in the chosen window, then subtracts the estimated left-side intercept from the right-side intercept. With a kernel, observations are weighted according to their distance from the cutoff.
For example, with weights \(w_i(h)\), the right-side fit minimizes weighted squared residuals among observations with \(0\leq X_i\leq h\); the left-side fit does likewise for \(-h\leq X_i<0\). The estimated discontinuity is the difference between their fitted values at zero:
This is a finite-sample estimate of the cutoff discontinuity, not a claim that the effect is constant throughout the window. Under the RD identification conditions—most importantly, continuity of the relevant potential-outcome regression functions at the cutoff, with no manipulation that undermines local comparability—the limiting cutoff contrast has a causal interpretation. Bandwidth sensitivity does not establish those conditions. The earlier material on sorting, covariate continuity, and placebo checks concerns complementary diagnostics.
A smaller bandwidth generally reduces the influence of observations far from the cutoff, which can reduce approximation bias when the regression functions are smooth. But it also uses fewer observations and tends to increase variance. A larger bandwidth can improve precision while making a straight-line approximation less credible over the wider window. These are tendencies, not guarantees: uneven support, outliers, or a change in the data distribution can make estimates behave irregularly.
Worked Example: Recomputing the Estimate as the Window Expands
Worked Example: Recomputing the Estimate as the Window Expands
Suppose a hypothetical sharp RD study has cutoff \(c=0\), and the analyst uses unweighted local linear fits (equivalently, a uniform kernel within each window). The running-variable offsets and outcomes are:
| Side | \(X_i\) | \(Y_i\) |
|---|---|---|
| Left | -4, -3, -2, -1 | 18, 18.5, 19, 19.5 |
| Right | 1, 2, 3, 4 | 20, 16, 16, 3 |
All observations are hypothetical. At \(h=2\), the left observations lie exactly on \(Y=20+0.5X\), so the left intercept is 20. On the right, the two points are \((1,20)\) and \((2,16)\). Their slope is \((16-20)/(2-1)=-4\), so their fitted intercept is \(20-(-4)(1)=24\). Thus \(\widehat{\tau}(2)=24-20=4\) outcome units.
At \(h=3\), the left intercept remains 20. On the right, the mean running variable is \(\bar{X}=2\), and the mean outcome is \(\bar{Y}=(20+16+16)/3=52/3\). The right-side slope is
The right intercept is \(\bar{Y}-(-2)\bar{X}=52/3+4=64/3\). Therefore \(\widehat{\tau}(3)=64/3-20=4/3\), or about 1.333 outcome units.
At \(h=4\), the left-side line still has intercept 20. On the right, \(\bar{X}=2.5\) and \(\bar{Y}=(20+16+16+3)/4=13.75\). The centered cross-product sum is \[ (-1.5)(6.25)+(-0.5)(2.25)+(0.5)(2.25)+(1.5)(-10.75)=-25.5, \] and the centered running-variable sum of squares is \(2.25+0.25+0.25+2.25=5\). The slope is \(-25.5/5=-5.1\), giving a right intercept of \(13.75-(-5.1)(2.5)=26.5\). Hence \(\widehat{\tau}(4)=26.5-20=6.5\) outcome units.
The estimates are 4, 1.333, and 6.5 outcome units at \(h=2,3,4\), respectively. The large change is a warning to investigate the wider-window fit: the right-side observations at \(X=4\) and \(X=3\) help pull its line down, changing the extrapolated value at the cutoff. This example illustrates sensitivity; by itself, it does not show which estimate is more credible or establish a causal effect.
Inspecting Support on Both Sides
Support means the observed running-variable values available to inform each side's fit. Begin with side-specific raw counts inside each bandwidth. Then inspect the actual values: a count alone can conceal a gap, repeated mass points, or observations concentrated far from the cutoff. Useful summaries include the closest observed value on each side, the number of distinct running-variable values, and the distribution of distances from the cutoff.
For weighted fits, raw counts can overstate how much information is available. One summary is the effective sample size of the weights:
This equals the raw count when all included weights are equal; unequal weights generally make it smaller. It is a useful warning signal, not a substitute for examining the local regression design. In particular, it does not account fully for the leverage of running-variable values or guarantee that a local linear slope is estimable.
Worked Example: Raw Counts Can Hide Weak Weighted Support
Consider a hypothetical cutoff \(c=100\) and a triangular kernel with bandwidth \(h=5\), so an observation at distance \(d=|R_i-c|\) receives weight \(w_i=1-d/5\) inside the window. The left-side values are 95, 97, and 98; the right-side values are 102, 103, 104, and 105.
There are 3 left observations and 4 right observations in the window. The closest left observation is \(100-98=2\) units below the cutoff; the closest right observation is \(102-100=2\) units above it. The nearest observations are therefore equally distant, but the raw counts alone do not describe the weights.
On the left, the distances are 5, 3, and 2, giving weights 0, 0.4, and 0.6. The boundary observation at distance 5 has zero weight. Thus \[ n_{\mathrm{eff,left}}=\frac{(0+0.4+0.6)^2}{0^2+0.4^2+0.6^2} =\frac{1}{0.52}\approx1.923. \] On the right, distances 2, 3, 4, and 5 give weights 0.6, 0.4, 0.2, and 0, so \[ n_{\mathrm{eff,right}}=\frac{(0.6+0.4+0.2+0)^2}{0.6^2+0.4^2+0.2^2+0^2} =\frac{1.44}{0.56}\approx2.571. \]
Although there are 3 and 4 raw observations, the effective weight counts are much smaller. The fit should be treated cautiously, and the analyst should inspect the locations and leverage of the nonzero-weight points. With \(h=3\), only values 97 and 98 on the left and 102 and 103 on the right are within the window. Under the same triangular kernel, the points at distance 3 receive zero weight; only one distinct running-variable value per side has positive weight. A separate local linear slope on each side cannot be identified from one running-variable value. This is a support failure, not merely a large-standard-error problem.
Reading a Sensitivity Table Without Overclaiming
A useful analysis presents estimates for a main bandwidth and a modest range of plausible narrower and wider alternatives. Keep the outcome, cutoff, polynomial order, kernel, covariate specification, and inference procedure fixed where possible. Report the bandwidths, side-specific counts, and uncertainty measures alongside the estimates. If a procedure includes bias correction, keep that procedure consistent across the comparison rather than mixing corrected and uncorrected inference.
The following hypothetical results illustrate how to interpret sensitivity and uncertainty together. The standard errors are assumed to be robust bias-corrected standard errors from the same procedure at each bandwidth. The intervals use the normal critical value 1.96; they are illustrative, and the counts are observations in the window.
Worked Example: Stable Estimates Are Not the Same as Precise Estimates
| \(h\) | Left/right counts | Estimate | Robust SE |
|---|---|---|---|
| 0.5 | 8 / 9 | 2.4 | 0.9 |
| 1.0 | 24 / 26 | 2.1 | 0.62 |
| 1.5 | 49 / 50 | 1.9 | 0.51 |
For \(h=1\), the illustrative 95% interval is \[ 2.1\pm1.96(0.62)=2.1\pm1.2152, \] or approximately \((0.885,3.315)\) outcome units. For \(h=1.5\), it is \[ 1.9\pm1.96(0.51)=1.9\pm0.9996, \] or approximately \((0.900,2.900)\) outcome units. The point estimates are fairly close, and the displayed intervals overlap substantially. This pattern supports describing the estimated discontinuity as reasonably stable across these choices, while also reporting the uncertainty. It does not prove the identifying assumptions, and overlapping intervals are not a formal test that the estimates are equal.
A bandwidth comparison is most informative when it includes the support diagnostics beside the estimates. If estimates move as \(h\) expands, ask whether the wider window adds many observations, adds only a few distant points, changes the balance of support, or introduces running-variable gaps. Also inspect the fitted values and residual pattern on each side. A dramatic estimate change paired with weak support on one side is especially concerning.
Use a principled primary bandwidth and plausible alternatives, not a long search for a desirable estimate.
Change the window while keeping other choices consistent so that differences are interpretable.
Tabulate counts, distinct values, nearest distances, gaps, and—when weighted—effective sample size on each side.
Describe the pattern without treating one bandwidth's significance or point estimate as a selection rule.
Common Pitfalls and Limits
- Choosing the bandwidth that gives the preferred result. This makes the reported estimate depend on the outcome of the analysis and can distort inference. Specify a primary choice on methodological grounds and show sensitivity transparently.
- Calling an estimate robust because the sign stays the same. Estimates may change substantially in magnitude, uncertainty, or support while retaining their sign. Report these features, not just direction.
- Treating the raw count as adequate support. Several observations can carry nearly all the weight, or share the same running-variable value. Check where the observations lie and whether the local regression has enough distinct support to fit its terms.
- Assuming bandwidth stability proves identification. Estimates can be stable even if sorting, measurement error, or another violation undermines the design. Sensitivity analysis complements, but does not replace, design diagnostics.
- Reading overlapping intervals as a formal equality test. Overlap is descriptive, not a test that estimates from different bandwidths are equal. The estimates are also based on overlapping data, so they are not independent.
Bandwidth comparisons have limits. They cannot repair a lack of observations near the cutoff, manufacture support where the running variable has a gap, or prove continuity of potential outcomes. If a narrower window leaves too little information to estimate the local regressions, report that limitation rather than presenting an unstable estimate as decisive. If a wider window changes the estimated discontinuity, explain how the additional observations and local fit contribute.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- If the bandwidth increases, what two competing changes in bias and variance are generally expected?
- At cutoff 50, the closest observed running-variable values are 47 on the left and 52 on the right. What are their distances from the cutoff?
- A triangular-kernel fit has weights 0.5 and 0.5 on one side. Calculate the effective sample size.
- Why is a change in estimate across bandwidths not, by itself, evidence that one estimate is causal and another is not?
- What should an analyst do if a narrow bandwidth leaves only one distinct positively weighted running-variable value on one side?
Solutions. (1) A wider window generally adds information and can reduce variance, but can increase approximation bias if the local line is less adequate farther from the cutoff. (2) The distances are \(50-47=3\) and \(52-50=2\) running-variable units. (3) \(n_{\mathrm{eff}}=(0.5+0.5)^2/(0.5^2+0.5^2)=1/0.5=2\). (4) Estimates can change because the fitted local approximation and data support change; bandwidth sensitivity neither proves nor disproves the RD identification assumptions. Diagnose support and report the pattern. (5) A separate local linear slope cannot be estimated from a single distinct running-variable value. Do not treat that fit as an ordinary viable local linear estimate; report the support limitation and reconsider the defensible analysis window or available design.