Inference With Weak First Stages in Fuzzy Regression Discontinuity

Why a Small First Stage Changes the Inference Problem

In a fuzzy regression discontinuity design, crossing the cutoff changes the probability of treatment, but does not assign treatment perfectly. The local effect is commonly estimated by dividing the outcome jump at the cutoff by the treatment-probability jump. This Wald ratio is informative when the denominator is estimated precisely and is meaningfully different from zero. When the treatment-probability jump is small or imprecise, however, small sampling changes in the denominator can produce large changes in the ratio. Ordinary confidence intervals based on a normal approximation to the ratio can then give a misleading account of uncertainty.

A safer strategy is to avoid dividing by the estimated first stage. For each proposed treatment effect, form a contrast between the outcome jump and that effect multiplied by the treatment-probability jump. Test whether the contrast is zero, using its standard error, and retain the proposed effects that are not rejected. Inverting these tests yields a confidence set that remains valid under a weak first stage, provided the joint inference for the jumps is valid.

Definition: Let $\hat{\Delta}_Y$ be the estimated discontinuity in the outcome at the cutoff and $\hat{\Delta}_D$ the estimated discontinuity in treatment probability. The fuzzy-RD Wald estimate is $\hat{\tau}=\hat{\Delta}_Y/\hat{\Delta}_D$, when the estimated first stage $\hat{\Delta}_D$ is nonzero. A weak first stage is one for which the treatment-probability jump is small relative to its sampling uncertainty; it is not defined by a universal numerical cutoff.

The target $\tau$ is the local treatment effect identified by the fuzzy design under its identifying assumptions, including the relevant continuity conditions, a nonzero population treatment-probability jump, and the exclusion and monotonicity conditions used for the local Wald interpretation. Those assumptions give the ratio its causal interpretation; the inference problem here is whether the observed first-stage jump is precise enough to support conventional ratio inference.

Why Conventional Ratio Intervals Can Fail

A standard approximate interval applies the delta method to the ratio. Treating the estimated jumps as jointly approximately normal, its standard error is based on the local change in the ratio as either jump changes. With an estimated first stage near zero, these derivatives contain large inverse powers of $\hat{\Delta}_D$. The approximation is therefore unstable precisely when denominator uncertainty matters most. Moreover, ratio distributions can be strongly skewed or have long tails, whereas this method represents uncertainty by a finite, approximately symmetric interval.

For estimates with estimated variances $\widehat{\operatorname{Var}}(\hat{\Delta}_Y)$ and $\widehat{\operatorname{Var}}(\hat{\Delta}_D)$, and estimated covariance $\widehat{\operatorname{Cov}}(\hat{\Delta}_Y,\hat{\Delta}_D)$, the delta-method variance is

\widehat{\operatorname{Var}}(\hat{\tau}) \approx \frac{\widehat{\operatorname{Var}}(\hat{\Delta}_Y)}{\hat{\Delta}_D^2} + \frac{\hat{\Delta}_Y^2\widehat{\operatorname{Var}}(\hat{\Delta}_D)}{\hat{\Delta}_D^4} - \frac{2\hat{\Delta}_Y\widehat{\operatorname{Cov}}(\hat{\Delta}_Y,\hat{\Delta}_D)}{\hat{\Delta}_D^3}.

The formula makes the problem visible: as $\hat{\Delta}_D$ approaches zero, its terms can become large, and a first-order approximation to the ratio need not describe its sampling distribution well. A large reported standard error alone does not repair the approximation.

Worked Example: A Weak First Stage and a Finite Delta-Method Interval

Suppose a hypothetical fuzzy-RD analysis estimates an outcome jump of $\hat{\Delta}_Y=0.020$ outcome units and a treatment-probability jump of $\hat{\Delta}_D=0.025$. Suppose the standard errors are $0.012$ outcome units and $0.015$, respectively, and their estimated covariance is zero. The point estimate is

\hat{\tau}=\frac{0.020}{0.025}=0.800.

The first-stage estimate is only $0.025/0.015=1.67$ standard errors from zero. The delta-method variance is

\widehat{\operatorname{Var}}(\hat{\tau}) = \frac{0.012^2}{0.025^2} + \frac{0.020^2(0.015^2)}{0.025^4} = \frac{0.000144}{0.000625} + \frac{0.000000090}{0.000000390625} = 0.2304+0.2304=0.4608.

Thus the approximate standard error is $\sqrt{0.4608}=0.6788$, and a nominal 95% delta-method interval is $0.800\pm1.96(0.6788)$, or approximately $[-0.530,\,2.130]$ effect units. The calculation is internally consistent, but it relies on a local normal approximation to a ratio whose denominator is imprecisely estimated. A test-inversion interval can instead be unbounded or disconnected, shapes that a finite symmetric interval cannot express.

Test a Candidate Effect Without Dividing

For a proposed effect $\tau_0$, the Wald-ratio null hypothesis can be written without a ratio: $H_0:\Delta_Y-\tau_0\Delta_D=0$. Here $\Delta_Y$ is the population outcome jump and $\Delta_D$ is the population treatment-probability jump. The sample contrast is $\hat{\Delta}_Y-\tau_0\hat{\Delta}_D$. Its estimated variance must account for uncertainty in both jumps and their covariance:

\widehat{\operatorname{Var}}(\hat{\Delta}_Y-\tau_0\hat{\Delta}_D) = \widehat{\operatorname{Var}}(\hat{\Delta}_Y) +\tau_0^2\widehat{\operatorname{Var}}(\hat{\Delta}_D) -2\tau_0\widehat{\operatorname{Cov}}(\hat{\Delta}_Y,\hat{\Delta}_D).

If the joint estimates have an appropriate large-sample normal approximation and the variance estimator is consistent, the test statistic is the contrast divided by its estimated standard error. For a two-sided test at significance level $\alpha$, reject $H_0$ when its absolute value exceeds the standard normal critical value $z_{1-\alpha/2}$. In practice, the jump estimates and their covariance must come from a coherent RD inference procedure for the chosen specification and bandwidth. If bias correction is used, the variance must correspond to that bias-corrected procedure.

Theorem: Inverting the two-sided tests of $H_0:\Delta_Y-\tau_0\Delta_D=0$ gives the confidence set $$ \mathcal{C}_{1-\alpha} = \left\{\tau_0: \left|\hat{\Delta}_Y-\tau_0\hat{\Delta}_D\right| \leq z_{1-\alpha/2} \sqrt{ \widehat{\operatorname{Var}}(\hat{\Delta}_Y) +\tau_0^2\widehat{\operatorname{Var}}(\hat{\Delta}_D) -2\tau_0\widehat{\operatorname{Cov}}(\hat{\Delta}_Y,\hat{\Delta}_D)} } \right\}. $$ Under the stated joint large-sample approximation and valid variance estimation, this procedure has asymptotic coverage $1-\alpha$ without requiring the first-stage jump to be bounded away from zero.

This is often called an Anderson–Rubin-type test inversion. Its logic is direct: a candidate effect remains in the set exactly when the corresponding null is not rejected. The method does not require a precise denominator to estimate a ratio. Instead, it carries the uncertainty in both jumps into every candidate-effect test.

Why the Confidence Set Can Have Different Shapes

Squaring both sides of the confidence-set inequality produces a quadratic inequality in $\tau_0$. Its leading coefficient depends on the squared estimated first stage relative to the uncertainty in that jump. Consequently, inversion need not produce an ordinary finite interval. It can produce a bounded interval, two unbounded rays, a half-line, the whole real line, or, in some cases, an empty set. These are meaningful features of the test inversion, not computational errors to be removed by forcing a conventional interval.

Worked Example: A Bounded Set From Test Inversion

Consider hypothetical jump estimates $\hat{\Delta}_Y=0.200$ outcome units and $\hat{\Delta}_D=0.200$, with estimated variances and covariance defined by

\widehat{\operatorname{Var}}(\hat{\Delta}_D)=\frac{0.040000-0.010000}{3.8416}, \qquad \widehat{\operatorname{Var}}(\hat{\Delta}_Y)=\frac{0.040000-0.009689276}{3.8416},

\widehat{\operatorname{Cov}}(\hat{\Delta}_Y,\hat{\Delta}_D) = \frac{0.040000-0.010310730}{3.8416}.

These are approximately $0.00780925$, $0.00789013$, and $0.00772836$, respectively. The covariance matrix is positive definite: its determinant is approximately $0.00000189>0$. For a 95% set, $z_{0.975}=1.96$, so $z_{0.975}^2=3.8416$. Substitution into the test-inversion inequality and squaring yields

(0.200-0.200\tau_0)^2 \leq 3.8416\left( \frac{0.030310724}{3.8416} -2\tau_0\frac{0.029689270}{3.8416} +\tau_0^2\frac{0.030000}{3.8416} \right),

or, after collecting terms and moving the right side to the left,

0.010000\tau_0^2-0.02062146\tau_0+0.009689276\leq0.

The discriminant is

$$ (-0.02062146)^2-4(0.010000)(0.009689276) = 0.0004252446-0.00038757104 = 0.00003767357. $$

Its square root is approximately $0.00613707$. The roots are therefore

\frac{0.02062146-0.00613707}{0.020000}\approx0.7242, \qquad \frac{0.02062146+0.00613707}{0.020000}\approx1.3379.

Because the quadratic opens upward, it is nonpositive between its roots. The 95% confidence set is approximately $[0.724,\,1.338]$ effect units. The ratio estimate is $0.200/0.200=1.000$, which lies in this set. The exact endpoints depend on the unrounded inputs; the displayed values are rounded to three decimals.

Worked Example: An Unbounded Confidence Set

In a second hypothetical analysis, let $\hat{\Delta}_Y=0.100$ outcome units and $\hat{\Delta}_D=0.050$. Suppose their estimated standard errors are $0.020$ and $0.040$, with estimated covariance zero. For a 95% set, the test-inversion inequality is

(0.100-0.050\tau_0)^2 \leq 3.8416(0.0004+0.0016\tau_0^2).

Expanding and subtracting the right side gives

-0.00364656\tau_0^2-0.010000\tau_0+0.00846336\leq0.

The roots are approximately $-3.42$ and $0.678$. Since the quadratic opens downward, it is nonpositive outside the roots. Thus the 95% set is approximately $(-\infty,-3.42]\cup[0.678,\infty)$. The ratio estimate is $0.100/0.050=2.00$, which belongs to the set, but so do arbitrarily large positive effects. This unboundedness records that the data do not rule out effects along an unbounded range at this confidence level; it should not be replaced with a finite interval merely for convenience.

Justification: Inversion Preserves the Test's Error Control

The coverage logic follows from the definition of a confidence set. Suppose the true local effect is $\tau$. By the fuzzy-RD moment restriction, $\Delta_Y-\tau\Delta_D=0$. If the test of this true candidate has size at most $\alpha$, it rejects with probability at most $\alpha$, under the model and inference approximation. The confidence set contains $\tau$ whenever that test does not reject. Therefore

P(\tau\in\mathcal{C}_{1-\alpha}) = 1-P(\text{test of the true }\tau\text{ rejects}) \geq 1-\alpha.

This argument does not divide by $\hat{\Delta}_D$, which is the key protection against weak-first-stage behavior. It does not make the design assumptions unnecessary, nor does it rescue invalid standard errors or a poor approximation for the jumps. It addresses the ratio's denominator problem conditional on valid inference for the underlying discontinuities.

Practical Interpretation and Common Pitfall

Report the confidence set in the units of the treatment effect, state its confidence level, and describe its shape. If it is unbounded or disconnected, say so plainly. A set that includes many candidate effects may be scientifically uninformative, but this is different from evidence that every included value is equally plausible. Likewise, non-rejection means the data do not reject that candidate under the test; it does not establish that the candidate is true.

Common mistake: reporting a Wald-ratio estimate and a symmetric normal interval as reliable simply because a software package returned them.
Correction: assess first-stage precision and use test inversion when weak-first-stage behavior is a concern. Show whether the confidence set is bounded, disconnected, or unbounded.
Common mistake: dropping an unbounded part of the set because it looks implausible.
Correction: report the set implied by the stated test. Any restriction to a scientifically plausible parameter range must be justified and reported as an additional restriction, not silently imposed.

Weak-first-stage-robust inference is valuable because it distinguishes the point ratio from what the data can establish about the local effect. A precise treatment-probability jump may support a concise ratio interval. When that jump is weak, test inversion can honestly show the resulting lack of precision rather than concealing it behind a finite normal approximation.

Check Your Understanding

Use the definitions and test-inversion logic to answer these questions.

Why can a small estimated treatment-probability jump make a delta-method interval for a fuzzy-RD Wald ratio unreliable?
For a candidate effect $\tau_0$, write the null restriction tested by comparing the outcome jump with the treatment-probability jump.
Why must the variance of the candidate-effect contrast include the covariance between the two estimated jumps?
What does an unbounded confidence set indicate, and why should it not be replaced automatically by a finite interval?
Does a candidate effect's inclusion in the inverted set prove that it is the true effect? Explain.