Worked Fuzzy RD Analysis of Scholarship Offers

From Eligibility to a Fuzzy RD Effect

A test-score threshold may determine who receives a scholarship offer without determining who actually enrolls. Some eligible students decline, while some ineligible students enroll elsewhere. The outcome can therefore change at the threshold even though enrollment does not switch perfectly from zero to one. A fuzzy regression discontinuity uses the eligibility-induced change in enrollment as a first stage and compares it with the eligibility-induced change in an outcome.

The ratio of these discontinuities is informative only under substantive assumptions. It is not automatically the causal effect of enrollment: the design must make eligibility a valid local instrument for enrollment, and the first-stage jump must be nonzero.

Definition: Let $R$ be a running variable, such as a test score, and let $c$ be its eligibility cutoff. Define $Z=1$ for scores at or above $c$, and $Z=0$ below $c$. Let $D$ indicate actual enrollment and $Y$ be an outcome measured for everyone. For any variable $V$, let $\mu_{V,+}$ and $\mu_{V,-}$ denote its conditional mean limits at the cutoff from above and below. The first stage is $\tau_D=\mu_{D,+}-\mu_{D,-}$; the reduced form is $\tau_Y=\mu_{Y,+}-\mu_{Y,-}$. When $\tau_D\ne0$, the fuzzy RD Wald estimand is $\tau_{\mathrm{FR}}=\tau_Y/\tau_D$.

These are discontinuities in conditional means at the cutoff, not necessarily differences between the raw averages of all students above and below it. In practice, local RD methods estimate the two limits using observations near the threshold, with a specified bandwidth and appropriate inference. Here, the score is centered at the cutoff for convenience: a score of zero is exactly eligible at the threshold, negative values are below it, and positive values are above it.

What the Ratio Can Identify

A causal interpretation requires more than a visible outcome jump. The core conditions are continuity of the relevant potential outcomes and treatment propensities at the cutoff in the absence of the eligibility change; no precise sorting around the cutoff that undermines that continuity; and a nonzero discontinuity in enrollment. These conditions make the threshold a locally credible source of variation in eligibility.

To interpret the ratio as an enrollment effect, eligibility must also affect $Y$ only through $D$ near the cutoff (the exclusion restriction), and eligibility must not cause some students to become less likely to enroll while causing others to become more likely (monotonicity). Under these conditions, the ratio identifies a local average treatment effect for compliers: students near the cutoff whose enrollment status is changed by eligibility. It does not generally identify the effect for every student, nor an average effect far from the cutoff.

The exclusion restriction deserves particular scrutiny in a scholarship setting. If crossing the threshold changes the amount of financial aid, and that aid changes persistence or graduation even among students whose enrollment would not change, eligibility may affect the outcome through a channel other than enrollment. In that case, the ratio cannot be interpreted as the causal effect of enrollment alone. The treatment would need to be defined to include the relevant scholarship package, or the design would need another defensible identification strategy.

Worked Example: Estimating the Two Discontinuities

Suppose a hypothetical scholarship program uses a test-score cutoff. Researchers use local RD fits to estimate the limits in enrollment and six-year degree completion. The fitted enrollment limits are $0.42$ below and $0.62$ above the cutoff. The corresponding degree-completion limits are $0.31$ and $0.34$. Treat these as illustrative estimates from the same local specification, not as raw averages.

First stage. The estimated jump in enrollment is $\hat{\tau}_D=0.62-0.42=0.20$. Thus, eligibility raises the local enrollment probability by 0.20, or 20 percentage points.

Reduced form. The estimated jump in degree completion is $\hat{\tau}_Y=0.34-0.31=0.03$. Thus, degree completion rises by an estimated 3 percentage points at the cutoff.

These are two distinct findings. The 20-point jump describes how strongly eligibility changes enrollment; the 3-point jump describes how strongly eligibility changes the outcome. Neither should be reported as the enrollment effect on degree completion by itself. The ratio is needed for that interpretation, and only the assumptions above justify interpreting it causally.

Calculating and Interpreting the Wald Ratio

The ratio rescales the outcome discontinuity by the change in treatment receipt. If eligibility increases enrollment by 0.20 and increases degree completion by 0.03, the outcome jump per unit of enrollment jump is $0.03/0.20$. This has the units of the outcome per unit of treatment. With a binary outcome and binary enrollment indicator, it is a probability difference, expressed as a proportion or percentage-point effect.

Worked Example: The Local Complier Effect

Using the discontinuities just estimated, the fuzzy RD ratio is $\hat{\tau}_{\mathrm{FR}}=\hat{\tau}_Y/\hat{\tau}_D=0.03/0.20=0.15$.

Under continuity, no sorting that invalidates the design, exclusion, monotonicity, and a nonzero first stage, the estimate is interpreted as a 15-percentage-point increase in six-year degree completion for local compliers, the students near the cutoff whose enrollment is induced by eligibility. It is not a 15-point effect for all students offered scholarships, all students who enroll, or all students in the program.

The direction of the ratio depends on the signs of both jumps. If eligibility reduced enrollment and also reduced degree completion, the ratio could still be positive: dividing two negative discontinuities gives a positive effect per induced enrollment. Always report the first stage and reduced form alongside the ratio so readers can see what generates it.

Uncertainty in a Ratio

The uncertainty of a fuzzy RD estimate depends on uncertainty in both discontinuities and on their covariance. Because the first stage and reduced form are usually estimated from the same observations, treating them as independent without checking is generally unjustified. A common large-sample approximation uses the delta method for the function $g(a,b)=b/a$, where $a=\hat{\tau}_D$ and $b=\hat{\tau}_Y$.

The gradient of this function is $\nabla g=(-b/a^2,\;1/a)$. A first-order Taylor approximation therefore gives the following variance estimate, using the estimated variances and covariance of the two discontinuities:

\widehat{\operatorname{Var}}\left(\frac{\hat{\tau}_Y}{\hat{\tau}_D}\right) \approx \frac{\widehat{\operatorname{Var}}(\hat{\tau}_Y)}{\hat{\tau}_D^2} + \frac{\hat{\tau}_Y^2\widehat{\operatorname{Var}}(\hat{\tau}_D)}{\hat{\tau}_D^4} - \frac{2\hat{\tau}_Y\widehat{\operatorname{Cov}}(\hat{\tau}_D,\hat{\tau}_Y)}{\hat{\tau}_D^3}.

The covariance term can raise or lower the estimated variance, depending on the covariance and the signs of the jumps. A conventional approximate 95% confidence interval is the ratio estimate plus or minus $1.96$ times its estimated standard error. This approximation can be unreliable when the first stage is weak; in that situation, inference designed to remain valid with weak instruments, covered in the earlier material on weak first stages, is important.

Worked Example: A Delta-Method Confidence Interval

Continue with $\hat{\tau}_D=0.20$ and $\hat{\tau}_Y=0.03$. For a transparent calculation, suppose the estimated first-stage variance is $0.0025$, the reduced-form variance is $0.00185626$, and their estimated covariance is $0$. The zero covariance is an explicit simplifying assumption for this arithmetic example; in an analysis, it should be estimated rather than presumed.

Substituting into the delta-method formula gives $\widehat{\operatorname{Var}}(\hat{\tau}_{\mathrm{FR}})\approx 0.00185626/(0.20)^2+(0.03)^2(0.0025)/(0.20)^4$. The first term is $0.00185626/0.04=0.0464065$. The second is $0.0009(0.0025)/0.0016=0.00140625$. Thus the estimated variance is approximately $0.04781275$, and the standard error is $\sqrt{0.04781275}\approx0.2187$.

The approximate interval is $0.150\pm1.96(0.2187)$, or approximately $[-0.279,\;0.579]$ after rounding the endpoints to three decimal places. In context, the interval ranges from a 27.9-percentage-point decrease to a 57.9-percentage-point increase in degree completion for local compliers. It is wide and includes zero, so these estimates do not rule out either a negative effect or no effect at the 5% two-sided level. The interval is not a statement that there is a 95% probability that the fixed effect lies inside these particular endpoints.

The interval uses a normal approximation and the stated covariance assumption. An applied analysis should use a variance estimate appropriate to its RD procedure, including the uncertainty induced by estimating the local discontinuities. It should also assess whether the first stage is strong enough for conventional ratio inference to be informative.

Units, Assumptions, and a Second Outcome

The units of the ratio are determined by the outcome and treatment. When $Y$ is binary and $D$ is binary, a ratio of probability jumps is a probability difference. When $Y$ is measured in dollars and $D$ is binary, the ratio is in dollars per enrollment induced by eligibility. Rescaling an outcome changes the numerical ratio but not the underlying substantive contrast when the units are converted consistently.

Worked Example: Earnings Measured in Dollars

Consider a second hypothetical outcome measured as annual earnings five years after the score threshold. Suppose local RD estimates give an earnings reduced-form jump of $ \$600$ per year and the same enrollment first stage of $0.20$. The ratio is $\$600/0.20=\$3{,}000$ per year.

If the identification assumptions hold, this is an estimated increase of $ \$3{,}000$ in annual earnings per enrollment induced by eligibility for local compliers. It is not the earnings increase for every enrolled student. As before, if the scholarship itself changes later earnings through a channel other than enrollment, the exclusion restriction for an enrollment-only effect fails; the arithmetic cannot repair that design problem.

Why the Two Jumps Must Be Reported

The ratio alone hides important design information. A large ratio can result from a small first-stage denominator, and a small reduced-form jump can still produce a large ratio when the denominator is smaller still. Reporting the first stage and reduced form makes the mechanism and scale of the estimate visible. Their uncertainty also matters: uncertainty in either jump, and their covariance, contributes to uncertainty in the ratio.

The estimates should come from compatible local specifications. For example, using different bandwidths or different local samples for the first stage and reduced form can make the ratio harder to interpret as a common local contrast. The choice of bandwidth, local functional form, and inference method should be justified and examined for sensitivity, rather than selected to produce a preferred ratio.

Common Mistakes

Calling the reduced form the treatment effect. A 3-percentage-point outcome jump is an effect of eligibility on the outcome. The enrollment effect is the ratio, subject to the fuzzy RD assumptions.
Calling the ratio an effect for everyone. Under the usual assumptions, it is a local effect for compliers near the cutoff, not an average effect for all eligible students or all enrollees.
Ignoring direct effects of the offer. If scholarship eligibility affects the outcome other than through the defined treatment, the exclusion restriction for an enrollment effect is not met.
Ignoring the denominator or its uncertainty. The ratio can be unstable when the first stage is weak. Report its size and uncertainty, and use suitable inference rather than relying automatically on a normal ratio interval.
Assuming covariance is zero. The two discontinuities are often estimated from the same data. Their covariance belongs in the delta-method variance unless the chosen estimator or a justified calculation makes it zero.

Key takeaway: A fuzzy RD analysis has two essential jumps: eligibility's effect on treatment receipt and eligibility's effect on the outcome. Their ratio is a local complier effect only when the RD continuity conditions and the instrumental-variable assumptions are credible. Uncertainty in both jumps, including their covariance, must be reflected in inference.

Check Your Understanding

Try these questions using the definitions and assumptions above.

In a scholarship RD, the enrollment limit is 0.35 below the cutoff and 0.55 above it. What is the estimated first stage, in percentage points?
If the reduced-form jump in a binary outcome is 0.04 and the first stage is 0.20, calculate the Wald ratio and state its units.
Why does an exclusion-restriction violation prevent interpreting the ratio as the effect of enrollment alone?
In the delta-method variance, what role does the covariance between the first-stage and reduced-form estimates play?
Give a correct interpretation of a fuzzy RD ratio when the outcome is annual earnings and the treatment is enrollment.