Comparing Instrument-Specific Wald Estimates

Why Compare Instruments One at a Time?

With more than one instrument, a single instrumental-variables estimate can conceal an important feature of the data: each instrument may shift treatment uptake for a different set of people. The ratio of an instrument’s effect on the outcome to its effect on treatment uptake is its instrument-specific Wald estimate. Comparing these ratios can reveal whether the instruments point to similar effects or to different local effects. But a difference alone does not establish why they differ.

A careful comparison separates three possibilities. The instruments may identify effects for different complier groups whose treatment effects vary. One or both first stages may be too weak for stable ratio estimation. Or one or both instruments may fail a causal assumption, such as the exclusion restriction. Sampling variation also matters: a visible difference between point estimates may be imprecise.

Instrument-Specific Reduced Forms and Wald Ratios

Consider a binary treatment $D$, an outcome $Y$, and two candidate instruments, $Z_A$ and $Z_B$. For each instrument, define the reduced form as its effect or contrast on $Y$, and the first stage as its effect or contrast on $D$. For a binary instrument, these may be expressed as the difference in mean outcomes and the difference in treatment proportions between the instrument’s two levels. When covariates are used, the reduced form and first stage must be defined using compatible adjustment and estimation procedures.

Definition: For instrument $Z_j$, let $\widehat{RF}_j$ denote its estimated reduced form and $\widehat{FS}_j$ its estimated first stage. If $\widehat{FS}_j\ne 0$, its instrument-specific Wald estimate is $$ \widehat{\theta}^{W}_j=\frac{\widehat{RF}_j}{\widehat{FS}_j}. $$ The estimate is measured in outcome units per unit of treatment. For a binary treatment, this is commonly interpreted as an effect in outcome units per treatment received, under the assumptions that give the ratio a causal interpretation.

The ratio is not automatically causal. A standard interpretation for a binary instrument and binary treatment requires instrument independence from potential treatment and outcome outcomes, an exclusion restriction (the instrument affects the outcome only through treatment), a nonzero first stage, and monotonicity (the instrument does not move some people toward treatment while moving others away). These assumptions apply to each instrument being interpreted. Under them, the Wald ratio identifies an average treatment effect for the people whose treatment status is changed by that instrument, often called its compliers. If the instruments change treatment for different groups, their ratios need not be equal even when both instruments are valid.

Worked Example: Calculating Two Wald Estimates

Suppose a hypothetical study estimates the following contrasts. The outcome is a symptom score measured in points, and the treatment indicator is on a zero-to-one scale. Instrument A has a reduced form of $0.80$ score points and a first stage of $0.20$. Instrument B has a reduced form of $0.54$ score points and a first stage of $0.15$.

For A, the Wald estimate is $$ \widehat{\theta}^{W}_A=\frac{0.80}{0.20}=4.00 $$ score points per treatment received. For B, it is $$ \widehat{\theta}^{W}_B=\frac{0.54}{0.15}=3.60 $$ score points per treatment received. The point estimates differ by $4.00-3.60=0.40$ score points per treatment.

These calculations describe the ratios, not yet their precision or causal meaning. A difference of $0.40$ may be sampling noise; the instruments may also identify different complier effects. The ratios have a causal interpretation only if the relevant assumptions are credible for each instrument.

Uncertainty in a Ratio and in the Difference

A Wald estimate is a ratio of estimated quantities, so its uncertainty depends on uncertainty in both the reduced form and the first stage. If the first stage is small, even modest sampling variation in the numerator or denominator can produce substantial variation in the ratio. The delta method gives a large-sample approximation when the first stage is sufficiently far from zero and the joint estimates behave regularly.

For a ratio $R/F$, the gradient with respect to $(R,F)$ is $(1/F,-R/F^2)$. If $\widehat{\operatorname{Var}}(\widehat{RF})$, $\widehat{\operatorname{Var}}(\widehat{FS})$, and $\widehat{\operatorname{Cov}}(\widehat{RF},\widehat{FS})$ are estimated jointly, the delta-method variance is

\widehat{\operatorname{Var}}(\widehat{\theta}^{W}) \approx \frac{\widehat{\operatorname{Var}}(\widehat{RF})}{\widehat{FS}^{\,2}} + \frac{\widehat{RF}^{\,2}\widehat{\operatorname{Var}}(\widehat{FS})}{\widehat{FS}^{\,4}} - \frac{2\widehat{RF}\widehat{\operatorname{Cov}}(\widehat{RF},\widehat{FS})}{\widehat{FS}^{\,3}}.

The covariance term should not be dropped unless the estimates are uncorrelated or the approximation deliberately assumes that. For comparing two ratios from the same sample, their covariance also matters. Writing $\widehat{\Delta}=\widehat{\theta}^{W}_A-\widehat{\theta}^{W}_B$,

\widehat{\operatorname{Var}}(\widehat{\Delta}) = \widehat{\operatorname{Var}}(\widehat{\theta}^{W}_A) + \widehat{\operatorname{Var}}(\widehat{\theta}^{W}_B) - 2\widehat{\operatorname{Cov}}(\widehat{\theta}^{W}_A,\widehat{\theta}^{W}_B).

A joint covariance estimate can be obtained by applying the delta method to the jointly estimated reduced forms and first stages, or by an appropriate resampling or robust covariance procedure. Treating the two ratios as independent simply because they use different instruments is generally unjustified: both estimates may be computed on the same people and share outcome and treatment variation.

Worked Example: Comparing Wald Estimates With Joint Uncertainty

Continue the hypothetical comparison above. Suppose a joint delta-method calculation, using the covariance of the reduced forms and first stages, gives standard errors $0.566$ for A and $0.500$ for B, and covariance $0.100$ between the two Wald estimates. Assume a large-sample normal approximation is reasonable for this illustration.

For reference, if A’s reduced-form and first-stage estimates have standard errors $0.08$ and $0.02$, respectively, and their covariance is zero, its delta-method standard error is $$ SE(\widehat{\theta}^{W}_A) \approx \sqrt{\left(\frac{0.08}{0.20}\right)^2+ \left(\frac{0.80(0.02)}{0.20^2}\right)^2} = \sqrt{0.16+0.16} = \sqrt{0.32} \approx 0.566. $$ Both squared contributions are $0.16$; the resulting standard error is about $0.566$, not $0.4$.

The estimated difference is $0.40$. Its variance and standard error are $$ \widehat{\operatorname{Var}}(\widehat{\Delta}) =0.566^2+0.500^2-2(0.100) \approx 0.370, \qquad SE(\widehat{\Delta})\approx\sqrt{0.370}=0.608. $$

The resulting normal statistic for testing equal Wald estimates is $0.40/0.608\approx0.658$, with a two-sided $p$-value of about $0.511$. A 95% normal-approximation interval for the difference is $$ 0.40\pm1.96(0.608)\approx[-0.792,\ 1.592] $$ score points per treatment. Under this approximation, the data do not provide strong evidence that the two ratios differ. The interval is compatible with either ordering and with equality; it does not establish that the instruments identify the same causal effect.

Why Valid Instruments Can Have Different Ratios

Under the binary-instrument assumptions, a Wald ratio averages treatment effects for people whose treatment is changed by that instrument. This is a local effect for that instrument’s complier group, rather than necessarily an average effect for the entire population. If effects vary across people, two instruments can have different valid Wald ratios because they induce treatment changes among different people.

Worked Example: Different Complier Groups, Different Effects

Imagine two valid encouragement instruments in a hypothetical population. There are two types of people with different treatment effects on an outcome measured in points: a lower-effect type with effect $2$, and a higher-effect type with effect $8$. Among the people induced to take treatment by A, suppose 80% are lower-effect types and 20% are higher-effect types. Among those induced by B, suppose 25% are lower-effect types and 75% are higher-effect types.

A’s complier average effect is $$ 0.80(2)+0.20(8)=1.60+1.60=3.20 $$ points. B’s is $$ 0.25(2)+0.75(8)=0.50+6.00=6.50 $$ points. If A’s first stage is $0.20$, its reduced form is $0.20(3.20)=0.64$ points and its Wald ratio is $0.64/0.20=3.20$. If B’s first stage is $0.12$, its reduced form is $0.12(6.50)=0.78$ points and its Wald ratio is $0.78/0.12=6.50$.

The instruments can both be valid and strongly relevant while producing distinct ratios, because their complier groups have different compositions. Thus, a difference in estimates is evidence to investigate, not by itself evidence that either instrument is invalid or that a calculation is wrong.

Reading a Difference Without Overclaiming

A difference between instrument-specific estimates is not a diagnostic with a unique explanation. It must be read alongside first-stage strength, uncertainty, and the substantive case for instrument validity. Weak relevance is especially important: when a first stage is close to zero, the ratio can be unstable and its usual normal approximation unreliable. A large or erratic ratio is not, by itself, evidence of a large causal effect.

Invalidity can also generate disagreement. If an instrument affects the outcome through a route other than treatment, its reduced form includes that additional pathway; dividing by the first stage does not remove it. For example, an encouragement that changes both treatment uptake and access to other care could have a reduced form partly driven by the other care. Different direct effects across instruments could therefore produce different Wald ratios even without complier-effect heterogeneity. Agreement is not proof of validity either: violations may happen to shift estimates similarly.

A comparison should therefore distinguish a point-estimate contrast from a causal conclusion. First report each instrument’s reduced form and first stage, with units and uncertainty. Then report the Wald ratios and their joint uncertainty, accounting for covariance. Finally, state the assumptions needed for the local causal interpretation and explain whether a difference is plausibly attributable to different complier populations, weak relevance, invalidity, or imprecision. Overidentification tests and weak-instrument diagnostics address related questions but do not make this interpretive work unnecessary.

Key takeaway: Instrument-specific Wald estimates are ratios of reduced forms to first stages. Their comparison is informative only when ratio uncertainty is handled jointly and the assumptions behind each instrument are kept explicit. Different valid instruments can identify different complier effects; different ratios alone do not identify the reason for disagreement.

Common Mistakes

Calling every ratio a population treatment effect. The causal interpretation is generally local to the instrument’s complier group under the required assumptions.
Comparing ratios as if they were independent. Estimates from the same sample can be correlated. Use their joint covariance when estimating the uncertainty of their difference.
Reading a different ratio as proof of invalidity. Valid instruments can shift treatment for different groups with different effects. Invalidity is another possible explanation, not a conclusion from disagreement alone.
Trusting a ratio despite a near-zero first stage. A weak denominator can make the ratio highly unstable and can undermine ordinary delta-method or normal approximations.
Using agreement as a validity certificate. Two instruments can yield similar estimates and still share a violation, or have offsetting violations.

Check Your Understanding

Use the definitions and comparison principles above to answer these questions.

An instrument has a reduced form of $0.45$ outcome points and a first stage of $0.15$. What is its Wald estimate, with units?
Why can two valid instruments have different Wald estimates when treatment effects vary across people?
When comparing two Wald estimates from the same sample, why can their covariance not generally be ignored?
In the delta-method standard error calculation, what happens to the ratio’s stability as the first stage approaches zero?
Name two explanations, other than treatment-effect heterogeneity, for a difference between instrument-specific ratios.