Relevance as a First-Stage Contrast

Why the First Stage Matters

An instrumental-variable analysis begins with a practical question: does the proposed instrument actually shift the probability that people receive the treatment? For example, an encouragement may increase enrollment in a program without making enrollment certain. Comparing treatment receipt across encouragement groups measures that shift. It is called the first-stage contrast because it describes the instrument–treatment relationship, before considering how the instrument relates to an outcome.

This contrast is a specific, measurable feature of an instrument. A difference in treatment-receipt probabilities supports relevance, but it does not by itself show that the instrument is independent of potential outcomes or that it affects the outcome only through treatment. Relevance is one requirement to examine, not a complete argument for causal interpretation.

Definition: Let $Z$ be a binary instrument, with levels 0 and 1, and $D$ a binary treatment indicator, where $D=1$ denotes treatment receipt. The population first-stage contrast is $$ \Delta_D = P(D=1\mid Z=1)-P(D=1\mid Z=0). $$ The instrument is relevant for this treatment if $\Delta_D\ne 0$: treatment-receipt probabilities differ across instrument levels. A positive contrast means receipt is more probable at $Z=1$; a negative contrast means it is less probable.

What the Contrast Measures

Each term in $\Delta_D$ is a probability of treatment receipt within one instrument group. Their difference is on the probability scale. For instance, a contrast of $0.20$ means that the treatment-receipt probability is 20 percentage points higher in the $Z=1$ group than in the $Z=0$ group. It does not mean that 20 percent more people received treatment in a relative sense; a relative percentage change would use a different calculation.

With binary $D$, a probability of receipt is also the mean of the treatment indicator in that group. This follows directly from the two possible values of $D$:

E[D\mid Z=z] =1\cdot P(D=1\mid Z=z)+0\cdot P(D=0\mid Z=z) =P(D=1\mid Z=z).

Consequently, the same contrast can be written as $E[D\mid Z=1]-E[D\mid Z=0]$. In a sample, the corresponding estimate is the difference between the two observed treatment proportions:

\widehat{\Delta}_D = \frac{\text{number with }D=1\text{ among }Z=1} {\text{number with }Z=1} - \frac{\text{number with }D=1\text{ among }Z=0} {\text{number with }Z=0}.

The population definition concerns probabilities; the sample calculation is an estimate based on observed data. A sample contrast is not guaranteed to equal the population contrast. In particular, a nonzero estimate alone does not establish that the population contrast is nonzero.

Worked Example: Randomized Encouragement

Suppose, hypothetically, 250 people are randomly assigned either an encouragement to attend a health program ($Z=1$) or no encouragement ($Z=0$). Of the 120 assigned encouragement, 78 attend, so $D=1$ for 78 people. Of the 130 assigned no encouragement, 39 attend. Calculate the sample first-stage contrast.

The treatment-receipt proportion in the encouragement group is

\hat p_1=\frac{78}{120}=0.650.

In the no-encouragement group it is

\hat p_0=\frac{39}{130}=0.300.

Subtract the $Z=0$ proportion from the $Z=1$ proportion:

\widehat{\Delta}_D=0.650-0.300=0.350.

In this hypothetical sample, attendance was 35 percentage points more common among those assigned encouragement. The positive sign gives the direction of the contrast. Because assignment was randomized in this example, this comparison can describe the effect of assignment on treatment receipt under the randomization design. It does not, by itself, establish the other conditions needed to interpret an instrumental-variable analysis.

Calculating and Interpreting the Contrast

The calculation is simple, but its interpretation depends on using the correct groups and keeping the subtraction order clear. For binary $Z$ and $D$, proceed in three steps: find the fraction treated at $Z=1$, find the fraction treated at $Z=0$, and subtract the second fraction from the first. The denominators are the numbers of people in the corresponding instrument groups, not the total sample size.

Identify the coding.
Confirm which instrument level is $Z=1$, which is $Z=0$, and that $D=1$ means treatment receipt.

Calculate each group proportion.
Within each instrument group, divide the number receiving treatment by the number in that group.

Subtract in the stated order.
Compute the $Z=1$ proportion minus the $Z=0$ proportion, then report the result in context and in percentage points.

Reversing the subtraction changes the sign but not whether the contrast is zero. Therefore, always state the order and the coding. A positive number is meaningful only alongside the statement that $Z=1$ has the higher treatment-receipt probability.

Worked Example: Access to a Treatment

Imagine a hypothetical health system where some patients live in areas with convenient access to a specialized clinic ($Z=1$) and others live in areas without convenient access ($Z=0$). Among 200 patients with convenient access, 110 receive the treatment. Among 180 patients without convenient access, 72 receive it. Compute the observed first-stage contrast.

For convenient access, the treatment proportion is

\hat p_1=\frac{110}{200}=0.550.

For no convenient access, it is

\hat p_0=\frac{72}{180}=0.400.

Thus,

\widehat{\Delta}_D=0.550-0.400=0.150.

The observed treatment-receipt probability is 15 percentage points higher in the convenient-access group. This is an observed contrast in the hypothetical data. Because access was not described as randomly assigned, the arithmetic alone does not show that access caused the difference: the groups may differ in other ways. The calculation establishes the size and direction of the observed first-stage contrast, not the validity of access as an instrument.

Nonzero Is Not the Same as Strong

The formal relevance condition is whether the population contrast is nonzero. A contrast close to zero indicates that the instrument produces little change in treatment receipt, even if the exact value is not zero. In practical analyses, the magnitude and uncertainty of the estimated first stage matter: a small estimated difference may provide little information about treatment effects and may be hard to distinguish from sampling variation.

There is no universal numerical cutoff in this definition that turns an instrument from irrelevant to relevant or from weak to strong. The contrast should be reported with its group-specific proportions and, when making an inferential claim, appropriate uncertainty information. The core definition remains a population condition, $\Delta_D\ne 0$; a particular sample estimate is evidence about that condition, not a replacement for it.

Worked Example: A Small Positive Contrast

In another hypothetical sample, 80 of 160 people with $Z=1$ receive treatment, while 56 of 140 people with $Z=0$ receive treatment. Calculate and interpret the observed contrast.

The proportions are

\hat p_1=\frac{80}{160}=0.500,\qquad \hat p_0=\frac{56}{140}=0.400.

Their difference is

\widehat{\Delta}_D=0.500-0.400=0.100.

Treatment receipt is 10 percentage points more common at $Z=1$ in this sample. The estimate is positive and nonzero, but it is smaller than the contrast in the encouragement example. It would be incorrect to conclude from the value $0.100$ alone that the instrument is sufficiently informative for every analysis: that judgment also depends on sampling uncertainty and the analytic setting. Nor does the positive sign say anything by itself about whether $Z$ meets other instrumental-variable requirements.

Why Relevance Is Only One Requirement

A first-stage contrast answers a limited question: are treatment-receipt probabilities different across instrument levels? An instrument can satisfy this condition and still be unsuitable for causal analysis if other requirements fail. For example, a variable might predict treatment because it is associated with factors that also affect the outcome, or it might affect the outcome through a route other than the treatment. Those concerns are not resolved by a large or precisely estimated first stage.

The distinction is especially important when comparing randomized encouragement with observational access. Random assignment of an encouragement can support a causal interpretation of the encouragement–receipt contrast, subject to the design being carried out as described. An observed access contrast is still calculable, but it may reflect differences between people in the two access groups. In either setting, the first-stage calculation itself does not establish how the instrument was assigned or whether it has a direct effect on the outcome.

Key takeaway: For binary instrument $Z$ and binary treatment $D$, relevance is expressed by the difference $P(D=1\mid Z=1)-P(D=1\mid Z=0)$. Compute the two within-group treatment proportions, subtract in the stated order, and interpret the result in percentage points. A nonzero first stage is necessary for an instrumental-variable analysis to use this source of treatment variation, but it is not sufficient for causal validity.

Common Mistakes

Using the wrong denominators. Dividing the number treated in a group by the total sample does not give that group’s treatment-receipt probability. Use the total number at that instrument level.
Reporting only a difference without its direction. A result of $0.15$ should be described as a 15-percentage-point higher receipt probability at $Z=1$ than at $Z=0$, given the defined subtraction order.
Calling a 0.15 contrast a 15 percent increase. It is a 15-percentage-point difference. A relative increase would compare the difference with a baseline proportion and answer a different question.
Treating a nonzero sample estimate as proof of population relevance. Sampling variation can produce a nonzero estimate even when the population contrast is zero. The estimate is evidence, and its uncertainty matters.
Equating relevance with instrument validity. A contrast shows that instrument levels are associated with different treatment-receipt probabilities. It does not establish that the instrument meets the other assumptions required for causal interpretation.

Check Your Understanding

Use the binary-instrument contrast and distinguish its calculation from its causal interpretation.

In a hypothetical sample, 45 of 90 people at $Z=1$ receive treatment and 24 of 80 people at $Z=0$ receive treatment. Calculate the first-stage contrast and interpret its sign and size in percentage points.
Why are the denominators in the two treatment proportions different in general?
A contrast is reported as $-0.08$, with the definition $P(D=1\mid Z=1)-P(D=1\mid Z=0)$. Which instrument level has the higher treatment-receipt probability, and by how many percentage points?
What is the distinction between a nonzero sample contrast and the population relevance condition?
Suppose access to a clinic is associated with treatment receipt in observed data. Why does that contrast alone not establish that clinic access is a valid instrument?