Bounding Average Selection Bias

Why Bound Selection Bias?

A difference in observed mean outcomes between treated and untreated groups combines treatment effects with differences in who receives treatment. The untreated potential outcomes of treated people are not observed, so their mean cannot generally be compared directly with the untreated potential-outcome mean of untreated people. A restriction on the size of that difference can nevertheless narrow the range of causal effects consistent with the data.

The restriction alone does not identify the average treatment effect (ATE). To bound the ATE here, we also require a known range for the outcome. That range limits the unobserved treated potential outcomes among untreated people. The resulting interval is a partial-identification result: it describes values consistent with the observations and stated restrictions, rather than selecting one value as the answer.

Definition: Let $D\in\{0,1\}$ indicate treatment, with $D=1$ for treated individuals. Let $Y(1)$ and $Y(0)$ denote the potential outcomes under treatment and no treatment, respectively, and let $Y=D Y(1)+(1-D)Y(0)$ be the observed outcome. The mean selection bias in untreated potential outcomes is $$ \delta_0=E[Y(0)\mid D=1]-E[Y(0)\mid D=0]. $$ A symmetric bias limit $\Gamma\geq 0$ imposes $|\delta_0|\leq\Gamma$. A positive $\delta_0$ means that, even without treatment, the treated group would have had a higher mean outcome than the untreated group.

Assumptions and the Bound

Write $p=P(D=1)$, $m_1=E[Y\mid D=1]$, and $m_0=E[Y\mid D=0]$. Consistency gives $m_1=E[Y(1)\mid D=1]$ and $m_0=E[Y(0)\mid D=0]$. Suppose the outcome is known to lie in $[L,U]$ for everyone under either treatment state. In addition, assume the sensitivity restriction $|\delta_0|\leq\Gamma$. These are restrictions on potential-outcome means and support; they do not require treatment assignment to be random or independent of potential outcomes.

The bias restriction locates the missing mean $q=E[Y(0)\mid D=1]$: it must be within $\Gamma$ of $m_0$. The outcome support also requires $q\in[L,U]$. Thus the feasible interval for $q$ is

q_{\min}=\max(L,m_0-\Gamma),\qquad q_{\max}=\min(U,m_0+\Gamma).

The other missing mean for the ATE is $r=E[Y(1)\mid D=0]$. The outcome support implies $L\leq r\leq U$. By partitioning the population according to treatment status,

\begin{aligned} ATE &=E[Y(1)-Y(0)]\\ &=p(m_1-q)+(1-p)(r-m_0). \end{aligned}

The ATE decreases as $q$ increases and increases as $r$ increases. Therefore, its smallest feasible value uses $q=q_{\max}$ and $r=L$, while its largest uses $q=q_{\min}$ and $r=U$.

Theorem: If $q_{\min}\leq q_{\max}$, the ATE is bounded by $$ p(m_1-q_{\max})+(1-p)(L-m_0) \ \leq\ ATE\ \leq\ p(m_1-q_{\min})+(1-p)(U-m_0). $$ Under the stated support and mean restrictions alone, these endpoints are sharp: each endpoint can be attained by setting the relevant missing group means to their feasible extremes. The result does not assume the bias restriction is true; it describes what follows if that restriction is imposed.

Derivation and Interpretation

The derivation follows directly from the population decomposition. In the treated group, the observed mean under treatment is $m_1$, while the unobserved mean under no treatment is $q$; that group’s mean treatment effect is $m_1-q$. In the untreated group, the observed mean under no treatment is $m_0$, while the unobserved mean under treatment is $r$; that group’s mean treatment effect is $r-m_0$. Weighting these two group-specific effects by their population shares gives the ATE.

For the lower endpoint, the smallest treated-group effect uses the largest allowable $q$, and the smallest untreated-group effect uses the lowest allowable $r$. For the upper endpoint, the choices reverse. Because these are restrictions on separate group means and impose no further relationship between $Y(1)$ and $Y(0)$, those endpoint choices can be attained together. For example, potential outcomes in a group can be set to constants at the chosen means; those constants lie within $[L,U]$. This justifies sharpness under these assumptions.

If $q_{\min}>q_{\max}$, the specified data and restrictions are incompatible: no mean $q$ can satisfy both the support and bias limit. That is not evidence for a zero-width causal effect. It signals that at least one input, such as the proposed bias limit or support, cannot hold alongside the observed mean $m_0$.

Worked Example: Bounded Continuous Outcome

Suppose, in a hypothetical population, $p=0.40$, the observed treated mean is $m_1=8$ points, and the observed untreated mean is $m_0=5$ points. The outcome is known to lie between $L=0$ and $U=10$ points, and the assumed bias limit is $\Gamma=1.5$ points. Check the range for the missing untreated mean among treated people:

$q_{\min}=\max(0,5-1.5)=3.5$ points and $q_{\max}=\min(10,5+1.5)=6.5$ points. The range is nonempty and satisfies the support restriction.

For the lower endpoint, use $q_{\max}=6.5$ and $r=L=0$:

ATE_{\min}=0.40(8-6.5)+0.60(0-5) =0.60-3.00=-2.40\text{ points}.

For the upper endpoint, use $q_{\min}=3.5$ and $r=U=10$:

ATE_{\max}=0.40(8-3.5)+0.60(10-5) =1.80+3.00=4.80\text{ points}.

Under these assumptions, the ATE lies between $-2.40$ and $4.80$ points. The observed mean difference is $8-5=3$ points, but that difference is not itself the ATE without further assumptions. In particular, the interval includes zero, so these bounds do not establish that the population average treatment effect is positive.

Do Not Confuse ATE Bounds With ATT Bounds

The average treatment effect on the treated is $ATT=E[Y(1)-Y(0)\mid D=1]=m_1-q$. Since the observed difference $m_1-m_0$ equals $ATT+\delta_0$, the bias limit also gives

m_1-m_0-\Gamma\ \leq\ ATT\ \leq\ m_1-m_0+\Gamma,

with endpoints tightened if needed by requiring $q\in[L,U]$. This calculation concerns the treated group, not the whole population. For the ATE, the untreated group’s missing treated-outcome mean $r$ also matters, which is why the outcome support enters the ATE formula. Treating the ATT interval as an ATE interval silently assumes away the difference between treatment effects in the treated and untreated groups.

Worked Example: Binary Outcome

In a hypothetical study, let $Y$ be a binary outcome, so its known support is $[0,1]$. Suppose $p=0.30$, $m_1=0.70$, $m_0=0.20$, and the assumed bias limit is $\Gamma=0.10$. Then

$q_{\min}=\max(0,0.20-0.10)=0.10$, and $q_{\max}=\min(1,0.20+0.10)=0.30$.

The lower bound uses $q_{\max}=0.30$ and $r=0$:

ATE_{\min}=0.30(0.70-0.30)+0.70(0-0.20) =0.12-0.14=-0.02.

The upper bound uses $q_{\min}=0.10$ and $r=1$:

ATE_{\max}=0.30(0.70-0.10)+0.70(1-0.20) =0.18+0.56=0.74.

The ATE is bounded by $-0.02$ and $0.74$, in probability points on the zero-to-one outcome scale. These endpoints are not probabilities of treatment benefit; they are bounds on the population mean difference in potential outcomes. Since the interval includes zero, the restrictions do not determine the sign of the ATE.

Diagnostics, Comparisons, and Limitations

Before interpreting an interval, check that the inputs are on the same scale and in the same units. The outcome support, observed means, and bias limit must all use the same outcome units. For a binary outcome on the zero-to-one scale, for example, a bias limit of $0.10$ is ten percentage points; it is not $0.10$ percentage points. Check also that the observed means satisfy the support, $L\leq m_0,m_1\leq U$, and that $q_{\min}\leq q_{\max}$.

The bound is more informative when the bias limit is smaller, but a small $\Gamma$ is an assumption, not a consequence of the observed data. If $\Gamma$ is so large that $q_{\min}=L$ and $q_{\max}=U$, the bias restriction adds no restriction beyond the outcome support on $q$. The resulting interval may still reflect the observed means and group proportions, but it can be wide. A bound that excludes zero is informative only conditional on the support and bias restriction being credible.

This approach differs from assuming no selection on untreated potential outcomes, which sets $\delta_0=0$. That stronger restriction fixes $q=m_0$, but it still does not determine $r$; the ATE generally remains bounded rather than point-identified unless further information or assumptions are available. It also differs from using only the known outcome support: the bias limit can narrow the feasible range for $q$. The earlier material on worst-case bounds for bounded outcomes develops support-based restrictions; here the added restriction concerns a particular difference in untreated potential-outcome means.

The limitation is substantive: a bound on selection in $Y(0)$ does not restrict how treatment effects vary across treatment groups. Nor does it identify the unobserved $Y(1)$ mean among untreated individuals more tightly than the outcome range does. Narrower bounds may require additional credible information or assumptions, but they should not be introduced implicitly. In particular, neither a favorable observed difference nor a narrow assumed bias limit proves that treatment assignment is unconfounded.

Worked Example: A Wide Bias Limit

Consider hypothetical outcome scores on $[0,10]$, with $p=0.50$, $m_1=6$, $m_0=4$, and $\Gamma=8$. The support clips the feasible untreated mean among treated people:

$q_{\min}=\max(0,4-8)=0$ and $q_{\max}=\min(10,4+8)=10$. The bias limit therefore does not narrow $q$ beyond its support.

The lower endpoint is

ATE_{\min}=0.50(6-10)+0.50(0-4) =-2-2=-4\text{ points}.

The upper endpoint is

ATE_{\max}=0.50(6-0)+0.50(10-4) =3+3=6\text{ points}.

Thus the ATE lies between $-4$ and $6$ points under these restrictions. The example illustrates a diagnostic, not a recommendation for a particular bias limit: when clipping reaches both support endpoints, the chosen $\Gamma$ contributes no additional information about $q$.

Common Pitfall

Wrong reasoning: “The observed treated-minus-untreated difference is the ATE, with at most $\Gamma$ of selection bias, so subtracting or adding $\Gamma$ gives ATE bounds.”
Correction: The direct identity $m_1-m_0=ATT+\delta_0$ concerns the ATT. For the ATE, decompose by treatment group and account for both missing means: $q=E[Y(0)\mid D=1]$ and $r=E[Y(1)\mid D=0]$. The bias limit constrains $q$, while the outcome support constrains $r$. Use the ATE formula and bounds derived above.

Key takeaway: A symmetric limit on mean selection bias in untreated potential outcomes restricts $q$, but does not by itself identify the ATE. Combining that restriction with a known outcome support yields sharp ATE bounds under the stated assumptions. Always report the outcome scale, the assumed bias limit, and the resulting interval together.

Check Your Understanding

Answer each question, then compare with the brief solutions below.

In this tutorial, what does a positive value of $\delta_0$ mean?
Suppose $m_0=3$, $[L,U]=[0,8]$, and $\Gamma=2$. What is the feasible interval for $q$?
Why is $m_1-m_0$ not generally the ATE under a bound on $\delta_0$?
For $p=0.25$, $m_1=6$, $m_0=4$, $[L,U]=[0,10]$, and $\Gamma=1$, calculate the ATE bounds.
What does it mean if the proposed bias limit and support produce $q_{\min}>q_{\max}$?

Solutions. (1) The treated group would have a higher mean untreated potential outcome than the untreated group. (2) $q_{\min}=\max(0,3-2)=1$ and $q_{\max}=\min(8,3+2)=5$, so $q\in[1,5]$. (3) The observed difference is $ATT+\delta_0$, not generally the ATE; the ATE also depends on the unobserved treated-outcome mean among untreated people. (4) Here $q_{\min}=3$ and $q_{\max}=5$. The lower bound is $0.25(6-5)+0.75(0-4)=0.25-3=-2.75$. The upper bound is $0.25(6-3)+0.75(10-4)=0.75+4.50=5.25$. Thus the ATE lies in $[-2.75,5.25]$ outcome units under the stated restrictions. (5) The inputs are incompatible: there is no $q$ satisfying both the support and the bias limit.