Covariate-Specific Limits On Selection Bias

Why Bound Selection Bias Separately?

An observed difference in mean outcomes between treated and untreated people need not equal the causal effect of treatment. Even after dividing the population into covariate strata, treatment may still be selected: within a stratum, people who receive treatment may differ from those who do not in their potential outcomes. A single bound on this discrepancy can be convenient, but it may be implausible if selection is expected to differ across strata.

Covariate-specific limits make that variation explicit. For example, a selection process may be thought to create only a small discrepancy among one group, but a larger or differently directed discrepancy among another. The resulting causal-effect interval depends on both the observed contrasts and the population proportions of the strata. The key is to combine intervals on a common target-population scale, rather than to average treatment-group summaries.

This definition uses selection bias in the sense of the discrepancy between the observed contrast and the stratum’s average causal effect. It is an exact decomposition, \(d_s=\tau_s+B_s\), not an assumption that selection bias has a particular sign. The bounds \([\ell_s,u_s]\) are assumptions or sensitivity restrictions; they are not learned merely by observing \(d_s\).

Let \(q_s=P(S=s)\) be the proportion of the target population in stratum \(s\), with \(\sum_s q_s=1\). The target average treatment effect is \(\tau=\sum_s q_s\tau_s\). These are population proportions for the target of inference. In general, they are not the proportions of strata among treated people or among untreated people.

From Stratum Restrictions to an Aggregate Interval

The decomposition gives an immediate interval for each stratum’s causal effect. Since \(\tau_s=d_s-B_s\), the largest permitted bias produces the smallest permitted effect, and the smallest permitted bias produces the largest permitted effect:

Weight these endpoints by the target-population stratum proportions and add them. Under the assumptions that the stratum restrictions are valid and there are no additional restrictions linking the possible biases across strata, the aggregate interval is:

The endpoints are sharp under those conditions: choosing every stratum’s bias at its upper endpoint attains the aggregate lower endpoint, and choosing every bias at its lower endpoint attains the upper endpoint. If the assumptions impose cross-stratum restrictions, those endpoint choices might not be jointly feasible; the formula then remains an outer bound, but need not be sharp.

Comparison With a Population-Wide Bias Limit

A population-wide restriction instead limits the aggregate discrepancy \(B=\sum_s q_sB_s\) directly, for example by assuming \(B\in[-\Delta,\Delta]\). The weighted observed contrast is \(d=\sum_s q_sd_s\), so the corresponding effect interval is \([d-\Delta,d+\Delta]\). That one restriction does not say how bias is distributed among strata.

Separate restrictions imply an aggregate restriction: if \(B_s\in[\ell_s,u_s]\), then \(B\in[\sum_s q_s\ell_s,\sum_s q_su_s]\). With symmetric stratum limits \([-\delta_s,\delta_s]\), this becomes \(B\in[-\sum_s q_s\delta_s,\sum_s q_s\delta_s]\). Therefore, if a single population-wide limit uses \(\Delta=\sum_s q_s\delta_s\), the two approaches give the same aggregate interval when the stratum restrictions are otherwise independent. The stratum-specific formulation is not automatically narrower just because it uses more than one limit.

It can be more informative when the credible stratum limits imply a smaller aggregate range than a separately chosen global limit, or when asymmetric stratum limits yield an asymmetric aggregate interval. Conversely, if one stratum has a wide permissible bias range and a substantial population weight, its uncertainty can dominate the result. A global limit may also be narrower than the weighted range implied by the separate limits. The comparison depends on the actual restrictions, not on the number of strata.

Conditions, Diagnostics, and Limitations

The calculation is simple, but its interpretation depends on substantive and data conditions. Potential outcomes and treatment must be well-defined, and the strata must be based on pre-treatment covariates if the target is to retain its usual causal interpretation. Consistency links the observed outcome to the potential outcome under the treatment received. No assumption of no unmeasured confounding is required for these bounds; residual confounding is represented by the bias limits.

To calculate \(d_s\) from observed data, both treated and untreated outcomes must be observed in each stratum of positive target-population weight. This is a within-stratum overlap requirement for the observed contrast. If a stratum has no treated or no untreated observations, its contrast is not directly estimable from the data, and the formula cannot supply a data-based interval for that stratum without an additional restriction. An empty or very small treatment group also makes the estimated contrast unstable.

The bias limits must be defensible on the outcome scale and for the target population. They may be informed by subject-matter knowledge or sensitivity analysis, but a narrow interval should not be treated as established merely because it produces a precise-looking answer. A useful diagnostic is to inspect each stratum’s observed contrast, population weight, bias interval, and resulting effect interval. This reveals whether a low-weight stratum has an unexpectedly large influence, whether one broad bias restriction drives the aggregate width, and whether the assumed limits have a plausible direction and scale.

These are identification bounds given the restrictions, not confidence intervals for estimated endpoints. In an application, the observed contrasts and population weights are estimated from data and have sampling uncertainty as well. A reported interval that ignores this uncertainty describes the plug-in calculation, not the full inferential uncertainty. Also, the bounds can be uninformative: if the bias limits are wide, the resulting effect interval may include meaningfully different effects, including effects of opposite signs.

Common Pitfall

A frequent error is to average stratum-specific treatment effects using the proportions of treated people in each stratum. That weighting targets the effect for a treatment-defined population, not generally the average effect in the full target population. For the target ATE defined here, use \(q_s=P(S=s)\). Another error is to assume that separate limits must tighten the result. They tighten the aggregate interval only relative to a global restriction that permits a wider aggregate bias range; they do not do so automatically.