Why Outcome-Dependent Inclusion Matters
A study sample is not necessarily a miniature version of the population from which it came. In particular, researchers may deliberately include people with an observed outcome more often than people without it. Case-control sampling is a familiar example: cases are recruited at a higher rate than controls. Such a design can be useful, but the sample outcome frequency is then partly set by the recruitment procedure rather than by the population.
For causal analysis, this distinction matters because inclusion occurs after the outcome is observed. Comparing outcomes among included treated and control participants without accounting for the selection process can produce a misleading estimate, even when treatment was randomized in the underlying source population. To transport an effect, the analyst must first recover the relevant source-population outcome information and then combine it with information about the target population.
The inclusion probability may also vary by treatment group or covariates. Define it as \(\pi(a,y,x)=P(R=1\mid A=a,Y=y,X=x,\text{source population})\). If inclusion probabilities are known from the design, or can be estimated using an appropriate sampling frame, they can be used to undo the outcome-dependent distortion. If the relevant probabilities are unknown, the observed sample distribution alone will generally not reveal the source population’s absolute outcome risks.
How Selection Changes the Outcome Distribution
For fixed \(A=a\) and \(X=x\), people with outcome value \(y\) are represented in the sample in proportion to their source-population frequency multiplied by their chance of inclusion. In density notation, this relationship is
Thus, an outcome value with a higher inclusion probability appears more often in the sample than it would in the source population. The sample conditional distribution is not, in general, the source conditional distribution. The same point applies to binary outcomes: oversampling people with \(Y=1\) increases the observed sample risk.
Worked Example: Correcting Two Treatment-Group Risks
Consider a hypothetical randomized source cohort with 500 people in each treatment arm. Among the 500 assigned treatment, 50 have the outcome and 450 do not; among the 500 assigned control, 100 have the outcome and 400 do not. The true source risks are therefore \(50/500=0.10\) under treatment and \(100/500=0.20\) under control. Suppose the sampling design includes 80% of people with the outcome and 20% of people without it in each arm.
The selected treatment group contains \(0.80(50)=40\) people with the outcome and \(0.20(450)=90\) without it. Its sample risk is
The selected control group contains \(0.80(100)=80\) people with the outcome and \(0.20(400)=80\) without it, giving a sample risk of \(80/160=0.5000\). The sample risk difference, treatment minus control, is \(0.3077-0.5000=-0.1923\), which is substantially different from the source-population risk difference of \(0.10-0.20=-0.10\).
Weight each included person by the inverse of their inclusion probability. In the treatment arm, the weighted outcome count is \(40/0.80=50\), and the weighted non-outcome count is \(90/0.20=450\). The corrected risk is \(50/(50+450)=0.10\). In the control arm, the corresponding weighted counts are \(80/0.80=100\) and \(80/0.20=400\), giving a corrected risk of \(100/500=0.20\). The corrected source risk difference is \(0.10-0.20=-0.10\), or 10 percentage points lower risk under treatment in this hypothetical cohort.
The design probabilities are the same in both arms here, but the sample risks are still distorted because the outcome distribution differs between arms. Randomization in the underlying cohort does not undo selection that happens after outcomes are observed.
Recovering Source Conditional Means
When the outcome is numerical, or when a binary outcome is summarized by its mean risk, a normalized inverse-inclusion-weighted mean recovers the source conditional mean. Within a treatment and covariate stratum, the formula is
The denominator normalizes the weights within the stratum. This is important: the goal is to restore the source distribution of outcomes among people with \(A=a,X=x\), not merely to calculate an average in the selected sample. The formula requires positive inclusion probability for every outcome value that occurs in that source stratum. If some relevant outcome values could never be sampled, reweighting cannot reconstruct their contribution.
This argument also shows what information is needed. The analyst needs the relative inclusion probabilities across outcome values within the treatment and covariate strata used for the analysis. Absolute probabilities are useful for reconstructing totals, but the common scale cancels in a normalized conditional mean. In practice, probabilities may come from the sampling design or from data that include the numbers eligible and sampled in relevant strata. If selection also depends on other variables related to the outcome, those variables and the corresponding sampling information may need to be included as well.
From Corrected Source Outcomes to a Target Effect
Correcting the sample distribution is only one part of transport. Let \(Y(1)\) and \(Y(0)\) denote the potential outcomes under treatment and control. Suppose the goal is the target-population average causal effect on the mean-difference scale:
A standard identification strategy first estimates treatment-specific outcome means in the source population conditional on pre-treatment covariates, then averages those means over the target covariate distribution. It requires consistency, conditional treatment exchangeability in the source population, treatment positivity, conditional transportability of the potential-outcome means given \(X\), and target support for the covariate values being transported. In words, treatment assignment must be unconfounded given \(X\) in the source, both treatments must be possible in the relevant source strata, and the conditional potential-outcome means must agree between source and target populations at each relevant \(X\).
Under those conditions, and after correcting outcome-dependent inclusion, the target mean under treatment \(a\) is identified by
where \(F_{\text{target}}(x)\) is the target-population distribution of the measured covariates. Subtracting the two treatment-specific means gives the target average causal effect. The target covariate distribution is therefore needed even if the source outcome data have been corrected perfectly: outcome correction does not make the source covariate mix equal to the target mix.
Worked Example: Standardizing Corrected Risks to a Target Population
Suppose a hypothetical randomized source study has outcome-dependent sampling, but the inclusion probabilities are known and the source risks have been corrected within each covariate stratum. Let \(X\) identify two pre-treatment strata. Assume treatment is randomized within the source, the conditional potential-outcome means transport to the target, and both strata are represented in the source and target. The corrected source risks are as follows.
| Covariate stratum | Control risk | Treatment risk | Target proportion |
|---|---|---|---|
| Lower-risk stratum | 0.10 | 0.04 | 0.70 |
| Higher-risk stratum | 0.20 | 0.12 | 0.30 |
The target mean risk under treatment is the target-weighted average of the corrected conditional treatment risks:
The corresponding target mean risk under control is
Therefore, the target risk difference is \(0.064-0.130=-0.066\). Under the stated assumptions, treatment lowers the target-population risk by 6.6 percentage points on average. The calculation uses the target proportions, not the proportions in the selected study sample; using the latter would generally target a different population.
What Case-Control Data Do and Do Not Provide
Outcome-dependent sampling is often designed to make the sample contain a chosen number of cases and controls. The resulting proportion of cases is not an estimate of the source-population outcome prevalence. If the sampling fractions among cases and non-cases are known within the relevant strata, the source risks can be recovered by correcting for those fractions. If the fractions are unknown, the sample’s case-control composition does not by itself identify absolute source risks.
Worked Example: Why the Case Fraction Is Not a Population Risk
Suppose a hypothetical case-control sample contains 50 cases and 50 controls. Among cases, 30 are exposed and 20 are unexposed; among controls, 20 are exposed and 30 are unexposed. The sample odds ratio comparing exposure and outcome is
The sample has a 50% case fraction, but that does not establish a 50% source-population risk. For instance, if the source prevalence were 10%, a design could sample cases at probability 0.90 and controls at probability 0.10, producing nine times as high an inclusion probability for cases. A different source prevalence of 50% could be paired with equal case and control inclusion probabilities. Both designs can yield a sample with equal numbers of cases and controls, so the sample case fraction alone cannot distinguish these source prevalences.
Under outcome-only sampling, the case-control odds ratio can retain the source odds ratio when selection within outcome groups does not additionally depend on exposure. That fact does not supply the source prevalence or identify absolute risks. For a causal risk difference or risk ratio, the outcome-specific sampling fractions, or equivalent source information, are needed in addition to assumptions that identify the causal contrast.
Common Mistakes and Practical Checks
- Treating the selected sample risk as a population risk. If cases were recruited more often, the observed case fraction reflects both the source prevalence and the sampling design. Use the inclusion probabilities or sampling fractions to correct it.
- Assuming randomization protects against outcome-based selection. Randomization addresses confounding of treatment in the randomized population. It does not make the selected sample’s outcome distribution representative when inclusion depends on observed outcomes.
- Correcting outcomes but ignoring target composition. Corrected source conditional means still need to be averaged over the target covariate distribution for a target effect.
- Using weights without checking support. If a relevant treatment-covariate-outcome stratum has zero inclusion probability, the observed data contain no information from which weighting can recover that stratum’s outcomes.
- Calling a weighted estimate causal without stating the assumptions. Sampling correction addresses selection distortion; it does not by itself establish treatment exchangeability or transportability of conditional effects.
Check Your Understanding
Answer each question using the distinction between the selected sample, the source population, and the target population.
- Why can the observed case fraction in a case-control sample fail to estimate source-population prevalence?
- In the inverse-inclusion-weighted conditional mean, what part of the sample distribution is corrected by dividing by \(\pi(a,Y,x)\)?
- What sampling information is needed to recover absolute risks when inclusion depends on the observed binary outcome?
- Why does randomization in the source population not, by itself, correct outcome-dependent selection?
- After correcting the source conditional outcome means, what additional population information is needed to estimate a target average effect?