Why Selection Into a Sample Matters
An observational analysis usually concerns a target population, but its data come from people who entered a study, remained under follow-up, or had all required records available. Those steps may be related to exposure, outcome, or their causes. Consequently, the exposure–outcome association in the analyzed sample need not represent the association in the target population. In some situations, selection can create an association even when none exists in the target population.
Causal diagrams help identify one important mechanism: conditioning on a common effect of two variables can make those variables associated. This can happen when analysts restrict an analysis to people who were included, retained, or complete. The diagram indicates a possible route to bias; numerical information and substantive knowledge are needed to determine its size and direction.
Reading Selection in a Causal Diagram
In a directed acyclic graph, an arrow represents a direct causal influence as specified by the proposed causal model. A collider is a variable with two arrows pointing into it. For example, \(X \rightarrow S \leftarrow U\) makes \(S\) a collider of \(X\) and \(U\). If \(S\) indicates study inclusion, restricting analysis to \(S=1\) conditions on that collider.
Before conditioning on \(S\), the path \(X \rightarrow S \leftarrow U\) is blocked at the collider. Conditioning on \(S\) can open that path, making \(X\) and \(U\) associated among included participants even if they were independent in the target population. If \(U\) also affects the outcome, the induced association between \(X\) and \(U\) can carry over to an exposure–outcome association.
This is a structural warning, not an automatic numerical conclusion. A diagram does not specify the strengths of causal influences or the probabilities of selection. In addition, conditioning on a selection variable may change an association through mechanisms other than opening a collider path. The examples below supply explicit probabilities so that these consequences can be calculated.
Worked Selection Examples
Worked Example: Study Inclusion Creates an Association
Suppose a hypothetical target population has binary exposure \(X\) and binary prognostic factor \(U\), each with probability \(0.5\), and suppose \(X\) and \(U\) are independent. Let the binary outcome be \(Y=U\): the outcome is determined by \(U\), with no causal effect of \(X\). A study includes a person if \(X=1\) or \(U=1\). What exposure–outcome association appears among included people?
The four combinations of \(X\) and \(U\) each have probability \(0.25\). Inclusion excludes only the combination \(X=0,U=0\), so the inclusion probability is \(0.75\). Each of the three included combinations therefore has conditional probability \(0.25/0.75=1/3\).
Among included people with \(X=1\), the equally likely combinations are \((X,U)=(1,0)\) and \((1,1)\), so half have \(Y=1\). Among included people with \(X=0\), inclusion requires \(U=1\), and thus everyone has \(Y=1\). The selected-sample risk difference comparing \(X=1\) with \(X=0\) is \(0.5-1=-0.5\).
In the target population, \(X\) is independent of \(U\), so the outcome risk is \(0.5\) at either exposure level and the risk difference is \(0\). The apparent protective association in the sample is induced by selection. The diagram is \(X \rightarrow S \leftarrow U \rightarrow Y\): inclusion is a collider, and conditioning on inclusion connects exposure with the outcome’s cause.
Worked Example: Follow-Up Can Connect Exposure and Outcome
In a second hypothetical population, binary exposure \(X\) and binary outcome \(Y\) are independent, each with probability \(0.5\). Let \(R=1\) mean that a person is retained through follow-up. Suppose retention probability is \(0.8\) when \(X=Y\) and \(0.2\) when \(X\ne Y\). What association is seen among retained people?
Each of the four exposure–outcome combinations has source-population probability \(0.25\). The probability of a combination and retention is its source probability multiplied by its retention probability. Thus, the joint probabilities for retained people are \(0.25(0.8)=0.20\) for each matching combination, and \(0.25(0.2)=0.05\) for each mismatching combination. The total retention probability is \(0.20+0.20+0.05+0.05=0.50\).
Among retained people with \(X=1\), the joint retained probabilities for \(Y=1\) and \(Y=0\) are \(0.20\) and \(0.05\), respectively. Their total is \(0.25\). Among retained people with \(X=0\), the corresponding probabilities for \(Y=1\) and \(Y=0\) are \(0.05\) and \(0.20\), also totaling \(0.25\). Therefore:
The retained-sample risk difference is \(0.80-0.20=0.60\), although the source-population risk at each exposure level is \(0.50\), giving a source-population risk difference of \(0\). A diagram with \(X \rightarrow R \leftarrow Y\) represents the two causes of retention. Conditioning on \(R=1\) has produced an association between variables that were independent in the source population. The retention probabilities in this example are deliberately specified to make the mechanism and arithmetic visible; they are not claims about typical follow-up.
Worked Example: Complete Records Distort a Risk Difference
Suppose, hypothetically, exposure \(X\) is assigned at random in a target population. The outcome risk is \(0.20\) for \(X=0\) and \(0.80\) for \(X=1\). Thus, the target-population risk difference comparing \(X=1\) with \(X=0\) is \(0.80-0.20=0.60\). Let \(C=1\) mean that all variables required for analysis are recorded. Assume a complete record is available with probability \(0.90\) when \(X=Y\), and with probability \(0.10\) when \(X\ne Y\). Calculate the risk difference among complete records.
With random assignment at probability \(0.5\), the target-population probabilities of \((X,Y)=(0,1),(0,0),(1,1),(1,0)\) are \(0.5(0.20)=0.10\), \(0.5(0.80)=0.40\), \(0.5(0.80)=0.40\), and \(0.5(0.20)=0.10\). Multiplying each by its completion probability gives the joint probabilities of the combination and \(C=1\):
- \((0,1)\): \(0.10(0.10)=0.01\).
- \((0,0)\): \(0.40(0.90)=0.36\).
- \((1,1)\): \(0.40(0.90)=0.36\).
- \((1,0)\): \(0.10(0.10)=0.01\).
For each exposure level, the probability of a complete record is \(0.37\). Among complete records at \(X=0\), the outcome risk is \(0.01/0.37\approx0.0270\). At \(X=1\), it is \(0.36/0.37\approx0.973\). The complete-record risk difference is therefore \(0.973-0.0270\approx0.946\), appreciably larger than the target-population difference of \(0.60\).
In this hypothetical mechanism, both \(X\) and \(Y\) affect complete-record status, represented by \(X \rightarrow C \leftarrow Y\), as well as the causal path \(X \rightarrow Y\). Restricting the analysis to \(C=1\) changes the exposure-specific outcome risks. A complete-case result need not represent the target-population contrast simply because both exposure groups have complete records.
A Short Justification for Collider-Induced Association
The arithmetic in the first example illustrates a general probability fact. Suppose \(X\) and \(U\) are independent before selection, and let \(S\) indicate selection. Among selected people, their joint distribution is obtained by weighting each source-population combination by its probability of selection:
If selection probabilities do not factor into an \(x\)-only term times a \(u\)-only term, the weighting can disrupt the factorization of the independent source variables. In the inclusion example, \(P(S=1\mid X=0,U=0)=0\), while selection probability is \(1\) for the other combinations. After selection, the combination \((0,0)\) is impossible, so \(X\) and \(U\) cannot be independent. Since \(Y=U\), this dependence appears as an exposure–outcome association.
This argument explains why conditioning on a collider can open a path; it does not say that every selection mechanism produces bias, nor that a particular direction of bias follows from a diagram alone. The selection probabilities, the outcome process, and the estimand all matter.
How to Assess a Selection Concern
A useful assessment starts by specifying the target population and the association or causal contrast of interest. Then identify the exact restriction defining the analyzed sample: study inclusion, completion of follow-up, or availability of all analysis variables. Draw a plausible diagram that includes the selection indicator and ask what causes it. If exposure and outcome, or their causes, both influence selection, conditioning on selection may alter the association.
Name the population to which the result is intended to apply and the exposure–outcome quantity being compared.
Specify what \(S=1\), \(R=1\), or \(C=1\) means in the data process. Distinguish people who were never recruited from those later lost or excluded for incomplete records.
Include exposure, outcome, and plausible common causes that affect whether a person is included, retained, or complete.
Look for a collider made relevant by conditioning on selection. Describe the possible distortion without claiming a direction or magnitude that the available probabilities do not establish.
Selection bias is not repaired automatically by fitting a regression model or adjusting for a customary covariate set. Whether adjustment addresses a particular selection mechanism depends on the causal structure, the variables observed, and the target estimand. A causal diagram helps diagnose the issue, but it cannot establish that an assumed structure is correct or recover information about excluded people by itself.
Common Mistake
Wrong reasoning: “The study has many participants, and the regression adjusts for measured confounders, so selection into the analyzed sample cannot matter.” A large sample may reduce sampling variability, but it does not undo a systematically altered sample. Nor does adjustment for confounders necessarily block a path opened by conditioning on selection.
Correction: Examine how entering the sample, remaining under follow-up, or having complete records relates to exposure, outcome, and their causes. Compare the selected-sample contrast with the stated target, and use a causal diagram to show plausible routes of distortion. Treat the diagram as a guide to assumptions, not as proof that bias exists or a calculation of its size.
Check Your Understanding
Answer each question using the distinction between the target population and the selected sample.
- In a diagram \(X \rightarrow S \leftarrow U \rightarrow Y\), which variable is the collider, and what can conditioning on \(S=1\) do to the path from \(X\) to \(Y\)?
- Can selection create an exposure–outcome association when exposure and outcome are independent in the target population? Explain the role of the selection mechanism.
- Why does a causal diagram identify a possible selection-bias mechanism but not determine the numerical size or direction of bias?
- In the follow-up example, what were the source-population and retained-sample risk differences?
- Does the fact that both exposure groups have complete records guarantee that a complete-case estimate represents the target population? Explain.