Why Draw a Selection Diagram?
A treatment effect estimated in one trial population does not automatically describe the effect in another. The populations may differ in baseline characteristics, treatment assignment, or the mechanisms that generate outcomes. A selection diagram makes these possible differences explicit: it marks which parts of the data-generating process may change between the source population, where evidence is available, and the target population, where the effect is needed.
The diagram is a structured statement of scientific assumptions, not a test that proves those assumptions. Its practical value is to expose what must be measured, what may be transportable as observed, and where the evidence may be insufficient. The intended treatment contrast and outcome scale must be specified first; a covariate can matter for one contrast but not another.
Ordinary arrows describe causal or predictive parent relationships within the system, such as \(A\to Y\) for treatment \(A\) affecting outcome \(Y\), or \(X\to Y\) for baseline covariate \(X\) contributing to the outcome. Selection arrows describe possible cross-domain changes in the mechanism for a variable. For example, \(D\rightsquigarrow X\) can represent a change in the distribution of a baseline characteristic, while \(D\rightsquigarrow Y\) flags a possible change in the outcome-generating mechanism.
A selection arrow does not by itself say that the treatment effect differs. The diagram must be interpreted in relation to the causal contrast, relevant covariates, and the assumptions used to connect the domains. In particular, a graph's ordinary arrows alone do not encode every interaction or treatment-effect modification. Those features may need to be stated as part of the outcome model.
Separating Three Kinds of Population Difference
For a treatment effect, it is useful to distinguish three locations where domains can differ. First, the distribution of baseline characteristics may change. Second, the treatment-assignment process may change, for example because routine care and a randomized trial allocate treatments differently. Third, the outcome-generating mechanism may change, for example because clinical practice, measurement, or background care differs.
| Selection arrow | What it flags | Implication for a treatment-effect bridge |
|---|---|---|
| \(D\rightsquigarrow X\) | The distribution or mechanism for baseline \(X\) may differ. | If \(X\) modifies the treatment effect on the chosen scale, its distribution matters; measurements relevant to the bridge are needed. |
| \(D\rightsquigarrow A\) | The treatment-assignment mechanism may differ. | Observed treatment groups may be formed differently. This is not, by itself, evidence that an effect under an intervention changes across domains. |
| \(D\rightsquigarrow Y\) | The outcome mechanism may differ beyond what the displayed common mechanisms explain. | A source outcome model may not carry over automatically. The reason for the shift and the evidence available in the target need scrutiny. |
The distinction between assignment and outcome is important. The question is usually about the effect of setting treatment to a specified value, not about the effect of the treatment-assignment policy. A change in assignment can complicate what the observed source data identify, but it need not change the causal effect of the same treatment intervention. Conversely, an outcome-mechanism shift may matter even if the treatment is assigned identically in both domains.
From a Diagram to Measurement Requirements
Start with the source and target definitions and the already aligned estimand. Then draw the ordinary relationships relevant to treatment and outcome, and add a selection arrow wherever scientific knowledge suggests that a generating mechanism may differ. Finally, ask which changed variables can affect the treatment contrast on the specified scale. A variable that differs in prevalence but is purely prognostic may affect an average outcome without changing a relative or absolute treatment contrast in the same way that an effect modifier would.
For a discrete baseline covariate \(X\), suppose the target effect is an average contrast in potential outcomes, \(Y^1\) versus \(Y^0\). If the conditional causal effect is the same in source and target at each value of \(X\), then the target average can be expressed using the source conditional effects and the target distribution of \(X\):
This expression presumes that the conditional effects are available from the source evidence and that the relevant values of \(X\) occur in the source as well as the target. It shows why a distribution-shifting effect modifier must be measured in the target: without \(P(X=x\mid D=1)\), the target average cannot be assembled from those conditional effects. If \(X\) is continuous, the corresponding expression is an integral, not a sum:
Here \(f(x\mid D=1)\) is the target density of \(X\). The equations are a justification for the measurement logic, not a complete analysis procedure: the causal effects, their cross-domain comparability, and the availability of relevant covariate values require scientific and statistical support.
Worked Example: A Shift in an Effect Modifier
Suppose a hypothetical source trial estimates 12-month treatment effects for a binary endpoint as risk differences within two baseline renal-function groups. The estimated contrast is \(+8\) percentage points in the lower-function group and \(+2\) percentage points in the higher-function group. Assume, for illustration, that these conditional causal effects apply in both domains. The lower-function group makes up 70% of the source population but 30% of the target population.
Source average contrast. Using the source group's proportions gives \(0.70(8)+0.30(2)=5.6+0.6=6.2\) percentage points.
Target average contrast. Using the target proportions gives \(0.30(8)+0.70(2)=2.4+1.4=3.8\) percentage points.
The two averages differ by \(6.2-3.8=2.4\) percentage points, even though the assumed conditional effects are unchanged. A diagram with \(D\rightsquigarrow X\), \(X\) included as a relevant modifier of the treatment effect, and \(A\to Y\) makes the measurement need visible: the target distribution of renal function is needed to describe the target average. These hypothetical calculations do not establish that the conditional effects truly transport; that is a separate scientific assumption.
Worked Diagram Decisions
Worked Example: Assignment Changes, but the Intervention Is the Same
Consider a hypothetical randomized source trial and a target health system where treatment is prescribed more often to patients with severe disease. Let \(X\) denote baseline severity, \(A\) treatment, and \(Y\) a clinical outcome. The source trial has randomized \(A\); the target treatment-assignment process depends on \(X\).
A suitable starting diagram includes \(X\to A\), \(X\to Y\), \(A\to Y\), and a selection arrow \(D\rightsquigarrow A\) if assignment differs by domain. The diagram should not be read as saying that the causal effect of setting \(A\) must differ. It says that observed treatment allocation differs and may require a different analysis of observed treatment-outcome data.
If the target question is the effect of the same treatment intervention, the assignment arrow alone does not show that the target outcome mechanism changed. Severity still needs attention because it predicts outcome and influences target assignment; whether its target distribution also needs to enter the effect transport depends on how it relates to the treatment contrast. The diagram therefore separates an assignment problem from a claim of treatment-effect nontransportability.
Worked Example: A Selection Arrow into the Outcome
Suppose a hypothetical source trial and target region use the same drug and measure the same baseline covariates, but the target region has a different background-care protocol that may affect the clinical endpoint. Represent the treatment effect with \(A\to Y\), and use \(D\rightsquigarrow Y\) to flag the possible outcome-mechanism change. If baseline \(X\) explains this difference, make that relationship explicit rather than treating the selection arrow as resolved merely because \(X\) was measured.
The diagram warns that source outcome relationships may not suffice. It does not prove that the treatment contrast changes: background care could shift both treatment outcomes equally on a risk-difference scale, for example, or it could interact with treatment. The scientific question and effect scale determine which possibility matters. If the diagram leaves a direct domain-related change in the outcome mechanism unexplained, measuring \(X\) alone cannot justify transport; target outcome information or a defensible mechanistic account may be needed.
Construction, Diagnostics, and Alternatives
A disciplined construction process reduces both omissions and overstatement:
State what defines source and target, the treatment intervention, outcome, time point, and effect scale. A diagram cannot repair mismatched estimands.
Include the baseline variables that cause or predict treatment and outcome, and the treatment-to-outcome relation relevant to the question.
Add selection arrows for distribution or mechanism changes supported by clinical, design, or operational knowledge. Keep assignment shifts distinct from outcome shifts.
Ask which shifted variables alter the target treatment contrast and whether they are measured comparably in source and target. Flag unexplained outcome shifts rather than assuming adjustment solves them.
Consider omitted effect modifiers, different outcome definitions, and plausible alternative arrows. Record which conclusions change when an assumption is relaxed.
A useful diagnostic is to compare each proposed selection arrow with the data collection plan. If a variable is asserted to explain domain differences but is missing from the target data, the diagram exposes a gap. If outcomes are measured differently, an arrow to \(Y\) may be less informative than first harmonizing the endpoint. If the diagram contains no route from a domain difference to the treatment contrast, ask whether that is a justified scientific conclusion or simply an omitted interaction.
Selection diagrams complement, rather than replace, alternatives such as pooled regression, mechanistic modeling, or design-based bridging. A pooled regression can represent measured covariates and interactions, but may hide the assumptions about which relationships are common across domains. A mechanistic model can explain how a changed physiological factor affects exposure or response, but its transport claims depend on model structure and validation. A new target-population trial can provide direct evidence, though it may be costly or infeasible. The diagram helps compare these strategies by showing which mechanisms each must address.
Common Mistakes and Limitations
- Treating every difference as a treatment-effect difference. A shift in \(D\rightsquigarrow A\) concerns assignment; a shift in the distribution of a prognostic variable may change average outcomes without changing a treatment contrast on every scale.
- Assuming that measuring a covariate resolves every selection arrow. Measurement helps only when the variable explains the relevant shift and the needed relationships are supported. A direct or unexplained \(D\rightsquigarrow Y\) cannot be removed by labeling an unrelated covariate as an adjustment variable.
- Reading arrows as empirical proof. A diagram records assumptions and knowledge. Data may reveal incompatibilities, but cannot generally verify all cross-domain invariance claims.
- Using an overly sparse diagram. Omitted effect modifiers, endpoint differences, or changes in background care can make an apparently simple bridge misleading. Conversely, unsupported arrows can make a diagram unhelpfully pessimistic.
The main limitation is that a selection diagram is qualitative: it does not provide the target effect, quantify uncertainty, or guarantee that source data cover the target covariate patterns. Its role is to make the proposed bridge auditable and to guide measurement and modeling. The diagram should be revised when clinical evidence, data diagnostics, or endpoint review changes the account of how the populations differ.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions below.
- A target population has a different distribution of a baseline covariate \(X\). What additional feature would make that distribution especially important for transporting an average treatment effect?
- In a diagram, what does \(D\rightsquigarrow A\) say, and why does it not alone establish that the effect of an intervention on \(A\) changes?
- Why is a sum over values of \(X\) not the general expression for averaging conditional effects when \(X\) is continuous?
- A diagram contains an unexplained \(D\rightsquigarrow Y\). Why is measuring a prognostic covariate not automatically enough to resolve the concern?
Solutions. (1) The distribution matters especially when \(X\) modifies the treatment effect on the chosen scale; then different target and source proportions can yield different average contrasts. (2) It flags a possible change in the treatment-assignment mechanism. The intervention contrast concerns setting treatment, whereas assignment describes how observed treatment groups are formed, so the arrow alone does not imply a change in the intervention effect. (3) A continuous covariate is averaged by integrating the conditional effect against its density, \(\int \tau(x)f(x)\,dx\), rather than summing over discrete categories. (4) The covariate must explain the relevant outcome-mechanism difference and the required conditional relationships must be supported. An unexplained outcome shift may require target outcome evidence or a defensible mechanistic explanation.