Why Treatment Selection Matters
When people receive a treatment through ordinary clinical or social processes, the treated and untreated groups may differ before treatment begins. A clinician may prescribe more often to patients with severe symptoms; patients with better access to care may be more likely to obtain a treatment; and people with a favorable prognosis may be more able to complete it. If the same characteristics also predict the outcome, the observed difference between groups mixes the effect of treatment with differences that were already present.
This is treatment-selection confounding: treatment status is related to baseline features that also predict the outcome, so the observed treatment groups do not provide a like-for-like comparison. The central issue is not simply that the groups look different. It is that their differences affect the outcomes they would have had under the treatment being compared.
The potential outcomes \(Y(1)\) and \(Y(0)\) describe what would happen to the same unit under each treatment condition. Only one is observed for each unit: the observed outcome is \(Y=Y(1)\) when \(A=1\), and \(Y=Y(0)\) when \(A=0\). This consistency statement connects observed data to potential outcomes, but does not imply that the treated and untreated groups are comparable.
A causal interpretation also presumes that the treatment is sufficiently well-defined and that one unit's treatment does not alter another unit's outcome in a way relevant to the analysis. Even under those conditions, treatment selection can leave a crude comparison confounded. In a randomized trial, assignment is determined by a random mechanism; in an observational study, treatment often depends on prognosis, access, preferences, or clinical judgment.
A Useful Decomposition of the Crude Difference
For a binary treatment, the observed difference in mean outcomes is the mean outcome among treated units minus the mean outcome among untreated units. By consistency, those means can be written using potential outcomes. Add and subtract the mean untreated potential outcome among treated units:
The first term is the average treatment effect among treated units: their average difference between their two potential outcomes. The bracketed term is a selection difference: how the treated and untreated groups would have differed, on average, had neither received treatment. It is a counterfactual contrast, so it cannot generally be calculated directly from observed outcomes alone.
This decomposition explains why a crude difference need not equal a causal effect. If treated people would, even without treatment, have had better outcomes than untreated people, the selection difference is positive when higher outcomes are better. It pushes the observed contrast upward. If treated people would have had worse outcomes without treatment, it pushes the contrast downward. If treatment effects vary across units, the first term is specifically the effect among treated units, not automatically the effect for the full population.
Worked Example: Better Prognosis Makes Treatment Look More Effective
Suppose a hypothetical group of 100 patients is split evenly between high and low baseline prognosis. A treatment is more often given to patients with high prognosis. The outcome is a recovery score, with higher scores better. Assume the untreated and treated potential-outcome means are as follows; for every patient, treatment increases the score by 10 points.
| Baseline prognosis | Number treated | Number untreated | Mean \(Y(0)\) | Mean \(Y(1)\) |
|---|---|---|---|---|
| High | 40 | 10 | 80 | 90 |
| Low | 10 | 40 | 40 | 50 |
The treated group contains 40 high-prognosis and 10 low-prognosis patients. Its observed mean is therefore
The untreated group contains 10 high-prognosis and 40 low-prognosis patients. Its observed mean is
The crude difference is \(82-48=34\) points. But the treatment effect is 10 points for every patient, so the average treatment effect is also 10 points. The selection difference is \(24\) points: without treatment, the treated patients would average \([40(80)+10(40)]/50=72\) points, compared with 48 points among the untreated patients. The decomposition gives \(34=10+24\). In this example, the treated group's better prognosis makes treatment appear substantially more effective than it is.
Selection Through Access to Care
Treatment selection is not limited to clinicians choosing treatment according to disease severity. Access can affect both the chance of receiving treatment and the outcome. For example, transportation, time away from work, insurance coverage, or proximity to a clinic may affect whether a person obtains treatment. Those same resources may support follow-up care or otherwise influence recovery. In that case, a crude comparison can attribute some advantage associated with access to the treatment itself.
Worked Example: Access Creates an Apparent Treatment Benefit
Consider 100 hypothetical patients, 50 with high access to care and 50 with low access. Higher access makes treatment more likely. The outcome is a health score, with higher values better. Assume treatment adds 5 points for every patient, while access also predicts the score even without treatment.
| Access level | Number treated | Number untreated | Mean \(Y(0)\) | Mean \(Y(1)\) |
|---|---|---|---|---|
| High | 40 | 10 | 70 | 75 |
| Low | 10 | 40 | 50 | 55 |
The observed mean among treated patients is \([40(75)+10(55)]/50=3550/50=71\) points. Among untreated patients it is \([10(70)+40(50)]/50=2700/50=54\) points. The crude contrast is 17 points, although the treatment effect is 5 points for every patient.
To see the source of the difference, compare the groups' untreated potential-outcome means. The treated patients would average \([40(70)+10(50)]/50=66\) points without treatment; untreated patients average 54. Thus the selection difference is 12 points and the decomposition is \(17=5+12\). Access is associated with treatment selection and with outcomes apart from treatment. The 17-point observed difference cannot be interpreted as the treatment's 5-point causal effect.
The direction of confounding is not always toward an apparently beneficial result. In clinical settings, treatment is often given more frequently to people who are sicker. If severity predicts a worse outcome even without treatment, treatment recipients can have worse observed outcomes despite benefiting from treatment. This is sometimes called confounding by indication.
Worked Example: Confounding by Indication Hides a Benefit
Suppose 100 hypothetical patients are evenly divided between severe and mild disease. Forty severe patients and 10 mild patients receive treatment; the other 10 severe and 40 mild patients do not. The outcome is a score for which higher values are better. Treatment raises every patient's score by 15 points.
| Severity | Number treated | Number untreated | Mean \(Y(0)\) | Mean \(Y(1)\) |
|---|---|---|---|---|
| Severe | 40 | 10 | 30 | 45 |
| Mild | 10 | 40 | 70 | 85 |
The treated group's observed mean is \([40(45)+10(85)]/50=2650/50=53\) points. The untreated group's observed mean is \([10(30)+40(70)]/50=3100/50=62\) points. The crude contrast is \(53-62=-9\) points, apparently favoring no treatment. Yet the treatment effect is \(+15\) points for every patient.
Without treatment, the treated group would average \([40(30)+10(70)]/50=38\) points, while the untreated group would average 62. The selection difference is \(38-62=-24\) points. Thus \(-9=15+(-24)\): treatment helps, but the treated patients' worse underlying prognosis more than offsets that benefit in the crude comparison.
What to Look For, and What It Cannot Establish
A useful first diagnostic is to describe important pretreatment characteristics by treatment group. If the groups differ in prognosis, disease severity, prior health, access, or other plausible outcome predictors, a crude outcome contrast deserves particular caution. Also ask how treatment was chosen: were clinical indications, patient preferences, resource constraints, or eligibility rules involved? Such features can reveal a plausible route from baseline characteristics to both treatment and outcome.
These checks are warnings, not proof. Similar observed baseline summaries do not establish that the groups are comparable in every relevant respect. A characteristic may be measured coarsely, omitted, or unknown; and a displayed balance table alone cannot rule out differences in potential outcomes. Conversely, an observed baseline difference does not by itself quantify the amount of confounding. The decomposition describes the issue conceptually, but its selection-difference term involves unobserved counterfactual outcomes.
Random assignment offers a different design-based starting point. In a randomized trial, the assignment mechanism does not select treatment based on participants' baseline prognosis or access, although chance can still produce imbalances in a particular trial. In an observational comparison, treatment selection can systematically depend on those features. Describing baseline differences therefore helps characterize an observational comparison, but does not turn it into a randomized one.
Adjustment methods aim to make comparisons more relevant by accounting for pretreatment differences, but their validity depends on substantive assumptions and appropriate data. Merely including many variables or obtaining similar observed groups does not guarantee that the comparison isolates a causal effect. The introductory material on potential outcomes and identification distinguishes the causal quantity from the observed contrast; here the key point is that selection can separate them before any statistical model is fitted.
Common Mistakes
- Calling every group difference a treatment effect. The groups may have differed in their potential outcomes even without treatment. Separate the treatment contrast from the selection difference.
- Assuming confounding must make treatment look beneficial. Selection of higher-prognosis patients can exaggerate benefit, while preferential treatment of sicker patients can mask a benefit or make treatment look harmful.
- Equating a baseline difference with a numerical bias estimate. A baseline imbalance signals a concern; it does not tell us the counterfactual outcome difference or the size of the resulting confounding.
- Assuming consistency ensures comparability. Consistency links the observed outcome to the potential outcome under the treatment actually received. It does not make treated and untreated groups alike.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions below.
- In one sentence, distinguish treatment selection confounding from a treatment effect.
- In the prognosis example, what are the crude difference, treatment effect among treated patients, and selection difference?
- A clinic gives a treatment more often to patients with severe disease, and severe disease predicts poorer outcomes. If the treatment is beneficial, which direction might the selection difference push the crude contrast?
- Why does finding similar measured baseline characteristics across treatment groups not prove that a crude comparison is unconfounded?
- What feature of randomized assignment distinguishes it from ordinary treatment selection, and what imbalance can still occur in a particular randomized trial?
Solutions. (1) A treatment effect compares potential outcomes under treatment and no treatment, whereas treatment-selection confounding occurs when treatment groups differ in their potential outcomes because of how treatment was chosen. (2) The crude difference is \(34\) recovery-score points, the treatment effect among treated patients is \(10\) points, and the selection difference is \(24\) points; \(34=10+24\). (3) The treated group has worse prognosis without treatment, so its untreated potential outcome mean is lower; this negative selection difference can pull the crude contrast downward, masking some or all of a benefit. (4) Similar measured characteristics cannot rule out unmeasured, coarsely measured, or otherwise unrecognized differences in prognosis. (5) Randomized assignment uses a random mechanism rather than selecting treatment according to baseline features; chance can nevertheless produce baseline imbalances in a particular trial.