From a Study Description to a Causal Contrast
An observational data set can show that people who received a treatment had different outcomes from those who did not. To interpret that difference causally, however, one needs more than a calculation: the treatment and outcome must be defined, the target population must be clear, and the assumptions connecting observed outcomes to potential outcomes must be defensible. This tutorial works through those decisions for a hypothetical study of a preventive medication after surgery.
Let \(A=1\) indicate that a patient receives the medication and \(A=0\) indicate that the patient does not. Let \(Y\) be an indicator of infection within 30 days, with \(Y=1\) for infection and \(Y=0\) otherwise. For each patient, \(Y(1)\) and \(Y(0)\) denote the potential infection outcomes under medication and no medication, respectively. The target is the average treatment effect (ATE) in the population of eligible surgical patients:
Because infection is coded as 1, a negative ATE means lower average infection risk under medication. The ATE is a difference in probabilities, so it is measured in risk points (or can be expressed in percentage points). The observed data reveal only \(Y=Y(A)\) for each person; the other potential outcome is not observed.
State the Assumptions Before Calculating
These assumptions have different roles. Consistency connects recorded treatment and outcome to the potential-outcome notation. Exchangeability makes the treated and untreated patients comparable within covariate levels. Positivity ensures that both treatment-specific conditional outcome means can, in principle, be learned from the data at those levels. None is established merely by finding a treatment-outcome association or by fitting a model.
For the medication example, suppose \(X\) is infection severity measured before surgery and categorized as lower or higher. The covariate must be measured before treatment, and the categories and measurement rules should be specified. If other pretreatment factors, such as immune status or procedure type, affect both treatment decisions and infection outcomes, severity alone may not make exchangeability credible. Selecting a measured covariate set requires subject-matter knowledge; a data table cannot certify that there is no unmeasured confounding.
Consistency also requires a sufficiently well-defined treatment contrast. “Medication” should specify the drug, dose, timing, and relevant administration protocol, or otherwise justify treating the versions used as equivalent for the outcome. If one group receives several materially different regimens, a single \(Y(1)\) may not correspond to a coherent intervention. The outcome window and ascertainment should also be the same for both groups.
Worked Example: Define the Study and Audit Its Assumptions
Worked Example: Define the Study and Audit Its Assumptions
Hypothetical study. An observational hospital study follows 500 eligible surgical patients. Some receive a specified preventive medication before incision; others do not. The outcome is infection within 30 days. The records include severity category measured before the medication decision. State a suitable treatment contrast and assess the key assumptions.
Treatment and outcome. Define \(A=1\) as receiving the specified medication at the specified dose and time, and \(A=0\) as receiving no preventive medication. Define \(Y=1\) if infection is recorded within 30 days. This definition makes the timing explicit and gives the outcome a common measurement window.
Target and covariate. The target population is the 500 eligible patients represented by the study, if that is the intended population. Let \(X\) be the recorded pretreatment severity category. The desired contrast is the average infection risk under medication minus the average infection risk under no medication in that target population.
Consistency assessment. The stated dose, timing, and outcome window make consistency more plausible: a patient who received the defined regimen has observed outcome \(Y=Y(1)\), and a patient who did not has \(Y=Y(0)\). If “medication” actually combines different drugs or timings with different effects, that assessment fails unless the estimand is redefined to cover a specified treatment policy.
Exchangeability assessment. Severity is a plausible confounder if it influences both medication receipt and infection risk. But the study description alone does not show that conditioning on severity suffices. For example, if immune status also influences both, then \(\{Y(1),Y(0)\}\perp A\mid X\) may be false. The analyst must justify the covariates using knowledge of how treatment is assigned and how infection risk arises.
Positivity assessment. Check whether both medicated and unmedicated patients occur in each severity category represented in the target population. The total of 500 patients does not answer this question; the treatment counts within each category do. Thus the study can be specified, but a causal calculation should wait until the subgroup counts and assumptions have been assessed.
Why Standardization Identifies the Contrast
Let \(X\) denote the covariates used for adjustment, allowing \(X\) to be discrete or continuous. Under consistency, conditional exchangeability, positivity, and suitable measurement of the target covariate distribution, the mean potential outcome under treatment \(a\) is identified by averaging the observed conditional outcome mean among people with \(A=a\) over the target distribution of \(X\):
For discrete covariate strata, this is a weighted sum. The weights are the proportions of the target population in each stratum, not the proportions of treated or untreated patients in that stratum. The resulting difference of the two standardized means estimates the ATE for the target population, provided the assumptions hold.
A short justification makes the role of each assumption explicit. First, average the potential outcome over covariate values:
Conditional exchangeability implies \(E[Y(a)\mid X]=E[Y(a)\mid A=a,X]\). Among people with \(A=a\), consistency gives \(Y=Y(a)\), and therefore \(E[Y(a)\mid A=a,X]=E[Y\mid A=a,X]\). Combining these equalities gives the standardization formula. Positivity is needed because the conditional mean \(E[Y\mid A=a,X=x]\) must be supported by observations at the covariate values included in the target average. In an actual data analysis, the conditional means and target covariate distribution are estimated; the formula itself does not remove sampling uncertainty or model error.
Worked Example: Calculate a Standardized Risk Difference
Worked Example: Calculate a Standardized Risk Difference
Hypothetical data. Suppose severity is the only adjustment covariate considered. In the target population, 60% of patients have lower severity and 40% have higher severity. The observed infection risks and treatment counts are:
| Severity | Target proportion | Medication: patients | Medication: infection risk | No medication: patients | No medication: infection risk |
|---|---|---|---|---|---|
| Lower | 0.60 | 60 | 0.08 | 240 | 0.12 |
| Higher | 0.40 | 160 | 0.20 | 40 | 0.30 |
Each severity category contains patients in both treatment groups, so there is observed support for the two conditional risks in these strata. The cell counts do not prove population positivity, and the risks do not establish exchangeability. For this calculation, assume the treatment is consistently defined, conditional exchangeability holds given severity, and the target proportions are appropriate.
Standardized risk under medication. Weight the medication risk in each severity category by that category’s target proportion:
The estimated standardized infection risk under medication is 0.128, or 12.8%.
Standardized risk under no medication.
The estimated standardized infection risk under no medication is 0.192, or 19.2%. The standardized risk difference is
Under the stated assumptions, this estimate corresponds to 6.4 fewer infections per 100 patients in the target population under medication than under no medication. It is not a conclusion warranted by the arithmetic alone: if exchangeability or consistency is implausible, the calculation remains a standardized association rather than an identified causal effect.
Worked Example: A Positivity Failure Changes What Can Be Identified
Worked Example: A Positivity Failure Changes What Can Be Identified
Hypothetical data. Suppose a hospital gives medication to every patient in the higher-severity category and gives either treatment to patients in the lower-severity category. The target population is still 60% lower severity and 40% higher severity.
In the higher-severity stratum, \(P(A=0\mid X=\text{higher})=0\) in the observed population. The untreated conditional infection risk for higher-severity patients is therefore not observed. The standardization formula for \(E[Y(0)]\) requires that risk with weight 0.40, but the data provide no value to multiply by that weight. The population ATE is not identified by this adjustment using these data alone.
It would be incorrect to substitute the lower-severity untreated risk for the missing higher-severity risk: that assumes the two strata have the same untreated risk, a new modeling restriction not supplied by the study. Restricting the target to lower-severity patients could define a different contrast for a population with treatment overlap, but it would not recover the ATE for the original target population.
Diagnostics, Alternatives, and Limits
A practical audit begins with a cross-tabulation of treatment by the covariate strata used in standardization. Empty cells reveal a direct positivity failure for the corresponding treatment and stratum. Very small cells signal weak empirical support: conditional risks may be highly variable, and estimates can depend strongly on a few observations. A large overall sample does not resolve a lack of overlap within important subgroups.
Observed covariate balance is useful descriptive information, but it cannot establish exchangeability. Likewise, a statistical test that finds no difference between treated and untreated groups on measured variables does not show that unmeasured common causes are absent. Exchangeability is an assumption supported or challenged by study context and covariate measurement, not a condition that can be conclusively verified from the observed outcome table.
The crude difference in overall outcome proportions is simpler, but it compares the groups as actually composed. If severity distributions differ and severity predicts infection, that contrast can mix treatment differences with compositional differences. Standardization instead applies the same target severity distribution to both treatment-specific risks. With a discrete covariate, direct stratification and standardization are two ways to implement this same weighted comparison when they use the same stratum-specific means and weights. If no adjustment assumptions are credible, the data support a descriptive association, not a causal interpretation.
Standardization also has limits. Its result depends on the chosen target population and on adequately estimating conditional outcome means. With many covariates, some combinations may have little or no support even if each covariate separately appears well represented. A model may produce predictions for unsupported combinations, but prediction is not evidence that the missing treatment-specific outcomes are identified. Finally, restricting the target population or changing the treatment definition can sometimes make a question more answerable, but it changes the estimand and should be stated explicitly.
Common Mistake
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- In a study with binary treatment \(A\) and covariates \(X\), state conditional exchangeability and positivity in words or symbols.
- A target population is 25% in stratum 1 and 75% in stratum 2. The observed risks under treatment are 0.10 and 0.18; risks without treatment are 0.16 and 0.24. Calculate both standardized risks and their difference.
- In the second stratum of a target population, every person receives treatment. Which assumption fails for estimating the target-population ATE by standardization, and why?
- Why is a medication definition that combines different doses and timings a potential consistency problem?
- Does finding similar measured covariate distributions in the treated and untreated groups prove conditional exchangeability? Explain.
Solutions. (1) Conditional exchangeability is \(\{Y(1),Y(0)\}\perp A\mid X\): within covariate values, treatment assignment is independent of both potential outcomes. Positivity requires both treatment values to have positive probability at every covariate value in the target population. (2) The standardized treated risk is \(0.25(0.10)+0.75(0.18)=0.025+0.135=0.160\). The untreated risk is \(0.25(0.16)+0.75(0.24)=0.040+0.180=0.220\). Their difference is \(0.160-0.220=-0.060\), or 6 percentage points lower under treatment, conditional on the identification assumptions. (3) Positivity fails because the untreated outcome risk in stratum 2 is unsupported, yet that stratum has positive weight in the target population. (4) If versions have different effects, the treatment indicator may not correspond to one coherent intervention and potential outcome \(Y(1)\); define a specific regimen or a clear treatment policy. (5) No. Similarity in measured covariates cannot rule out unmeasured common causes or establish independence of treatment and potential outcomes within covariate strata.