Why Causal Wording Needs a Separate Justification
A comparative observational study can estimate a difference between groups without establishing that treatment caused that difference. Matching or weighting may make measured characteristics more comparable, but the method does not randomize treatment or guarantee that treated and untreated individuals are comparable in every relevant respect. A careful report therefore distinguishes the estimated contrast from the causal interpretation attached to it.
The appropriate wording depends on the whole evidential record: the study design and population, the assumptions needed to identify a causal effect, the diagnostics relevant to those assumptions, and the sensitivity evidence about remaining bias. A result should not be described as causal merely because an estimate is statistically distinguishable from zero, the measured covariates appear balanced, or several analyses give similar estimates.
State the Target and the Assumptions
Let \(A=1\) denote receipt of treatment and \(A=0\) denote the comparison treatment. Let \(Y(1)\) and \(Y(0)\) be the outcomes an individual would have under treatment and comparison, respectively. Only one of these potential outcomes is observed for each individual. A common target is the average treatment effect in a specified population, \(E[Y(1)-Y(0)]\). Another is the average treatment effect among treated individuals, \(E[Y(1)-Y(0)\mid A=1]\). These are different quantities; a report should name the target and the population to which it applies.
A measured-covariate adjustment strategy, including propensity-score matching or weighting, supports a causal interpretation only under assumptions that connect the observed contrast to the potential-outcome contrast. The key assumptions are:
- Consistency: the observed outcome equals the potential outcome corresponding to the treatment actually received, and the treatment is defined well enough that its versions do not make this correspondence ambiguous.
- Conditional exchangeability: given the measured pre-treatment covariates \(X\), treatment assignment is independent of the potential outcomes. In symbols, \((Y(1),Y(0))\perp A\mid X\). This assumption rules out residual confounding after conditioning on \(X\).
- Positivity: within covariate patterns represented in the target population, there is a nonzero probability of receiving either treatment. Without this support, the data cannot provide a direct comparison for those patterns.
- No relevant interference: one individual’s outcome is not affected by other individuals’ treatment assignments, unless the estimand and analysis explicitly account for such effects.
These assumptions are not all verifiable from the observed data. Balance diagnostics can reveal remaining differences in measured covariates; overlap diagnostics can reveal limited treatment support. Neither can establish that all confounders were measured or that conditional exchangeability holds. Reports should distinguish evidence about observable features of the data from assumptions about unobserved features.
Why Adjustment Does Not Remove the Need for Assumptions
The identification argument shows what the assumptions accomplish. Under consistency, conditional exchangeability, and positivity, the mean potential outcome under treatment \(a\) can be written as an average of observed outcomes among individuals with that treatment, conditional on their covariates, and then averaged over the target population’s covariate distribution:
To see the key step, conditional exchangeability gives \(E[Y(a)\mid X]=E[Y(a)\mid A=a,X]\). Consistency then gives \(E[Y(a)\mid A=a,X]=E[Y\mid A=a,X]\). Averaging both sides over \(X\) in the target population yields the displayed expression. Positivity is needed so the conditional mean among individuals with \(A=a\) is defined across the relevant covariate patterns. If exchangeability fails, the observed conditional means may still differ from the potential-outcome means; balancing measured covariates does not repair that failure.
In practice, matching may also change which individuals contribute to the estimate. The target then may be the matched population or a population represented by the retained observations, rather than everyone initially eligible. The conclusion should name that population rather than quietly generalizing beyond it.
Worked Example: Reporting a Matched Risk Difference
Hypothetical study. A health system matches adults who began a new medication to adults who began a comparison medication, using recorded pre-treatment covariates. In the matched sample, 12% of the new-medication group and 16% of the comparison group had an unplanned admission within one year. The estimated risk difference, new medication minus comparison, is \(-0.04\), with a 95% confidence interval from \(-0.07\) to \(-0.01\).
State the estimate and scale. The risk difference is \(0.12-0.16=-0.04\), or 4 percentage points lower in the new-medication group. The interval corresponds to differences from 7 percentage points lower to 1 percentage point lower. This is an estimated contrast in the matched sample; by itself it does not establish a treatment effect.
Check what the diagnostics support. Suppose measured baseline covariates have small post-matching standardized mean differences and both groups retain observations across most of the matched sample’s covariate range. Those findings support the claim that the matching procedure improved measured balance and that gross overlap problems were not evident in that range. They do not demonstrate exchangeability with respect to unmeasured severity, clinician judgment, or other unrecorded factors.
Include sensitivity evidence. Suppose a prespecified sensitivity analysis allowing for residual unmeasured confounding moves the estimated difference to \(-0.02\), with a 95% interval from \(-0.06\) to \(0.02\). This result indicates that the evidence for a lower admission risk is sensitive to the stated confounding scenario: the interval includes no difference. It does not prove that such confounding exists, nor that it does not.
Calibrated conclusion. “Among adults represented in the matched sample, the new-medication group had a 4-percentage-point lower observed one-year admission risk than the comparison group. The estimate is compatible with a causal benefit only if the measured covariates adequately control confounding and the other identification assumptions hold; the sensitivity analysis shows that a modest residual-confounding scenario could make the result inconclusive.” This wording separates the observed estimate from the conditional causal interpretation.
Match the Claim to the Evidence
A useful reporting discipline is to build the conclusion in layers. First state the design, population, outcome, time window, and estimate. Then describe the evidence that bears on the assumptions. Finally say whether a causal interpretation is plausible under the stated assumptions and identify the principal threats. A concise conclusion can still be cautious without becoming vague.
Specify the treatment groups, outcome scale, follow-up period, and analysis population. Use “was associated with” or “had a lower observed risk” when reporting the direct empirical contrast.
Summarize measured-covariate balance, overlap, exclusions, and relevant sensitivity or falsification analyses. State what each check assessed rather than treating a favorable result as proof of no bias.
Say that the result can be interpreted causally only under the specified assumptions, and identify which plausible violations matter most in this setting.
Do not extend the result to populations, treatment versions, outcomes, or follow-up periods that the design and analysis did not represent.
The wording should also reflect the kind of evidence available. A sensitivity analysis that changes little under a range of specified scenarios strengthens the claim that the estimate is robust to those scenarios. It does not establish robustness to every possible unmeasured confounder. A negative-control result consistent with no association can be reassuring for the bias mechanism it probes, but cannot rule out other sources of bias. Conversely, an unexpected negative-control association can weaken a causal interpretation by suggesting residual bias, measurement problems, or chance variation.
Worked Example: When the Estimate Applies to a Narrower Population
Hypothetical study. Investigators compare a specialty treatment with usual care. Before matching, the treatment is rarely used among patients with mild disease and is nearly universal among patients with very severe disease. After restricting to patients with adequate overlap and matching, the estimated six-month symptom-score difference is 3 points lower with specialty treatment (95% confidence interval: 1 to 5 points lower). Lower scores indicate fewer symptoms.
The numerical contrast is a 3-point lower mean score in the analyzed matched population. The lack of support for comparing treatments across all disease-severity levels means this estimate should not be presented as the average effect for every eligible patient. The restriction and matching define a narrower population with treatment options represented in both groups.
A defensible conclusion is: “Among patients in the overlap-supported matched population, specialty treatment was associated with a mean six-month symptom score 3 points lower than usual care. A causal interpretation requires no important residual confounding within this population; the estimate should not be generalized to severity levels excluded for inadequate overlap.” Even excellent balance in the retained sample does not justify extrapolation to patients who were not represented.
Principal Threats and Common Overstatements
The main threats should be named in relation to the study, not listed as boilerplate. For example, treatment may have been chosen in response to disease severity that was measured poorly, creating residual confounding. A covariate recorded after treatment began may be a consequence of treatment rather than a baseline confounder; adjusting for it can distort the target contrast. Differential outcome measurement or missing follow-up can create group differences unrelated to the treatment effect. Selective inclusion, treatment crossover, and ill-defined treatment versions can also complicate interpretation. The report need not claim that a threat definitely caused bias, but should explain why it is plausible and what direction or scope of distortion is possible when that can be assessed.
A common mistake is to write, “Matching eliminated confounding, so treatment caused the observed improvement.” This reasoning is wrong because matching balances only the covariates measured and handled by the procedure, and observed balance does not establish conditional exchangeability. A more accurate statement is that matching improved balance in measured baseline covariates, while causal interpretation still depends on adequate confounder measurement, overlap, consistency, and the other design assumptions.
Another overstatement is to treat a confidence interval that excludes zero as proof of causation. Statistical uncertainty quantifies sampling variability under the analysis; it does not automatically account for unmeasured confounding, selection bias, or outcome misclassification. Likewise, agreement across several model specifications is useful robustness evidence, but those analyses may share the same underlying bias. A randomized comparison, when feasible and appropriate, uses a different design basis for causal interpretation; it is not interchangeable with observational adjustment. The earlier material on sensitivity analysis and negative controls provides tools to probe specific threats, not a general certificate of causal validity.
Worked Example: A Negative-Control Warning
Hypothetical study. A matched observational analysis estimates that a preventive service is associated with 2 fewer complications per 100 patients over one year. The investigators also examine a negative-control outcome that the service is not expected to affect, but which shares some of the same care-seeking and recording processes. The matched groups differ on that outcome by 1.5 events per 100 patients, with an interval that excludes zero.
The negative-control finding is not evidence that the service caused the control outcome. Instead, it raises concern that group differences in care-seeking, recording, or another shared process may remain after matching. Because the negative control probes only certain pathways and may itself have limitations, it does not quantify the bias in the primary estimate.
A cautious report would state: “The primary analysis found 2 fewer complications per 100 patients in the service group. However, the groups also differed on a negative-control outcome, suggesting that residual differences in care processes may affect interpretation. The primary estimate should therefore be viewed as an adjusted association rather than definitive evidence of a causal reduction.” The conclusion identifies the warning without claiming the control result settles the causal question.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions below.
- A matched study reports a lower observed event risk and good balance on recorded covariates. Does this establish conditional exchangeability? Explain.
- An estimate applies to a matched population after patients without treatment overlap were excluded. What must the conclusion say about generalizability?
- A sensitivity analysis yields a similar estimate under several specified unmeasured-confounding scenarios. What does this support, and what does it not prove?
- A negative-control outcome shows a group difference. How should that finding affect causal wording?
- Rewrite this claim cautiously: “The treatment reduced symptoms because the adjusted confidence interval excluded zero.”
Solutions. (1) No. Good measured-covariate balance supports comparability on those recorded variables, but cannot establish that all confounders were measured or that conditional exchangeability holds. (2) State that the estimate concerns the overlap-supported matched population; do not present it as an effect for excluded patients or the full eligible population without additional justification. (3) It supports robustness to the scenarios examined. It does not rule out other confounding structures or prove the causal assumptions. (4) Treat it as a warning that residual bias or another shared process may remain, while noting that the control does not identify the size of bias in the primary estimate. (5) For example: “The treatment group had a lower adjusted symptom score than the comparison group. A causal interpretation is possible only if the measured covariates adequately control confounding and the other identification assumptions hold; exclusion of zero from the interval addresses sampling uncertainty, not all sources of bias.”