From a Published Result to a Causal Claim
A study summary may report that people who used a program had better outcomes than people who did not, then say the program “improved” those outcomes. The reported difference and the causal conclusion are not the same thing. A difference between groups is an observation; saying the program produced that difference is an interpretation that depends on how the study was conducted.
When assessing a causal claim, begin with the methods rather than the strength of the wording or the size of the reported difference. Ask how participants came to receive each treatment, what group provides the comparison, whether treatment preceded the outcome, and whether other differences between groups could explain the result. A small p-value or a large observed difference does not, by itself, answer these design questions.
The central design question is whether the groups being compared would have had similar outcomes, on average, apart from their treatment status. Random assignment can support this comparison by making treatment assignment independent of participants' pre-existing characteristics, apart from chance variation. In an observational study, people receive or choose their exposures without random assignment, so group differences may have causes other than the exposure itself.
A Practical Review of the Methods
For a short study summary, use the following sequence. Each question helps determine what the design can support; no single phrase in an abstract replaces the answers.
State what exposure or intervention is being compared, what outcome was measured, and which participants form each group.
Look for random assignment to treatments. If participants chose a treatment or received it through usual practice, the study is observational for that comparison.
Check that the groups are meaningfully comparable and that the treatment or exposure came before the outcome being attributed to it.
Ask whether a third variable could be related to both treatment status and outcome, and whether missing data, different measurement, or other design features could distort the comparison.
Use causal wording only when the design and its assumptions provide a credible basis for it; otherwise describe the observed association.
A confounder is a variable related to both the exposure or treatment and the outcome, such that the comparison may mix the exposure's effect with the confounder's effect. It must be a plausible alternative explanation in the context, not merely any variable measured in the study. For example, age could be related both to use of a health service and to the outcome being studied.
A comparison group matters because an outcome observed in one group alone does not reveal what would have happened to those same participants without the treatment. A control group can provide a comparison, but its existence does not automatically make the groups comparable. In particular, a control group created by self-selection may differ systematically from the treated group.
Why Random Assignment Supports Causal Interpretation
A brief justification clarifies what random assignment does and does not establish. Imagine \(n\) participants in a randomized experiment. For each participant \(i\), let \(Y_i(1)\) be the outcome that would occur under the treatment and \(Y_i(0)\) the outcome that would occur under the control. Only one of these outcomes can be observed for each participant. With random assignment, each participant has a known chance of being assigned to either group, rather than treatment being chosen according to their characteristics.
In a completely randomized experiment with \(n_1\) participants assigned to treatment and \(n_0\) assigned to control, the observed difference in average outcomes estimates the average treatment effect for those participants. In expectation over the random assignments, this difference equals the difference between the groups' average potential outcomes:
To see why, random assignment gives every participant the same chance \(n_1/n\) of entering the treatment group. Therefore, the expected treatment-group average is the average of all \(Y_i(1)\) values: \(E(\bar{Y}_{\text{treatment}})=\frac{1}{n}\sum_{i=1}^{n}Y_i(1)\). Likewise, each participant has chance \(n_0/n\) of entering control, so \(E(\bar{Y}_{\text{control}})=\frac{1}{n}\sum_{i=1}^{n}Y_i(0)\). Subtracting these expectations gives the displayed result. A particular random assignment can still produce chance imbalances, but systematic pre-existing differences are not used to decide who receives treatment.
This justification relies on actual random assignment and on a clear, consistently defined treatment and outcome. It does not say that the treatment groups will be identical in every respect, that measurement or follow-up problems cannot occur, or that results automatically generalize to people outside the experiment. If some assigned participants do not receive their treatment, a comparison by assigned group supports a causal claim about the effect of assignment; a claim about the effect of actually receiving treatment needs additional care.
Worked Example: A Randomized Program for Recovery
A hypothetical hospital report describes 120 adults recovering from a minor procedure. The researchers randomly assigned 60 to a guided breathing program and 60 to usual care. All participants followed their assigned condition, and recovery days were recorded for everyone. The average was 8.4 days in the program group and 10.1 days in the usual-care group. The summary says, “The breathing program shortened recovery.”
Assignment and comparison. The report says that researchers randomly assigned participants, rather than letting them choose their group. The usual-care group provides a comparison, and both groups' recovery times were measured in the same units. The outcome was observed after assignment.
Difference. Using program minus usual care, the observed difference in average recovery time is
Thus the program group averaged 1.7 fewer recovery days in this hypothetical experiment. Because allocation was randomized, follow-up was complete, and participants followed their assigned conditions, the design supports a causal interpretation for the effect of assignment to the program among these participants, subject to the usual requirement that the treatment and outcome were implemented and measured consistently. The result does not establish that the program shortens recovery by exactly 1.7 days for every patient, nor does this sample alone establish the effect in every hospital or patient population.
When the Groups Were Not Randomly Assigned
In observational studies, the exposed and unexposed groups can differ before the exposure or treatment. The study may still measure an important association, and researchers may try to account for measured differences, but those steps do not guarantee that all confounding has been removed. A variable that was not measured, was measured poorly, or was not adequately accounted for can still offer an alternative explanation.
Worked Example: Exercise and Blood Pressure
A hypothetical retrospective study uses clinic records for 800 adults. It finds that adults who reported exercising regularly had lower average systolic blood pressure than adults who reported little exercise. The authors write, “Regular exercise lowered blood pressure.”
Assignment and groups. The researchers did not assign exercise. Participants' exercise habits were already present in the records, so this is an observational comparison. The two groups may differ in age, diet, medication use, health status, or other characteristics related to blood pressure.
Timing and conclusion. The summary does not establish whether the exercise reports preceded the blood pressure measurements, or whether participants changed their exercise after health problems arose. Even if the timing were clear, plausible differences between groups could still help explain the association. The causal wording is therefore stronger than the described design supports.
A more accurate statement is: “In these clinic records, adults reporting regular exercise had lower average systolic blood pressure than adults reporting little exercise.” This describes the observed relationship without claiming that exercise caused the difference.
Sometimes a study uses an unusual policy change, eligibility rule, or other event to create treatment groups without direct random assignment. Such designs may support causal conclusions under additional assumptions, but the word “natural” or “quasi” does not by itself establish those assumptions. For a brief summary, identify how the groups arose and what alternative explanations remain; do not treat a nonrandom comparison as randomized.
Worked Example: A New Appointment Reminder
A hypothetical network of clinics introduces text-message appointment reminders at some locations. A report compares missed-appointment rates over the next month at those clinics with rates at clinics that have not yet introduced reminders. The reminder clinics have a lower rate, and the report says, “Text reminders reduced missed appointments.”
What the design shows. The report describes a comparison across clinics, but does not say that clinics were randomly assigned to introduce reminders. The groups may differ in patient populations, staffing, appointment availability, or other practices that affect missed appointments. The comparison group is useful, but its presence alone does not remove these possible confounders.
What can be concluded. Based on the information given, the safe conclusion is that clinics using reminders had a lower missed-appointment rate during the reported month than clinics not yet using them. To support the causal claim, the report would need to explain why differences between clinics and other changes over time do not plausibly account for the result. Without that information, the association statement is better supported.
Writing a Careful Conclusion
A useful conclusion names the population or setting, the groups or exposure being compared, the outcome, and whether the wording is causal or associational. Avoid stronger language than the methods justify. For an observational comparison, phrases such as “was associated with,” “had a higher average,” or “was more common among” accurately describe patterns without assigning a cause.
| Study feature | What it supports | Careful wording |
|---|---|---|
| Random assignment with a suitable comparison and sound follow-up | A causal interpretation for the assigned treatment in the studied setting, subject to design assumptions | “Assignment to the program caused…” or “The experiment provides evidence that…” |
| Exposure observed without random assignment | An association in the observed groups; confounding may remain | “Exposure was associated with…” |
| Only one group measured after treatment | A description of that group’s outcome, without a treatment comparison | “Participants had an average outcome of…” |
Even for a well-designed randomized experiment, avoid implying that one result settles every question. Reported effects apply most directly to the treatment, participants, setting, and outcome measured. The study may provide strong evidence for a causal effect in that setting while leaving generalization to other settings uncertain.
Common Mistake
Wrong reasoning: “The treated group did better, and the study included a control group, so the treatment caused the improvement.”
Correction: A control group is not enough. Check how treatment status was assigned and whether the groups could differ in ways that affect the outcome. If treatment was self-selected or otherwise not randomized, describe the observed association unless the design supplies a credible additional basis for causal inference. Conversely, random assignment is important evidence, but conclusions still depend on sound implementation, measurement, and follow-up.
Check Your Understanding
For each question, focus on what the study methods allow you to conclude.
- A study randomly assigns patients to two treatments and measures the same outcome in both groups. What feature most directly supports a causal comparison?
- A survey finds that people who take a supplement report fewer headaches. Give one reason this association alone does not establish that the supplement prevented headaches.
- Why does the existence of a control group not, by itself, show that a treatment caused an observed difference?
- Rewrite this conclusion cautiously: “People who used the service had higher employment rates, so the service increased employment,” when participants chose whether to use the service.
- In a randomized experiment, why might a causal conclusion about participants in the study not automatically apply to a different population?