Why the Intervention Must Be Named Precisely
A treatment policy and an individual treatment are different interventions. A clinic might adopt a policy of offering therapy to all eligible patients, while each patient decides whether to start it. The policy is assigned at the clinic or patient level; receipt depends on what happens after the offer. Comparing those assigned to the offer policy with those assigned to usual care answers a policy question. Comparing people who received therapy with people who did not answers a different question and may not have a causal interpretation without further assumptions.
This distinction matters when translating a trial protocol into an observational study. The intervention must be defined as the action that could be assigned at time zero. If the intended question is whether a health system should recommend therapy, defining groups by later treatment receipt changes the question and can introduce selection bias. The earlier material on Intention-to-Treat Analogs in Observational Data provides relevant context; here the emphasis is on naming the policy intervention and separating it from treatment received.
For a binary outcome, a common policy estimand is the difference in risks:
This contrast includes the consequences of offering or recommending treatment: who accepts, who declines, and any other consequences of the policy that affect the outcome. The treatment-receipt contrast \(E[Y^1]-E[Y^0]\), using the superscript to denote an intervention on receipt in that expression, instead asks what would happen if treatment itself were set. The superscripts refer to different interventions in the two expressions, so it is safer to name the interventions in words whenever there could be ambiguity.
Specify the Target Trial Around the Policy
A target trial comparing policies should define eligibility, assignment, follow-up, outcome, and analysis in terms of the policy being considered. In particular, the protocol must say what counts as an offer or recommendation: who makes it, to whom, at what time, and with what support or follow-up. “Offered therapy” is not a sufficiently precise intervention if some clinics provide a detailed consultation and others merely make a treatment available.
Worked Example: Writing a Policy Comparison
Suppose a health system is deciding whether to implement a policy that offers a counseling program to adults with newly diagnosed hypertension. A suitable target-trial protocol could specify the following.
Eligibility and time zero. Enroll adults meeting the same hypertension criteria at their first eligible visit, before the offer is made. This aligns eligibility, assignment, and the start of follow-up.
Policies. Assign eligible patients to either (1) an offer of the program, with a defined explanation and a route to enrollment, or (2) usual care without that offer during the assignment period. The protocol does not require every person offered the program to accept it.
Outcome and follow-up. Measure whether blood pressure is controlled at 12 months using the same ascertainment method in both groups.
Estimand. Compare the 12-month risk of control under assignment to the offer policy with the risk under assignment to usual care. This is the effect of implementing the offer policy, not the effect of completing the program.
If the offer is not documented but program attendance is recorded, attendance alone does not reveal who was offered the program and declined. The available data may therefore be insufficient to emulate this policy trial.
When Observational Data Identify a Policy Effect
For observational emulation, let \(X\) denote measured baseline covariates. The following result gives a standard identification formula. It assumes a well-defined policy, consistency, no interference between individuals, conditional exchangeability of policy assignment, positivity, and outcome ascertainment. Conditional exchangeability means \(Y^z\) is independent of \(Z\) given \(X\), for each \(z\). Positivity means \(P(Z=z\mid X=x)>0\) for covariate values \(x\) in the target population. Consistency means that a person assigned policy \(z\) has observed outcome \(Y=Y^z\). If policies have materially different versions, consistency requires that those versions be defined or handled explicitly.
for discrete \(X\). For continuous \(X\), replace the sum with an integral against the target population distribution of \(X\).
Proof. By iterated expectation,
Conditional exchangeability gives \(E[Y^z\mid X=x]=E[Y^z\mid Z=z,X=x]\). By consistency, among those with \(Z=z\), \(Y=Y^z\), so \(E[Y^z\mid Z=z,X=x]=E[Y\mid Z=z,X=x]\). Positivity ensures the conditional mean on the right is defined for each covariate stratum being averaged. Substitution gives the stated formula. \(\square\)
This identifies a policy effect by comparing outcomes across policy-assignment groups within baseline covariate strata, then averaging over the target population’s covariate distribution. It does not justify replacing \(Z\) with \(A\): treatment receipt may be affected by prognosis, preferences, or access, and may itself be affected by policy assignment.
Worked Example: Standardizing Policy Risks
Suppose \(X\) marks whether a patient has a high baseline risk. In the target population, 40% have high risk and 60% have low risk. Observational data provide the following outcome risks by policy and risk stratum.
| Baseline risk | Population proportion | Risk under offer policy | Risk under usual care |
|---|---|---|---|
| High | 0.40 | 0.30 | 0.40 |
| Low | 0.60 | 0.10 | 0.15 |
Assuming the identification conditions hold, standardize each policy’s risks to the same population:
The estimated policy risk difference is \(0.18-0.25=-0.07\), or 7 percentage points lower risk under the offer policy. This is not the unadjusted difference between the observed policy groups unless their baseline-risk distributions happen to match. Nor does it estimate the effect of receiving the program: the table groups people by policy assignment, not by receipt.
Policy Effects Need Not Equal Treatment Effects
A policy can change the proportion who receive treatment without changing treatment for everyone. It can also affect outcomes through processes other than receipt, such as counseling, monitoring, or access to follow-up care. Consequently, policy effects and treatment effects generally differ.
Worked Example: An Offer Policy With Partial Uptake
Consider a setting where an offer changes only treatment uptake. Under the offer policy, 60% receive treatment; under usual care, 10% receive it. Suppose, for this example, treatment has a risk of 0.10 and no treatment has a risk of 0.20 for everyone, regardless of policy. The assumptions of no other policy effect and common treatment risks are deliberately strong.
Under the offer policy, the outcome risk is the weighted average across receipt:
Under usual care it is:
Thus the policy risk difference is \(0.14-0.19=-0.05\), a 5-percentage-point reduction. The treatment risk difference in this example is \(0.10-0.20=-0.10\), a 10-percentage-point reduction. The policy effect is smaller because the policy changes uptake by \(0.60-0.10=0.50\), rather than changing treatment for everyone. Under the stated assumptions, the policy difference is \(0.50(-0.10)=-0.05\), agreeing with the direct calculation. Without those assumptions, this simple multiplication need not hold.
One clear special case does make policy and treatment contrasts coincide. This is a result about potential outcomes, not a general property of offers.
Derivation. Under perfect compliance, \(A^1=1\) and \(A^0=0\). The exclusion condition gives \(Y^1=Y^{A^1}=Y^{1}\) under the receipt intervention, and \(Y^0=Y^{A^0}=Y^{0}\) under the receipt intervention. Taking expectations and subtracting yields equality of the two contrasts. If either perfect compliance or the exclusion condition fails, this argument does not apply. \(\square\)
Diagnostics, Alternatives, and Limitations
The central diagnostic is whether the data record the assigned policy, rather than merely what people eventually did. Check how the offer was defined, whether its timing is known, whether policy versions varied across providers, and whether a recorded “no offer” really represents the comparison policy. Compare treatment uptake across policy groups as a description of implementation; uptake differences do not by themselves validate the causal estimate.
Also inspect whether both policies occur across important baseline covariate patterns. If some types of patients are always offered therapy and others are never offered it, positivity may fail, and the standardized contrast relies on unsupported extrapolation. Poor outcome measurement, differential follow-up, or unmeasured causes of both policy assignment and outcome can also undermine identification. These checks can reveal design problems, but they cannot prove conditional exchangeability.
| Question | Policy comparison | Treatment-receipt comparison |
|---|---|---|
| What defines the groups? | Offer or recommendation assigned at baseline | Actual treatment received |
| What effect is targeted? | Consequences of implementing the policy, including its uptake | Consequences of intervening on receipt, if identified under its own assumptions |
| Common interpretive risk | Calling an offer effect the effect of treatment itself | Assuming receipt groups are comparable despite selection into receipt |
The comparison is not a choice between a “better” and a “worse” estimand. The policy question is appropriate for decisions about offering or recommending care; a receipt intervention may be more relevant to a clinical question about treatment itself. Each requires its own clearly defined intervention and assumptions. If assignment is not observed, an analysis based only on receipt cannot generally recover the effect of an offer policy. If a policy has direct effects beyond treatment receipt, it is especially important not to interpret its estimated effect as a treatment-only effect.
Common Mistakes
- Defining the offer group by treatment received. This replaces policy assignment with a post-assignment behavior. Preserve the offer indicator as \(Z\), and describe receipt \(A\) separately.
- Assuming an offer is equivalent to treatment. An offer may be declined, and a policy may affect outcomes through more than receipt. Interpret the policy contrast as a policy effect.
- Using an undefined intervention. “Recommend therapy” can represent different actions in different settings. Specify who recommends what, when, and with what accompanying support.
- Treating overlap in receipt as proof of policy positivity. Positivity concerns policy assignment within baseline covariate patterns. Similar receipt rates do not establish it.
Check Your Understanding
Answer each question, then compare with the solution immediately below it.
- A clinic assigns patients to receive a written recommendation, but some patients decline the drug. Which variable represents assignment, and which represents receipt?
Solution. Let \(Z\) represent assignment to the recommendation policy and \(A\) represent actual drug receipt. Declining does not change the assigned policy group.
- Suppose the standardized risks are 0.12 under a policy offer and 0.20 under usual care. What is the policy risk difference, and what does it mean?
Solution. The risk difference is \(0.12-0.20=-0.08\). The offer policy is associated with an 8-percentage-point lower outcome risk under the stated identification assumptions. This does not by itself estimate the effect of receiving treatment.
- Name two assumptions used in the observational identification result and explain their roles.
Solution. Conditional exchangeability means that, within levels of measured baseline \(X\), policy groups have comparable potential outcomes; it supports replacing potential-outcome means with observed conditional means. Positivity requires a positive probability of each policy within the relevant \(X\) strata, so those conditional means can be estimated. Consistency is another required assumption: observed outcomes under assigned policy must equal the corresponding potential outcomes.
- Under what conditions does the policy contrast equal the treatment-receipt contrast in the stated special case?
Solution. The policy must perfectly set receipt—everyone receives treatment under one policy and no one under the other—and policy assignment must affect outcomes only through receipt. Then the policy potential outcomes reduce to the corresponding receipt potential outcomes. Without these conditions, equality is not guaranteed.