Why the Effect Must Be Defined Before It Is Estimated
When a policy is introduced in one region or an intervention is adopted by one institution, the central causal question is not simply whether its outcome changed afterward. The outcome might have changed even without the intervention. The causal effect compares what happened under the intervention with what would have happened to that same treated region or institution, over the same period, without it.
That comparison involves one observed path and a counterfactual path that cannot be observed at the same time for the treated unit. Defining the paths precisely establishes the estimand: the causal quantity the analysis aims to learn. It also makes clear why an effect at one time, an average across times, and a sum across times are different quantities.
Potential Outcomes for an Aggregate Unit
Let \(i\) index aggregate units such as regions or institutions, and let \(t\) index time periods. Suppose unit \(i=1\) is exposed to an intervention beginning after period \(T_0\). The outcome \(Y_{it}\) might be a regional rate, an institutional total, or a mean for people served by the institution. The unit of analysis here is the region or institution; the outcome’s definition determines what its values and units mean.
An intervention often has a time path rather than being a one-time switch. For example, it may begin on a particular date and continue thereafter. To keep the notation compact, define the treated regime as the specified path in which the intervention begins after \(T_0\) and is implemented as intended through time \(t\). Define the comparison regime as the path in which that intervention is not introduced through \(t\). The intervention’s content, timing, and duration must be specified for these regimes to have a clear meaning.
The subscript \(1\) identifies the treated unit, while \(t\) identifies the time. The arguments \(1\) and \(0\) distinguish the regimes; they do not mean that the outcome itself is binary. If the outcome is a rate, the effect is a difference in rates. If the outcome is a count, it is a difference in counts.
The consistency condition says that the observed outcome for the treated unit at a post-intervention time equals its potential outcome under the regime it actually received: \(Y_{1t}=Y_{1t}(1)\). The corresponding \(Y_{1t}(0)\) is unobserved. It is a counterfactual for the treated unit, not the outcome of a different region. If spillovers between aggregate units are relevant, potential outcomes may also depend on other units’ intervention assignments; the intervention and comparison regimes must then specify those assignments as well. Defining an estimand does not, by itself, establish that the counterfactual can be identified from data.
This definition concerns an aggregate outcome. If \(Y_{1t}\) is a regional mean, then \(\tau_{1t}\) is an effect on that mean, not automatically the effect on each resident. If \(Y_{1t}\) is a total count, the effect concerns that total. A well-defined causal contrast requires the same outcome definition and scale under both regimes.
Worked Example: Effects at Three Post-Intervention Times
Suppose a hypothetical region introduces a school-support program after the baseline period. Its outcome is a regional mean test score, measured in score points. For illustration, suppose the region’s two potential outcome paths are as follows. These stipulated values are hypothetical; in an actual application, only the outcome under the implemented regime is observed.
| Post-intervention period | Mean score under intervention \(Y_{1t}(1)\) | Mean score without intervention \(Y_{1t}(0)\) |
|---|---|---|
| 1 | 83 | 80 |
| 2 | 76 | 78 |
| 3 | 86 | 80 |
At period 1, the effect is \(83-80=3\) score points. At period 2, it is \(76-78=-2\) score points. At period 3, it is \(86-80=6\) score points. Thus, the effect path is \(3,-2,6\) score points. The negative effect in period 2 is part of the path; it should not be dropped simply because the other two effects are positive.
These are effects on the region’s mean score for the particular periods. They do not establish that the intervention raised scores for every student, nor do they describe a change from the region’s own baseline. Each is a comparison between two potential outcomes for the same period.
Average Effects Across Post-Intervention Time
A summary across periods can be useful when the question concerns the typical effect over a defined post-intervention horizon. For \(K\) post-intervention periods, the equal-weight average effect is the arithmetic mean of the time-specific effects:
In the three-period example, the average effect is
The result describes the equal-weight mean of the three period-specific effects. It is not an effect at a particular period, and it does not say that the effect was 2.333 points in each period. The negative effect in the second period offsets part of the positive effects in the first and third.
Equal weighting is a choice of estimand, not a universal rule. If periods should contribute different amounts—for example, because the target is a population-weighted average over periods—one can define nonnegative weights \(w_t\) that sum to 1 and use \(\sum_t w_t\tau_{1t}\). The weights and the target they represent should be stated. Without weights, the ordinary time average gives every included period equal importance.
Cumulative Effects Across Post-Intervention Time
A cumulative effect adds the period-specific effects over a stated horizon:
For equal-weight periods, the cumulative effect is \(K\) times the average effect. In the three-period example, it is \(3+(-2)+6=7\) score-point periods, equivalently \(3(7/3)=7\) score-point periods. This sum describes the net accumulated difference across the three periods; it is not the effect in any one period.
The word “cumulative” does not change the underlying outcome definition. If the outcome is a count of events per month, summing effects over months gives a difference in the total number of events across those months. If the outcome is a rate, summing rate differences gives rate-points per period; it does not automatically give a number of people or events. Converting rates to totals requires the relevant denominators and an explicit aggregation rule.
Worked Example: Aggregating Effects Across Population Groups
Suppose a hypothetical region reports a mean outcome across three population groups. The groups make up fixed shares of the target population: 50%, 30%, and 20%. At a particular post-intervention time, the group-specific means under the intervention are 83, 76, and 86 score points. Under no intervention, the corresponding means are 80, 78, and 80 score points. The target estimand is the intervention effect on the regional mean using those same population shares under both regimes.
The regional mean under the intervention is
The regional mean under no intervention is
Therefore, the effect on the weighted regional mean is
The same answer follows by first calculating the group-specific effects and applying the shares:
This equality relies on using the same shares for both regimes. If the target instead uses regime-specific shares, the difference between the two weighted means need not equal a weighted average of group-specific effects using one common set of weights. The chosen population and weighting rule are part of the estimand.
What the Definitions Do—and Do Not—Establish
The time-specific effect, average effect, and cumulative effect answer different questions. The time-specific effect preserves the intervention’s evolution over time. The average summarizes a defined set of periods using stated weights. The cumulative effect totals the differences over that horizon. None is automatically preferable: the relevant summary depends on the scientific or policy question and on the outcome scale.
These quantities are estimands, not estimates. Their definitions use potential outcomes, including outcomes that may be unobserved. To calculate an estimate from data, an analysis needs a defensible way to learn the treated unit’s no-intervention counterfactual path. A numerical difference between an observed post-intervention outcome and a pre-intervention outcome is not, by definition, the causal effect; it also reflects whatever would have changed over time without the intervention. The potential-outcome contrast makes that distinction explicit without claiming that the missing path has already been recovered.
Common Mistakes
- Calling a before-and-after change the causal effect. The causal contrast is \(Y_{1t}(1)-Y_{1t}(0)\), not simply the post-intervention outcome minus a pre-intervention outcome. The latter compares different times and does not supply the same-time no-intervention counterfactual.
- Confusing an average with a cumulative effect. An average divides the sum of effects by the number of included periods; a cumulative effect does not. State which summary is being reported and name its horizon.
- Interpreting an aggregate effect as an individual effect. A change in a region’s mean or total is a result for that defined aggregate outcome. It does not imply that every resident or institution experienced the same change.
- Changing weights between regimes without noticing. A difference between two weighted means is a clean weighted average of group-specific effects only when the same weights are used for both regimes. State the target population and weights.
- Ignoring the outcome’s units when summing over time. A sum of monthly event-count effects is a difference in event counts across months; a sum of monthly rate effects remains a sum of rate differences, not a count of affected people.
Check Your Understanding
Use the definitions and distinctions in this tutorial to answer the following questions.
- A treated hospital’s outcome is the number of admissions per quarter. In words, what do \(Y_{1t}(1)\) and \(Y_{1t}(0)\) represent for a post-intervention quarter?
- For effects of 4, \(-1\), and 3 score points across three periods, calculate the equal-weight average effect and the cumulative effect. State the units for each.
- Why is a regional mean effect not automatically an effect experienced by every resident in the region?
- Two population groups have different means under each regime. What must be held fixed if a difference in weighted means is to equal a weighted average of group-specific effects?
- Explain why a cumulative sum of monthly rate differences is not automatically a cumulative number of events.