Why Make Assumptions Explicit?
An empirical result is never just a number. It is a number interpreted as evidence about a particular quantity, for a particular group, using particular measurements and a particular data-generating process. The same calculation can support one claim while failing to support another. A mean calculated from a set of records, for example, describes those records; interpreting it as the mean for a broader population requires additional assumptions.
The key question is not whether an analysis uses assumptions. Every empirical analysis does. The useful questions are: What is the claim? Which assumptions connect the observed data to that claim? What would the data mean if one of those assumptions failed? Making this connection explicit separates a calculation from the interpretation placed on it.
For instance, an observed average can be calculated without assuming that the observations were randomly selected. But using that average to estimate a population mean may depend on how the observations entered the data. Similarly, a difference between two observed group means is calculable whether or not treatment was assigned randomly; interpreting it as a causal effect requires more.
Four Places Assumptions Enter
A practical way to audit an empirical claim is to sort its assumptions by what they concern. These categories are not mutually exclusive: one conclusion may rely on several at once.
- Sampling assumptions describe how observed units relate to the population or set of units the claim concerns. They matter when a sample summary is used to describe a larger group.
- Measurement assumptions connect recorded values to the concepts or quantities named in the claim. They matter when a recorded variable is treated as an accurate measure of a target quantity.
- Assignment assumptions describe how treatment or exposure is allocated across units. They matter when observed outcome differences are interpreted causally.
- Population-relationship assumptions describe whether relationships learned in one group apply to another group. They matter when results are transported beyond the population directly studied.
The first step is to state the estimand in context. The next is to say which assumption connects the observed data to that estimand. Finally, ask how the conclusion would change under a plausible violation. The examples below apply this sequence to different kinds of claims.
Sampling Assumptions and a Population Mean
Suppose the target is the average hourly wage among a fixed group of workers. A sample average estimates that target under some sampling processes better than others. The sampling process is part of the claim, not an incidental detail.
Worked Example: A Mean Under Uniform Pair Sampling
Consider four workers whose hourly wages, in dollars, are 10, 12, 16, and 22. The target is the mean wage of these four workers. A sample of two is selected uniformly from the six possible pairs, and the sample mean is calculated. Find the target mean and the expected sample mean under this process.
The population mean is
The six possible samples and their sample means are: (10, 12), mean 11; (10, 16), mean 13; (10, 22), mean 16; (12, 16), mean 14; (12, 22), mean 17; and (16, 22), mean 19. Because each pair is equally likely, the expected sample mean is
Here the sample mean is unbiased for this finite-group mean under the stated uniform sampling process: its expectation across possible samples equals the target, 15 dollars per hour. Each worker appears in three of the six possible pairs, so each worker has probability \(3/6=1/2\) of appearing in the sample. More generally, writing the four wages as \(x_1,\ldots,x_4\), each appears three times in the sum of the six pair means. Therefore,
The equality depends on uniform selection among the six pairs. If the selection rule always chose the pair with wages 10 and 12, the sample mean would always be 11 dollars per hour, not 15. The observed calculation is possible either way, but the claim that it estimates the four-worker mean relies on the sampling assumption.
This example makes a precise point: a sample mean is not automatically representative of a population just because it is an average. The assumption must match the data collection process and the target group. The example establishes unbiasedness under a specified process; it does not say that every realized sample mean equals the population mean.
Measurement Assumptions and the Meaning of a Recorded Value
A measurement assumption links a recorded value to the quantity named in the estimand. If the claim concerns actual hours worked, a calculation using recorded hours supports that claim only if the records measure actual hours adequately for the intended purpose. A numerical result can be computed perfectly while describing a different quantity than the one of interest.
Worked Example: Recorded Hours and Actual Hours
Suppose the target is the average number of hours actually worked by three employees during a week. Their actual hours are 35, 40, and 45. A timekeeping system records 35, 40, and 50. Compare the actual mean with the mean of the recorded values.
The target mean of actual hours is
The mean calculated from the records is
The third record exceeds actual hours by 5 hours, so the recorded mean exceeds the target mean by \(5/3\approx 1.667\) hours. The calculation of the recorded mean is correct, but treating it as the average of actual hours requires an assumption that the records correspond adequately to actual hours. In this hypothetical example, that assumption does not hold exactly.
A measurement assumption should be stated at the level needed by the claim. If the goal is to describe what a particular administrative system recorded, the recorded values may be the target themselves. If the goal is to describe actual hours worked, the relationship between records and actual hours matters. The relevant question is not simply whether the data are numerical, but what those numbers represent.
Assignment Assumptions and Causal Claims
An observed difference between groups is a descriptive comparison. To interpret it as the effect of treatment, the comparison must distinguish the treatment from other differences between the groups. A clear way to state this issue is with potential outcomes. For unit \(i\), let \(Y_i(1)\) be the outcome that would occur under treatment and \(Y_i(0)\) the outcome that would occur without treatment. The average treatment effect for a fixed group of \(n\) units is
Only one of these two potential outcomes is observed for each unit. Consequently, an observed treated-versus-untreated difference is not automatically the average of the individual treatment effects. An assignment assumption is needed to connect the comparison to that causal target.
Worked Example: A Group Difference Without a Treatment Effect
Suppose four workers have the following potential earnings, in dollars per hour. Two higher-paid workers have potential earnings of 20 with or without a training program; two lower-paid workers have potential earnings of 10 with or without the program. The program is given to the higher-paid workers and not to the lower-paid workers. Calculate the observed difference and the average treatment effect.
For each worker, treatment changes no potential outcome: \(Y_i(1)-Y_i(0)=0\). Thus the average treatment effect is
The observed treated mean is 20 dollars per hour, while the observed untreated mean is 10 dollars per hour. Their difference is
The 10-dollar difference does not represent a treatment effect in this example. Treatment assignment selected the higher-paid workers, who would have had higher earnings even without treatment. The assignment is therefore related to the potential outcomes, and the observed group comparison does not identify the average treatment effect.
For a randomized assignment to identify the average treatment effect in the assigned group, the assignment mechanism must make treatment independent of the potential outcomes, each unit must have a positive chance of receiving each treatment condition, and the observed outcome must equal the potential outcome corresponding to the treatment actually received. In addition, the treatment must be sufficiently well-defined that one unit's assignment does not change another unit's outcome. These conditions clarify what a causal interpretation requires; merely observing two groups with different average outcomes does not establish them.
Population Relationships and Applying Results Elsewhere
Even when a relationship is identified in a study population, applying it to a different target population requires an additional argument. The populations may differ in the mix of people or in how a relevant relationship operates. A common assumption for transporting a treatment effect is that effects within relevant groups are the same in the study and target populations, and that the groups represented in the target are also represented in the study.
Worked Example: Changing Population Composition
Imagine a study with two groups, A and B. Under valid treatment assignment within each group, the estimated effect is 5 dollars per hour in group A and \(-1\) dollar per hour in group B. Group A makes up 80% of the study population, but only 20% of the target population. Assuming those group-specific effects apply in both populations, calculate the average effect for each population.
The study-population average effect is
The target-population average effect, using its different group proportions, is
The study average of 3.80 dollars per hour is not the target average of 0.20 dollars per hour. The difference arises from population composition, even though the group-specific effects were assumed to remain the same. If those effects also changed between populations, reweighting alone would not suffice. The interpretation therefore relies both on appropriate group-specific evidence and on the assumption that the relevant relationships carry over.
A Practical Assumption Audit
Before interpreting an empirical result, write down the claim in a way that makes its target explicit. Then identify the assumptions that connect the data to that target. The following sequence is useful:
State what quantity is being claimed, for which group, and in what units. A population mean, an average treatment effect, and a descriptive sample mean are different targets.
Ask whether the link depends on sampling, measurement, treatment assignment, or a relationship carrying over to another population.
Replace vague phrases such as “the data are good” with a checkable statement, such as “each pair was equally likely to be selected” or “recorded hours equal actual hours.”
Ask whether the same observed result could occur while the target claim is false. If it could, identify which additional assumption rules out that counterexample.
This audit does not prove that an assumption is true. It clarifies what the conclusion depends on, makes the scope of the claim visible, and suggests which parts of the data collection or research design deserve scrutiny.
Common Mistakes
- Confusing calculation with identification. A software package can calculate a difference in means without establishing that it is a causal effect. The correction is to name the estimand and state the assumptions that connect the calculation to it.
- Calling a sample “representative” without specifying why. A sample mean can differ systematically from the target mean under some selection processes. State the sampling condition that justifies the intended population claim.
- Treating a recorded value as the concept itself. A value in a data file may measure a related quantity rather than the target quantity. State what the variable records and what measurement link the interpretation requires.
- Assuming a study result automatically applies elsewhere. A different population composition or a different group-specific relationship can change the target effect. State the transport assumption and account for relevant population differences.
Check Your Understanding
For each question, identify the relevant assumption and explain how its failure could affect the claim.
- A researcher uses the mean wage of a selected group of workers to claim the mean wage of all workers at a firm. What sampling assumption connects the observed mean to that population claim?
- A study claims to estimate actual weekly work hours using values from a timekeeping system. What measurement assumption is required?
- In the training example, why is the observed 10-dollar-per-hour difference not the average treatment effect?
- A study finds different effects in two population subgroups. Why might its overall average effect differ from the effect in a target population with a different subgroup mix?
- Explain in your own words how identification differs from calculating an estimate.