Why the Level of the Data Matters
A relationship between regional averages is not automatically a relationship between people in those regions. If regions with more parks also have higher average physical activity, that describes a pattern across regions. It does not, by itself, show that the people who live near more parks are more active. The distinction matters because a regression conclusion should answer the question supported by the observations actually analyzed.
The same issue arises with schools, hospitals, workplaces, and other groups. Data may describe each group as a whole, or they may describe the individuals within each group. Those are different levels of measurement, and they can support different conclusions. A group-level association can be useful for questions about groups, but it cannot simply be translated into a claim about individuals.
The level of analysis should be identified before interpreting a relationship. If each row represents a region, a regression of regional averages describes differences among regions. If each row represents a person, a regression of individual measurements describes differences among people in the population represented by those data. Even when individual records are available, a question about regional patterns may call for regional summaries; the appropriate level depends on the question.
What Group-Level Data Can and Cannot Show
Suppose a data set contains one value for each region: its mean daily outdoor activity and the percentage of residents with access to a park. A relationship between those variables is an association across regions. It may help describe which regions have higher averages or guide a question about regional planning. But the data do not reveal whether, within any one region, people with park access are more active than people without it. Individual measurements and an individual-level analysis would be needed to answer that question.
This is not a defect in the regional analysis. It is a limit on what its results address. A group-level analysis can support group-level conclusions when groups are the units relevant to the question. The mistake is to make a claim at a different level without evidence at that level.
Worked Example: Comparing Regional Averages
Imagine a hypothetical data set for four regions. For each region, analysts record average nearby park area per resident and average minutes of physical activity per resident per day.
| Region | Park area per resident (square metres) | Average activity (minutes per day) |
|---|---|---|
| North | 2 | 50 |
| East | 4 | 55 |
| South | 6 | 61 |
| West | 8 | 64 |
Identify the units. There are four observations, and each observation is a region. The recorded quantities are regional summaries, not measurements on individual residents.
Describe the observed pattern. Park area increases from 2 to 4 to 6 to 8 square metres per resident. The corresponding regional activity averages increase from 50 to 55 to 61 to 64 minutes per day. The data therefore show a positive association across these four regions.
State a supported conclusion. In this hypothetical data set, regions with more park area per resident also have higher average activity. The table does not show whether residents with greater personal access to parks are more active than other residents. It also does not establish that increasing park area would increase activity. Those are different claims requiring evidence suited to them.
Averages can hide variation among the people who contribute to them. Two regions can have the same mean activity even if one contains many residents with similar activity levels and the other contains a wide range of levels. Likewise, knowing a region's average park area does not tell us which residents have access to parks. Group summaries do not preserve all the relationships among individual measurements.
The Ecological Fallacy
The ecological fallacy is the error of using a relationship observed for groups to draw an unsupported conclusion about individuals in those groups. It often occurs when group summaries are the only data available, or when an analysis is reported without making the level of observation clear.
For example, if schools with higher average attendance have higher average test scores, it would be an ecological fallacy to conclude from those school averages alone that students who attend more often score higher. The school-level pattern may reflect differences between schools, such as student intake or other school characteristics. To describe a relationship among students, the analysis needs student-level attendance and score data. Even then, its conclusions should be limited to the population and measurements represented by those data.
Worked Example: School Averages and Student Conclusions
Suppose three hypothetical schools report average weekly attendance and mean reading score. Scores are on a 100-point scale.
| School | Average attendance (days per week) | Mean reading score (points) |
|---|---|---|
| Elm | 4.4 | 68 |
| Maple | 4.7 | 74 |
| Oak | 4.9 | 81 |
What the data show. The three observations are schools. Across them, higher average attendance accompanies a higher mean reading score: the values rise from 4.4 days and 68 points at Elm to 4.9 days and 81 points at Oak. The measured association is between school averages.
What the data do not show. The table does not give any student's attendance or score. It cannot establish that within each school the students with higher attendance tend to have higher scores, nor can it show that the same student would score higher if their attendance changed. Those statements concern students, not schools.
Appropriate conclusion. The reported values show a positive association between average attendance and mean reading score across these three hypothetical schools. A student-level conclusion would require student-level observations and an analysis of the relationship among students. The school-level association alone does not warrant that conclusion.
The same reasoning applies when the groups are regions. A region with higher average income and lower average illness prevalence does not, from those summaries alone, show that higher-income individuals are less likely to be ill. Regional means and rates can be used to compare regions, but they do not identify how income and illness are related among people within a region.
Aggregation Can Conceal or Reverse a Pattern
A particularly clear warning comes from data in which both group-level and individual-level comparisons can be calculated. The overall relationship among people may differ from the relationship within each group. One reason is that groups can have different baseline outcome rates and different proportions of people in the categories being compared. Thus, an aggregate pattern can partly reflect which groups contribute people to each category.
The arithmetic below shows why the levels must be kept separate. A percentage for a region is calculated using everyone in that region as the denominator; a percentage for an exposure category is calculated using everyone in that category as the denominator. Changing the denominator changes the question being answered.
Worked Example: Regional Rates and Individual Rates
Consider hypothetical data on an exposure and a binary outcome in two regions. The outcome is recorded as Yes or No. The counts are:
| Region | Exposure | Outcome Yes | Outcome No | Total people |
|---|---|---|---|---|
| A | Yes | 80 | 10 | 90 |
| A | No | 9 | 1 | 10 |
| B | Yes | 0 | 10 | 10 |
| B | No | 9 | 81 | 90 |
Compare the regions. In Region A, 90 of 100 people have the exposure, so the exposed proportion is \(90/100=0.90\), or 90%. The outcome rate in A is \((80+9)/100=89/100=0.89\), or 89%. In Region B, the exposed proportion is \(10/100=0.10\), or 10%, and the outcome rate is \((0+9)/100=9/100=0.09\), or 9%. Across these two regions, the region with the higher exposed proportion also has the higher outcome rate.
Compare individuals within each region. In Region A, the outcome rate among exposed people is \(80/90\approx0.889\), or 88.9%. Among unexposed people it is \(9/10=0.90\), or 90%. The exposed group has a slightly lower outcome rate. In Region B, the exposed rate is \(0/10=0\), or 0%, while the unexposed rate is \(9/90=0.10\), or 10%. Again, the exposed group has a lower outcome rate within the region.
Pool the people, then compare exposure categories. There are \(80+0=80\) outcomes among \(90+10=100\) exposed people, giving \(80/100=0.80\), or 80%. There are \(9+9=18\) outcomes among \(10+90=100\) unexposed people, giving \(18/100=0.18\), or 18%. In the pooled individual data, the exposed group has a higher outcome rate.
The regional comparison is positive: Region A has both higher exposure prevalence and a higher outcome rate than Region B. Yet within each region the exposed group has a lower outcome rate than the unexposed group. The pooled individual comparison also differs from the within-region comparisons. These are distinct comparisons, with distinct denominators and group composition. None, by itself, establishes a causal effect of exposure.
Why Aggregation Loses Information
The example illustrates a general point: group summaries do not retain the full arrangement of individual values. To see this, imagine knowing only that a group has average \(x\) equal to 5 and average \(y\) equal to 10. Those two averages do not tell us whether the individuals with larger \(x\) also tend to have larger \(y\), smaller \(y\), or neither. Many different sets of paired individual values can have the same two averages.
A brief algebraic identity makes the distinction precise. Let group \(g\) contain \(n_g\) individuals, with group means \(\bar{x}_g\) and \(\bar{y}_g\). Let \(\bar{x}\) and \(\bar{y}\) be the overall individual means. The sum of cross-products across individuals can be split into within-group and between-group parts:
To verify the identity, write \(x_{ig}-\bar{x}=(x_{ig}-\bar{x}_g)+(\bar{x}_g-\bar{x})\), and write the corresponding decomposition for \(y\). Multiplying produces four terms. Within group \(g\), \(\sum_i(x_{ig}-\bar{x}_g)=0\) and \(\sum_i(y_{ig}-\bar{y}_g)=0\), so the two cross-terms sum to zero. The remaining terms are the within-group cross-products and \(n_g(\bar{x}_g-\bar{x})(\bar{y}_g-\bar{y})\). Summing over groups gives the displayed identity.
The first part on the right describes individual variation around each group's means. The second describes variation among group means. A group-level analysis sees the second kind of variation, not the first. Since the overall individual relationship combines both parts, a pattern among group means cannot determine the individual-level pattern on its own.
How to State a Conclusion at the Right Level
Before interpreting a regression relationship, ask what one observation represents and what population the question concerns. Then name the variables and the level of the comparison in the conclusion. This keeps the conclusion aligned with the information analyzed.
Ask whether each observation represents a person, a school, a region, or another unit.
Check whether they are individual values or group summaries, such as means, percentages, or rates.
Describe a relationship among regions when regions are the observations; describe a relationship among individuals only when individual-level data support it.
This discipline does not make group-level findings unimportant. Regional and school comparisons can address questions about regions and schools. It simply prevents a group pattern from being presented as if it directly described every person in those groups. The target population and intended use also matter: a result should be generalized only to the population represented by the data and the study's recruitment and measurement choices.
Common Mistakes
- Wrong reasoning: “Regions with more park area have higher activity, so residents with more park access are more active.” Correction: The regional data describe an association among regional summaries. They do not show the relationship among residents.
- Wrong reasoning: “The school with the highest mean score must have students who each score higher than students at the other schools.” Correction: A mean does not describe every student or show how individual scores are distributed or paired with other characteristics.
- Wrong reasoning: “The pooled rate answers the same question as the within-region rates.” Correction: The pooled rate combines people across regions; within-region rates compare people in the same region. State the denominator and comparison being used.
- Wrong reasoning: “A group-level association is meaningless because it cannot establish an individual-level relationship.” Correction: It can be informative for a question about groups. Its limitation is that it does not, by itself, answer a question about individuals.
Check Your Understanding
For each question, identify the level of the data and state what conclusion is supported.
- A data set records each region's average income and percentage of residents with a health condition. What is the observational unit, and what does an association describe?
- Why would a relationship between school mean attendance and mean test scores not, by itself, establish a student-level relationship?
- In the two-region example, calculate the outcome rate among unexposed people pooled across both regions.
- Explain in your own words what the ecological fallacy is.
- Can group-level data be useful even when the research question concerns regions rather than people? Explain.