Why the Sampling Frame Matters
An empirical question refers to a population, but data are usually collected from a list, register, database, or set of locations that can be used to find units. That operational source is the sampling frame. If it does not correspond to the population named by the question, even a carefully conducted survey can leave out relevant units or include units that do not belong. The resulting problem is called coverage error.
The distinction matters because an estimate calculated from observed records may describe the frame’s units rather than the population the researcher intended to study. A university email list, for example, may cover students with active accounts but not every student enrolled at the reference date. The issue is not necessarily that any recorded answer is inaccurate; the list and the intended population may simply differ.
A useful description specifies more than a broad label such as “workers” or “businesses.” It identifies the unit (person, household, firm, or another entity), eligibility criteria, geographic scope, and reference time. For example, “people who were employed and resident in the city on 1 April” is more precise than “city workers.” A frame built from workplace payroll records might identify people working in the city, including commuters, rather than people residing there.
Undercoverage, Overcoverage, and Mismatch
Undercoverage occurs when one or more target-population units have no corresponding representation in the frame. Examples include renters omitted from a property-owner list, informal businesses absent from a registration database, or households without internet access omitted from an online-only survey frame. Undercoverage can affect a small or large share of the target population; its importance also depends on whether the omitted units differ on outcomes relevant to the study.
Overcoverage occurs when the frame includes units that are not members of the target population. A register may retain businesses that have closed, include addresses outside the study area, or list people who no longer meet an eligibility rule. An entry is not automatically an error just because it is inconvenient to contact: the question is whether the unit belongs to the defined target population at the relevant time.
A frame can have both problems. It may omit eligible units while also including ineligible ones. More generally, a frame–population mismatch can arise from different definitions of the unit, eligibility, place, or time. A household survey frame built from utility accounts, for example, does not automatically align with a target population of households: accounts and households are different kinds of units, and their counts need not match.
| Diagnosis | Question to ask | Illustrative case |
|---|---|---|
| Undercoverage | Which target units cannot be found in the frame? | Eligible tenants are missing from an owner-based address list. |
| Overcoverage | Which frame units do not meet the target definition? | A business register still lists firms that closed before the reference date. |
| Definition mismatch | Do the frame and target use the same unit and eligibility rules? | A file of utility accounts is used to represent households. |
| Geographic or time mismatch | Do both refer to the same place and period? | A current address list is used to study residents at a date several years earlier. |
Worked Examples: Diagnosing Coverage in Practice
Worked Example: An Online Survey of University Students
A university wants to estimate average weekly paid work hours among all 1,200 students enrolled on 1 October. Its survey invitation goes only to the 900 students with an active university email account on that date. Assume all 900 listed students are enrolled, and that the other 300 enrolled students have no active account. Diagnose the frame and calculate the share of the target population omitted.
Target and frame. The target population is the 1,200 enrolled students. The frame is the list of 900 students with active accounts. Because every listed student is eligible but 300 eligible students are absent, this is undercoverage, with no overcoverage under the stated assumptions.
The omitted share of the target population is
Thus, 25% of the target population is not represented on the frame. This calculation describes frame coverage; it does not tell us whether the omitted students work more or fewer hours. If, hypothetically, the frame students averaged 8 paid work hours per week and the 300 omitted students averaged 14, the target-population mean would be
The frame-group mean of 8 hours would then be 1.5 hours per week below the target mean. That direction follows from the hypothetical group means, not from undercoverage alone.
Worked Example: An Administrative Register of Businesses
Suppose the target is all active businesses operating in a district on 31 December. An administrative register contains 1,000 records. A review establishes that all 900 target businesses appear in the register, while the other 100 records refer to businesses that closed before 31 December. The average annual sales recorded for the 900 eligible businesses is $500,000, and the average for the 100 closed businesses is $200,000. What is the coverage issue, and what would the mean across all register records be?
All target businesses are represented, but the frame also includes 100 units that do not meet the target’s active-on-31-December rule. This is overcoverage. The mean across all 1,000 records is
The target-population mean is $500,000, whereas the mean over the uncorrected frame records is $470,000, a difference of $30,000 per business. Here the extra records pull the frame mean downward because their sales average is lower. If their average were higher instead, the direction could reverse. An administrative database is not automatically a suitable frame merely because it is comprehensive for its own administrative purpose.
Worked Example: A Household Target and a Utility-Account Frame
A city wants to study households that lived within its boundaries on 1 July. The frame contains 460 utility-account records. Investigators determine that 430 records correspond to distinct target households, 30 refer to vacant or commercial premises, and 70 target households have no account in the frame. Assume these classifications are exhaustive. Describe both coverage errors and calculate the relevant shares.
The 70 target households without a frame record indicate undercoverage. Relative to the 500 target households identified by the review (430 represented plus 70 omitted), the omitted share is
So 14% of target households are missing from the frame. The 30 vacant or commercial premises are not target households, so they indicate overcoverage. Their share of the 460 frame records is
or about 6.52%. The two percentages use different denominators: undercoverage is expressed relative to the target population, while overcoverage here is expressed relative to the frame. The example also reveals a unit mismatch: an account is not necessarily a household. Treating accounts as if they were households would obscure both errors.
Worked Example: Residents or People Working in the City?
A researcher asks about earnings among people who lived in a city on 1 April. The available administrative file lists people employed by firms with a workplace in the city. A cross-check identifies 700 people who both lived and worked in the city, 200 city residents whose jobs were outside the city, and 150 people who worked in the city but lived elsewhere. Assume these groups are correctly classified and that the target consists of the 900 city residents represented by the first two groups.
The target population is residents, so the 200 residents working elsewhere are missing from the workplace-based frame: undercoverage relative to the target. The 150 nonresident workers are in the frame but outside the target: overcoverage. Among the target residents, the omitted share is
Thus, about 22.22% of the target residents are absent from the frame. The frame has 850 people in total (\(700+150\)), of whom \(150/850\approx 17.65\%\) are not target residents. The mismatch is not repaired by having many records: the file answers a workplace-based question unless it is adjusted to match the resident-based target.
Why Coverage Can Change an Estimate
Coverage error matters when the frame and target populations differ in outcomes relevant to the estimand. The basic relationship can be shown for undercoverage in a finite population. Suppose the target has \(N\) units, \(N_c\) of them are covered, and \(N_u=N-N_c\) are uncovered. Let \(\bar{Y}_c\) and \(\bar{Y}_u\) denote the mean outcome among covered and uncovered target units, respectively. The target mean is the weighted average of those group means:
If the frame corresponds exactly to the covered units and the frame mean is \(\bar{Y}_c\), subtracting it from both sides gives
This identity shows two things. First, if no target units are omitted, the difference is zero under these assumptions. Second, omission alone does not determine the direction or size of the difference: that also depends on how the uncovered units’ outcome mean compares with the covered units’ mean. If those means are equal, undercoverage does not shift this particular mean, although the frame still fails to cover the whole target population. The identity is a diagnostic relationship, not a general guarantee that frame differences can be corrected from observed data.
Overcoverage can similarly change a frame-based summary if out-of-target units have different outcomes. Its effect depends on their number and outcome values, as the business-register example illustrates. In practice, those values may not be known without further information, which is why a clear frame audit is important before interpreting estimates.
How to Diagnose a Frame
State the unit, eligibility rule, geography, and reference time in precise terms.
Identify how records are created and what their units and dates mean; do not assume a database’s label matches the research target.
Ask which eligible target units are absent and which frame units fall outside the target. Check whether records correspond to the intended observational units.
Consider whether coverage differences could be associated with the outcome of interest, and report the scope of the population the data can directly describe.
Coverage error is distinct from other data problems. Nonresponse concerns eligible units that are approached but do not provide data; coverage concerns whether the frame represents the target units in the first place. Measurement error concerns inaccurate recorded values; a frame can cover the right units while measuring their earnings incorrectly, or measure values accurately for a frame that misses part of the target. These problems can occur together, but diagnosing one does not resolve the others.
Common Mistake
A defensible empirical description names both the intended population and the data source’s coverage limits. If the frame cannot be aligned with the target, the researcher may need a different source or a narrower, explicitly defined target. The key is not to silently treat the available records as equivalent to the population the research question originally named.
Check Your Understanding
For each question, identify the relevant target–frame distinction and explain your reasoning.
- A study targets all renters in a county, but its address list is built from property owners. What possible coverage problem should be investigated?
- A business register contains all currently active firms in a district plus records for firms that closed last year. Name the coverage problem and state the target-definition detail that determines it.
- A survey targets residents of a city, but the frame lists everyone whose workplace is in the city. Which groups might be omitted or added?
- In the finite-population identity, what determines the direction of the difference between the covered-unit mean and the target mean?
- Explain why an eligible person who is on the frame but does not answer presents a different issue from a target person who is absent from the frame.