From a Campus Question to a Sample Plan
A campus survey can be carefully worded and still give a misleading picture if the wrong people could be selected in the first place. A sound plan begins before anyone is contacted: define whose views the survey is meant to describe, decide how to identify those people, and choose a selection procedure that fits the population and the available resources.
Suppose a university wants to estimate what proportion of its undergraduate students support keeping the library open later on weeknights. The plan should make clear whether “undergraduate students” means everyone enrolled on a specified date, only students taking in-person classes, or some other group. It should also state what list will be used to draw the sample. Those decisions determine the reach of the survey; a random selection from an incomplete list cannot include people absent from that list.
A frame does not have to be a single spreadsheet. For example, a university might combine a registrar list with a separate list of eligible visiting students, then reconcile the lists to remove duplicates. What matters is whether every target unit has a clear, nonzero chance of selection and whether the selection probabilities are understood.
Specify the Population and Check the Frame
A useful plan defines the target group precisely enough that another person could decide whether a particular person belongs to it. Include a relevant time period and eligibility rules. “Students at the university” may be ambiguous: it could include graduate students, students on leave, or people who attended last term but are not currently enrolled.
Next, compare the frame with that definition. Look for three practical problems:
- Omissions: eligible people who do not appear on the list, such as visiting students missing from the registrar file.
- Ineligible entries: people on the list who do not meet the target definition, such as students who withdrew before the survey’s reference date.
- Duplicates or ambiguous records: a person listed more than once, or records that cannot reliably be matched to people.
Omissions create undercoverage: some members of the target population have no chance of selection through that frame. Removing ineligible records and resolving duplicates can improve frame quality, but it does not by itself fix omissions. If a useful complete frame is unavailable, the plan should say what is missing and narrow or qualify the population claim rather than imply that the frame covers everyone.
Worked Example: Defining the Population and Auditing a Frame
A hypothetical university wants to survey support for later library hours among all undergraduate students enrolled on October 1. It has a registrar file with 7,600 eligible records, while an enrollment audit indicates that 8,000 undergraduates meet the definition. The registrar file does not include 400 visiting undergraduates.
Target population. State it as all undergraduates enrolled at the university on October 1. This is precise about who is included and when.
Initial frame. The registrar file covers 7,600 of the 8,000 target students. Its coverage fraction is \(7600/8000=0.95\), or 95%. The remaining \(400/8000=0.05\), or 5%, have no chance of selection from that file.
Plan adjustment. If a separate, suitable list of the 400 visiting students exists, combine it with the registrar file and remove any duplicates. If no such list exists, describe the frame as covering registered non-visiting undergraduates only, or report the coverage limitation when presenting results. A random draw from the 7,600 records does not make the 400 omitted students represented.
This audit does not tell us whether the omitted students support later hours more or less often. It establishes a risk of undercoverage, not the direction or size of any resulting error.
Choose a Probability Design That Fits
A probability sampling design uses a chance mechanism to select units, with each eligible unit having a known, nonzero probability of selection. This supports a defensible description of how the sample was selected. It does not guarantee that every sample will match the population on every characteristic, and it does not prevent problems caused by frame omissions or nonresponse.
The familiar designs offer different practical advantages. A simple random sample selects units so that each possible sample of the specified size has the same chance. Stratified sampling divides the population into groups and randomly samples within each group; it is useful when representation from important groups is needed. Systematic sampling selects units at a regular interval from an ordered frame after a random start, so the order should be checked for patterns related to the interval. In cluster sampling, groups are selected at random, and units within selected groups are surveyed or sampled; it can reduce travel or contact costs when units are naturally grouped.
For a campus survey, the choice should follow the question and logistics. If class year is important and the roster records it accurately, stratifying by year can ensure that each year is included. If students are widely dispersed and contacting them individually is costly, a cluster design may be operationally attractive. If the frame is manageable and there is no strong reason to organize selection by subgroups, a simple random sample may be straightforward.
Worked Example: Proportional Stratification by Class Year
Suppose a hypothetical campus has 4,800 undergraduates on a complete, deduplicated roster: 1,200 first-year students, 1,120 sophomores, 1,240 juniors, and 1,240 seniors. A survey team plans to invite a stratified random sample of 480 students, allocating the sample in proportion to the four class-year counts.
For stratum \(h\), proportional allocation is \(n_h=n(N_h/N)\), where \(N_h\) is the population size of that stratum, \(N=4800\), and \(n=480\). The sampling fraction is \(480/4800=0.10\), so the allocation is:
The allocations sum to \(120+112+124+124=480\). The first-year share of the selected invitations is exactly \(120/480=0.25\), matching the population share \(1200/4800=0.25\). The corresponding shares for sophomores, juniors, and seniors are also exactly proportional.
Within each class-year list, use a random selection procedure to choose the specified number of students. Because these fixed allocations determine how many invitations come from each year, chance in the within-year draws cannot change the sample’s class-year counts. Other characteristics, such as major or library use, can still differ between the realized sample and the population.
Justifying Proportional Allocation
The reason proportional allocation preserves group shares is algebraic. If a stratum has \(N_h\) of the \(N\) population members, its population share is \(N_h/N\). Under proportional allocation, its sample size is \(n_h=nN_h/N\). Dividing by the total sample size gives:
Thus, when the allocations can be made exactly, each stratum’s share among selected invitations equals its share in the population. This statement concerns the selected sample’s stratum counts, not every other trait and not necessarily the completed surveys after nonresponse. If allocations must be rounded to whole people, the shares may be very close rather than exact.
Proportional allocation is not the only defensible choice. A team might deliberately sample more students from a small class year to ensure enough responses for a separate description of that year. In that case, the sample’s class-year composition will not mirror the population by design. The plan should state the reason and avoid treating unadjusted sample proportions as if they automatically reflected population proportions.
Worked Example: When a Cluster Design Fits the Fieldwork
Suppose a separate survey concerns the 1,200 undergraduates living in campus housing. A complete housing frame lists 12 residence halls, each with 100 eligible residents. The team has limited time for in-person contact and considers selecting three halls at random and surveying every eligible resident in those halls.
Target and frame. The target population is campus-housing undergraduates on the specified date. The housing roster must include all 12 halls and their eligible residents. This design would not, by itself, cover commuters, so it should not be used to make claims about all undergraduates.
Selection probability. Under a simple random selection of 3 of the 12 halls, each hall has probability \(3/12=0.25\) of selection. Since all 100 residents in a selected hall are surveyed, each resident’s chance of selection is also 0.25. The design therefore gives each of the 1,200 residents a known, nonzero selection probability.
Practical trade-off. The plan contacts people in only three locations, which may reduce travel. But the selected 300 residents are concentrated in three halls rather than spread across the housing population. If residents in the same hall tend to have similar views, that clustering can make the collected information less varied than a sample of 300 spread across all halls. The team should choose this approach for its operational advantage, not assume that selecting halls eliminates all sampling uncertainty.
Plan for Invitations, Nonresponse, and Follow-Up
The number selected is not necessarily the number who complete the survey. A plan should distinguish the invited sample from the respondents, state how and when invitations will be sent, and provide a reasonable follow-up procedure. Earlier material on nonresponse establishes that a response rate alone cannot show whether nonrespondents differ in ways relevant to the survey. Still, expected response is useful for planning how many invitations may be needed.
For a stratified survey, response rates may differ by class year. Then the completed surveys need not retain the selected sample’s class-year composition, even if invitations were allocated exactly in proportion to the population. Proportional allocation fixes the selected invitation counts; it does not force selected students to respond. A team should monitor responses by stratum, use planned reminders consistently, and report the final response pattern rather than silently treating the invitation design as the completed-sample design.
Worked Example: Projecting Completes by Class Year
Using the same campus counts as the proportional-allocation example, suppose a team plans to invite 800 students. To obtain whole-number allocations close to the population proportions, it invites 200 first-year students, 187 sophomores, 206 juniors, and 207 seniors. Assume, solely for planning, expected response rates of 70%, 60%, 55%, and 50%, respectively.
The expected number of completed surveys in each group is the invitation count multiplied by the assumed response rate:
The total expected number of completes is \(140+112.2+113.3+103.5=469\). This is an expectation under the stated assumptions, not a promise that exactly 469 people will respond. The expected class-year composition among completes is also different from the invitation composition: for example, first-year students account for \(140/469\approx0.2985\), about 29.9% of expected completes, compared with 25% of the target population.
The team should not infer that response differences necessarily cause bias in support estimates; that depends on how response is related to the survey outcome. But the unequal assumed rates are a reason to monitor participation and to avoid claiming that the completed surveys remain proportionally composed by class year.
A Practical Planning Sequence
A useful plan can be written as a sequence of explicit decisions. Each decision constrains the next one, and documenting them makes the final claim about whom the survey represents easier to evaluate.
Name the people, eligibility conditions, and reference date relevant to the survey question.
Compare the available list with the target; identify omissions, ineligible records, and duplicates, and decide how to resolve them or qualify the population claim.
Select a design that gives eligible units a known chance of selection and is workable for the question, frame, and fieldwork constraints.
Record how random selection will occur, how many units will be drawn from each group or cluster, and whether any allocations intentionally depart from population shares.
Estimate likely response for operational planning, set a consistent follow-up process, monitor response by relevant groups, and describe limitations when reporting results.
Common Planning Mistakes
- Equating the frame with the target. A registrar list is a tool for selecting people; it is not proof that every member of the target population appears on it. Check coverage against the population definition.
- Calling a sample random because selection was computerized. A random draw from a list with omissions remains unable to select omitted people. The frame and the draw both matter.
- Assuming proportional invitations guarantee proportional responses. Fixed stratum allocations determine selected invitation counts, not how many people in each stratum complete the survey.
- Choosing clusters without matching the claim to the population. A residence-hall sample can describe residents under its design; it does not automatically describe commuters or all students.
- Treating expected completes as certain. A planning response rate gives an expectation, not a fixed outcome. Actual response can be higher or lower.
Check Your Understanding
Use the definitions and planning logic in this tutorial to answer the following.
- A survey aims to describe all students enrolled on October 1, but its frame lists only students registered for in-person classes. What is the target population, and what coverage concern should be checked?
- A campus has 2,000 first-year students and 6,000 other undergraduates. A proportional sample of 400 is planned. How many first-year students should be selected, and what share of the selected sample will they form?
- Why can a sample selected proportionally by class year have a completed-survey composition that is not proportional by class year?
- A team randomly selects 2 of 8 residence halls and surveys everyone in each selected hall. If all halls have equal numbers of eligible residents, what is each resident’s probability of selection?
- What does an estimated response rate of 60% tell a survey team for planning, and what does it not establish about nonresponse bias?