Start by Asking What Was Random
In “Why Conditions Matter in Mean Inference,” we saw that randomness helps justify an inference. The next step is to look closely at a study description and identify what was randomized. Random selection and random assignment are different design features. They answer different questions, so naming one without explaining the other can lead to an overconfident conclusion.
A random sample is selected from a defined population using a chance process. A randomized experiment assigns experimental units to treatments using chance. A study might use one of these methods, both, or neither. For each method, ask what claim the randomness supports: generalizing from a sample to a population, or assessing a treatment’s effect on the units in an experiment.
This tutorial focuses on the randomness condition, not every condition for a mean procedure. Even when a study has an appropriate random process, you must still consider the other relevant conditions before completing an inference.
A Three-Question Check
When you read a study description, avoid stopping at phrases such as “the researchers used a random method.” Identify the units, the target of the claim, and the specific role of chance. A short, organized check makes it easier to decide what the design supports.
Name the population or the experimental units the researchers want to say something about. “People in the study” and “all adults in the region” are different targets.
Look for how participants or other units entered the study, and separately look for how they were assigned to treatments. A random number generator used for selection is not the same as one used for assignment.
Randomly selected units can support generalization to the population they came from. Randomly assigned treatments can support a causal comparison for the experimental units. Explain the link in context.
Descriptions often leave out details. If a study says only that researchers “surveyed 300 people,” that number tells you the sample size, not how people were selected. Do not assume the selection was random. If the method is not described, state that the randomness condition cannot be verified from the information given.
Random Selection: Who Can the Sample Represent?
Random selection concerns how units are chosen from a population. For example, if a school wants to estimate the mean time its enrolled students spend on homework, randomly selecting students from a complete list of those students can support generalizing to that school’s enrolled students. It does not, by itself, support a claim about students in every school.
A sample gathered from people who happen to be nearby is a convenience sample. A survey that people decide for themselves whether to answer is a voluntary-response sample. Neither method gives the researcher the same chance-based selection process as a random sample. People who are easier to reach or more motivated to respond may differ from those who are missed or do not respond.
A large sample does not change how the units were selected. A thousand voluntary responses may give a detailed description of the people who responded, but the number alone does not establish that they represent the target population. As “Does a Larger Sample Reduce Bias in the Mean” explained, more observations can reduce variability when a design supports that conclusion; sample size cannot, by itself, remove selection bias.
Random Assignment: What Can the Treatment Comparison Show?
Random assignment concerns how experimental units are placed into treatment groups. Suppose participants are assigned by a chance process to use one of two study plans, and the researchers compare their outcomes. Because the assignment process helps make the groups comparable, a difference in their results can support a cause-and-effect conclusion about the plans for those experimental units.
But consider who agreed to participate. If the participants volunteered, the experiment does not automatically represent people who did not volunteer. Random assignment does not undo the way participants entered the study. It supports a different kind of conclusion from random selection: the treatment comparison may be causal for the experimental units, while a claim about a wider population is not necessarily justified.
Worked Examples
Worked Example: Homework Time at One High School
A high school has 1,200 enrolled students. A counselor uses a random-number generator to select 60 student ID numbers from the complete enrollment list, then records each selected student’s usual nightly homework time. The counselor wants to estimate the mean homework time of all students enrolled at the school. Does the study meet the randomness condition for that population claim?
Identify the target. The target population is the 1,200 students enrolled at this high school. The counselor is estimating the mean homework time for that population.
Identify what was randomized. The 60 students were selected from the enrollment list using a random-number generator. This is random selection, not treatment assignment; there is no treatment comparison in the description.
Judge the claim. The selection method gives students on the list a chance-based route into the sample, so the data come from a random sample of the school’s enrolled students. This supports generalizing the sample’s information about mean homework time to students at this school. It does not, by itself, support generalizing to students at other schools.
Conclusion. The randomness condition is supported for the stated population claim. The counselor should still check the other conditions for the intended mean inference; random selection alone does not establish that every condition holds.
Worked Example: A Study App Tested by Volunteers
A team advertises a study app online. Twenty-four students volunteer and are assigned by drawing names from a hat: 12 use the app and 12 use their usual study routine. After two weeks, the team compares the groups’ mean quiz scores. It wants to know whether using the app affects quiz scores for the participating students and whether the result applies to all students in the region.
Identify the targets. There are two claims to assess. One concerns whether the app affects quiz scores for the participating students. The other asks whether the result can be generalized to all students in the region.
Identify what was randomized. The students volunteered to join. Their assignment to the app or usual routine was randomized by drawing names from a hat. The description does not say that students were randomly selected from all students in the region.
Judge each claim. Random assignment supports a cause-and-effect comparison of the two study conditions for the experimental units. However, volunteering is not random selection from all students in the region. The treatment assignment therefore does not, on its own, justify generalizing the result to that wider population.
Conclusion. The design has a randomized experiment, which supports a causal claim for the participants if the other relevant conditions are met. It does not establish that the participants represent all students in the region. The two claims require different reasoning, even though they arise from the same study.
Worked Example: Clinic Patients Assigned to Two Routines
A clinic wants to compare the mean number of minutes patients spend completing a breathing routine. Over one week, staff invite patients arriving for appointments to join a study. The first 30 patients who agree are enrolled. A computer then randomly assigns 15 to Routine A and 15 to Routine B. The clinic wants to claim that Routine A causes a different completion time for all patients served by the clinic.
Identify the target. The proposed broad target is all patients served by the clinic. The study also has a more limited set of units: the 30 enrolled patients.
Identify what was randomized. The first patients who agreed were enrolled, so the description gives no random selection from all clinic patients. The computer randomly assigned the 30 enrolled patients to the two routines.
Judge the claim. Random assignment supports comparing the routines as treatments for the enrolled patients. It does not turn the first 30 willing patients into a random sample of all clinic patients. The study’s assignment method alone therefore does not support generalizing the result to every patient served by the clinic.
Conclusion. The design supports a causal comparison for the experimental units, subject to the other conditions for the analysis. The stated population-wide claim is not supported by random selection because no random sample of clinic patients is described. A carefully worded conclusion should keep the experimental units and the wider target population distinct.
Worked Example: An Online Survey With Many Responses
A neighborhood group posts an online survey asking residents how many minutes they spend walking each day. Within a week, 850 people have responded. The group calculates the sample mean and wants to use a one-sample t procedure to estimate the mean walking time for every adult in the neighborhood. Does the number of responses establish the randomness condition?
Identify the target. The target is the mean walking time of all adults in the neighborhood, not just the adults who completed the survey.
Identify what was randomized. The description says that the survey was posted online and residents responded. It does not describe random selection. The adults chose whether to participate, so the responses form a voluntary-response sample.
Judge the claim. The 850 responses may describe the respondents, but the large count does not show that they were randomly selected from neighborhood adults. People with especially strong interest in walking, or more time to complete the survey, might respond at different rates from others.
Conclusion. The randomness condition for generalizing to all neighborhood adults is not established. A calculator can produce a t interval from the responses, but the calculation does not make the voluntary-response sample random or justify the intended population claim.
Common Mistakes and AP Exam Tips
- Using “random” without naming the process. State whether units were randomly selected or treatments were randomly assigned, and describe the method given in the problem.
- Confusing selection with assignment. Selecting participants by chance addresses how the sample relates to a population. Assigning participants to treatments by chance addresses the treatment comparison. Full-credit reasoning identifies the right one for the claim.
- Generalizing from volunteers because they were randomly assigned. Random assignment does not make volunteers representative of people who did not volunteer. Limit the generalization unless a random selection method also supports a broader population claim.
- Assuming a large sample is random. The number of observations does not reveal how the observations were obtained. Explain the selection method; if it is absent, say that randomness cannot be verified.
- Claiming a design supports nothing when one claim is too broad. A volunteer experiment may still support a causal comparison for its experimental units. Identify the conclusion the randomization does support, then explain what it does not establish.
- Writing only “the random condition is met.” A stronger AP response names the target, the random process, and the scope of the conclusion that process supports.
A useful response pattern is: “The units were [selected or assigned] by [specific method]. This supports [population generalization or a causal comparison] for [specific target]. It does not, by itself, support [a different claim].” That wording makes clear that randomness is a feature of a design, not a label that automatically approves every conclusion.
Check Your Understanding
For each situation, identify what was randomized, if anything, and explain which claim the design can support.
- A city selects 75 names at random from its list of registered library users and records their mean visit duration. What population can the sample support a claim about?
- Twenty volunteers are randomly assigned to two exercise plans. Does that assignment make them a random sample of all adults in the city? Explain.
- A school surveys the first 40 students who enter the cafeteria and wants to estimate the mean number of hours all students sleep. Identify the randomness concern.
- A report says that 500 customers completed a product survey but does not explain how they were recruited. What can you conclude about the randomness condition from the description?
- A randomized experiment uses volunteers and randomly assigns treatments. State one type of conclusion the assignment can support and one broader claim it does not establish by itself.