Why the Study Design Matters
A chi-square calculation can compare observed counts with expected counts, but the calculation alone does not tell us whether the data support conclusions about a population or an experiment. That depends in part on how the observations were obtained. In this tutorial, we focus on the random condition: the requirement that the study use an appropriate random sample or, for an experiment, random assignment.
As covered in “Chi-Square Test for Independence: Purpose and Setting,” an independence test uses one sample in which each individual is classified according to two categorical variables. A homogeneity test, introduced in “Chi-Square Test for Homogeneity: Purpose and Setting,” compares the distribution of one categorical response across groups or populations. These procedures use the same kind of chi-square calculation, but their study designs—and the role of randomness in those designs—can differ.
The random condition is only one part of checking whether a chi-square procedure is appropriate. You must also consider whether observations are independent and whether the expected counts are large enough, as discussed in earlier tutorials. The 10% condition for sampling without replacement is addressed in the next tutorial.
Random Sampling for a Test of Independence
A chi-square test of independence begins with one sample from a population. Each sampled individual is classified according to two categorical variables—for example, preferred type of exercise and usual time of day for exercising. The random condition asks whether that one sample was selected randomly from the population about which the researchers want to draw a conclusion.
A random sample supports generalizing conclusions from the sample to the population from which it was drawn. It does not establish that one variable causes the other. In an independence study, the researcher observes and records the two variables rather than assigning individuals to categories. Even a well-designed random sample can provide evidence of an association, but an association found in an observational study does not by itself establish causation.
The phrase “random sample” should refer to the selection process, not to a sample that merely seems varied or representative. If researchers ask people who happen to be nearby, invite volunteers to respond to an online poll, or select participants because they are easy to reach, the sample is not a random sample just because it includes people with different characteristics.
Randomness for a Test of Homogeneity
A test of homogeneity compares the distribution of one categorical response across two or more groups. The groups can come from separate populations, or they can be treatment groups in an experiment. Accordingly, the random condition can be supported in two different ways.
- Separate random samples: Researchers independently select a random sample from each population or group they want to compare. This design supports generalizing the results to those populations, subject to the other conditions.
- Random assignment: Researchers assign experimental units to treatment groups using a chance process, then record a categorical response for each unit. This design supports a cause-and-effect conclusion about the treatments for the experimental units in the study, subject to the other conditions.
These methods of introducing randomness are not interchangeable. Random sampling concerns which individuals enter the study. Random assignment concerns which treatment each experimental unit receives. A study can use one, both, or neither.
For example, assigning a group of volunteers randomly to two treatments can help determine whether the treatments caused a difference in responses among those volunteers. It does not, by itself, justify claiming that the same result applies to all people in a broader population. To support both a causal conclusion and broad generalization, a study would need both random assignment and an appropriate random sample from that population.
A Design Checklist for the Random Condition
Before using a chi-square procedure, describe the design in ordinary language. Then connect that description to the procedure. The following sequence helps keep the two chi-square settings straight.
Is there one sample classified by two variables, separate samples from different populations, or an experiment with treatment groups?
For a sample, check how individuals were selected. For an experiment, check whether experimental units were assigned to treatments at random.
Random sampling can support generalization to the population sampled. Random assignment can support a causal conclusion about the treatments.
Randomness does not guarantee independent observations or sufficiently large expected counts. Those requirements still need attention.
A good condition statement names the design detail. “The data are random” is not enough. Say who or what was randomly selected or assigned, and explain how that supports the procedure being used.
Worked Example: A Random Sample for Independence
Worked Example: A Random Sample for Independence
A fictional city recreation department uses a computer to select 240 residents at random from its current resident list. Each selected resident reports their usual way of getting to work—walking, cycling, or driving—and whether they usually work a daytime or evening schedule. The department wants to investigate whether commute method and work schedule are associated among city residents.
Identify the design. This is one random sample of residents. Each resident is classified according to two categorical variables: commute method and work schedule. That is the setting for a chi-square test of independence.
Check the random condition. The residents were selected at random from a list of city residents. If that list adequately represents the intended population, the design supports using the sample to draw conclusions about an association between commute method and work schedule among city residents. The random condition is met based on the stated selection process.
Keep the conclusion within the design. The department did not assign residents to commute methods or work schedules. Therefore, even if a later test found convincing evidence of an association, the result would not show that a work schedule causes a particular commute method or that a commute method causes a particular work schedule.
This assessment addresses only the random condition. Before carrying out the test, the department would also need to check the other chi-square conditions, including the expected counts.
Answer: The random condition is supported because the study uses one random sample from the population of interest. The design can support generalizing an association to that population, but it cannot establish a cause-and-effect relationship.
Worked Example: Separate Random Samples for Homogeneity
Worked Example: Separate Random Samples for Homogeneity
A fictional environmental research team wants to compare how residents in three regions sort household food scraps: compost them, place them in regular trash, or use another method. The team selects a separate random sample of 150 households from each region and asks one adult in each household which method is usually used.
Identify the design. There are three separate samples, one from each region, and one categorical response—usual food-scrap method. The question asks whether the response distribution is the same across regions, so the setting is a chi-square test of homogeneity.
Check the random condition. The team selected a random sample from each of the three regional populations. This supports generalizing the response distributions to those populations, assuming the sampling frames and selection procedures are appropriate. Separate random samples are the relevant form of randomness for this design.
Do not confuse the groups with treatments. The researchers did not assign households to regions or assign a food-scrap method. Region is a naturally occurring grouping variable, so the study is observational. Random sampling supports population comparisons, but it does not justify claiming that living in a particular region causes a household to choose a particular method.
Before using the chi-square test, the team must still check that each household contributes appropriately to the data and that expected counts satisfy the chi-square condition. The fact that each sample was random does not settle those separate questions.
Answer: The random condition is met for a test of homogeneity because the researchers took separate random samples from the three populations. The design supports comparing and generalizing the response distributions, but not making a causal claim about region.
Worked Example: Random Assignment in a Homogeneity Test
Worked Example: Random Assignment in a Homogeneity Test
A fictional school nutrition team studies whether the type of reminder affects students’ lunch-order response. The team recruits 180 students from one school and randomly assigns 90 to receive a text reminder and 90 to receive a printed reminder. Each student’s response is recorded as ordered a lunch, did not order a lunch, or other.
Identify the design. This is an experiment with two treatment groups and one categorical response with three categories. Comparing the response distributions across the treatments is a chi-square test of homogeneity.
Check the random condition. The students were randomly assigned to the two reminder types. Therefore, random assignment supports a cause-and-effect conclusion about the reminders for the students in this experiment, if the other conditions are met.
State the limit on generalization. The students were recruited from one school; the description does not say they were randomly selected from a wider population. Random assignment does not make them a random sample. The experiment alone does not justify generalizing the result to all students or to students at other schools.
In a full test, the team would also verify the other chi-square conditions. Random assignment meets the random condition for this experimental comparison; it does not automatically ensure that every other condition is satisfied.
Answer: Random assignment supports investigating whether reminder type caused a difference in response distributions among the participating students. Without random sampling from a broader population, it does not support generalizing that effect to all students.
Worked Example: A Nonrandom Sample Is Not Made Random by Its Size
Worked Example: A Nonrandom Sample Is Not Made Random by Its Size
A fictional museum posts a poll on its website asking visitors whether they prefer audio guides, printed guides, or no guide. The museum receives 2,000 responses and compares guide preference across age categories. The poll is open to anyone who visits the website.
Identify the design. Respondents are classified by two categorical variables, so the table may resemble an independence-test setting. But the poll is a voluntary-response sample: people choose for themselves whether to answer.
Check the random condition. The museum did not randomly select visitors. A large number of responses does not change the selection method, and it does not establish that the respondents represent all museum visitors. Therefore, the random condition is not met for using this poll to make a formal chi-square inference about the population of museum visitors.
The museum can describe the responses it received—for example, the counts or percentages in its poll. It should not treat a small chi-square p-value as evidence that the same relationship exists among all museum visitors, because the random condition is not supported.
Answer: The poll does not meet the random condition. Its 2,000 responses can be summarized descriptively, but the voluntary-response design does not justify a population-level chi-square inference.
Common Mistakes and AP Exam Tip
- Calling any sample random: A sample is random because of how individuals were selected, not because it is large or appears diverse. Full-credit wording identifies the random selection method.
- Mixing up sampling and assignment: Random sampling selects participants; random assignment allocates experimental units to treatments. Explain which one the study actually used.
- Using random assignment to claim broad generalization: Assignment supports causal conclusions about treatments, not representativeness of a larger population. Generalization requires an appropriate random sample.
- Using random sampling to claim causation: A random sample can support generalization, but an observational association does not show that one variable caused the other.
- Assuming the random condition is the only condition: A random sample or random assignment does not guarantee independent observations or adequate expected counts. Check those requirements separately.
- Writing only “random condition met”: Name the random process and connect it to the intended conclusion. For example, say that the individuals were randomly selected from the named population, or that units were randomly assigned to the treatments.
Check Your Understanding
For each situation, identify the source of randomness and explain what conclusion the design can support.
- A random sample of residents reports both its preferred news source and whether it usually reads news in the morning or evening. What chi-square setting fits, and what does the random sample support?
- Researchers randomly select households from each of four towns and record each household’s main source of drinking water. What form of randomness supports a test of homogeneity?
- Students volunteer for a study and are then randomly assigned to one of two reminder types. What can random assignment support, and what does volunteering not establish?
- A large online poll asks visitors to choose one of three options. Does the large number of responses meet the random condition? Explain.
- Why does random sampling not, by itself, allow a researcher to claim that one categorical variable causes another?