From a School Question to a Data-Collection Plan
A statistical question is useful only if we can say what information would answer it. In Writing Investigative Questions That Data Can Answer, we learned that a statistical question anticipates variation across observations. In Defining the Population of Interest and Deciding Which Variables to Measure, we learned to specify the group and the information to collect. Now we combine those choices into a short, workable plan for a school question.
A plan is not a result. It does not say what the data will show, and it does not guarantee that a study will be free of bias. It states who or what the investigation is about, what will be recorded, and how the information will be obtained. The plan should be brief enough to follow, but specific enough that another person can understand what to do.
A reliable way to draft one is to move through four connected decisions: state the question, bound the population, define the variables, and choose the method. Then check that the method can actually produce the information the question asks for. This alignment check is the key skill: a plan can have a reasonable method and still fail if it measures the wrong thing or studies the wrong group.
A Compact Planning Sequence
Make clear what you want to describe, compare, or investigate. Avoid a question that relies on a vague idea such as “better” unless you specify what outcome would represent better.
Give clear boundaries, such as which students, school, and time period. Identify what one row of data represents: a student, a class, a school day, or another unit.
For each unit, state exactly what will be recorded. Specify units for numerical variables or the categories for categorical variables. As in Deciding Which Variables to Measure, use definitions that can guide consistent recording.
In an observational study, record existing conditions without assigning treatments; a survey is one way to collect information in such a study. Use an experiment when the question asks about the effect of an imposed treatment.
State how participants or records will be identified, when data will be collected, and who will collect them when those details matter. Confirm that the recorded variables can answer the question as written.
The population is the group the question is about; it is not automatically the same as the people who actually provide data. A plan might target every student enrolled at a school during a particular semester but collect information from a random sample of those students. In Census Versus Sample, we distinguished collecting information from everyone in a population from collecting it from a subset. Whichever approach is proposed, say what group it covers.
Likewise, do not list a topic as if it were already a variable. “Sleep” could mean hours slept last night, usual hours on school nights, or a response category about sleep quality. A plan needs one clear measurement definition. The method must also fit: a question about students’ reported sleep habits calls for asking students, while a question about recorded tardiness could use attendance records.
Worked Example: How Long Is the One-Way Trip to School?
A student council wants to answer: “How long does it usually take students at Cedar Grove High School to get to school?” Write a short plan that states the population, variables, and method.
First, bound the population. “Students” could refer to students from several schools or to graduates as well as current students, so the plan specifies current enrollment and a time period. The observational unit is one student. The variable should measure travel duration consistently, rather than mixing one-way and round-trip times.
| Plan element | Specification |
|---|---|
| Population | All students enrolled at Cedar Grove High School during the fall semester. |
| Observational unit | One enrolled student. |
| Variables | Usual one-way travel time to school on a school day, reported in minutes; usual main travel mode, recorded as a category such as walking, bicycle, car, or bus. |
| Method | Use the current enrollment roster to select a random sample of students. Ask each selected student the same clear survey questions about a typical school-day trip. |
This is a survey because students can report their usual travel time and travel mode. The plan measures two variables, either of which could describe the group; collecting both also allows the council to explore how reported travel time varies by mode. It does not claim that travel mode causes a difference in travel time.
A complete short plan could read: “We will study all students enrolled at Cedar Grove High School during the fall semester by selecting a random sample from the enrollment roster. Each selected student will report their usual one-way school commute time in minutes and their usual main travel mode on a brief survey.” This states the population, measurements, and collection method without pretending the data have already been collected.
Choose the Method to Fit the Question
In Choosing Between a Survey, Observational Study, and Experiment, we compared the purposes of these methods. When planning, focus on what the question asks and what information is realistically available. A question about an opinion or a person’s usual routine often calls for a survey. A question about existing behavior or recorded outcomes may be answered by an observational study. A question asking whether an intervention changes an outcome may call for an experiment, if assigning the intervention is appropriate and ethical.
A method should not be selected just because it sounds more scientific. If a school asks whether students who bring lunch spend a different amount of time eating than students who buy lunch, researchers can record existing lunch choices and times. They should not claim that lunch source causes a difference merely because the groups’ times differ; as explained in Association Versus Causation in Study Conclusions, an observational study alone does not establish cause and effect.
Be explicit about the source of information. “Measure study habits” leaves open whether students will report their habits, researchers will observe them, or the school will use records. “Ask each selected student how many days they studied biology for at least 20 minutes during the previous week” is more specific. The method and the variable now fit together.
Worked Example: Study Time and Biology Quiz Scores
A biology teacher asks: “Among students in this semester’s Grade 10 biology classes, how is study time related to the next unit quiz score?” Propose a plan that can investigate the relationship without assigning students to study.
The question is about a relationship between two variables, so the plan needs both variables recorded for each student. Its population is not all high-school students; it is the clearly bounded group named in the question. One student is the observational unit because study time and a quiz score will be paired for each student.
| Plan element | Specification |
|---|---|
| Population | All students enrolled in the Grade 10 biology classes at the school during the current semester. |
| Observational unit | One student enrolled in those classes. |
| Variables | Number of minutes the student reports studying biology during the seven days before the quiz; the student’s score on the next unit quiz, recorded as points earned out of the stated total. |
| Method | Ask students to report study time before the quiz, then use the teacher’s score records to obtain each student’s quiz score. Pair the two values by student while protecting private information. |
This is an observational study: the teacher records study time and quiz scores but does not assign students different amounts of study. The two variables are measured on the same observational units, so the data can show whether study time and score are associated in this group. The study cannot, by itself, establish that additional study time caused a higher or lower score. Other variables, such as prior preparation, could be related to both.
The plan is more useful than “compare studying with grades” because it defines the time window, specifies how both variables are obtained, and identifies the population. It also avoids an imprecise variable such as “effort,” which different students or teachers might interpret in different ways.
Plans for Questions About an Intervention
If the school question asks whether a change makes a difference, consider whether an experiment is suitable. In Experiments and Why They Can Show Cause, we learned that an experiment imposes treatments and measures responses; random assignment can support a cause-and-effect conclusion. A plan should identify the treatment, comparison condition, response variable, and experimental units. Keep the treatment and outcome specific enough that someone could implement and record them consistently.
For example, “Does a new review format improve learning?” needs more detail. Which review format? What will represent learning? Will students be assigned to conditions, or will the school only compare classes that already use different formats? Those decisions determine whether the plan proposes an experiment or an observational study. Ethical and practical limits matter too; as discussed in Ethical Considerations in Data Collection, participation and treatment choices must be handled responsibly.
Worked Example: Testing a New Vocabulary Review Routine
A language department asks: “Does a five-minute retrieval-practice routine improve vocabulary quiz performance for Grade 9 students?” Write a plan that tests the effect of the routine while keeping the outcome clear.
The population is all Grade 9 students at the school during the semester in which the routine is tested. The experimental units are the students assigned to conditions. The treatment is using the five-minute retrieval-practice routine before each of four vocabulary lessons; the comparison condition is using the department’s usual review activity for those lessons. The response variable is each student’s score, in points, on the same vocabulary quiz given after the four lessons.
The department could randomly assign participating students to the two review conditions, teach both conditions over the same four lessons, and give all students the same quiz. It would record each student’s assigned condition and quiz score, then compare the groups’ score distributions. The plan should also consider whether students assigned to different routines can be kept from sharing materials in a way that changes their assigned condition.
This is an experiment because the review routine is deliberately assigned. Random assignment helps make the groups comparable, but it does not automatically make them a random sample of all Grade 9 students. The conclusion’s scope depends on how participants are recruited and assigned, as discussed in Scope of Inference: Four Combinations. The plan should not promise a particular result; it only describes how data could be collected to evaluate the question.
Common Mistakes and What a Strong Plan Includes
- Using a population label that is too broad. “Students” does not state which school or time period. A strong plan identifies a bounded group, such as all students enrolled at a named school during a specified semester.
- Confusing the population with the sample. The population is the group the question concerns; a sample is the subset that provides data. State both when the plan uses a sample, and explain how the sample will be identified or selected.
- Naming a topic instead of defining a variable. “Well-being” or “study habits” is not a complete measurement definition. Say exactly what value or response will be recorded for each unit.
- Choosing a method that cannot answer the question. Attendance records cannot directly reveal students’ reasons for being absent. If reasons are needed, plan a suitable survey question rather than treating a count as an explanation.
- Implying cause from an observational plan. Recording existing choices or behaviors can show an association, but it is not the same as assigning a treatment. Use an experiment when the question concerns an imposed change and an ethical, practical experiment is possible.
- Leaving the collection process vague. “Get data about students” does not say who will be asked, what records will be used, or when measurements will be made. Include the essential steps another person would need to carry out the plan.
A full-credit response does not need a long proposal. It needs a clear match. For example: “The population is all students enrolled at Westview High School during the spring semester. We will randomly select students from the enrollment roster and ask each to report their usual one-way commute time in minutes. This survey will measure reported commute time for each selected student.” The group, variable, and method are all explicit.
Check Your Understanding
For each question, sketch a short plan that names the population, observational unit, variables, and method. Where relevant, explain what the method can and cannot establish.
- A principal wants to know how many minutes students spend waiting for the late bus after school. What population and variable should the plan specify, and how might the information be collected?
- A school nurse asks whether students who bring reusable water bottles drink more water during the school day. Name two variables and propose a method that records existing behavior without assigning students to bring a bottle.
- A teacher asks whether a new way of organizing group review changes the next test score. What additional treatment and response details should a plan define, and what type of study could test the effect?
- A student proposes, “We will study whether school is stressful by asking some students if they feel stressed.” Identify two details that would make the population and variable more precise.
- A plan uses cafeteria records to answer whether students like the new lunch menu. Explain why the recorded data may not measure the variable in the question, and suggest a better collection method.