Read the Study Description, Not Just the Headline
A news headline might say that a new study “shows that morning exercise boosts focus” or “reveals what local shoppers prefer.” Before accepting a claim, ask what information the study actually collected. Did researchers record exercise, ask people how focused they felt, give participants a test, or use some other measure? The details of the data collection determine what the study can describe.
In Choosing Between a Survey, Observational Study, and Experiment, we learned to distinguish data-collection methods by what researchers do. This tutorial uses that background to read short study summaries critically. The goal is not to decide whether a study is good or bad based on a few sentences. It is to list what the summary tells us was measured, how the information was obtained, and who obtained it—and to notice what remains unknown.
A news summary is not always a complete description of a study. It may omit the exact question wording, measurement units, dates, or identity of the people who recorded the data. An important skill is to leave those gaps visible rather than quietly filling them with assumptions.
A Four-Part Audit: Who, What, How, and By Whom?
Start with the people or things that supplied the data. These are the observational units. They might be people, classrooms, devices, or measurements taken at different times. Then identify the variables: the characteristics recorded for each unit. A variable can be categorical, such as a selected response, or numerical, such as minutes spent exercising.
Next, describe how each variable was measured. A summary might say that people completed an online questionnaire, that researchers read values from a device, or that staff used existing attendance records. Be as specific as the account allows. “Researchers measured sleep” is less informative than “participants reported their usual sleep duration in an online questionnaire,” but do not add that specificity unless the summary supports it.
Finally, identify who collected or recorded the information. This could be the study’s research team, participants themselves, school staff, or an automated device. The person who wrote the news article is not necessarily the person who collected the study data. If the summary does not name the data collectors, write “not stated.”
It can also help to note the population or setting the article mentions, and when or where data collection took place. These details provide context, but they do not replace the four main questions. In Defining the Population of Interest, we learned to describe a population with clear boundaries. When a news summary gives only a vague phrase such as “young people,” do not invent an age range or location that it does not provide.
Separate What Is Reported From What Is Inferred
A useful audit has three labels. Reported means the summary directly says the detail. Implied means the detail is a reasonable reading of the wording, but the summary does not state it fully. Not stated means the summary provides no basis for deciding. These labels help prevent a reader from treating a guess as a study fact.
For example, “participants answered questions about their commute” reports that questions were used and that participants supplied information. It may imply self-report, but it does not tell us whether the questions were on paper, online, or asked in an interview. It also does not reveal the exact response choices, the dates covered, or whether travel time was recorded in minutes. Those specifics belong in the “not stated” column unless the summary supplies them.
A headline may also describe an idea more broadly than the study’s measured variables. “The study measured happiness” could mean the researchers used a particular questionnaire score, asked one question about mood, or recorded something else. A careful reader names the measure described, not a larger concept that the measure may only approximate. This connects to Deciding Which Variables to Measure: a variable needs a clear meaning and a method of recording it.
Identify who or what supplied observations. Do not confuse the article’s audience or subject with the study’s actual observational units.
Look for verbs such as asked, reported, recorded, tested, observed, counted, or measured. These often reveal what data were collected and how.
Write what the value or category represents. Include units or response categories only if the summary gives them.
Distinguish participants who reported their own information from researchers, staff, or devices that recorded it. Mark the collector as not stated if necessary.
Record important details the summary leaves unclear. Do not infer a particular instrument, question wording, or data collector without evidence.
Worked Example: A News Brief About Commute Time
A fictional news brief says: “A team at North Valley College surveyed 180 students about their trips to campus. Students reported how they usually traveled and how many minutes their one-way trip took. The team found that students who usually biked reported shorter trips than students who usually rode the bus.” Audit the description by listing the units, measured variables, collection method, and data collectors. Also identify details that are not stated.
The observational units are the 180 students surveyed. The summary does not say whether they were a random sample of all North Valley College students, so the audit should not claim that they represent every student there.
| Audit question | What the summary supports |
|---|---|
| Who or what was observed? | 180 students who took part in the survey. |
| What was measured? | Usual travel method, a categorical variable, and reported one-way commute time in minutes, a numerical variable. |
| How was it measured? | Students were surveyed and reported their usual travel method and trip duration. The precise survey format and questions are not stated. |
| By whom? | The summary says a team at North Valley College conducted the survey. It does not identify which team members administered it or whether students entered answers themselves. |
The comparison in the brief is about reported trip times for students grouped by their usual travel method. The summary does not give exact times or say whether the comparison uses means, medians, or another summary. It also does not state when the survey occurred, how “usually” was defined, or whether the researchers checked travel time against an independent record.
A careful audit does not rewrite the result as “biking causes shorter commutes.” The account describes measured travel method and reported time, but this brief alone does not tell us enough about how groups were formed or what other factors might differ. Our task here is to identify the data and collection details the summary actually provides, not to add a stronger conclusion.
Headlines and Measurements May Not Match Exactly
A study summary can use a broad word in its headline while the methods describe a narrower measurement. For example, “stress” might be represented by a response to one survey item, a score on a questionnaire, or a recorded physical measurement. Those are not automatically interchangeable. When auditing a description, state the operational detail provided—for example, “a score on a 12-item questionnaire”—rather than treating the broad topic as if it were measured directly in every possible sense.
Similarly, a news story may present a claim about a whole population when the summary only describes people who participated. In Generalizing Results to a Population, we learned that the method of selecting a sample matters for generalization. For this audit, record the stated group and selection details; if the summary does not explain how participants were chosen, say so rather than assuming a random sample.
A source can also be unclear about the collector. “Researchers recorded activity” indicates that the study team recorded something, but may not tell us whether a researcher watched participants, a sensor captured movement, or a software system stored readings. Write the broad detail that is supported, then list the unresolved method. This is more useful than choosing one plausible method and presenting it as fact.
Worked Example: A Summary About Garden Soil
A fictional local article reports: “For a neighborhood garden project, volunteers collected soil from 24 planting beds. A lab tested each sample for acidity and reported a pH value. The article says the older beds had lower pH readings.” Identify the observational units, variables, collection steps, and data collectors. Point out at least two details that the summary leaves open.
The observational units for the reported measurements are soil samples from 24 planting beds. The article links each sample to a bed, so the beds are also the groups being compared. The measured variable is pH, a numerical measurement of soil acidity. The summary does not specify the number of samples taken from each bed or whether the reported pH is based on one reading or several.
The volunteers collected soil, and a lab tested the samples. Thus the summary identifies different roles: volunteers gathered the material, while lab personnel or lab equipment produced the pH readings. It does not name the volunteers or lab workers, and it does not say whether the lab used an automated instrument or another method.
The summary also leaves the definition of “older beds” unclear. It does not state the age cutoff, when samples were collected, or whether the beds were chosen randomly. A complete audit preserves these uncertainties. It would be inaccurate to claim, for example, that the lab randomly selected 24 beds or that every bed in the neighborhood was sampled.
A concise audit might say: “The study measured pH in soil samples associated with 24 planting beds. Volunteers collected the soil and a lab tested the samples; the testing instrument, sampling locations within each bed, and definition of ‘older’ are not stated.” This sentence answers the main questions while clearly marking the limits of the account.
Distinguish the Study Team From the News Source
News summaries have at least two possible sets of people: those involved in the study and those who reported the story. A journalist may interview a researcher and summarize a published project. That journalist is responsible for the news account, but usually did not collect the study’s data. If the article names only a university or research group, do not assume that every person in that organization collected data.
Look for phrases such as “researchers asked,” “participants wore,” “school staff recorded,” or “the team reviewed records.” They indicate different data sources and roles. Participants may supply self-reports; staff may transcribe existing information; devices may automatically register measurements; and researchers may observe behavior. A study can use more than one method, so list each relevant variable alongside its source rather than forcing all measurements into one category.
The “by whom” question sometimes has a limited answer. If a summary states only that “a survey was conducted,” the method is reported but the person or group who administered it may not be. If participants filled in the survey themselves, they supplied their answers, but the account may still omit who distributed it or entered the responses. State the distinction as precisely as the evidence permits.
Worked Example: A School Library Use Report
A fictional district newsletter says: “Researchers reviewed library-entry records for 12 middle schools during October. They counted visits by grade level and compared the totals with the previous October. The report said visits increased in most schools.” Create a study-description audit and identify what cannot be determined from the summary.
The observational units could be individual visits if each recorded entry is counted, while the summary also organizes results by school, grade, and month. Because the account does not fully explain its analysis unit, a cautious audit should describe both levels: the records contain library entries, and the reported comparisons concern visit totals for schools or grade levels.
The measured information includes entry records, grade level, school, and month or year of the recorded visit. The researchers reviewed existing records and counted visits; this is a records-based collection method rather than a survey of students. The summary identifies researchers as the people who reviewed the records, but does not name them or explain who originally recorded each library entry.
We cannot tell whether every library visit was recorded correctly, whether the same counting rules were used in both Octobers, or what “increased in most schools” means numerically. The summary also does not tell us whether the researchers counted unique students or total visits; those are different variables. A student who visits repeatedly contributes several visits to a total, unless the method specifically counts each student once.
A careful audit therefore avoids saying that “more students used the library.” The summary reports increased visit counts, not necessarily an increase in the number of distinct students. Identifying this difference is part of reading the measured variable accurately.
Common Mistakes and AP Exam Tips
- Repeating the headline instead of naming variables. “The study measured wellness” is vague if the summary says participants answered a question about sleep or completed a particular score. A strong answer names the reported measure and its units or categories when given.
- Inventing missing methods. “The researchers interviewed participants” is not supported if the account only says “surveyed.” State that a survey was used and note that its format or exact questions are not stated.
- Confusing the news writer with the data collector. The journalist may describe the study without collecting any measurements. Identify study researchers, participants, staff, devices, or records as the summary permits.
- Leaving out who or what supplied the data. Name the observational units, not just the number in the report. “180 students” is more useful when paired with what those students reported or had recorded about them.
- Turning an unknown into a certainty. If sampling, timing, instruments, or collection responsibilities are omitted, say “not stated.” A complete answer can accurately identify uncertainty rather than resolving it by guessing.
- Claiming a broader outcome than the measurement supports. If records count visits, do not call the result a count of individual visitors unless the summary says that people were counted once.
For an AP-style response, connect each statement to a detail in the description. For example: “The students reported their one-way travel time in minutes, so the measured numerical variable was self-reported commute duration. The summary does not say whether the survey was online or administered in person.” This is more precise than saying only that the researchers “studied transportation.”
Check Your Understanding
For each fictional description, practice identifying the units, measurements, collection method, data collectors, and any important missing details.
- A news brief says that 75 hikers wore a step-counting device during a weekend trail event. The organizers later reported the number of steps for each hiker. What was measured, how, and by whom? Which collection detail remains unclear?
- A report says that employees answered questions about their preferred work schedule using an online form. Identify the observational units and variable, and state one fact about the questions that cannot be known from this sentence.
- A summary says a team counted bicycle trips using a counter on a bridge. Who or what supplied the data? Does the summary establish how many distinct cyclists used the bridge? Explain.
- A headline says a new program “improved confidence,” while its summary states that participants gave a confidence rating before and after a workshop. What variable was measured, and what measurement detail would make the description more precise?
- A local article reports that researchers reviewed clinic records but does not name the people who first entered the information. How should an audit describe who collected or recorded the data?