Limitations Should Be Specific and Relevant
A good regression report does more than present a fitted line and its statistics. It also tells readers what boundaries matter when interpreting the analysis. In “Writing a Conclusion Without Overclaiming,” you learned to anchor conclusions in the observed cases and setting. Here, the focus is how to identify and describe four common limitations: extrapolation, unusual points, sampling, and confounding.
A limitation is useful when it clarifies what a particular result can support. For example, a prediction beyond the observed \(x\)-range raises a different concern from a sample of volunteers or a possible confounding variable. Simply adding “the study has limitations” does not tell a reader what to be cautious about. Name the issue, connect it to the data or model, and explain how it affects the claim.
A limitation does not automatically make the regression useless. It may instead narrow a conclusion: the association is supported for the observed cases, but not necessarily for a broader population; the line describes the data in the observed range, but not beyond it; or a strong association is present, but the analysis does not establish its cause.
A Four-Part Limitation Audit
Before writing a report, ask four questions. These are not four required disclaimers for every analysis. They are prompts to help you find the limitations that matter to the conclusion you plan to make.
Compare the requested \(x\)-value with the minimum and maximum \(x\)-values used to fit the line. A prediction outside that range is extrapolation.
Look for points with unusually large residuals or unusual \(x\)-positions. If a point could affect the conclusion, compare the fit with and without it, as in “Testing Influence by Removing a Point.”
Ask who or what is represented by the data, and whether the sampling method supports applying the finding to a larger group.
For observational data, consider whether another variable could be related to both \(x\) and \(y\), helping explain their association.
The audit is most effective when each answer leads to a specific reporting sentence. For example, “the sample consisted of volunteers from one clinic” is more informative than “the sample may be biased.” A sentence should state the limitation and its consequence without suggesting more than the available evidence shows.
| Limitation | Evidence to check | What the report can qualify |
|---|---|---|
| Extrapolation | Requested \(x\)-value compared with the observed \(x\)-range | Whether a particular prediction is supported by the data |
| Unusual point | Residual, \(x\)-position, and with-versus-without fit comparison | How sensitive the fitted relationship is to that observation |
| Sampling | Who was selected, how they were selected, and who is missing | Which population or setting the results describe |
| Confounding | Whether another variable could relate to both \(x\) and \(y\) | Whether an association can be interpreted as a causal effect |
Describe Each Limitation Without Overstating It
Extrapolation. As explained in “Reliability Within the Range of Data” and “Writing an Extrapolation Critique,” compare the requested explanatory-variable value with the observed range. Being outside the range means the prediction is extrapolation. Explain why the pattern seen in the data may not continue there. A strong \(r\) does not remove this limitation, and a prediction’s numerical plausibility does not make it well supported.
Unusual points. An observation may be unusual because it has a large residual, high leverage, or substantial influence on the fitted line. These are different properties, as covered in “Outlier, High-Leverage, and Influential Points Defined.” A report should not call a point influential just because it looks unusual: influence is assessed by comparing the regression with and without that point. Nor should a valid observation be discarded just because it changes the result. Investigate it and explain the sensitivity if it matters.
Sampling. A regression summarizes the cases used to fit it. The method used to obtain those cases affects how far a report can responsibly generalize. A volunteer sample from one location, for example, may not represent people who did not volunteer or people in other settings. A larger sample is not automatically representative; size alone does not correct a selection process that leaves out important groups. Describe how the sample was obtained when that information is available, and limit the claim to a defensible group.
Confounding. In observational data, an association between \(x\) and \(y\) may partly reflect another variable related to both. That variable is a possible confounder. For instance, an association between time spent outdoors and reported mood might also be connected to season: season can affect outdoor time and may be related to mood. Unless the analysis provides a way to separate these effects, identify season as a possible alternative explanation—not as a proven cause of the observed pattern. A fitted slope describes the association in the data; it does not, by itself, establish what would happen if \(x\) were changed.
Worked Example: Qualify a Prediction Beyond the Data
Worked Example: Cooling Time and Room Temperature
Original AP-style question. In an invented data set, a technician records the cooling time \(x\), in minutes, and the temperature \(y\), in degrees Celsius, for 20 identical sample containers under one laboratory setup. The observed cooling times range from 4 to 26 minutes. A fitted line is \(\hat{y}=82-1.4x\). A report uses the line to predict the temperature after 35 minutes. Describe the limitation that should be reported.
State. Identify whether the prediction is within the observed range and explain what that means for interpreting the model’s result.
Plan. Compare 35 minutes with the observed range of 4 to 26 minutes. If it is outside that range, calculate the fitted value but distinguish the arithmetic result from how well the data support it.
Do. Since \(35>26\), 35 minutes is outside the observed cooling-time range, so this is extrapolation. Substituting into the fitted line gives
The line calculates a predicted temperature of \(33^\circ\text{C}\), but the observations only show the relationship from 4 to 26 minutes. The data do not establish that the fitted linear pattern continues to 35 minutes; the cooling process might not keep decreasing at the same rate.
Conclude. The fitted line predicts \(33^\circ\text{C}\) after 35 minutes, but this prediction is extrapolation because the observed cooling times ranged only from 4 to 26 minutes. It should be treated cautiously because the data do not show whether the linear pattern continues beyond 26 minutes.
Worked Example: Report Sensitivity to an Unusual Point
Worked Example: Practice Time and Free-Throw Success
Original AP-style question. In an invented observational data set, a coach records weekly practice time \(x\), in hours, and free-throw success \(y\), as a percentage, for 22 players. A scatterplot shows a generally positive linear association. One player has an unusually large positive residual. The original fit has slope \(2.4\) percentage points per hour and \(r=0.68\). After investigating the point and refitting without it, the slope is \(1.9\) percentage points per hour and \(r=0.61\). The player’s record is verified as accurate. Write a limitation for the report.
Solution. The positive association remains after the verified observation is removed, but the reported slope changes from \(2.4\) to \(1.9\) percentage points per hour, and \(r\) changes from \(0.68\) to \(0.61\). These comparisons indicate that the estimated relationship is somewhat sensitive to this observation. Since the record is verified, the point should not be deleted merely because it affects the results. The comparison also does not show that the observation is an error or that the relationship is absent.
A useful limitation would be: “The estimated strength and slope of the association between weekly practice time and free-throw success are somewhat sensitive to one player with a large positive residual. That player’s record was verified, so the primary analysis retains the observation; conclusions should account for the change in fit when it is omitted.”
This wording names the evidence, the with-versus-without comparison, and the reporting consequence. It does not confuse a large residual with influence: the change in the fitted results is the evidence of sensitivity.
Worked Example: Limit a Claim About a Volunteer Sample
Worked Example: Screen Brightness and Battery Life
Original AP-style question. In an invented observational data set, 32 volunteers from one after-school robotics club record their usual phone-screen brightness \(x\), measured as a percentage, and battery life \(y\), in hours, during a typical day. The scatterplot shows a negative linear association. The report concludes, “Higher screen brightness is associated with shorter battery life for all phone users.” Identify a sampling limitation and revise the conclusion.
State. Decide what group the observed cases represent and whether the sampling method supports generalizing to all phone users.
Plan. Use the reported association to describe the data, but name the volunteer group and setting. Explain that volunteers from one club may differ from phone users generally, and that observational data do not establish causation.
Do. The 32 volunteers represent members of one after-school robotics club who chose to participate. They are not described as a random sample of phone users, so the report cannot assume that the same relationship applies to all phone users. Differences in phone models, battery condition, usage habits, or other factors may also matter. Since the data are observational, the negative association does not show that increasing brightness alone causes battery life to decrease.
Conclude. Among the 32 volunteers from this robotics club, screen brightness and battery life have a negative association. Because the cases were volunteers from one club rather than a representative sample of phone users, the finding should not be generalized to all phone users. The observational association also does not establish that brightness causes shorter battery life.
Worked Example: Name a Possible Confounder Carefully
Worked Example: Outdoor Activity and Mood Ratings
Original AP-style question. In an invented observational data set, 40 adults record weekly hours of outdoor activity \(x\) and a mood rating \(y\) on a fixed survey scale during a two-month period. The scatterplot shows a positive linear association. Participants were not assigned activity levels, and the analysis did not include the week or season in which each rating was recorded. Write a limitation that addresses confounding without claiming more than the information supports.
Solution. A positive association between outdoor activity and mood ratings is present in the observed data, but the observational design does not establish that outdoor activity causes higher mood ratings. Week or season is a possible confounding variable: it could be related to how much outdoor activity people do and could also be related to mood. The information given does not show whether season actually explains the association, so the report should describe it as a possibility rather than a demonstrated cause.
A careful report sentence is: “Weekly outdoor activity and mood ratings had a positive linear association among these participants. Because activity was not assigned and the analysis did not account for week or season, this association does not establish a causal effect; season is one possible alternative explanation that was not assessed.”
The qualification is specific: it names the observational design and an omitted variable that could matter. It does not say that the association is false or that season definitely produced it.
Common Mistakes and AP Exam Tips
- Listing a limitation without its consequence. “The sample is small” or “there is an outlier” is incomplete on its own. Explain what claim or model feature may be affected, and connect it to evidence when available.
- Calling every prediction outside the data unreasonable. Extrapolation means the requested \(x\)-value is outside the observed range. Explain that the pattern may not continue; do not claim the calculated value is automatically impossible.
- Deleting an unusual observation automatically. First investigate whether it is a recording error or a valid case. If valid, retain it in the primary analysis and describe any important change in the fit when it is omitted.
- Assuming a large sample guarantees broad representation. A large volunteer sample can still differ from the population the report hopes to describe. State who participated and avoid generalizing beyond what the sampling process supports.
- Presenting a possible confounder as a proven explanation. Say that another variable “could be related to both” or “is a possible alternative explanation.” Do not claim it caused the association unless the analysis supports that conclusion.
- Using a generic disclaimer instead of a relevant one. “Results may not be accurate” does not identify a limitation. Specify the range, point, sampling method, or possible confounder and the part of the conclusion it affects.
For full credit, connect each limitation to the situation in the question. Identify the observed range for extrapolation, use a with-versus-without comparison to support a claim about influence, name the sampled group when discussing generalization, and frame unmeasured confounders as possibilities. Keep the report direct: a limitation qualifies the evidence; it does not replace describing what the data show.
Check Your Understanding
For each situation, identify the relevant limitation and write one sentence that explains its consequence for a regression report.
- A model is fitted using \(x\)-values from 10 to 40. A report predicts \(y\) at \(x=52\). What should the report say about this prediction?
- A point has a large residual, but removing it barely changes the slope or \(r\). What can the report say about the point, and what should it avoid calling it?
- A survey uses volunteers from one neighborhood to study the association between weekly exercise and resting heart rate. What sampling detail should limit the report’s scope?
- An observational analysis finds that people who use a community garden more often report eating more vegetables. Name one possible confounding variable and phrase it as a possibility, not a finding.
- A verified observation changes the fitted slope noticeably when omitted. What comparison supports mentioning sensitivity, and should the observation automatically be discarded?