Introduction
A regression diagnostic can draw attention to a case that has an unusual residual, high leverage, or a substantial effect on fitted results. That warning is useful, but it does not tell us what happened. The observation might reflect a recording problem, a valid but uncommon situation, or a weakness in the model’s description of the data. Those possibilities call for different responses.
The aim of investigating a flagged observation is therefore not to find a reason to remove it. It is to gather evidence in a disciplined order: verify what was recorded, understand how it was measured, and consider whether the model is appropriate for the pattern. Only then can we judge whether further sensitivity analysis would clarify how much the case matters.
Start With the Model and the Question
Before investigating a case, state what the regression is intended to describe. For \(n\) observations, a multiple linear regression model can be written as
Here \(Y_i\) is the response for case \(i\), \(x_{ij}\) is its value on predictor \(j\), and the \(\beta\) terms describe the model’s conditional mean. The error \(\varepsilon_i\) represents variation not explained by that mean. For ordinary least squares regression, the diagnostic interpretation rests on having a meaningful response and predictor definition, an adequately specified conditional mean, and errors with mean zero and an appropriate independence and variance structure for the intended analysis. Normality is additionally used for some small-sample inferential procedures; it is not a requirement for simply checking whether a recorded value matches its source.
A case can look unusual because it is unusual relative to the fitted model, even when its recorded values are accurate. The right question is not merely, “Is this point extreme?” It is, “What does this case represent, are its values trustworthy, and does the model describe the relevant relationship adequately?”
A Structured Investigation
Use the following sequence, recording what each check establishes and what remains uncertain. Begin with the case identifier and the specific diagnostic that raised the flag. Do not assume that two different diagnostics are flagging a case for the same reason.
Confirm the identifier, row, and variables involved. Keep an unchanged copy of the original data, and record any correction separately with its source and rationale.
Compare the analysis file with the original form, instrument output, or other authoritative source. Check decimal placement, data types, category codes, missing-value codes, and the units used for each variable.
Ask when, where, and how the measurement was obtained. Look for documented protocol changes, instrument limits, unusual conditions, or a difference in the population or process that generated the case.
Check whether the response and predictors mean what the analysis assumes they mean for this case. Consider whether a model limitation could account for the discrepancy, without treating a single case as proof of a particular model defect.
If a concern is unresolved or the case materially affects a conclusion, decide whether an explicitly described sensitivity analysis is useful. Keep the primary analysis tied to the stated data and scientific question.
This order matters. A calculation performed on an incorrectly entered value can be arithmetically correct but scientifically misleading. Conversely, a value that looks surprising should not be changed simply because doing so makes the regression look more regular.
Why a Flag Is Not a Verdict
A residual is the observed response minus the response fitted by the model for that case. A large residual means the observed response is poorly predicted by the fitted conditional mean. It does not identify whether the cause is a recording problem, natural variation, a missing feature of the model, or some combination.
This distinction follows directly from the model’s structure. The observed value is represented as a model mean plus an error, so a discrepancy from the fitted mean can arise from the error term even when the recorded response and predictors are correct. A diagnostic measures a feature of the case relative to the fitted model; it does not independently verify the data source or explain the measurement process. Therefore, a flag alone cannot justify editing or excluding a case.
A useful investigation keeps three questions separate:
- Record question: Does the analysis file faithfully represent the original measurement and its units?
- Context question: Does the measurement have a known interpretation for the case and collection conditions?
- Model question: Is the fitted relationship a reasonable description for the population and range being studied?
Evidence answering one question does not automatically answer the others. For example, confirming a value against a source record supports its accuracy, but does not establish that the model is adequate for that case.
Worked Example: Checking a Possible Unit Mismatch
Suppose a hypothetical model relates a device reading \(x\), recorded in centimeters, to a response \(Y\). One flagged row lists \(x=1.72\) cm, while nearby readings are around 170 cm. The fitted line predicts \(Y=42\) units for this row, but the observed response is 9 units. The discrepancy prompts a data check; it does not prove an error.
Check the source. The original measurement form for the case lists “172 cm.” The analysis file contains “1.72.” This is evidence of a possible unit or conversion issue, but the source and data-processing history must be reconciled before changing the analysis value.
Check the arithmetic implied by each interpretation. If the recorded value is 1.72 cm, the gap from a typical value of 170 cm is \(170-1.72=168.28\) cm. If the intended value is 172 cm, the gap is \(172-170=2\) cm. These are very different predictor values, so the ambiguity is important to resolve.
Document the result. The original form supports 172 cm, but the discrepancy should be traced through the data entry or conversion steps. Until that is confirmed, do not silently replace the value or present either interpretation as settled. Once resolved, assess the regression using the documented measurement and consider sensitivity analysis if uncertainty remains about the effect on conclusions.
Measurement Context and Model Fit
After checking the record, ask whether the measurement was produced under conditions comparable to those assumed by the analysis. Relevant context can include the instrument used, the timing of measurement, the eligibility rules for inclusion, or a change in the procedure. These details help determine whether the case is comparable to the others in the intended analysis. They should be documented rather than inferred from the diagnostic statistic alone.
Then consider whether the model’s response and predictors describe the scientific question for this case. A single surprising observation might be consistent with ordinary variability. Alternatively, it might draw attention to a broader limitation, such as a relationship that changes across groups or a predictor that does not capture an important feature. Establishing a broader model-fit problem generally requires examining the data and scientific context as a whole, not assigning a cause to one flagged row.
This tutorial’s task is to identify what needs checking, not to diagnose every possible pattern in residuals. If several cases share an unusual pattern, or if concerns arise across the data rather than for one record, treat that as a reason to examine overall model fit. Do not respond to a possible model problem merely by removing whichever individual case looks least compatible with the fitted line.
Worked Example: A Surprising but Context-Dependent Reading
In a hypothetical analysis of study time \(x\), measured in hours per week, and a test score \(Y\), one flagged student has \(x=24\) hours and a score of 51 points. The fitted model predicts 76 points for 24 hours. The residual is
Check the recorded values. The entry for study time is 24 hours and the score is 51 points on a 100-point scale. The original records agree with the analysis file, so the initial check finds no discrepancy in those values.
Check context. The measurement notes indicate that the student missed several days of instruction during the study period. That information may matter to interpretation, but it does not change the recorded study time or test score. It also does not, by itself, establish that the case should be excluded: the analysis question must determine whether students with interrupted instruction belong to the target population.
Check model scope. The regression uses study time as its predictor; it does not include attendance. The unusual residual could reflect individual variation, the omitted context, or limitations of the model. The available facts do not determine which explanation is correct.
Decide what to do next. Retain the verified values in the primary analysis if the student meets the stated inclusion criteria. If the intended conclusion is sensitive to the case, a clearly labelled sensitivity analysis may help show that dependence. Do not describe the residual alone as proof that the measurement is invalid or that attendance caused the discrepancy.
When Is Sensitivity Analysis Warranted?
Sensitivity analysis asks whether a conclusion changes under a specified alternative analysis. It is most informative when the alternative corresponds to a clearly described uncertainty or analytic choice. For a flagged observation, that might mean checking the consequence of a documented ambiguity, or assessing how conclusions change under a defensible alternative treatment. The alternative must be explained; trying many arbitrary changes until a preferred result appears is not a sound investigation.
A flag may warrant sensitivity analysis when the case has a substantial influence on results, when a meaningful data or context question remains unresolved, or when the conclusion seems to depend heavily on the case. A flag may still be worth documenting even if the primary conclusion does not materially change. Conversely, a flagged case with verified values and no material effect need not trigger a sequence of alternative analyses just because a diagnostic crossed a screening threshold.
In every case, distinguish the primary analysis from the sensitivity analysis. State what changed, why that alternative was considered, and whether the interpretation changed. Sensitivity results describe dependence on an analytic choice; they do not establish that the alternative is the truth.
Worked Example: Deciding Whether to Check a Conclusion
A hypothetical regression estimates the association between \(x\), distance in kilometers, and \(Y\), travel time in minutes. A flagged case has a verified distance of 38 km and time of 84 minutes. The original records and units agree, and the case meets the study’s inclusion rules. The fitted-model diagnostics indicate that the case affects the estimated slope substantially.
What is known? The values have been checked against their source, the units are consistent, and the case belongs to the defined population. These checks provide no basis for correcting or excluding the observation.
What is not known? The diagnostic shows that the estimated relationship is sensitive to this case, but it does not show that the case is erroneous. Its unusual influence is a feature of the relationship between this case and the fitted analysis.
What is warranted? Because the slope is sensitive, report the primary analysis using the stated data and conduct a clearly labelled sensitivity analysis if it helps communicate the dependence of the conclusion on this case. The sensitivity analysis should not replace the primary result merely because it produces a more convenient slope. The final interpretation should describe both what the primary analysis estimates and how robust the conclusion is to the stated alternative.
Common Mistakes
- “The diagnostic flagged it, so it must be wrong.” A diagnostic describes how a case relates to a fitted model; verify the record and context before making claims about its accuracy.
- “The value is extreme, so it should be removed.” Extremeness is not an exclusion rule. Use the study’s inclusion criteria and documented evidence, not the desired shape of the fitted results.
- “The source record matches, so the model is fine.” Data verification addresses the recorded value, not whether the model adequately describes the relationship.
- “A sensitivity analysis tells us which result is correct.” It shows how conclusions respond to a specified alternative; it cannot resolve factual uncertainty without relevant evidence.
- Changing the data without a record. Preserve the original value, document any correction and its source, and make the analysis choice explicit.
Check Your Understanding
Use the investigation sequence to answer each question.
- A flagged predictor value differs from the value on the original form. What should you check and document before changing the analysis file?
- Why does a large residual not, by itself, establish that a response was measured incorrectly?
- A verified case belongs to the target population but substantially affects the fitted slope. What can a sensitivity analysis show, and what can it not establish?
- Give one example of measurement context that could help interpret a flagged case without proving that it should be excluded.
- Why is it important to distinguish a record question from a model-fit question?