From an Unusual Point to a Defensible Diagnosis
A plot can show that an observation deserves attention; it cannot, by itself, tell you why the observation is unusual. A long right tail might contain genuine large values, a separate cluster might represent a distinct type of case, and a single extreme value might be a recording mistake. Those possibilities have different implications for analysis, so visual diagnosis should combine the shape of the distribution, relationships with other variables, and the way the data were collected.
This tutorial uses three complementary views. A histogram shows concentrations, gaps, and tail behavior in one numeric variable. A boxplot summarizes its center and spread and may mark unusually distant observations. A scatterplot places a candidate observation in relation to another numeric variable, which can reveal whether it follows a pattern or belongs to a separate pattern. None of these plots establishes an error on its own.
A Three-Plot Diagnosis
Begin by locating the observation and checking the variable’s units and meaning. Then ask what each plot contributes. The histogram answers whether the value sits in a tail, across a gap, or in a second concentration. The boxplot shows its position relative to the middle half of the data. The scatterplot shows whether the observation is unusual in one variable only, unusual in the joint relationship, or part of a separate pattern.
For a conventional boxplot, the interquartile range is \(IQR=Q_3-Q_1\). The lower and upper fences are \(Q_1-1.5IQR\) and \(Q_3+1.5IQR\). A value beyond a fence may be displayed as a separate point. The fence is a screening convention, not a test of whether the value is erroneous. Quartile calculations can depend on the software’s quantile convention; when exact results matter, state the convention.
In the examples below, quartiles are calculated using R’s default sample quantile convention, type 7. For a sorted sample of size \(n\), the type 7 position for probability \(p\) is \(1+(n-1)p\); if the position is between two observations, interpolate linearly. This convention makes the numerical boxplot calculations reproducible.
Use a histogram and boxplot to find the observation’s location, its surrounding values, and whether the data show a tail, gap, or another concentration.
Use a scatterplot with relevant numeric variables and, when available, an independently recorded group label. Ask whether the observation follows the general pattern and whether its value is possible under the measurement process.
Distinguish what the plots show from what must be checked in records or with subject-matter knowledge. Do not delete, cap, or change a value solely because it looks unusual.
Why Histogram Scale and Bin Width Matter
A histogram groups numeric values into intervals, called bins. With equal-width bins, bar heights may represent counts. If bin widths differ, comparing raw heights can be misleading: a wider bin can contain more observations simply because it covers more of the number line. A density histogram instead scales height so that bar area represents the fraction of observations in the bin.
To see why, suppose a bin has width \(w\) and contains \(c\) of \(n\) observations. Its fraction of the data is \(c/n\). Setting its height to that fraction divided by its width makes the area equal to the fraction:
Thus, for unequal-width bins, interpret the areas rather than comparing bar heights as though they were counts. In R, inspect the bin boundaries as well as the bars: a value exactly on a boundary is assigned according to the software’s bin convention. When describing a gap, be precise about whether its endpoints contain observations.
Worked Example: An Extreme but Plausible Annual Distance
A hypothetical fleet analyst reviews annual distance, in thousands of kilometres, for 12 vehicles of the same broad class. The sorted values are 4, 5, 5, 6, 6, 6, 11, 12, 12, 13, 14, and 47. A histogram with equal-width bins from 0 to 50, in widths of 10, has counts 6, 5, 0, 0, and 1. A boxplot marks 47 separately. The maintenance record for that vehicle independently confirms a reading of 47,000 kilometres for the year. How should the analyst diagnose it?
Step 1: Describe the histogram. The first bin, [0, 10), contains 4, 5, 5, 6, 6, and 6, so its count is 6. The bin [10, 20) contains 11, 12, 12, 13, and 14, so its count is 5. The bins [20, 30) and [30, 40) each contain 0 observations; there are no observations in the interval [20, 40). The final bin, [40, 50), contains 47, so its count is 1. The value is separated from the main concentration, but the distribution also has a right tail.
Step 2: Calculate the boxplot summary. With \(n=12\), the type 7 position for \(Q_1\) is \(1+11(0.25)=3.75\). The third and fourth sorted values are 5 and 6, so \(Q_1=5+0.75(6-5)=5.75\). The median is the mean of values 6 and 7 in sorted order: \((6+11)/2=8.5\). The type 7 position for \(Q_3\) is \(1+11(0.75)=9.25\). The ninth and tenth values are 12 and 13, so \(Q_3=12+0.25(13-12)=12.25\).
Therefore, \(IQR=12.25-5.75=6.50\), and the upper fence is \(12.25+1.5(6.50)=22.00\). Since \(47>22\), the boxplot flags it. This calculation says the point is far from the central spread; it does not establish an error.
Step 3: Use context to classify it. The value is extreme relative to the other vehicles, but its units are consistent and the independent maintenance record confirms it. The defensible classification is a plausible extreme, not a suspected error. The vehicle may merit separate review, but removing its observation would discard a confirmed value.
Common mistake and correction. A mistaken calculation might use \(Q_3+1.5Q_1=12.25+1.5(5.75)=20.875\) as the upper fence. That expression incorrectly multiplies \(Q_1\) rather than the interquartile range. The correct calculation is \(Q_3+1.5(Q_3-Q_1)=12.25+1.5(6.50)=22.00\). Either way, 47 is flagged here, but only the correct calculation applies the boxplot rule.
When a Separate Cluster May Be Meaningful
A value can look extreme in a histogram while appearing quite ordinary within a subgroup. A scatterplot can help distinguish a single point that departs from a relationship from several points that form a coherent second pattern. If an independently recorded category aligns with that pattern, it offers a lead for investigation; it does not prove that the category caused the difference.
Worked Example: A Second Pattern in a Scatterplot
A hypothetical insurer explores two numeric measures for nine policies: an index of insured equipment size, \(x\), and a service-cost index, \(y\). The six policies in the main concentration have \((x,y)\) values (1, 2), (2, 4), (3, 6), (4, 8), (5, 10), and (6, 12). Three more policies have values (9, 25), (10, 27), and (11, 29). A pre-existing equipment category is recorded for each policy: the first six are standard equipment and the last three are specialist equipment. Diagnose the apparent separation.
Step 1: Compare marginal distributions. For \(x\), the six standard policies have values 1 through 6 and the specialist policies have 9, 10, and 11. No observations lie strictly between 6 and 9. For \(y\), the standard values are 2, 4, 6, 8, 10, and 12, while the specialist values are 25, 27, and 29. No observations lie strictly between 12 and 25. The marginal histograms therefore show separated concentrations in both variables.
Step 2: Check the joint pattern. For each standard policy, \(y-2x=0\): for example, the pair (4, 8) gives \(8-2(4)=0\). For each specialist policy, \(y-2x=7\): (9, 25) gives \(25-2(9)=7\), (10, 27) gives \(27-2(10)=7\), and (11, 29) gives \(29-2(11)=7\). The scatterplot thus shows two internally consistent linear patterns with a vertical separation of 7 index units, rather than one isolated point far from an otherwise shared pattern.
Step 3: Classify cautiously. Because the pre-existing equipment category matches the two clusters, the evidence supports a distinct population feature associated with the specialist category. This does not show why service costs differ, nor does it establish a causal effect. Before deciding how to model the policies, the analyst should confirm that the category and both measurements were recorded consistently. Treating the three specialist observations as errors merely because they are separate would ignore the structure visible in the data.
Common mistake and correction. Calling all three points errors because they sit apart from the main concentration confuses separation with invalidity. The correction is to inspect the joint pattern and recorded context: here, all three share a documented category and the same relationship offset. That is evidence to investigate a subgroup, not a reason to discard its observations.
When a Plot Suggests a Recording Problem
A likely data error requires more than distance from the center. Warning signs include an impossible value, a disagreement with source records, an unexpected unit change, or a point that conflicts with a known measurement process. A histogram may compress almost all ordinary values against one edge when a single number is far larger than the rest. A scatterplot can add context, but the source record is often needed to resolve the issue.
Worked Example: A Mixed Currency Unit
A hypothetical analyst reviews 11 policy premiums stored in dollars. The sorted values are 48, 49, 50, 51, 52, 52, 53, 54, 55, 56, and 5,200. The first ten values are from the same processing extract. A histogram puts ten observations between 40 and 60 and one near 5,200; a boxplot marks 5,200. The source system’s transaction record shows that the final value was 5,200 cents, or 52 dollars. Determine the appropriate diagnosis and action.
Step 1: Calculate the boxplot flag. For \(n=11\), the type 7 position for \(Q_1\) is \(1+10(0.25)=3.5\). The third and fourth values are 50 and 51, so \(Q_1=(50+51)/2=50.5\). The median is the sixth value, 52. The position for \(Q_3\) is \(1+10(0.75)=8.5\); the eighth and ninth values are 54 and 55, so \(Q_3=(54+55)/2=54.5\).
Thus \(IQR=54.5-50.5=4.0\), and the upper fence is \(54.5+1.5(4.0)=60.5\). Since \(5{,}200>60.5\), the value is flagged. The histogram also shows why the ordinary observations may appear compressed when plotted on the same scale.
Step 2: Check the evidence outside the plots. The transaction record states that the raw amount is 5,200 cents. Converting cents to dollars gives \(5{,}200/100=52\) dollars, which is consistent with the other premiums. The issue is a mixed unit in the stored data, not a confirmed premium of 5,200 dollars.
Step 3: Record the diagnosis and correction. This is a confirmed data error in the analysis field, supported by the source record. Correct the analysis value to 52 dollars using the documented conversion, retain the original raw value, and record the reason for the correction. The boxplot flag prompted the investigation, but the transaction record established the unit problem.
Common mistake and correction. Automatically deleting the flagged value would lose a valid premium and leave only ten records. Automatically changing any flagged value to 52 without checking its source would also be unjustified. Correct it to 52 only because the transaction record confirms the amount and unit.
Using R Plots as Diagnostic Evidence
In R, a basic workflow can place a histogram and boxplot beside one another for a numeric variable, then plot that variable against a relevant second numeric variable. Functions such as hist, boxplot, and plot provide the basic displays. For a scatterplot, use a group symbol or color only when the group label was recorded independently of the visual pattern; otherwise, the grouping can make a pattern appear more established than the data support.
The display settings matter. A histogram’s bin width and boundary can make a gap look larger or smaller, and a boxplot hides the detailed shape that the histogram shows. A scatterplot can reveal a pattern in two variables, but it may not explain why the pattern occurs. Inspect the plotted values and units, and consider whether the conclusion changes under a reasonable alternative histogram binning. When a conclusion depends on one arbitrary display choice, describe that uncertainty.
A disciplined report separates observation from interpretation. For example: “The histogram has a sparse upper tail, and the boxplot flags the largest value. The source record confirms the value and unit, so it is retained as a plausible extreme.” This statement identifies what the plots show, names the additional evidence, and explains the treatment. It is stronger than saying only that the value is “an outlier.”
Common Diagnostic Traps
- Equating a boxplot flag with an error. A fence identifies a value distant from the central spread under a convention; it does not verify a recording problem.
- Calling every gap a separate population. A small sample may have an empty interval by chance. Look for coherent clusters, relevant recorded context, and consistency across views.
- Ignoring the second variable. A marginal extreme may follow a clear relationship in a scatterplot, or a point that seems ordinary marginally may depart sharply from the joint pattern.
- Correcting from appearance alone. A suspected unit or entry error should be checked against collection rules or source records before values are changed.
- Using a diagnosis as an automatic modeling rule. Whether to retain, correct, or analyze a subgroup separately depends on the validated data and the modeling question, not solely on a visual label.
Check Your Understanding
For each problem, combine the stated plot evidence with the available context. A flag or gap is evidence for investigation, not proof of an error.
- A hypothetical delivery service records annual distance in thousands of kilometres for 11 vans: 8, 9, 9, 10, 11, 12, 13, 14, 15, 16, and 45. Using type 7 quartiles, calculate the upper fence and classify 45 if the vehicle log confirms 45,000 kilometres. (Answer: \(Q_1=9.5\), \(Q_3=14.5\), \(IQR=5\), upper fence \(=22\); 45 is flagged but is a plausible extreme if the log and units are verified.)
- A scatterplot of a hypothetical equipment index \(x\) and cost index \(y\) shows four observations (1, 3), (2, 6), (3, 9), and (4, 12), plus three observations (8, 30), (9, 33), and (10, 36). Each of the latter three has a previously recorded specialist-equipment label. What visual feature is present, and what is a cautious diagnosis? (Answer: Two separated, internally consistent patterns; the recorded label supports investigating a distinct population feature, not deleting the three observations.)
- A boxplot for 12 hypothetical repair amounts, sorted in dollars, shows 100, 105, 110, 115, 120, 125, 130, 135, 140, 145, 150, and 1,500. Using type 7 quartiles, calculate the upper fence. The source record has not yet been checked. What should be concluded? (Answer: \(Q_1=113.75\), \(Q_3=141.25\), \(IQR=27.5\), upper fence \(=182.5\); 1,500 is flagged and suspicious, but its status as an error is unconfirmed until investigated.)
- A density histogram uses bins of widths 5 and 20. The first contains 10 of 100 observations; the second contains 20 of 100. Calculate each density height and identify which bar is taller, without confusing height with count. (Answer: The heights are \(10/(100\cdot5)=0.020\) and \(20/(100\cdot20)=0.010\); the first bar is taller even though the second contains more observations.)