Why an Unusual Value Needs an Audit, Not an Automatic Deletion
An economic dataset may contain a negative age, a monthly wage stored among annual wages, or a very high income that is entirely genuine. All three values may look unusual, but they call for different responses. The first conflicts with the variable’s meaning, the second may have a unit or period error, and the third may be a valid observation with substantial influence on a summary.
The central task is to establish what each recorded value means before deciding what to do with it. An extreme value is not, by itself, evidence of a mistake. Conversely, a plausible-looking value can still be wrong if it uses a different unit, refers to a different time period, or is a code for missing information.
A useful audit begins with the measurement definition: what does the variable measure, for which observational unit, in what units, and over what period? Check the value against that definition, any documented valid range, and any logical relationships with other variables. Then seek evidence for a suspected error, such as a data dictionary, source record, or verified collection rule. A statistical flag is a reason to investigate, not a finding that the observation is wrong.
Check Meaning, Bounds, and Cross-Variable Consistency
Some variables have natural or design-based limits. A person’s age cannot be negative; hours in a person-week cannot exceed 168 if the variable counts elapsed hours in that week. Other limits depend on the study’s definitions. A recorded value of zero earnings might be valid for a person who did not work, or invalid if the dataset includes only people with positive earnings. The audit must use the actual inclusion rule rather than assume one.
Cross-variable checks can expose values that pass a single-variable range check. For example, annual earnings recorded for a full-time worker may be inconsistent with the hourly wage and hours worked, depending on how each variable is defined. Such a discrepancy is a flag: different periods, unpaid leave, overtime, or distinct definitions may explain it. Do not impose an equality unless the variables’ measurement rules justify it.
Worked Example: Applying a Plausibility Check
A hypothetical employment file defines weekly hours as hours worked by one employee during a calendar week, and records age in completed years. Two records are: employee A, age 42 and 40 weekly hours; employee B, age −2 and 168 weekly hours. Determine what the checks establish.
Age check. Under the stated definition, age must satisfy \(a \geq 0\). For employee B, \(a=-2<0\), so the recorded age violates a necessary bound. It is an impossible value under this definition and should be flagged for source verification. The check does not reveal the correct age; changing it to a guessed value would not be justified.
Hours check. A calendar week contains \(7 \times 24=168\) hours, so the stated elapsed-hours measure must satisfy \(0 \leq h \leq 168\). Employee B’s 168 hours meet the upper bound. They are extreme and merit review, but the bound alone does not prove an error. The value could reflect an unusual case or a problem in how “hours worked” was recorded; more evidence is needed.
Employee A’s values pass these two checks. Passing a range check does not certify a record as error-free; it only means these particular checks found no violation.
Look for Unit and Time-Period Mismatches
A unit inconsistency can create an extreme-looking value without any arithmetic mistake in the source record. Earnings might be recorded per hour, week, month, or year; prices may be in dollars or thousands of dollars; quantities may be measured in different physical units. Time periods matter as well: a monthly value is not interchangeable with an annual value merely because both are labelled “earnings.”
Before converting a value, confirm both the source unit and the target definition. A conversion is justified only when the relationship is known. For instance, multiplying monthly earnings by 12 assumes that the figure is monthly, that the target is annualized earnings, and that the intended calculation treats the monthly amount as representative for all 12 months. It is not automatically a correction for observed annual earnings, which may reflect unemployment, changing pay, or partial-year work.
Worked Example: Verifying an Earnings-Period Mismatch
A hypothetical file is defined as gross annual earnings in dollars. Three records contain $48,600, $51,000, and $49,200. A fourth contains $4,200. The source documentation confirms that the fourth figure is gross monthly earnings and that this variable should contain the annualized equivalent of that monthly rate.
The documented conversion is:
The converted value, $50,400, is on the same annualized basis as the other records. This calculation is supported by the documentation, not by the fact that the result looks plausible. If the source instead described actual earnings during the preceding calendar year, multiplying by 12 would not be justified without further evidence.
An audit should preserve the original recorded value and record the basis for any correction. If the source unit cannot be verified, flag the record as unresolved rather than silently converting it.
Identify Sentinel Codes Before Calculating Summaries
A sentinel code is a special recorded value used to represent a status such as “not reported,” “not applicable,” or “not in universe.” It can resemble a real number. Common examples include −9, −99, or −999, but no particular number has a universal meaning. A codebook or collection rule must establish whether a value is a sentinel and what status it represents.
Treating a sentinel as a measurement can distort summaries and analyses. Replacing every negative value with missing information is also unsafe: negative values might be valid for some variables, such as a change in employment or a net financial balance. Use the variable-specific code definition. Keep distinct missing statuses separate when the source distinguishes them, and do not infer an unreported value.
Worked Example: A Sentinel in Unemployment Duration
A hypothetical dataset records unemployment duration in months. Its codebook says that −999 means “duration not reported”; it is not a measured duration. Five recorded entries are 2, 4, 6, −999, and 3 months. Calculate the mean for the reported durations and show what goes wrong if the code is treated as a measurement.
Using the codebook. Exclude the sentinel from the calculation of the mean duration, while retaining its missing-status information. The four reported values sum to \(2+4+6+3=15\) months, so their mean is:
If the sentinel is mistakenly treated as a measurement, the sum becomes \(2+4+6-999+3=-984\), and the computed mean is:
That result is not a meaningful average duration; it is an artifact of using the code as if it were data. The reason for excluding −999 here is the documented codebook rule, not a general rule to discard negative numbers.
Unusual Values and Influence
An outlier is relative to a comparison set and a method of identifying unusual observations. A value may be far from most observations but still be valid. Conversely, an observation near the center of a distribution may contain a coding mistake. A rule such as a plot-based threshold can help locate values to inspect, but it cannot establish whether their measurement is correct.
Influence is also relative: ask which estimate or conclusion is being considered and how much it changes when the observation is included. For a sample mean, the effect of one observation can be derived directly. Let \(\bar{x}_{-i}\) be the mean of the \(n-1\) observations other than observation \(i\). Then:
Thus, an observation far from the mean of the others can move the full-sample mean substantially, especially in a small sample. This arithmetic describes influence on the mean; it does not show that the observation is erroneous. Influence on a fitted relationship or another estimator depends on that analysis and must be assessed for that target.
Worked Example: A Valid Extreme Income and the Sample Mean
A hypothetical set of annual incomes, in thousands of dollars, is \(30, 32, 34, 35, 36, 38, 40,\) and \(400\). Source verification confirms that the $400,000 income is correctly recorded and belongs to the target population. Compare the sample mean with and without that observation to assess its influence on the mean.
The first seven values sum to \(30+32+34+35+36+38+40=245\). Including the verified extreme value gives a total of \(245+400=645\), so:
Without the $400,000 observation, the mean is:
The observation changes the sample mean by \(80.625-35=45.625\) thousand dollars. It is influential for this mean, but it is not a data error: verification says it is a valid income. Removing it solely to bring the mean closer to the other values would change the data being summarized and could misrepresent the target population.
Whether this is an appropriate summary depends on the question. A mean is sensitive to large values; another summary may answer a different question. The audit’s job is to establish what the observation represents, not to erase valid heterogeneity.
A Defensible Audit Decision
A practical audit separates what is known from what is suspected. Record the original value, the check that flagged it, the evidence reviewed, and the disposition. A verified error may be corrected using an identifiable source or a documented rule. A valid unusual observation should generally remain in the dataset, with its unusualness and possible influence made clear. An unresolved case should remain flagged rather than being silently recoded or deleted.
After any justified correction, recompute the relevant summaries or estimates and state how the decision affects them. When a result is sensitive to a valid extreme value, report that sensitivity rather than describing the value as erroneous. The goal is a dataset and analysis whose treatment of unusual observations can be explained and reproduced.
- Wrong: “This value is far from the others, so it must be a mistake.” Correction: Unusualness identifies a case to investigate; source evidence and the variable definition determine whether it is an error.
- Wrong: “The converted amount looks realistic, so I multiplied by 12.” Correction: Convert only when the source unit, target unit, and relevant period are established.
- Wrong: “All negative values mean missing.” Correction: Apply the documented, variable-specific code definitions; a negative value may be a genuine measurement for some variables.
- Wrong: “The observation changes the mean a lot, so I should delete it.” Correction: Influence is not evidence of error. Verify the observation and interpret sensitivity in relation to the analysis target.
Check Your Understanding
Use the definitions and audit principles above to answer each question.
- A dataset defines age in completed years and contains an age of −1. What can the range check establish, and what can it not establish?
- A value of 3,600 appears in a variable defined as annual earnings in dollars. Name two pieces of information you would check before converting it.
- A codebook says −99 means “not applicable.” Why is it incorrect to treat −99 as a measured value, and why should the codebook be checked before excluding other negative values?
- A verified, unusually high income shifts the sample mean substantially. Does that make the record a coding error? Explain the distinction between unusualness and influence.
- What should be recorded when a flagged value cannot be verified or corrected?