Choose a Treatment for a Reason
An outlier rule identifies an observation for review; it does not decide what should happen to that observation. A valid extreme claim, a misplaced decimal, and a record accidentally joined to the wrong policy can all appear unusually large, but they call for different actions. Treating all flagged records alike can distort the data just as surely as overlooking a genuine error.
The decision should follow the data’s meaning and the model’s purpose. A plausible value that belongs to the population being studied is usually retained. A flag can preserve a record while making it possible to examine or model a distinct pattern. A value should be corrected only when reliable evidence establishes what it should have been. A record should be deleted only when evidence shows it is invalid or outside the defined population—not merely because it is extreme.
The earlier material on distinguishing errors from plausible extreme values establishes the key starting point: unusualness is a screening signal, not proof of error. Context, source records, units, definitions, and the intended target population provide the evidence for a treatment decision.
A Decision Framework
Before changing a data set, state the purpose of the analysis and the unit represented by each row. Then examine the flagged records and ask what the evidence supports. The steps below distinguish data correction from a modeling choice.
Specify which people, policies, transactions, or time periods belong in the analysis. A record outside that definition may be excluded, but it should not be described as an outlier deletion if the actual reason is ineligibility.
Look for unit errors, misplaced decimal points, impossible dates, duplicate joins, and conflicts with source documentation. A value that merely looks improbable is not enough evidence to edit it.
Retain a valid observation. Add a flag when its status needs to remain visible. Correct a confirmed error using a traceable source. Delete a record only when it is invalid or does not belong to the target population.
Ask whether the flag or the information that defines it will be available when predictions are made. A flag derived from the outcome after it occurs can describe historical data, but using it as a predictor for a future outcome may cause leakage.
Worked Examples
Worked Example: Retaining a Valid Extreme Claim
A hypothetical insurer reviews seven paid claim amounts, in thousands of dollars: 18, 22, 25, 27, 28, 30, and 150. Source files confirm each payment, and the largest claim resulted from a covered fire loss. The modeling task is to describe paid severity for this portfolio. Calculate the mean with all valid claims and the mean if the largest claim were deleted. Recommend a treatment.
Method. First calculate the summary using the defined portfolio. Then compare it with the result of deleting the verified extreme, not to justify deletion, but to show the size of the change.
The total of all seven claims is \(18+22+25+27+28+30+150=300\) thousand dollars. Thus the mean is \(300/7=42.8571\) thousand dollars, or about $42,857.14.
If the $150,000 claim were deleted, the remaining total would be \(300-150=150\) thousand dollars across six claims. The resulting mean would be \(150/6=25\) thousand dollars, or $25,000. The deletion reduces the mean by \(42.8571-25=17.8571\) thousand dollars, about $17,857.14.
Decision. Retain the claim because it is verified and belongs to the target portfolio. A flag can identify it for review or support a separate descriptive analysis. Whether that flag is appropriate as a predictor depends on whether the relevant information is available at prediction time. For example, a label assigned after a claim is paid cannot be used to predict that claim’s severity in advance.
Check. The retained mean is greater than the mean after deletion, as expected: the deleted value, 150, is much greater than the other six values. The common mistake is to call the record erroneous because it has a large effect on the mean. Influence on a summary is not evidence that a valid record is wrong.
Worked Example: Correcting a Confirmed Decimal Error
A hypothetical policy file records annual service costs, in thousands of dollars, as 1.2, 1.4, 1.5, 1.6, and 16.0. The source invoice for the last record shows $1,600, and the file’s unit is thousands of dollars. The value was entered as 16.0 rather than 1.6. Calculate the means before and after correction, and compare correction with deleting the record.
Method. Correct only the value that the source evidence establishes. Keep the record because the policy belongs to the population and the invoice confirms its cost.
Before correction, the sum is \(1.2+1.4+1.5+1.6+16.0=21.7\) thousand dollars. The mean is \(21.7/5=4.34\) thousand dollars.
The source supports changing 16.0 to 1.6. After correction, the sum is \(1.2+1.4+1.5+1.6+1.6=7.3\) thousand dollars, and the mean is \(7.3/5=1.46\) thousand dollars.
If the record were deleted instead, the other four values sum to \(1.2+1.4+1.5+1.6=5.7\) thousand dollars. Their mean is \(5.7/4=1.425\) thousand dollars. This is not the mean of the five verified costs: it excludes a valid policy and its correctly recorded $1,600 cost.
Decision. Correct the documented decimal error, retain the policy, and preserve an audit trail of the original and corrected values. The common mistake is to delete the whole row because one field is wrong. That discards valid information about the policy and is not supported by the evidence.
Check. The corrected total, 7.3, equals the sum of the five source-supported amounts, and \(7.3/5=1.46\). The deletion mean uses a different denominator because it has only four records; it answers a different question.
Worked Example: Deleting a Record Outside the Target Population
A hypothetical analysis is intended to describe annual payments for policies in force during a specified calendar year. A joined data file contains five payment values, in dollars: 800, 900, 950, 1,000, and 10,000. Review of the policy identifier and coverage dates shows that the $10,000 record belongs to a policy that was not in force during that year; it entered through an incorrect join. Calculate the file mean and the eligible-population mean, then state the action.
Method. Apply the stated eligibility definition, using source identifiers and dates as evidence. Do not make the decision based on the payment amount alone.
The five-record total is \(800+900+950+1{,}000+10{,}000=13{,}650\) dollars. Its mean is \(13{,}650/5=2{,}730\) dollars.
The first four records meet the in-force definition. Their total is \(800+900+950+1{,}000=3{,}650\) dollars, and their mean is \(3{,}650/4=912.50\) dollars.
Decision. Exclude the $10,000 row from this analysis because the verified coverage dates show it is outside the target population. Document the join error and retain the original source record for auditability. The common mistake is to describe this as deleting a large outlier. The decisive evidence is that the row is ineligible, not that its payment is unusually high.
Check. The eligible count is four, so the eligible mean uses denominator four. If the policy had been in force and the payment verified, its extremeness alone would not justify excluding it.
Why Deleting Flagged Values Changes a Mean
The arithmetic shows why deletion needs justification. Let \(n\) observations have mean \(\bar{x}\), and let \(k\) flagged observations have mean \(\bar{x}_F\). The remaining \(n-k\) observations have mean \(\bar{x}_U\). The original total is both \(n\bar{x}\) and the sum of the flagged and unflagged totals:
Divide by \(n\), and let \(p=k/n\), the fraction flagged. Then
The second equation follows by subtracting \(\bar{x}_U\) from both sides of the first. It shows that the change caused by removing the flagged group depends on both its proportion and how far its mean is from the unflagged mean. This is an arithmetic identity, not a claim that either mean is a better estimate of the target population value. If valid extreme cases are systematically more common in part of the population, deleting them can change which population the analysis describes.
Using a Flag Without Hiding the Observation
A flag can preserve the original value while making a distinction available for review or analysis. For example, a portfolio may retain all verified costs and add an indicator for records that pass a prespecified review rule. This can help analysts check whether flagged and unflagged records behave differently. It does not make the flag automatically useful, statistically stable, or suitable as a predictor.
For a binary flag \(F\) taking values 0 or 1, consider the simple least-squares model \(Y=\beta_0+\beta_1F+\epsilon\), fitted with an intercept and with observations in both groups. The fitted mean for \(F=0\) is \(\beta_0\), and the fitted mean for \(F=1\) is \(\beta_0+\beta_1\). Least squares chooses these group fitted values to equal the respective group means, so \(\beta_1=\bar{Y}_{F=1}-\bar{Y}_{F=0}\). Thus, in this simple model, the flag coefficient describes a difference in group means. It does not establish that the flag causes the difference, and it does not account for other predictors.
A flag should be defined consistently and recorded with enough information to reproduce why a value was marked. A threshold chosen after inspecting the outcome can make reported patterns look stronger than they are. A flag based on a future event, such as a later claim investigation result, is not available for a prediction made before that event. Using it as an input would give the model information it could not have at deployment.
Common Traps and Documentation
- Deleting because a rule flagged a value. A fence or other screening rule identifies a candidate for examination; it does not establish a data error or exclusion criterion.
- Correcting by intuition. Replacing a value with a more typical number is not a correction. A correction needs traceable evidence such as a source file, unit definition, or verified transaction record.
- Deleting a valid observation to improve fit. A better-looking model fit is not evidence that a record is invalid. Removing valid cases can change the population represented and the model’s predictions.
- Using a flag unavailable at prediction time. A flag derived from a realized outcome or a later investigation may be useful for historical review but can create leakage in a predictive model.
- Failing to record the decision. Document the record identifier, evidence, treatment, analysis scope, and any original value. This makes it possible to explain and reproduce the data preparation.
The defensible choice is the one that matches the evidence and target population. Retain valid extremes, use flags to make relevant distinctions visible, correct only established errors, and delete only records shown to be invalid or out of scope. These decisions are aligned with the SOA Exam PA (Predictive Analytics) syllabus because they connect data quality judgments to the meaning and reliability of a predictive analysis.
Check Your Understanding
For each situation, choose a treatment and check the numerical consequence where requested.
- A hypothetical portfolio has verified annual costs of 12, 14, 15, 18, and 81 thousand dollars. Calculate the mean with all records and the mean after deleting 81. Should the value be deleted solely because it is largest? (Answer: The full mean is \(140/5=28\) thousand dollars; after deletion it is \(59/4=14.75\) thousand dollars. No; verification and population membership support retaining it.)
- A file lists five verified eligible values, in dollars: 45, 48, 50, 52, and 500. The source shows that the last entry should be $50, not $500. What treatment is supported, and what is the corrected mean? (Answer: Correct the confirmed error and retain the record; the corrected mean is \((45+48+50+52+50)/5=49\) dollars.)
- A flagged group is 10% of a data set, with mean 240; the unflagged group has mean 40. What is the full-data mean, and how much does it exceed the unflagged mean? (Answer: \(0.10(240)+0.90(40)=60\); it exceeds the unflagged mean by 20.)
- A payment is unusually large but is supported by the source document and belongs to the defined policy-year population. A proposed flag uses a status assigned only after a later claim investigation. Should the payment be deleted, and can that flag be used to predict payment before the investigation? (Answer: Retain the verified in-scope payment; do not use the later investigation flag as a pre-investigation predictor because it is unavailable at prediction time.)
- A joined file includes a high payment record whose policy dates show it was never in force during the analysis year. What evidence supports exclusion, and how should the action be described? (Answer: The verified policy dates establish that the record is outside the target population; exclude it from this analysis as an ineligible record, not merely because it is an outlier.)