Why Input Anomalies Matter
A prediction is only as meaningful as the inputs supplied to the model. A value can be surprising because it is wrong, because it is valid but rare, or because a data pipeline has changed how a field is recorded. Those possibilities require different responses. Automatically deleting every unusual value can remove legitimate cases; accepting every value without investigation can produce predictions that the model was not designed to handle.
Input checks are therefore best treated as screening and diagnosis. They help identify records or batches that merit investigation. A flag is evidence that a stated rule was triggered; it is not, by itself, proof of a data error. This distinction is important when communicating limitations and operational risks in work aligned with the SOA Exam PA (Predictive Analytics) syllabus.
Three Useful Checks
A reliable check begins with what the field is supposed to represent, how it is encoded, and how that definition is expected to appear in the data. It is not enough to decide that a value “looks strange.” The rule and the reason for it should be documented.
- Range checks compare numeric values with defensible limits. A hard limit follows from the field’s definition or system constraints; a review threshold marks values that are unusual but can still be valid. These two types of limits should not be confused.
- Category checks compare recorded labels or codes with the documented set of allowed values. A new value can represent a typo, an undocumented category, or an intentional change to coding.
- Historical input comparisons compare current records with appropriately defined earlier records. These comparisons can reveal changed units, formats, defaults, or mappings. They do not establish the cause of a difference on their own.
Checks should be specific to a field and its use. For example, an age field might have a hard lower bound determined by the population being modeled, while an age near the upper end of the observed range could be valid and simply uncommon. A threshold based only on past observations should generally prompt review rather than automatic rejection: the historical range may not contain every valid future value.
For a batch of \(n\) records, the rate for a particular check is the number of records triggering it divided by \(n\). This is a descriptive measure of the batch, not a probability that any flagged record is wrong. Interpretation requires a clear denominator and a consistent rule for what counts as a record.
Worked Example: A Value Outside the Defined Range
A hypothetical model uses the applicant’s age in completed years. The documented input rule for this particular application population requires an integer age from 18 through 95, inclusive. A batch contains ages 34, 95, 96, 17, and 62. Identify the records that fail the hard range check, calculate the flag rate, and explain what the check does and does not establish.
Method. Apply the documented inclusive limits to each value: a record passes exactly when \(18 \leq \text{age} \leq 95\).
Age 34 passes because \(18 \leq 34 \leq 95\). Age 95 passes because the upper limit is inclusive. Age 96 fails because \(96>95\), and age 17 fails because \(17<18\). Age 62 passes. Thus, 2 of the 5 records are flagged.
The check establishes that two records conflict with the stated input rule. It does not establish whether either age was mistyped, whether the records belong in this application population, or whether the rule was implemented incorrectly. The appropriate next step is to check the source records and the field definition. Common mistake: treating age 95 as out of range by reading “through 95” as “less than 95.” The correction is to apply the stated inclusive condition; 95 passes.
Category Checks and Historical Comparisons
Category checks require an authoritative list of permitted values and, where applicable, a documented mapping from source codes to model inputs. A value that is not in the expected set should be flagged for review, not silently recoded to a convenient category. Otherwise, a new category, a changed source code, and a simple typo may all be treated as if they meant the same thing.
Historical comparisons can help distinguish these possibilities. Useful comparisons include field names, data types, units, category labels, ranges, and a small number of descriptive summaries. The comparison is informative only if the reference period and current data use compatible definitions. A difference may arise because the pipeline changed, because the population or operations changed, or because an earlier data problem was not detected. A historical match does not guarantee that current inputs are correct, and a historical difference does not prove that they are wrong.
Worked Example: Checking a New Category Code
A hypothetical service model expects the field “contact route” to contain one of three documented labels: “office,” “phone,” or “web.” Earlier records used these labels. In a current batch of 40 records, 17 contain “office,” 14 contain “phone,” 7 contain “web,” and 2 contain “mobile.” Determine the category-check result and the rate of records with an unrecognized label. State a reasonable first investigation.
Method. Compare each distinct recorded label with the documented allowed set. “Office,” “phone,” and “web” are permitted; “mobile” is not in that set. The number of records with an unrecognized label is therefore 2, not 1: there are two records carrying the label “mobile.”
The check flags two records. The first investigation is to inspect the source-system definition and the relevant records: “mobile” might be a new route, an undocumented label, or a coding error. If the source confirms that “mobile” is a valid new category, the model input process and its documented category mapping need review before predictions are relied on. Common mistake: counting the number of distinct unexpected labels as the number of flagged records. Here, there is one unexpected label but two flagged records.
Historical input comparisons are especially useful when values have plausible-looking magnitudes but a different unit or representation. A batch in which a monetary field has moved from thousands of dollars to dollars may still contain numbers that look numeric and valid. The scale change is a reason to investigate the pipeline, not a license to divide values automatically: some records could be mixed, or a different field could have been supplied.
Worked Example: Suspecting a Unit Change
A hypothetical model expects annual benefit amounts in thousands of dollars. Historical records for this field are typically between 20 and 90, with the documented unit “thousands of dollars.” In a current batch, four values are 42, 57, 63, and 51,000. A separate source-system note says that amounts from a recently changed feed may be stored in dollars. Identify the concern, calculate the share of records that exceed a review threshold of 500, and decide whether the flagged value should be automatically changed.
Method. Check the values against the expected representation and the stated review threshold. The first three values are within the familiar historical scale. The fourth, 51,000, exceeds 500, so one of the four records is flagged.
The value 51,000 is consistent with a possible dollar-versus-thousands-of-dollars discrepancy: \(51{,}000\) dollars would equal \(51\) thousand dollars because \(51{,}000 \div 1{,}000=51\). But that calculation only describes one plausible interpretation. It does not prove that the source value is 51,000 dollars or that dividing by 1,000 is the correct repair. Confirm the source field’s unit and the pipeline transformation, then correct the data through a documented process if the evidence supports it. Common mistake: changing 51,000 to 51 solely because that value resembles the historical range. The correction is to flag it, investigate its provenance, and transform it only when the unit change is confirmed.
Combining Flags Without Overstating Them
A record may trigger more than one check. For example, one row may have both an out-of-range amount and an unknown category. Adding the counts from separate checks would count that row twice. When the question is how many distinct records need review, count the union of the flagged sets.
Let \(A\) be the set of records failing a range check and \(B\) the set failing a category check. A record in both sets belongs in the total only once. The reason for the adjustment is that the overlap \(|A \cap B|\) is included once in \(|A|\) and again in \(|B|\); subtracting it once leaves each record in the union counted exactly once.
Worked Example: Counting Distinct Records for Review
A batch contains 120 records. A range check flags 9 records, a category check flags 7, and 3 records fail both checks. Calculate the number and proportion of distinct records that trigger at least one check.
Method. Use the union count, subtracting the overlap once:
Thus, 13 distinct records trigger at least one check. The proportion is \(13/120=0.10833\ldots\), or 10.83% rounded to two decimal places. As a check, the range-only count is \(9-3=6\), the category-only count is \(7-3=4\), and the overlap is 3; \(6+4+3=13\). Common mistake: reporting \(9+7=16\) distinct records. That double-counts the three records that fail both checks.
A Practical Investigation Process
A useful anomaly process separates detection from diagnosis and action. It also preserves enough information for a reviewer to understand what was checked and what was done. The specific follow-up depends on the field, the model, and the source system; a generic threshold cannot replace that context.
Record the field definition, unit, permitted range or categories, and any documented transformations. Distinguish hard constraints from review thresholds.
Run the checks on the same fields and under the same rules used to prepare inputs for prediction. Record both flagged counts and the denominator.
Check whether historical records use compatible definitions, units, and coding. Treat differences as clues to investigate, not as diagnoses.
Review source records, transformation logic, and change documentation. Correct, recode, or exclude data only when the evidence and approved process support that action.
Keep the rule, flag count, investigation, outcome, and any unresolved uncertainty. Explain how unresolved inputs affect the interpretation or use of predictions.
A clear report distinguishes what is known from what remains uncertain. “Two records contain a label not in the documented set” is a check result. “The source pipeline introduced a new category” is a causal explanation and should be stated only if the investigation supports it. If the cause cannot be confirmed, report the uncertainty and the operational consequence rather than implying certainty.
Common Traps
- Equating unusual with invalid. A rare but valid value can be important. Use hard limits only when the field definition supports them; use review thresholds to prompt investigation.
- Silently repairing flagged values. A plausible unit conversion is still an assumption until verified against source definitions or records.
- Comparing incompatible periods. A historical comparison is misleading if the field definition, unit, population, or extraction process differs. Check comparability before interpreting the difference.
- Reporting counts without denominators. “Eight records were flagged” is difficult to interpret without stating how many records were checked and which rule was applied.
- Double-counting records. When checks overlap, count distinct records for an overall review total and show individual check counts separately where helpful.
Check Your Understanding
For each problem, identify what the check supports and avoid assuming more than the stated evidence allows.
- A hypothetical model accepts severity scores from 0 through 500, inclusive. A batch has values 0, 42, 612, and 35. Which records fail the hard range check? (Answer: Only 612 fails; 0 is allowed by the inclusive lower bound.)
- A model expects a predictor as a proportion between 0 and 1. Historical values are recorded on that scale. A current batch contains 0.74, 74, 0.68, and 0.81. Calculate the share exceeding 1 and state whether 74 should automatically be changed to 0.74. (Answer: \(1/4=25\%\) exceed 1; do not automatically change 74 without confirming the unit or representation.)
- A documented category field permits “north,” “south,” and “west.” Among 18 current records, 8 are “north,” 6 are “south,” 2 are “west,” and 2 are “central.” What share of records have an unrecognized category? (Answer: \(2/18=0.1111\), or 11.11%.)
- In a batch of 100 records, 5 fail a range check, 4 fail a category check, and 2 fail both. How many distinct records fail at least one check, and what is the percentage? (Answer: \(5+4-2=7\) records, or 7%.)
- A hypothetical model expects monthly amounts in hundreds of dollars. Historical records are usually between 2 and 30. A new value is 1,800, and a source note says a feed may now use dollars. What is the value if 1,800 is confirmed to be dollars, expressed in hundreds of dollars, and what action is justified before confirmation? (Answer: \(1{,}800/100=18\) hundreds of dollars; flag and investigate the unit, but do not convert the value before confirming the feed’s representation.)