Why an Observed Range Needs Context
A minimum and maximum can quickly reveal whether a data set deserves closer inspection. But an endpoint is meaningful only in relation to what the variable measures, which records are included, and what values the intended population can have. A maximum age of 78 could be entirely reasonable in a data set of policyholders, but inconsistent with a data set guaranteed to contain only applicants aged 20 to 60. The number alone cannot tell you which description is correct.
This tutorial focuses on that comparison. It does not treat every unusually small or large value as an error. Instead, it separates values that violate a justified rule from plausible extremes and from clues that the data may represent a wider population than expected.
Set the Variable and Population Before Judging the Endpoints
Before comparing endpoints with domain knowledge, identify the exact variable definition and the intended population. A value may be reasonable for one definition and inconsistent with another. For example, an age recorded at policy issue is not the same variable as a person’s current age. Likewise, an insurance loss is not necessarily the same as the amount paid after a contractual limit.
A useful comparison requires a rule that actually applies to the recorded values. It might be a physical limit, a documented eligibility condition, a contractual cap on the specific field, or a data-collection rule. A general expectation, such as “most applicants are younger,” is not a hard bound. It may make an endpoint worth investigating, but it cannot establish that the value is impossible.
Confirm the unit, timing, and meaning of the recorded quantity. Check, for example, whether an amount is a loss, a payment, or a payment after a limit.
State who or what should be represented, including any eligibility rules and relevant time period.
Use documented requirements, not a typical value or personal expectation. Record whether each bound is inclusive.
Identify which endpoints conflict with the rules, then check the source, coding, units, and scope before deciding what happened.
If every nonmissing observed value \(x_i\) is required to satisfy \(L \leq x_i \leq U\), then the observed endpoints must satisfy \(x_{\min} \geq L\) and \(x_{\max} \leq U\). The condition concerns every nonmissing observed record, not merely records already judged valid. If an invalid value remains in the data, it can determine the observed minimum or maximum and violate the endpoint inequalities. Alternatively, the endpoints can be calculated after filtering to records known to be valid, but then they describe that filtered set rather than all observations.
The justification follows directly from the definitions. If every value in a nonempty set is at least \(L\), its smallest member cannot be less than \(L\); otherwise that member would contradict the condition. Similarly, if every value is at most \(U\), its largest member cannot exceed \(U\). This result checks consistency with a bound. It does not prove that a value is accurate, that the records are in scope, or that all parts of the target population were observed.
Three Interpretations of a Range
When an endpoint differs from an expectation, distinguish three possibilities. First, it may be impossible under a firm rule that applies to the field and all intended records. Second, it may be plausible but unusual: extreme values can occur even when most observations are much less extreme. Third, it may be plausible but out of scope, suggesting that records from a broader or different population have entered the data. These possibilities call for different follow-up; the minimum and maximum alone do not determine which one is correct.
“Impossible” should be used narrowly. A measurement may be outside a system’s permitted operating range, or a documented eligibility condition may exclude it. By contrast, a value that is rare, unexpected, or far from the average is not necessarily impossible. The earlier material on detecting suspicious values from summary output helps identify what to review; domain knowledge is needed to assess what the endpoints mean.
Worked Example: An Age Outside a Documented Eligibility Range
A hypothetical insurer prepares an analysis file described as containing applications for a product available only to people aged 18 through 64, inclusive. The numeric age-at-application field has the nonmissing values 22, 35, 18, 64, and 71, plus one missing value. Find the observed range and assess it against the stated population.
Method. Exclude the missing value when calculating endpoints. The sorted nonmissing values are 18, 22, 35, 64, 71. Therefore, \(x_{\min}=18\), \(x_{\max}=71\), and the width is \(71-18=53\) years.
The documented bounds are \(L=18\) and \(U=64\). The minimum satisfies \(18 \geq 18\), but the maximum does not satisfy \(71 \leq 64\). Thus at least one observed nonmissing value conflicts with the stated eligibility condition. In this list, the conflicting value is 71.
The correct conclusion is that the value and the data description need reconciliation. Possible explanations include a data-entry or transformation error, an incorrectly stated age definition, or records from another product or population. The range does not identify which explanation is correct. If the file is guaranteed to contain only eligible applications and age at application is recorded correctly in years, 71 cannot be valid under the stated rule.
Check. The lower bound is met exactly, as an inclusive bound allows age 18. The upper bound is exceeded by \(71-64=7\) years. The common mistake would be to label 71 a typo solely because it is high; the justified conclusion is a rule conflict requiring investigation, not a diagnosis of its cause.
Worked Example: A Contractual Maximum on the Recorded Payment
In a hypothetical data set, each value is the payment for one covered claim under a policy that caps payment per claim at $10,000. Assume the field excludes fees and additional payments, and each row represents exactly one claim. The nonmissing payments, in dollars, are 240, 875, 2,100, 9,650, and 12,400. Assess the observed range against the contract.
Method. The values in increasing order are 240, 875, 2,100, 9,650, 12,400. Thus \(x_{\min}=240\), \(x_{\max}=12{,}400\), and the observed width is \(12{,}400-240=12{,}160\) dollars.
Under the stated field definition and assumptions, the permitted payment interval is \(0 \leq x_i \leq 10{,}000\). Four payments satisfy the upper bound; the value $12,400 exceeds it by \(12{,}400-10{,}000=2{,}400\) dollars. The observed maximum therefore conflicts with the contractual cap as described.
This conclusion depends on recording payment, not underlying loss. An underlying loss can exceed a policy limit even when the payment does not. If the field instead included multiple payments, fees, or amounts aggregated across claims, the $10,000 rule might not apply to each recorded value. The problem explicitly rules those out, so they are not explanations consistent with its assumptions; in real analysis, confirm the definition and contract before classifying the value.
Check. The minimum is above zero and within the stated interval; the maximum is above the cap. The common mistake is to apply a payment limit to a loss field—or to overlook aggregation. Here, because the field and row definition match the cap, at least one record warrants investigation.
Plausible Extremes and Broader Population Coverage
A range can also point to a mismatch between the data and the intended population without showing an impossible measurement. Suppose a file is described as covering new applicants aged 20 through 60, but its age values run from 19 to 78. Those ages are possible human ages; the issue is whether the file really contains only the stated applicants. It could combine new applications with renewals, conversions, or records from another product. If the source documentation confirms that broader scope, the values may be correct even though the original description was incomplete.
Conversely, a wide range need not signal wider coverage. Some target populations genuinely contain diverse values, and rare but valid observations may set an endpoint. Avoid using “typical” as if it meant “permitted.” A high but plausible endpoint can be important to investigate, yet it does not establish data contamination or an error.
Worked Example: Plausible Ages That Suggest a Scope Question
A hypothetical extract is initially described as containing new term-life applications for ages 20 through 60. The nonmissing ages are 19, 28, 43, 60, and 78. A source note says the extract may also include renewal records, but does not confirm whether it does. Find the range and decide what can be concluded.
Method. The sorted values are 19, 28, 43, 60, 78. The observed minimum is 19, the maximum is 78, and the width is \(78-19=59\) years. Against the new-application bounds, 19 is one year below the lower bound and 78 is 18 years above the upper bound.
Neither endpoint is inherently impossible as a person’s age. Instead, both conflict with the stated new-application scope. The note about possible renewal records gives a plausible scope explanation, but it does not prove that explanation. The right next step is to check the extract’s inclusion rules or source records and clarify whether the file combines record types.
If renewal records are included, the data description should be revised and analyses should use the intended population explicitly. If the file is confirmed to contain only new applications, the two endpoint records conflict with eligibility and should be investigated. The range alone cannot tell whether the note is correct, whether the eligibility description is wrong, or whether the values were recorded incorrectly.
Check. The arithmetic is \(78-19=59\). The common mistake is to call the ages impossible or to remove them automatically. The evidence supports a scope question, not a final decision about validity.
Using R Without Misreading Missing Values
For a numeric vector named age, R’s range(age, na.rm = TRUE) returns the smallest and largest nonmissing values when at least one nonmissing value is present. Check that condition before interpreting the result: if every value is missing, range(age, na.rm = TRUE) returns Inf and -Inf, with a warning, rather than meaningful observed endpoints. An all-missing variable has no observed minimum or maximum.
One safe approach for numeric data is to test whether any nonmissing values exist before asking for the range. In R, the logic can be written as: if sum(!is.na(age)) == 0, return c(NA_real_, NA_real_); otherwise return range(age, na.rm = TRUE). The first branch represents unavailable endpoints; the second computes the range only when at least one value is available. Also check whether infinite values are possible in the data: Inf is not missing, so a nonmissing-value check alone does not establish that all observations are finite or valid.
R’s range is a summary of observed values, not a validation procedure. It does not know the eligible population, units, contractual terms, or field definition. A range calculation should therefore be followed by a comparison with documented rules and, when necessary, checks of record scope and source data.
Common Traps and Practical Conclusions
- Applying a bound only to records already called valid. That does not imply that the minimum and maximum over all observed records obey it. State that the bound applies to every nonmissing observed record, or calculate endpoints on a clearly defined valid subset.
- Treating an expected value as a hard limit. “Most ages are below 60” does not imply that no age can exceed 60. Use a hard bound only when the variable definition or population rule supports it.
- Confusing field definitions. A cap on payment may not cap loss; a date of birth may not be age at application. Confirm the recorded quantity before comparing it with a rule.
- Assuming an all-missing vector has a range. There are no nonmissing endpoints to report. Handle that case separately rather than interpreting R’s infinite sentinels as data.
- Claiming the range proves the data are clean. Endpoints within bounds do not verify every record’s accuracy, and a plausible endpoint does not prove that the record belongs to the intended population.
A careful range assessment states the observed endpoints, the relevant population or field rule, and exactly how they compare. If they conflict, report the conflict and identify what needs checking. If the endpoints are plausible but raise a scope question, verify the population description rather than assuming the records are errors. This disciplined interpretation is aligned with the SOA Exam PA (Predictive Analytics) syllabus.
Check Your Understanding
For each situation, use the stated field definition and population rules. Distinguish a rule conflict from evidence that merely calls for a scope check.
- A hypothetical sensor records temperatures in degrees Celsius, and its documented operating interval is inclusive from -20 to 50. The nonmissing readings are -3, 8, 49, and 52. State the observed minimum and maximum, the range width, and whether an endpoint violates the interval. (Answer: Minimum -3, maximum 52, width 55 degrees; the maximum exceeds the upper bound by 2 degrees.)
- A numeric vector contains three missing values and no other entries. What should be reported for its observed minimum and maximum, and why should the result of range(x, na.rm = TRUE) not be interpreted as data endpoints? (Answer: Both endpoints are unavailable because there are no nonmissing observations; R returns Inf and -Inf with a warning.)
- A hypothetical age field is measured at renewal. The intended population has a documented minimum age of 18 but no stated maximum age. Its nonmissing values are 17, 18, 44, and 92. Give the observed range and identify the rule conflict. (Answer: Minimum 17, maximum 92, width 75 years; age 17 violates the lower bound, while 92 does not violate any stated upper bound.)
- A hypothetical payment field is defined as one payment per covered claim, with a documented cap of $5,000 and no additional amounts included. Values are $800, $1,600, $5,000, and $4,800. Find the observed endpoints and decide whether the maximum conflicts with the cap. (Answer: Minimum $800, maximum $5,000; no value exceeds the cap, and the maximum meets the inclusive limit.)
- A file is described as containing applicants aged 18 through 65. Its nonmissing ages are 18, 26, 51, 65, 17, and 72. Calculate the range width, count the values outside the documented interval, and state what the range alone establishes. (Answer: Minimum 17, maximum 72, width 55 years; two values are outside the interval, but the range alone does not establish whether they are errors or out-of-scope records.)