Why Numeric Codes Need Care
A data column can contain integers without measuring a quantity. A value such as 4 might mean “Plan D,” “region 4,” or a particular claim type. If so, the difference between 4 and 2 is not necessarily a meaningful amount, and the value 4 is not necessarily “twice” the value 2. In R, leaving such a column numeric can encourage summaries and models to treat the codes as measurements.
The cleaning decision is therefore about meaning, not storage type. A count of claims is quantitative even if it is stored as an integer. A code that labels a product group is categorical even if the labels are integers. Before converting, confirm the codebook or other data definition: a safe conversion preserves valid categories, exposes unexpected codes, and does not silently recast missing or invalid values as legitimate groups. This data-cleaning practice is aligned with the SOA Exam PA (Predictive Analytics) syllabus.
Deciding Whether a Numeric Column Is a Category
Start by asking what one unit in the column represents. If the value records an amount or count, arithmetic may be meaningful. If it identifies a group, arithmetic on the code is generally not meaningful. A codebook can settle cases that the data alone cannot. For example, “number of dependents” may be a quantitative count, whereas “customer segment 1, 2, or 3” may use numbers solely as labels.
Also distinguish a category with a genuine order from a numeric measurement. Codes for “low,” “medium,” and “high” can have an order, but the gaps between the codes need not represent equal changes. A set of product identifiers may have no order at all. The fact that one code is larger than another does not establish either ordering or distance.
In R, `factor(x)` and `as.factor(x)` create a factor. For a numeric vector, `factor(x)` uses the distinct observed values as levels, ordinarily arranged in increasing order. That default can be convenient, but it does not verify that every observed code is valid or that the level order has substantive meaning. When the valid codes are known, specify them explicitly and inspect the result.
Use the variable definition or codebook to decide whether the integers measure a quantity or label groups.
Compare observed nonmissing values with the documented set before conversion.
Use explicit factor levels, and provide labels when they make the categories clearer.
Inspect levels, counts, and missing values; confirm that each original valid code maps to the intended category.
Worked Examples: Safe Conversion and Its Consequences
Worked Example: Mapping Severity Codes to Named Categories
A hypothetical insurer’s data dictionary defines severity code 1 as “minor,” code 2 as “moderate,” and code 4 as “major.” A sample column is `c(1L, 2L, 4L, 2L, NA, 4L)`. Convert it to a factor and verify that the mapping preserves the observed information.
Method. Use the documented codes as factor levels and the corresponding words as labels. In R, write `severity_f = factor(severity_raw, levels = c(1, 2, 4), labels = c("minor", "moderate", "major"))`.
The six input entries map in order to “minor,” “moderate,” “major,” “moderate,” missing, and “major.” The codebook’s numeric gap from 2 to 4 does not make “major” two units above “moderate”; the values are category codes.
A frequency check with `table(severity_f, useNA = "ifany")` gives minor: 1, moderate: 2, major: 2, and NA: 1. The count total is \(1+2+2+1=6\), matching the six input records. The valid codes and the missing entry are all accounted for.
Common mistake avoided. Treating code 4 as a measured severity of four, or assuming its distance from code 2 has quantitative meaning, would add an interpretation not supported by the codebook.
Worked Example: Catching an Unexpected Code Before Conversion
A hypothetical claims file uses codes 10, 20, and 30 for three service channels. The observed values are `c(10L, 20L, 30L, 20L, 40L, NA)`. Determine whether the column can be converted safely using the documented codes, and show why validation must come first.
Step 1: Compare observed codes with the valid set. Set `allowed = c(10, 20, 30)` and find nonmissing values not in that set. In R, `unexpected = setdiff(unique(channel_raw[!is.na(channel_raw)]), allowed)` returns 40. The observed valid-code counts are 10: 1, 20: 2, and 30: 1; there is also one missing value and one unexpected value.
Step 2: Do not silently accept the mismatch. If we immediately run `factor(channel_raw, levels = allowed)`, the 40 is converted to NA because it is not one of the specified levels. A frequency table would then show two missing values: the original NA and the invalid 40. That loses the distinction between “not recorded” and “recorded with an unrecognized code.”
Step 3: Resolve and verify. Check the source definition or correct the underlying record before conversion. If the codebook confirms that 40 is an error to be repaired, make that correction explicitly and document it. Only after validation should the conversion be made: `channel_f = factor(channel_raw, levels = allowed, labels = c("agent", "web", "phone"))`. For the four valid records, the resulting counts are agent: 1, web: 2, and phone: 1. The source NA remains missing.
Check. The four valid observations plus one original missing value total five records after the unexpected entry is resolved according to the data-quality decision. The original file had six records; the discrepancy makes clear why an unexpected code must not disappear without investigation.
Common mistake avoided. Specifying levels is not a substitute for validating them: unmatched values become missing, potentially masking an error.
Worked Example: Why a Factor’s Numeric Conversion Is Not the Original Code
A hypothetical portfolio field uses numeric identifiers 10, 20, and 50 for three coverage types. The observed codes are `c(10L, 20L, 50L, 20L)`. Show what happens when the values are factored and then passed to `as.numeric()`.
Step 1: Create the factor. With `coverage_f = factor(coverage_raw)`, R uses levels 10, 20, and 50. Internally, the four observations are represented by the indices 1, 2, 3, and 2, respectively. These indices refer to positions in the factor’s level list.
Step 2: Inspect the tempting conversion. `as.numeric(coverage_f)` returns `c(1, 2, 3, 2)`, not `c(10, 20, 50, 20)`. It returns the internal indices. The output sum is \(1+2+3+2=8\), whereas the original codes sum to \(10+20+50+20=100\). The different totals confirm that these are different quantities.
If the original numeric codes are specifically needed and the factor’s labels are still numeric strings, `as.numeric(as.character(coverage_f))` returns `c(10, 20, 50, 20)`. That operation parses the labels as numbers. It is not appropriate for semantic labels such as “basic” or “premium,” which are not numeric values.
Common mistake avoided. Applying `as.numeric()` directly to a factor does not recover the codes used to create it. Use the original vector when possible, or explicitly convert numeric text back to numbers only when that is genuinely the intended operation.
Why Numeric Modeling Can Misrepresent Categories
A numeric predictor in a simple linear model imposes a particular relationship between its values and the outcome. If \(C\) is used as a numeric predictor, the model’s conditional mean has the form
This formula implies that increasing the code by one changes the modeled mean by the same amount, \(\beta_1\), at every point. To see why, subtract the fitted means for any two codes:
The difference depends on the numerical distance \(c_2-c_1\). That is justified only if the codes represent a quantitative scale and a linear mean relationship is appropriate. For nominal category codes, the distance is arbitrary, so this constraint is generally unjustified. Treating the variable as a factor instead represents the groups as categories rather than asserting a linear effect per code unit. The particular model encoding and coefficient interpretation depend on the modeling method.
Worked Example: Checking Whether Code Spacing Has Meaning
A hypothetical service dataset assigns platform codes 1, 2, and 7 to “legacy,” “mobile,” and “partner,” respectively. The average annual service cost in a training sample is $100, $110, and $160 for those groups. Determine what a numeric-code interpretation would imply and whether the codebook supports it.
Numeric interpretation. The difference from code 1 to 2 is one code unit, and the observed mean difference is \(110-100=10\) dollars. The difference from code 2 to 7 is five code units, and the observed mean difference is \(160-110=50\) dollars. Both changes happen to equal $10 per code unit in these sample averages.
This arithmetic does not establish that the codes are quantitative. The codebook defines three platform labels, not a scale on which platform 7 is five units above platform 2. Relabeling the partner platform as code 3 would leave the real groups and their costs unchanged but would change the numeric distances. A numeric model could therefore change its fitted relationship merely because arbitrary labels changed.
Decision. Keep the platform field categorical unless the data definition supplies a meaningful quantitative scale. The sample averages are useful descriptions of the three observed groups; they do not turn identifier codes into measurements.
Common mistake avoided. Inferring a numeric trend from a convenient pattern in a small set of group averages confuses observed category differences with an effect per unit of a measured predictor.
Verification and Common Traps
A factor conversion is complete only when the resulting levels and counts agree with the intended meaning. Use `levels(x_factor)` to inspect the available levels and `table(x_factor, useNA = "ifany")` to inspect observed counts alongside missing values. These checks answer different questions: a level can be defined but have no observations, while a missing value is not a level unless it has been deliberately represented as an explicit category.
- Assuming all integers are quantities. Check whether the variable records an amount or merely identifies a group.
- Assuming numeric order is meaningful. Increasing codes do not prove either a real order or equal spacing.
- Letting unexpected codes become NA. Compare observed values with allowed values before factor conversion; investigate mismatches.
- Reading factor indices as original values. `as.numeric()` on a factor returns its internal level indices, not its original numeric labels.
- Relying on default level order without checking. Specify valid levels when appropriate and verify the resulting order and labels.
The central rule is simple: use the data definition to decide what a code means, then choose an R representation that preserves that meaning. Explicit levels, a pre-conversion validity check, and a post-conversion count check make this decision auditable and reduce the risk of encoding a category as a false measurement.
Check Your Understanding
For each problem, identify the appropriate factor-handling decision and check the final result.
- A hypothetical file records `c(3L, 5L, 3L, NA)` for a field whose codebook defines 3 as “urban” and 5 as “rural.” Give the factor counts, including missingness. (Answer: urban 2, rural 1, NA 1.)
- A numeric factor has levels 10, 30, and 90, and its observations are 10, 90, 30. What does `as.numeric()` return, assuming the default increasing level order? (Answer: 1, 3, 2.)
- A codebook permits 0 and 1, but the nonmissing observations are 0, 1, and 2. What should be checked before conversion, and what would a conversion with levels `c(0, 1)` do to code 2? (Answer: Investigate code 2 before conversion; with those specified levels, code 2 becomes NA.)
- A three-level categorical field uses codes 1, 2, and 8. Its groups have outcome means 40, 50, and 80. What assumption would a numeric linear predictor make about the mean change from code 2 to code 8? Is that assumption established by the codes alone? (Answer: It models a change of \(6\beta_1\), six times the per-code-unit slope; no, code spacing alone does not establish a quantitative scale.)
- A factor has levels “10,” “20,” and “40,” and observations corresponding to the first and third levels. What values result from `as.numeric()` on the factor, and what values result from `as.numeric(as.character(x))`? (Answer: The direct conversion gives 1 and 3; converting through character gives 10 and 40.)