Why the Distinction Matters
A factor value recorded as Unknown can look similar to a missing value: neither identifies an ordinary category such as “Standard” or “Preferred.” But they do not necessarily mean the same thing. A blank field may indicate that information was not collected or was lost. An explicit “Unknown” entry may mean that someone recorded a particular response or classification status. Treating these states as interchangeable can change summary denominators, which records are available to a model, and what a resulting category means.
In R, a factor can contain both ordinary category values and missing values. The missing value is represented by NA; the text label “Unknown” is an observed value if it is included as a factor level. The appropriate choice depends on the data’s meaning and the analysis goal, not on which representation is more convenient.
What R Treats as Missing and What It Treats as a Category
The distinction is operational as well as semantic. In a factor, `is.na(x)` identifies missing values. A value such as “Unknown” is not missing if it is an included factor level. For example, the levels `Standard`, `Preferred`, and `Unknown` describe three possible recorded categories, while NA indicates that no category value is available for that row.
When creating a factor, the set of levels determines which values are accepted as categories. If “Unknown” is meant to be a genuine recorded category, it must be included among the levels. A value that does not match any specified level is converted to NA; this can accidentally turn a label into missingness. Also distinguish the literal text “NA” from the special R value NA: the text is a character string, not R’s missing-value marker.
Missing values can affect model fitting. For many R modeling functions, the default missing-data action omits rows with missing values in variables used by the model, but the behavior depends on the function and the `na.action` setting. Check the fitted model’s row count and missing-data behavior rather than assuming every record was used. An explicit “Unknown” factor value is ordinarily a nonmissing predictor value, so its row can remain available to the model. Keeping a row does not, by itself, make the recorded category informative or appropriate.
Worked Example: Choosing the Denominator for a Category Proportion
A hypothetical service team reviews 120 applications. The recorded region values are North for 42 applications, South for 36, and Central for 24. The remaining 18 region fields are blank. Find the proportion of applications recorded as North among applications with a known region, and explain why it differs from the proportion of all applications known to be North.
Method. First identify which records belong in each denominator. The number with a recorded region is \(42+36+24=102\). The number of all applications is \(120\).
Among applications with a known region, the North proportion is \(42/102=0.4118\), or 41.18% to two decimal places. Among all applications, the fraction recorded as North is \(42/120=0.3500\), or 35.00% to two decimal places. The second fraction answers a different question: it counts all blank-region applications in the denominator, without claiming that they are non-North.
Check. The known-region counts sum to \(102\), and \(102+18=120\). The two proportions use the stated denominators. The common mistake is to call 35.00% the North share “among known regions”; its denominator includes 18 records without a region value.
Choosing Between NA and an Explicit Unknown Level
Use NA when the category is genuinely unavailable for the intended analysis—for example, a field was left blank, was not collected, or could not be recovered from the source. This representation preserves the fact that the category value is absent. It also makes missingness easy to count and inspect separately from the observed categories.
Use an explicit Unknown level when the data definition treats that status as a meaningful recorded category. For instance, a hypothetical review process might record “Unknown” when an assessor checked a file but could not verify its classification, while leaving the field blank when no assessment was completed. Those states could carry different information about the process that produced the data. Keep them distinct only if the source definitions support that distinction and the distinction is useful for the analysis.
An explicit Unknown level is not a universal fix for missing data. It keeps a row’s factor value from being missing, but it does not restore the unobserved category or explain why the information is unavailable. Conversely, replacing every explicit Unknown with NA can discard a recorded state and cause affected rows to be omitted from analyses that use the factor.
Check whether “Unknown” is an actual response or code, and whether blank or absent values have a separate documented meaning.
Keep NA for unavailable values and include an Unknown factor level only when it represents an observed, defined category.
Count NA values separately from each factor level, then verify the records retained by any summary or model.
Worked Example: Missing Predictor Values and Model Rows
A hypothetical claims study has 200 policies. The rating-zone factor is recorded for 170 policies; the other 30 have NA for rating zone. Among the 170 with a recorded zone, 17 had a claim. Among the 30 with NA, 9 had a claim. A model uses claim occurrence as its response and rating zone as a predictor. Compare the event rates in the complete cases and in the full data, and identify the row-count consequence if the model omits rows with missing predictors.
Method. A complete-case rate uses policies with a recorded rating zone. Its denominator is 170, so the rate is \(17/170=0.1000\), or 10.00%. The full-data event count is \(17+9=26\), and the full-data rate is \(26/200=0.1300\), or 13.00%.
If the modeling function applies row omission to the missing rating-zone values, it uses \(200-30=170\) rows, not all 200. The 9 claims among the omitted records are not included in that fit. The difference between 10.00% and 13.00% shows that the omitted records have a different observed event rate in this example; it does not establish that omission always causes bias or that an Unknown category is always preferable.
Check. The records divide into 170 with a rating zone and 30 without one, totaling 200. The event counts divide into 17 and 9, totaling 26. The common mistake is to report \(26/200\) as the complete-case rate or to assume the model used all 200 rows just because the response was observed.
Why Merging the States Changes a Summary
Suppose an analyst combines explicit Unknown values with NA values and then reports one event rate for the combined group. That rate is a weighted average of the rates in the two original groups, with weights proportional to their sizes. It need not equal either original rate. The arithmetic explains why recoding can change interpretation even when no event counts are changed.
Here \(n_U\) and \(e_U\) are the number of explicit Unknown records and their event count; \(n_M\) and \(e_M\) are the corresponding number and event count among records that were actually missing. The equality follows by multiplying each group rate by its group-size weight: the first term is \(e_U/(n_U+n_M)\), the second is \(e_M/(n_U+n_M)\), and their sum is the combined event rate. This calculation applies when both group sizes are positive. If either group has size zero, its group rate is undefined, although the combined rate can still be calculated from the available group.
Worked Example: Separating an Unknown Category From Blank Values
A hypothetical review dataset contains 150 applications. The classification factor has 80 Standard values, 45 Preferred values, 15 explicit Unknown values, and 10 NA values. Claim counts are 8, 3, 6, and 2 in those groups, respectively. The source documentation says Unknown means an assessor reviewed the application but could not verify its class; NA means the field was blank. Calculate the event rate for explicit Unknown, the rate after incorrectly merging Unknown and NA, and the complete-case rate when only NA rows are omitted.
Method. For the explicit Unknown group, the event rate is \(6/15=0.4000\), or 40.00%. If Unknown and NA are merged, the combined group has \(15+10=25\) applications and \(6+2=8\) claims. Its event rate is \(8/25=0.3200\), or 32.00%.
If only the 10 NA rows are omitted, the retained sample has \(150-10=140\) applications and \(8+3+6=17\) claims. The retained-sample event rate is \(17/140=0.1214\), or 12.14% to two decimal places. The explicit Unknown group remains a separate observed group in that calculation.
Check. The group sizes sum to \(80+45+15+10=150\), and the event counts sum to \(8+3+6+2=19\). The combined Unknown-and-NA rate is lower than the explicit Unknown rate because it adds 10 records with only 2 claims. The common mistake is to merge the states and then describe the resulting 32.00% as the rate for applications explicitly classified as Unknown.
Practical Checks Before Analysis
Before using a factor with missing values or an Unknown level, establish what each state means and verify how many observations it represents. In R, inspect the factor’s levels, count its observed values, and count missing values separately. A printed table should make clear whether NA is counted as a missing value rather than treated as an ordinary level. When a modeling function uses the factor, inspect how many rows it retained and confirm the configured missing-data action.
These checks support an interpretation that is consistent with the data. If a model retains an explicit Unknown category, its associated comparisons describe records carrying that label; they do not automatically describe every record whose true category is unavailable. If rows with NA are omitted, conclusions from the fitted data apply directly to the retained records, and the omitted records should be considered when assessing how well those conclusions address the full population.
Check Your Understanding
For each problem, distinguish the recorded category from missingness and track the relevant denominator.
- A hypothetical survey has 90 responses: 31 are East, 39 are West, and 20 have no region recorded. Find the West proportion among records with a known region and among all 90 records. (Answer: \(39/70=0.5571\), or 55.71%, among known regions; \(39/90=0.4333\), or 43.33%, of all records.)
- A model dataset has 240 records, of which 18 have NA for a factor predictor used by the model. If the model’s missing-data action is row omission, how many rows are used? What should be checked before reporting that number as the actual fitted sample size? (Answer: 222 rows; verify the model’s missing-data setting and fitted row count.)
- There are 12 explicit Unknown records with 3 events and 8 missing records with 1 event. Find the event rate for explicit Unknown and for the combined group if the states are merged. (Answer: \(3/12=25.00\%\) for explicit Unknown; \(4/20=20.00\%\) for the merged group.)
- A source guide defines “Unknown” as “reviewed but not verifiable” and a blank as “not reviewed.” There are 25 Unknown values and 7 blanks. Which should be an explicit factor level, and which should remain missing if this definition is followed? (Answer: “Unknown” should be an explicit level; the 7 blanks should remain NA.)
- A factor is intended to have levels Standard, Preferred, and Unknown, but the raw data also contain the label “Unk.” What is the risk if “Unk” is not reconciled or included in the allowed levels when creating the factor, and what should be decided first? (Answer: “Unk” can be converted to NA if it does not match a level; first determine from the source whether it means the explicit Unknown category or represents a different or erroneous value.)