Why Missing Categories Need a Deliberate Treatment
A categorical predictor might record a policyholder’s payment frequency, a member’s plan type, or an applicant’s occupation group. If some records have no value, the analyst has two common single-value choices: replace each missing value with the most frequent observed category, or represent missingness with a distinct level such as “Not recorded.” These choices preserve the rows, but they do not mean the same thing.
Modal imputation assigns missing records to a real, observed category. A distinct missing level keeps those records identifiable as having no recorded category. Neither approach recovers the unknown true values. The choice affects category counts, proportions, and the interpretation of comparisons made using the predictor. It should be made and documented as a data-treatment decision, not as a neutral formatting change.
Counts and Proportions Under the Two Choices
Let \(N\) be the number of records, \(m\) the number with a missing predictor value, and \(n_j\) the number observed in category \(j\). The observed category counts sum to \(N-m\). Let category \(k\) be a mode: its count is at least as large as the count of every other observed category. If there is a tie, a rule for choosing among tied modes is needed before modal imputation can be reproduced consistently.
After modal imputation, the count for category \(k\) increases by \(m\). The other observed category counts do not change, and the total remains \(N\). Under distinct-level treatment, each observed category count remains unchanged and a separate level receives count \(m\). In either case, when proportions are calculated over all records, the denominator is \(N\).
The distinction is visible in the category distribution. Modal imputation increases the recorded share of the chosen mode and removes the separate count of missing records from that distribution. A distinct level leaves the observed-category counts intact and displays the amount of missingness. A useful summary should therefore identify both the treatment and the denominator used.
Worked Example: Recalculating Category Shares After Modal Imputation
A hypothetical portfolio contains 24 records for payment frequency. Among the records with a value, 8 are Monthly, 6 are Quarterly, 4 are Annual, and 2 are Other. The remaining 4 values are missing. Compare modal imputation with a distinct “Not recorded” level, and report proportions to four decimal places.
Method. First identify the mode using observed values only. Monthly has the largest observed count, 8, so it is the modal category. The observed counts sum to \(8+6+4+2=20\), and \(20+4=24\) confirms the total record count.
Modal imputation. Assign all 4 missing records to Monthly. The new counts are Monthly \(8+4=12\), Quarterly \(6\), Annual \(4\), and Other \(2\). They sum to \(12+6+4+2=24\). Dividing each by 24 gives proportions \(12/24=0.5000\), \(6/24=0.2500\), \(4/24=0.1667\), and \(2/24=0.0833\). The displayed proportions sum to \(1.0000\).
Distinct-level treatment. Keep the four observed counts and add Not recorded with count 4. The counts are Monthly 8, Quarterly 6, Annual 4, Other 2, and Not recorded 4. Dividing by 24 gives \(0.3333\), \(0.2500\), \(0.1667\), \(0.0833\), and \(0.1667\), respectively, to four decimal places. The displayed proportions sum to \(1.0000\); before rounding, the proportions sum to 1.
Check and interpretation. Both treatments retain all 24 records, but modal imputation makes Monthly account for half the portfolio, whereas it accounts for 8 of the 24 records under the distinct-level treatment. The difference comes entirely from assigning the 4 unobserved values to Monthly in the first calculation. A common mistake is to divide modal-imputed counts by 20, the number originally observed; that gives counts summing to 24 divided by 20, or 1.2, not a valid distribution over the 24 records. The correct denominator after either treatment is 24.
Why Modal Imputation Changes the Distribution
The count change follows directly from the assignment rule. Every missing record is placed into category \(k\). Thus the number assigned to \(k\) is its observed count \(n_k\), plus the \(m\) reassigned records. No record is added or removed, so the total remains \(N\). Dividing the adjusted counts by \(N\) gives proportions that sum to one because the adjusted counts still sum to \(N\).
This arithmetic also explains a key interpretive limitation: after modal imputation, the data no longer distinguish records that originally had the modal value from records whose value was missing and assigned to it. The resulting category count is a mixture of observed and imputed values. A larger modal share is therefore not evidence that the underlying population truly had that category more often.
A distinct missing level preserves this distinction in the prepared data. It can be useful when the fact that a value was not recorded may carry predictive information, or when an analyst wants model summaries to make the missing group visible. It does not reveal what the missing values should have been, and it should not be described as though “Not recorded” were a real policy type, occupation, or customer preference.
Worked Example: Keeping Missing Records Visible
A hypothetical study of 40 applicants records preferred contact frequency. The observed values are Annual for 18 applicants, Quarterly for 10, Monthly for 6, and Other for 2; 4 values are missing. An analyst wants to report the distribution without assigning the missing applicants to a frequency.
Method. Use a distinct “Not recorded” level. Verify the count first: \(18+10+6+2+4=40\). The observed categories retain their counts, and the missing group has count 4.
Calculate proportions. Divide each count by all 40 applicants. Annual is \(18/40=0.4500\); Quarterly is \(10/40=0.2500\); Monthly is \(6/40=0.1500\); Other is \(2/40=0.0500\); and Not recorded is \(4/40=0.1000\). These proportions sum to \(0.4500+0.2500+0.1500+0.0500+0.1000=1.0000\).
Check and interpret. The report can say that 10% of applicant records lack a recorded contact frequency; it cannot say that those applicants prefer a frequency called “Not recorded.” If instead all 4 were assigned to Annual, its count would be 22 and its share would be \(22/40=0.5500\). That would answer a different question because it treats unknown values as Annual. The common mistake avoided here is presenting the modal-imputed share as if it were the observed Annual share.
Consequences for Group Comparisons and Models
The choice can affect more than a summary table. Suppose a categorical predictor is used to compare outcomes across groups. Modal imputation folds missing records into one observed group, so that group’s count and outcome summary combine genuinely observed values with assigned values. A distinct level permits a separate summary for records with missing predictor values, provided there are enough such records for a meaningful estimate.
For a regression model that uses a categorical predictor, the levels are represented through the model’s categorical encoding. Modal imputation makes a missing record contribute as a member of the selected observed level. A distinct level instead gives the model a separate group to estimate, subject to the model specification and the amount of data in that group. That separate estimate may be unstable when the missing group is small, and it describes an association with recording status rather than an effect of a substantive category. Neither result establishes that missingness caused an outcome.
Worked Example: Outcome Rates Can Be Blended by Modal Imputation
In a hypothetical applicant study, the categorical predictor “review tier” is observed for 90 applicants: 50 are Standard, 30 are Preferred, and 10 are Basic. Ten additional applicants have a missing tier. An outcome occurs for 5 Standard, 1 Preferred, 2 Basic, and 4 of the 10 applicants with a missing tier. Compare the Standard outcome rate under modal imputation with the separate rates under a distinct missing level.
Method. Standard is the mode among observed tiers because its count of 50 exceeds 30 and 10. For each group, calculate the outcome rate as the number with the outcome divided by the group count.
Observed Standard rate. Before imputation, Standard has 5 outcomes among 50 applicants, so its rate is \(5/50=0.1000\), or 10%.
Modal-imputed Standard rate. Assign the 10 missing tiers to Standard. The resulting Standard group has \(50+10=60\) applicants and \(5+4=9\) outcomes. Its rate is \(9/60=0.1500\), or 15%. The rate has changed because the imputed records contributed their outcomes to the Standard group.
Distinct-level rates. Keep the original Standard rate at \(5/50=10\%\), and report the missing-tier group separately as \(4/10=40\%\). For context, Preferred is \(1/30=3.333\ldots\%\), and Basic is \(2/10=20\%\). The missing group’s rate is based on only 10 applicants, so it should be interpreted with appropriate caution.
Check and interpretation. The applicant count is \(50+30+10+10=100\), and the outcome count is \(5+1+2+4=12\) under either treatment. Modal imputation does not change those totals; it changes which group receives the 10 applicants and their 4 outcomes. A common mistake is to retain the 5 Standard outcomes but divide by 60, or to add the 4 missing-group outcomes to the Standard denominator without adding them to its numerator. The correct modal-imputed rate uses both adjusted quantities: \(9/60=15\%\).
Implementation and Choice in R
In R, a factor’s missing value is ordinarily represented as NA, which is not automatically the same as a factor level named “Not recorded.” When auditing observed counts, make sure that missing values are excluded from the counts used to find the mode. For example, table(x, useNA = "no") counts observed levels without treating missing values as a category. A frequency table that includes missing values can be useful for auditing, but the missing count must not be mistaken for an observed substantive level when selecting the mode.
For modal imputation, calculate the most frequent observed level and replace missing entries with that level. For a distinct-level approach, assign missing entries an explicit label such as “Not recorded” and ensure that the factor includes that level. Choose a label that cannot be confused with a valid value; if “Not recorded” could already occur as a genuine response, use a different label or resolve the ambiguity before recoding. Retain the original variable or a clear record of the transformation so the treatment can be audited and reproduced.
In predictive work, calculate the modal value from the training data and use that same value to prepare later data. This avoids allowing information from held-out records to influence preprocessing. The same principle applies to the level definitions used for the model: the analysis must handle a level that appears in later data but was not present in training, rather than silently recoding it without a documented rule.
Choosing and Reporting a Treatment
There is no universally correct choice based only on which category is most frequent. First use the missingness audit and context of the data collection to understand how much is missing and whether it varies across outcomes or subgroups. Earlier material on missingness assumptions and subgroup diagnosis provides useful context: these single-value treatments do not, by themselves, establish why values are missing or eliminate bias caused by missingness.
| Choice | What happens to counts | Interpretive consequence | Important caution |
|---|---|---|---|
| Modal imputation | All missing counts are added to the observed mode | The mode combines observed and imputed records | May overstate the mode’s share and conceal the original amount missing |
| Distinct missing level | Observed counts stay unchanged; missing records form a separate level | Missingness remains visible as a recording-status group | The level is not a substantive category, and estimates may be imprecise if the group is small |
A defensible report states the number and proportion missing before treatment, the rule used to handle them, the mode and any tie-breaking rule if modal imputation was used, and whether reported category shares include all records. For a distinct-level treatment, label the level explicitly as missing or not recorded. For modal imputation, describe the adjusted modal count as including imputed records, rather than presenting it as a count of observed responses.
- Do not call imputed values observed. The assigned mode is a convenience for analysis, not a recovered answer.
- Do not confuse missing status with a real category. A “Not recorded” level describes the data-collection result.
- Do not choose the mode using missing values as if they were responses. Determine it from observed values and handle ties explicitly.
- Do not use the wrong denominator. When summarizing all retained records after either treatment, proportions use the total number of records.
Check Your Understanding
For each problem, check both the arithmetic and what the resulting categories mean.
- A hypothetical database has 30 records: Red 12, Blue 8, Green 4, and 6 missing. If the missing values are imputed to the mode, what are the adjusted counts and the Red proportion? (Answer: Red 18, Blue 8, Green 4; Red proportion \(18/30=0.6000\).)
- A survey has observed counts of Weekly 9, Monthly 7, and Rarely 4, plus 5 missing responses. Under a distinct missing-level treatment, what is the proportion labelled Not recorded, to four decimal places? (Answer: \(5/25=0.2000\).)
- In a portfolio of 50 records, an observed mode has count 20 and there are 10 missing values. An analyst reports an imputed modal proportion of \(30/40\). Identify the error and give the correct proportion over all retained records. (Answer: The denominator incorrectly excludes the 10 records; the correct proportion is \(30/50=0.6000\).)
- A hypothetical outcome occurs in 3 of 30 records with observed Standard tier and in 2 of 5 records with missing tier. If missing tiers are imputed to Standard, calculate the resulting Standard outcome rate. (Answer: \((3+2)/(30+5)=5/35=0.1429\), or 14.29% to two decimal places.)
- A predictor has observed counts A 14, B 14, and C 6, with 4 missing records. What must be specified before modal imputation, and what are the adjusted counts if the documented tie rule selects B? (Answer: A tie-breaking rule must be specified; with B selected, the counts are A 14, B 18, and C 6.)