Why Category Levels Need to Be Checked Across Subsets
A categorical variable may be recorded in several files, collected at different sites, or split into groups for separate summaries. Even when each file looks reasonable on its own, their categories may not line up. One file may contain a level that is absent from another; a level may be defined but have no observations in a subset; or similar-looking labels may refer to categories that are not actually the same. These differences matter when comparing summaries or assembling the data.
The key is to distinguish the allowed levels of a variable from the levels observed in a particular subset. The allowed levels describe the intended category set. The observed levels are the categories that occur among the records currently being examined. A category with zero observations in a subset can still be an allowed level; conversely, an observed label outside the intended set needs investigation.
State the Category Set Before Comparing Counts
Before comparing files, identify the category set the variable is supposed to use. This should come from the variable's definition and the data collection or study documentation, not merely from the labels that happen to appear in one file. For example, a referral-source variable might allow exactly “Phone,” “Web,” “Walk-in,” and “Other.” If a file contains “Email,” do not quietly treat it as “Web” or “Other.” First determine whether it is an unexpected label, a change in the definition, or a valid new category.
The same principle applies to labels that seem related. “Other” and “Unknown” have different meanings: “Other” can indicate a known response outside the named options, while “Unknown” indicates that the category was not established. They should not be combined simply because neither identifies one of the named substantive categories. Likewise, do not merge two categories because doing so would make a comparison easier.
For a single-response categorical variable, a count table is easiest to interpret when each record contributes to exactly one category. State this assumption when using the counts: each included unit has one recorded level for the variable, and any unclassified or missing records are tracked separately rather than silently omitted or placed in a substantive category. The category counts should then sum to the number of included records.
Worked Example: Comparing Referral Categories at Two Clinics
Suppose two hypothetical clinics record one referral source for each patient. The allowed levels are Phone, Web, Walk-in, and Other. Their counts are shown below. The categories are displayed in rows and the clinics in columns.
| Referral source | Clinic East | Clinic West |
|---|---|---|
| Phone | 18 | 11 |
| Web | 9 | 14 |
| Walk-in | 3 | 0 |
| Other | 5 | 4 |
For Clinic East, add the four category counts in the East column:
Thus, the Clinic East column total is 35. For Clinic West, add the four counts in the West column:
Thus, the Clinic West column total is 29. Across both clinics, the total number of records is \(35 + 29 = 64\). The zero for Walk-in at Clinic West means that this allowed level has no observed cases in that clinic's subset; it does not by itself show that the level is invalid or that the table is incomplete.
The calculation is appropriate if every included patient has exactly one referral source recorded and all 64 records are represented in the table. If some records have no known referral source, those records need a separate, clearly defined treatment before claiming that the four category counts account for every patient.
Check Labels and Level Sets Across Files
A practical comparison has two parts. First compare the intended allowed levels: do the files use the same category set, or does a documented reason explain a difference? Then compare the recorded labels with that set: does every observed value match an allowed level exactly? A label that appears in one file but not another is a prompt to check the definition and source, not permission to change it automatically.
A useful audit table places every allowed level in every subset, including levels with count zero. This makes the comparison explicit. If an extra label appears, include it in the audit as an item requiring resolution rather than disguising it in an existing category. Labels can differ in spelling or formatting, but deciding whether two labels mean the same thing requires evidence from the definitions or collection process.
Use the documented meaning of the variable and list each permitted category before comparing subsets.
Compare those labels with the allowed set, noting both unexpected values and allowed levels that do not occur.
Check source documentation or collection procedures. Keep distinct meanings distinct unless an explicit, justified category definition says otherwise.
For a single-response variable, verify that category counts sum to the number of included records, accounting separately for records not assigned to an allowed level.
Worked Example: A Label Appears in Only One File
Imagine two hypothetical survey files that record a participant's program group. The documented allowed levels are “Certificate,” “Diploma,” and “Degree.” A count audit gives the following results.
| Program group | File A | File B |
|---|---|---|
| Certificate | 12 | 8 |
| Diploma | 7 | 10 |
| Degree | 5 | 6 |
| Advanced Diploma | 0 | 2 |
File A has \(12 + 7 + 5 = 24\) records in the three documented levels. File B has \(8 + 10 + 6 = 24\) records in those levels, plus 2 recorded as “Advanced Diploma,” for a total of \(8 + 10 + 6 + 2 = 26\). The extra label is not an unused allowed level: it has two observations and is not on the documented list.
The correct conclusion is that File B contains a category requiring investigation. It might reflect a change in the program definitions, an error in recording, or an allowed level omitted from the documentation. The counts alone do not determine which explanation is right. Do not automatically combine “Advanced Diploma” with “Diploma”: the labels may represent distinct programs. Until the definition is clarified, report the discrepancy and keep the two labels distinguishable.
This check also prevents a misleading total. If a summary reports only the three allowed levels, it accounts for 24 of File B's 26 records, not all 26. The two-record difference must be made visible and explained.
Unused Levels and Subset Totals
An allowed level may be present in the variable's definition but absent from one or more subsets. Keeping it visible in a comparison table is often useful: it shows that the category was considered and that its count is zero, rather than leaving readers to wonder whether it was forgotten. This is particularly important when one set of levels is used to summarize several files on the same basis.
A zero count has a limited meaning. It says that no records in the subset were observed in that category. It does not establish that the category is impossible in the population, invalid in the data definition, or equivalent to another category. Nor does it show that the file was coded incorrectly. Those conclusions require information beyond the count.
Counts also provide a simple consistency check when subsets are disjoint. If a full data set is divided into mutually exclusive subsets, each record belongs to exactly one subset, and the category definition is unchanged, then the count for a level in the full data is the sum of its counts across the subsets. This follows because every record in that level is counted in exactly one subset.
Here \(n_j\) is the count in category \(j\) in the full data, and \(n_{jk}\) is the count in category \(j\) in subset \(k\). The equality is valid only when the subsets partition the records being counted: they do not overlap, and together they include all those records. If some records are excluded or unclassified, the corresponding accounting must be shown separately.
Worked Example: Reconcile Counts Across Two Time Periods
Suppose a hypothetical service records one contact method per client. The allowed levels are Phone, Online, and In person. The service has two disjoint monthly files.
| Contact method | Month 1 | Month 2 | Combined |
|---|---|---|---|
| Phone | 13 | 9 | 22 |
| Online | 7 | 11 | 18 |
| In person | 4 | 0 | 4 |
For Phone, the combined count is \(13 + 9 = 22\). For Online, it is \(7 + 11 = 18\). For In person, it is \(4 + 0 = 4\). The Month 2 count of zero for In person is retained because In person remains an allowed level even though it was not observed in that month's records.
The Month 1 total is \(13 + 7 + 4 = 24\), and the Month 2 total is \(9 + 11 + 0 = 20\). The combined total is \(22 + 18 + 4 = 44\), which agrees with \(24 + 20 = 44\). This agreement supports the count reconciliation, provided the monthly files are disjoint, the same contact-method definition was used in each, and each included client contributes exactly one recorded method.
If the same client could appear in both files, or if the definition changed between months, the arithmetic could still be performed but the combined counts would not have this straightforward interpretation. Check the subset structure and category definition, not only the sums.
Why This Check Matters
Category-level inconsistencies can change what a comparison says. If one subset has a category missing from its display, readers may mistake an unreported zero for an omitted category. If an unexpected label is silently assigned to a familiar category, counts no longer describe the recorded values faithfully. If two distinct levels are combined without justification, useful differences are erased.
For example, combining “Unknown” with “Other” may make a table shorter, but it changes the meaning of the resulting category. “Unknown” describes uncertainty about the response; “Other” describes a response outside the named options. The fact that either category might have a small count is not a reason to merge them. Decisions to create a combined category require an explicit definition and a traceable rationale, not an automatic fix during a cross-file check.
Common Mistakes
- Calling a zero-count level an error. The mistaken reasoning is that a category with no observations should not appear. The correction is to distinguish the allowed set from the observed values: a valid allowed level can have count zero in a particular subset.
- Treating an unexpected label as a synonym. The mistaken reasoning is that two labels seem close enough to combine. The correction is to consult the category definition and source documentation; do not infer equivalence from wording alone.
- Omitting an extra category from totals. A total based only on the documented levels may not account for every record. Compare the sum of category counts with the number of included records and make any discrepancy explicit.
- Confusing subset layout with category totals. In a table with clinics in columns and categories in rows, adding down one clinic's column gives that clinic's total. State clearly whether a reported sum is a row total or a column total.
- Assuming matching counts prove matching definitions. Two files can have similar totals while using different meanings for a level. Check the labels and definitions as well as the arithmetic.
Check Your Understanding
Answer each question using the distinctions between allowed levels, observed levels, and subset counts.
- A variable allows “A,” “B,” and “C.” One subset has counts 6, 4, and 0. Is C an invalid level? What does its count show?
- A second file contains two records labeled “Other response,” although the documented levels are “Other” and “Unknown.” What should be checked before assigning those records to either documented category?
- Two disjoint subsets have counts 8 and 5 for a particular level. Under what conditions can their combined count be reported as 13?
- A count table places departments in columns and response categories in rows. What does summing the entries down one department's column give?
- A file has 30 records, but counts across the documented levels sum to 28. Give two checks to make before reporting the category totals as complete.