Why Missing-Value Codes Need Special Attention
A data file can look complete even when some entries do not record the quantity or category they appear to contain. A survey might store \(-9\) when a respondent skips a question, or use the text “unknown” when a value is unavailable. If these entries are treated as ordinary measurements, they can distort summaries and later analyses. If legitimate values are mistakenly recoded as missing, real information is lost.
The key is not to guess from how a value looks. A number such as 99 could be a reserved code in one variable and a genuine measurement in another. A blank string might indicate no response, but it might also be an intended category or a formatting artifact. Missing-value codes must be identified using evidence about how each variable was collected and stored.
Verify the Meaning Before Recode
Before changing any entries, establish what the raw values mean. The most useful evidence is a codebook, questionnaire, data-entry guide, or other documentation describing the variable and its allowed codes. Check whether the document specifies distinct codes for “not known,” “not applicable,” “refused,” or other situations. The same token can have different meanings across variables, so verification must be variable-specific.
A range check can help flag a suspicious entry, but it does not establish that the entry is missing. For example, an unusual value might be a legitimate extreme, a measurement error, or a reserved missing code. Likewise, a plausible value may still be a missing code if the data collection system used it that way. When the coding cannot be verified, flag the value for review rather than silently recoding it.
A sound recoding decision rests on two conditions: the token is documented or otherwise reliably confirmed as representing absence for this variable, and the recode changes only entries that meet that rule. If either condition is uncertain, keep the original value and record the uncertainty until it can be resolved.
Find the definition, valid values, and any reserved codes for unavailable responses.
Check exact tokens, including signs, capitalization, spaces, and whether numbers are stored as numbers or text.
Map each documented code to the analysis system’s standard missing representation; leave valid values unchanged.
Retain the raw values when possible and record the variable, code, replacement, and supporting rule.
Numeric Placeholders Are Not Measurements
A numeric placeholder can be especially misleading because software may treat it as a real number without warning. Suppose a data system uses \(-9\) to encode a skipped measurement. If that code remains in a numeric variable, arithmetic uses it as though it were an observed value. The following calculation shows why a documented code must be recoded before calculating a mean.
Worked Example: Recode a Documented Numeric Code
In a hypothetical clinic training file, a documentation sheet states that systolic blood pressure is recorded in millimetres of mercury and that \(-9\) means “not recorded.” Three entries are 118, \(-9\), and 132. Recode the missing entry and compare the mean before and after recoding.
Check the rule. The codebook explicitly identifies \(-9\) as a missing-value code for this measurement. It is not being treated as a blood-pressure reading. Therefore, the two recorded measurements are 118 mmHg and 132 mmHg, while the \(-9\) entry should be represented as missing.
Mean if the code is incorrectly treated as a measurement.
This value is not the mean of the two observed readings: it includes a placeholder as if it were a measurement.
Mean after the documented recode. Replacing \(-9\) with a missing value leaves two observed readings:
The recoded-data mean is 125 mmHg, the mean of the two available readings. The example does not establish that the unavailable reading would have been 125 mmHg; it simply excludes the placeholder from this arithmetic. The recode is justified by the documented rule, not by which mean seems more plausible.
The general arithmetic explains the problem. Let \(n\) be the number of entries, let \(m\) entries be a placeholder \(c\), and let the sum of the genuine observed measurements be \(S\). If the placeholders are mistakenly included, the computed mean is \((S+mc)/n\). If those \(m\) entries are correctly identified as missing, the mean of the observed measurements is \(S/(n-m)\), provided at least one measurement remains. These are generally different quantities because the placeholder is not a measurement from the variable’s scale.
Text Tokens, Blanks, and Ambiguous Labels
Missingness may also be stored as text. Common tokens include “unknown,” “N/A,” “missing,” and strings containing only spaces. Before recoding, check spelling and formatting variations, such as “Unknown,” “unknown ”, or an empty string. These can be distinct values to a computer even when a person reads them as equivalent.
Do not assume every word that sounds like absence is a missing-value code. “Unknown” might be a documented way to record an unavailable answer, or it might be an intentional category in the dataset. “Not applicable” may mean that the question did not apply to an individual, which can be important to distinguish from “refused” or “not known.” Follow the study’s coding definitions, and preserve separate reasons when those distinctions are meaningful for the intended analysis.
Worked Example: Inspect Text Codes Before Replacing Them
A hypothetical questionnaire records preferred transport as one of “Walk,” “Bus,” “Car,” or “Unknown.” Its data guide says that “Unknown” is entered when the respondent did not provide an answer. The raw entries include “Bus,” “Unknown,” “Walk,” and “Unknown ” (with a trailing space). Decide what to recode.
Confirm the meaning. The guide defines “Unknown” as an unanswered item, not as a transport mode. The two entries that match this token after removal of surrounding spaces therefore represent missing responses.
Normalize the formatting for comparison. Treating “Unknown ” and “Unknown” consistently for this check identifies both documented codes. This formatting normalization does not itself decide that a value is missing; the guide supplies that meaning.
Recode the confirmed tokens. Replace those two entries with the standard missing representation used by the analysis system. Leave “Bus” and “Walk” unchanged as recorded categories. Do not create an “Unknown” transport category in the analysis variable unless the data guide or analysis question specifically calls for retaining it as a distinct category.
If the guide had not explained “Unknown,” it would not be safe to assume that it meant a missing answer. The appropriate action would be to consult the data owner or retain and flag the value until its meaning was confirmed.
A Code Can Be Valid in One Variable and Missing in Another
Missing codes are variable-specific. It is not safe to search the entire dataset for a number such as 99 and replace every occurrence. The value may be reserved in one column but a legitimate observation in another. Even within a single variable, several codes might have distinct documented meanings.
Worked Example: Protect a Legitimate Value
A hypothetical survey records age in completed years and a separate symptom-severity score. The age documentation says values from 18 through 99 are observed ages. The score documentation says that 99 means “not answered,” while scores from 0 through 10 are valid. The records include age 99, score 99, and score 0.
Age 99. The age guide identifies 99 as a valid age. Keep it as an observed age of 99 years. Replacing it with missing would erase a legitimate measurement.
Score 99. The score guide explicitly reserves 99 for “not answered.” Recode this entry as missing; it is not a severity score of 99.
Score 0. Zero is within the documented valid score range. Keep it as an observed score of 0. A zero is not missing merely because it may look like an absence of symptoms; it has the defined meaning assigned by the scale.
This example shows why recoding must be based on the combination of variable and code. A global rule such as “replace every 99” would incorrectly remove the age observation, while a rule such as “keep all plausible-looking numbers” would retain the score placeholder.
Choosing a Consistent Representation and Keeping a Record
After confirming a code, convert it to the missing-value representation expected by the analysis software or data format. The aim is to prevent the placeholder from being treated as an observed category or quantity. Keep the original file unchanged when possible, and create a prepared copy so that a recode can be reviewed or reversed.
If the source distinguishes reasons for missingness, decide whether those distinctions need to remain available. For example, “not known” and “declined” may both be unavailable measurements for a calculation, but they are not necessarily the same response. A common approach is to use a standard missing representation in the analysis variable while preserving a separate status field or an audit record of the original code. Do not collapse documented distinctions without a reason.
A concise recoding record should identify the variable, the original token, the replacement, the rule supporting the change, and whether the raw data were retained. This makes the decision auditable and helps prevent someone from repeating the same investigation or misinterpreting a recoded value later.
Common Mistakes
- Replacing every unusual number. An unusual value may be a real extreme. Recode it only when the variable-specific rule confirms it is a placeholder.
- Replacing every occurrence of 0, 99, or \(-9\). A token can be valid for one variable and a missing code for another. Apply rules to named variables, not indiscriminately to a whole file.
- Assuming text labels explain themselves. “Unknown” or “N/A” can have different meanings in different datasets. Check the data guide or ask the data owner.
- Changing the raw file without a record. Preserve the original when possible and document the mapping so the preparation can be checked.
- Combining distinct reasons automatically. A skipped answer and a not-applicable response may both lack a measurement but can represent different situations. Preserve that distinction when it matters.
Check Your Understanding
Use the variable definitions and coding rules in each question rather than judging a token by appearance alone.
- A temperature variable’s data guide says \(-9\) means “not measured.” The file contains \(-9\), 18, and 21 degrees. Which entry should be recoded, and why?
- A variable for age has documented valid values from 0 through 105. A row contains 99. What evidence would justify keeping or recoding it?
- A categorical field contains “Unknown,” but no documentation explains the label. What should you do before converting it to missing?
- A documented missing code is stored as “N/A ” with a trailing space. What should be checked before recoding it?
- Why is it unsafe to replace every 99 across every column in a dataset?