Missing-Value Representation in R

Why Missing-Value Representation Matters

A data field can look as though it contains a missing value without R treating it as one. A blank text entry, a numeric code such as -999, and R’s special value NA are not interchangeable. If a cleaning step recognizes only one representation, apparent missing values may remain in the data and affect summaries or model inputs. If it recodes too broadly, valid observations may be lost.

The first task is therefore not to replace values automatically. It is to identify how each column is stored, test for the representations that are meaningful for that type, and consult the field definition before treating an ordinary value as missing. This distinction is useful when preparing a hypothetical policy dataset with numeric exposure and categorical region fields, but the same checks apply to many predictive-analytics data sets.

Definition: In R, NA is a special missing-value marker. NaN (“not a number”) is a special numeric result, and R also recognizes it as missing with is.na(). An empty string such as "", a string containing spaces, and an ordinary numeric sentinel such as -999 are not NA values. They require separate interpretation and, if appropriate, explicit cleaning.

What R Does and Does Not Treat as Missing

R provides typed NA constants. Use NA_integer_ for an integer missing value, NA_real_ for a double-precision numeric missing value, NA_character_ for a character missing value, and NA_complex_ for a complex missing value. The untyped-looking NA is a logical value; R may coerce it when it is combined with another type, but an explicitly typed NA makes the intended column type clear.

The principal check is is.na(x). It returns a logical value for each element of x, with TRUE where that element is NA or NaN and FALSE otherwise. The narrower check is.nan(x) identifies NaN values; it does not identify an ordinary NA. NaN can arise from undefined numeric operations such as 0/0. It may be useful to distinguish it from a recorded NA when diagnosing how a numeric value was created.

A blank character value is still a character value. For a vector containing "North", "", and NA_character_, the empty string is not missing, while the third element is. A whitespace-only string, such as " ", is also not NA. If the data specification says that whitespace-only entries mean “no value supplied,” trim whitespace and check for an empty result.

A sentinel code is also an ordinary value until the data definition says otherwise. For example, -999 could be a placeholder for unavailable information in one field, but it could be a legitimate value in another. R cannot infer its intended meaning from the number alone.

Stored value	What R recognizes	Check or decision
`NA_real_`	Missing numeric value	`is.na(x)` is TRUE
`NaN`	Special numeric result; also recognized as missing by `is.na()`	`is.nan(x)` distinguishes it
`""`	Character string of length zero	Check explicitly if blanks are defined as missing
`" "`	Character string containing spaces	Trim, then check if the result is empty
`-999`	Ordinary number	Recode only when the field definition confirms it is a sentinel

Check the Type Before Choosing a Test

A check should match the stored type. typeof(x) reports the underlying storage type, while class(x) reports the object’s class. Functions such as is.numeric(x), is.character(x), and is.factor(x) help identify which checks are appropriate. This matters because a factor is stored internally using integer codes, but its values are category labels, not measurements.

For a numeric vector, is.na(x) detects both NA and NaN. is.nan(x) is useful when the distinction between those two matters. is.finite(x) identifies finite numeric values: it is FALSE for NA, NaN, positive infinity, and negative infinity. Infinity is not the same thing as a missing value, so a non-finite check is a broader diagnostic rather than a substitute for is.na().

For character data, test NA separately from blank text. A reliable blank-string condition excludes missing values before comparing the remaining text. The use of & is element-wise: it combines the two logical results at each position.

Key check: For character data in which empty or whitespace-only strings are defined as missing, use !is.na(x) & trimws(x) == "" as the replacement condition. The first part prevents an NA comparison from entering the replacement index; the second part identifies nonmissing entries that become empty after trimming.

Worked Example: Separating NA, NaN, Infinity, and a Sentinel

A hypothetical numeric claim field is stored as c(0/0, NA_real_, Inf, -999, 0). A data dictionary states that -999 is a placeholder for an unavailable amount, while zero is a valid amount. Identify what the standard R checks flag, then recode only the confirmed missing representations.

Method. Apply the numeric checks element by element, then use the dictionary to decide whether the sentinel should also be recoded. This avoids assuming that every unusual-looking number is missing.

The vector’s five values are, in order: NaN from 0/0, NA, positive infinity, -999, and zero. Therefore is.na(x) gives TRUE, TRUE, FALSE, FALSE, FALSE: it includes NaN and NA, but not infinity, -999, or zero. The check is.nan(x) gives TRUE, FALSE, FALSE, FALSE, FALSE. The check is.finite(x) gives FALSE, FALSE, FALSE, TRUE, TRUE.

The dictionary confirms that -999 is a placeholder in this field. Construct the replacement condition as is.na(x) | x == -999. For the five entries, that condition is TRUE, TRUE, FALSE, TRUE, FALSE. Replacing those positions with NA_real_ yields c(NA_real_, NA_real_, Inf, NA_real_, 0).

Check. The original NaN, NA, and confirmed sentinel are now missing; positive infinity remains because the stated rule did not define it as missing, and zero remains a valid amount. If the field specification also declared infinity invalid, it would need a separate documented rule. The common mistake avoided is recoding -999 merely because it looks extreme, without checking what it means for this particular field.

Cleaning Blank Character Values Safely

R’s treatment of NA has an important consequence for replacement. Comparisons involving NA usually return NA, not TRUE or FALSE. For example, if region_text contains "North", "", and NA_character_, then trimws(region_text) == "" evaluates to FALSE, TRUE, NA. A replacement subscript containing NA can cause an error; it does not mean “replace only the TRUE positions.”

The safe condition includes !is.na(region_text). At the NA position this first test is FALSE. Element-wise logical combination then makes the full condition FALSE at that position, even though the comparison on the right is NA: FALSE & NA is FALSE. Thus !is.na(region_text) & trimws(region_text) == "" evaluates to FALSE, TRUE, FALSE for these three values. Only the blank entry is selected.

Worked Example: Replacing Whitespace-Only Entries in a Factor

A hypothetical region field is recorded as a factor with observed values "North", "", and NA. The data specification says that an empty or whitespace-only region entry means the region was not recorded. Clean the blank entry while preserving the existing NA.

Method. Check the factor’s labels as character values, exclude existing NA values in the replacement condition, and assign a missing value. The explicit conversion is for checking labels; the column remains a factor.

First, the unsafe condition trimws(as.character(region_text)) == "" evaluates to FALSE, TRUE, NA. Using that vector directly as the replacement index can produce an error because it contains NA. This is the common trap: it is not enough that the intended blank has a TRUE condition.

Use the protected condition !is.na(region_text) & trimws(as.character(region_text)) == "". For the three observations, !is.na(region_text) is TRUE, TRUE, FALSE, and the trimmed comparison is FALSE, TRUE, NA. Combining them with & gives FALSE, TRUE, FALSE.

The safe replacement is region_text[!is.na(region_text) & trimws(as.character(region_text)) == ""] <- NA. The resulting observed values are "North", NA, and NA. The original missing value remains missing, and the blank category becomes missing as specified. A factor may retain an unused empty-string level after this assignment; that unused level does not mean an observation is still blank. The common mistake avoided is using an index that contains NA.

Sentinel Codes Need Field-Specific Rules

A sentinel is a convention agreed on by a data producer and user, not a special R value. Before recoding one, confirm the affected variable, the exact code, and whether the code can also be a valid observation. It is possible for -999 to mean “not available” in one numeric column and to be an observed value in another. Recode according to the field definition, not a global rule applied to every column.

Worked Example: Applying a Sentinel Rule to One Numeric Field

A hypothetical monthly exposure vector, measured in months, is c(12, -999, 8, NA_real_). The data dictionary explicitly says that -999 means “not recorded” for this exposure field, and that valid recorded exposures are nonnegative. Show a type-aware check and clean the confirmed representations.

Method. Confirm that the vector is numeric, then identify both R-missing values and the field-specific sentinel. The check is.numeric(exposure) is TRUE for this numeric vector. Its elements are 12, -999, 8, and NA.

The sentinel comparison exposure == -999 gives FALSE, TRUE, FALSE, NA. The last result is NA because NA cannot be compared as an ordinary number. Combine that comparison with the missing-value test using a condition that safely handles the final element: is.na(exposure) | (!is.na(exposure) & exposure == -999). The two parts identify the sentinel at the second position and NA at the fourth; the complete condition is FALSE, TRUE, FALSE, TRUE.

Assign NA_real_ at those positions. The cleaned vector is c(12, NA_real_, 8, NA_real_). The two observed exposures, 12 and 8 months, are unchanged, and the two unavailable entries have the numeric missing type. The result respects the stated valid range and the documented sentinel rule. The common mistake avoided is applying a sentinel replacement to other numeric columns without evidence that they use the same convention.

A Practical Decision Sequence

A disciplined sequence separates recognition from interpretation. The checks can identify candidates for cleaning, but only the data definition can establish whether a blank or ordinary numeric code represents missing information.

Inspect the column type.
Use typeof(), class(), and relevant type checks such as is.numeric(), is.character(), or is.factor(). Do not interpret factor storage codes as numeric measurements.

Identify R’s special missing values.
Use is.na() for NA and NaN. For a numeric diagnostic, use is.nan() when the NaN distinction matters, and is.finite() when checking for infinities as well.

Check text conventions explicitly.
For character labels, examine empty strings and, if relevant, whitespace-only strings using trimws(). Exclude existing NA values in a replacement condition.

Consult the field definition.
Verify whether a sentinel such as -999 has a missing-value meaning in this specific variable and whether that number could be valid.

Recode deliberately and check the result.
Use an NA constant matching the column type, and confirm that valid values were preserved and only defined missing representations were changed.

Common Mistakes and Their Corrections

Treating a blank as NA. "" is an ordinary character value. Use a blank-string check only when the data specification says blanks represent missing information.
Assuming is.na() catches every special numeric case but distinguishes them all. It returns TRUE for both NA and NaN. Use is.nan() if NaN itself must be identified.
Using a logical subscript that contains NA. A condition such as trimws(x) == "" can contain NA. Protect a replacement with !is.na(x) & ....
Recoding every -999 automatically. The number is not inherently missing. Confirm the convention for each field before replacing it.
Applying numeric tests to factor codes. Factors represent labels. Inspect their labels rather than interpreting their underlying integer storage as measurements.

Key takeaway: R’s NA and NaN values are detectable with is.na(), but blank strings and sentinel codes need explicit, type-aware checks. Interpret those candidates using the field definition, protect replacement conditions from existing NA values, and preserve values that the data specification considers valid.

Check Your Understanding

For each problem, distinguish what R detects automatically from what requires an explicit data rule.

A character vector contains "East", "", and NA_character_. What does is.na() return, and which value would a blank-string rule select? (Answer: FALSE, FALSE, TRUE; the blank-string rule selects only "".)
A numeric vector is c(0/0, NA_real_, Inf, 4). Give the results of is.na() and is.nan(). (Answer: is.na() is TRUE, TRUE, FALSE, FALSE; is.nan() is TRUE, FALSE, FALSE, FALSE.)
A text vector is c("West", " ", NA_character_), and whitespace-only strings are defined as missing. Give the safe replacement condition !is.na(x) & trimws(x) == "". (Answer: FALSE, TRUE, FALSE.)
A numeric duration field contains c(6, -999, 9). The dictionary confirms that -999 means “not recorded” in this field. What is the cleaned vector? (Answer: c(6, NA_real_, 9).)
A factor has observed values "Central", "", and NA. Blank labels are missing. Why can x[trimws(as.character(x)) == ""] <- NA fail, and what condition should be used instead? (Answer: the comparison can produce FALSE, TRUE, NA, so the replacement index contains NA; use !is.na(x) & trimws(as.character(x)) == "".)

What You'll Learn