Why Summarize Missingness?
Before preparing data for a regression analysis, it is useful to know which fields lack values and whether the omissions are concentrated in particular records. A single blank can be easy to overlook in a wide table; a small summary can show that a particular variable is often unrecorded or that one observational unit has several missing fields.
These summaries describe the data in hand. They do not establish why a value is absent, whether the omission is an error, or whether the data are suitable for a particular analysis. Their immediate purpose is practical: make missingness visible and identify records that may need checking against the original source.
A missing-value proportion is a count divided by the number of opportunities for that value to be present. For a variable, the denominator is the number of observational units included in the audit. For a unit, the denominator is the number of selected variables included in the audit. The proportion can be multiplied by 100 to report a percentage.
Be explicit about what is included in the audit. Usually, the selected columns are data fields of interest, not identifiers such as record numbers. If a field applies only to some units, its “not applicable” entries should not automatically be treated as missing observations. Report the applicable denominator for that field, or define a separate rule for it, rather than silently counting inapplicable cells as missing. The examples below use a simple rectangular table in which each selected field is intended to have a value for every unit.
For these summaries, first use a consistent representation of missing values, following the prior work of recognizing missing-value codes. Count only values classified as missing under that rule. A summary is only as clear as the definition it uses: a blank, a special code, and a legitimate recorded value should not be grouped together without justification.
Summarizing Missingness by Variable
A variable-level summary gives one row per selected data field, with its missing count, denominator, and proportion. The denominator matters: “two missing” alone does not show whether two of five values or two of five thousand values are absent.
Worked Example: Finding Missingness Rates by Variable
The following is a small, entirely hypothetical data set of six clinic visits. A dash represents a value already classified as missing. The record ID identifies a row and is not included among the four variables being summarized.
| Record ID | Age (years) | Lab result (units) | Symptom score | Visit group |
|---|---|---|---|---|
| V101 | 42 | 10.2 | 3 | Early |
| V102 | 51 | — | 2 | Late |
| V103 | — | 8.4 | — | Early |
| V104 | 47 | 9.8 | 4 | — |
| V105 | 39 | 9.1 | 1 | Late |
| V106 | — | 10.0 | 2 | Early |
There are \(n=6\) visits. For age, the missing entries are in V103 and V106, so
Thus, two of six age values are missing, or 33.33%. The lab result has one missing entry, so its proportion is \(1/6=0.1667\), or 16.67%. The symptom score also has one missing entry, giving \(1/6=0.1667\). The visit group has one missing entry, also \(1/6=0.1667\). These denominators are all six because each of the four selected fields is intended to be recorded for every visit.
The summary shows that age has the largest missing proportion in this data set. That observation does not tell us why age is absent or whether the entries are mistakes. It gives a clear starting point for checking the source records and deciding what needs further attention.
Summarizing Missingness by Observational Unit
A variable-level total can conceal how omissions are distributed across records. Row summaries show whether missing values are spread across many units or concentrated in a few. To calculate a row proportion, divide that unit’s missing count by the number of selected variables. Keep that variable set consistent when comparing row proportions.
Worked Example: Flagging Records With Multiple Missing Fields
Use the same six-visit table. There are \(J=4\) selected fields: age, lab result, symptom score, and visit group. For V103, age and symptom score are missing. Its row count and proportion are
V103 therefore has two missing fields out of four, or 50.00%. Applying the same calculation to each row gives the following summary.
| Record ID | Missing fields | Missing count | Proportion missing |
|---|---|---|---|
| V101 | None | 0 | 0/4 = 0.0000 (0%) |
| V102 | Lab result | 1 | 1/4 = 0.2500 (25%) |
| V103 | Age, symptom score | 2 | 2/4 = 0.5000 (50%) |
| V104 | Visit group | 1 | 1/4 = 0.2500 (25%) |
| V105 | None | 0 | 0/4 = 0.0000 (0%) |
| V106 | Age | 1 | 1/4 = 0.2500 (25%) |
V103 is a reasonable record to check first because it has more missing fields than any other row. The table also makes clear which fields to verify. V102, V104, and V106 each have one missing field; their omissions should not be ignored merely because their row proportions are lower. A useful audit list preserves both the record ID and the names of its missing fields.
A higher row count is a reason to inspect a record, not proof that the record is defective. The values may have been unavailable, not collected, or recorded elsewhere. Checking the source is what distinguishes a recoverable omission from a value that is genuinely unavailable.
A Consistency Check for the Counts
The variable summaries and unit summaries describe the same missing cells from opposite directions. Adding the missing counts down the columns must give the same total as adding the missing counts across the rows. This gives a simple arithmetic check on a hand-prepared summary.
The justification follows by writing each count using the missingness indicator \(m_{ij}\). The total from variable summaries is
The middle equality changes only the order of adding the same \(nJ\) indicators; it does not change their sum. In the six-visit example, the variable counts are \(2,1,1,1\), which sum to 5. The row counts are \(0,1,2,1,0,1\), which also sum to 5. If the totals disagree, recheck the selected columns, row entries, and counting arithmetic before relying on the summaries.
A related overall proportion can be calculated as total missing cells divided by all selected cells. For the six-visit example, there are \(6\times4=24\) selected cells and five missing cells, giving \(5/24=0.2083\), or 20.83%. This overall rate is a compact description, but it can hide differences between variables and records. The variable- and unit-level summaries show where the omissions occur.
Worked Example: Checking a Separate Audit Summary
Consider a second hypothetical register containing eight observational units and three selected fields: age, exposure, and outcome. A preliminary variable summary reports two missing age values, one missing exposure value, and three missing outcome values. The column counts total
Suppose the row audit lists missing counts of 0, 1, 2, 0, 1, 0, 1, and 1. These sum to
The totals agree, so the two summaries pass this arithmetic consistency check. There are \(8\times3=24\) selected cells, and the overall proportion missing is \(6/24=0.2500\), or 25.00%. The variable-specific proportions are \(2/8=0.2500\) for age, \(1/8=0.1250\) for exposure, and \(3/8=0.3750\) for outcome. These rates identify outcome as the field with the largest missing proportion; the row counts identify which units have multiple omissions.
Agreement does not prove that every missing value was classified correctly: a mistaken classification could appear in both summaries. It confirms only that the row and column counts account for the same number of missing cells. Source checks and a clear definition of missingness remain necessary.
Using the Summaries Carefully
A missingness summary supports a data-quality check, not a conclusion about the process that produced the omissions. For example, a high missing proportion for one variable does not establish that the missing values are random, and a row with several missing fields is not automatically invalid. These counts alone do not determine how missing values should be handled in a later analysis.
Use the summaries to make the next verification step specific. A practical review can record the observational-unit ID, the field or fields marked missing, and whether the source was checked. If the value can be confirmed from an authorized source, follow the established correction and documentation procedures. If it cannot be recovered, preserve its missing status rather than inventing a value.
Report the scope and denominators alongside proportions. “Age is 33.3% missing among six visits” is more informative than “age is 33.3% missing.” If the selected fields, included rows, or applicable denominators differ, state those choices so that the percentages can be interpreted correctly.
Common Mistakes
- Reporting counts without denominators. “Three missing” is difficult to interpret alone. Report the number of eligible values as well, such as three of twelve.
- Using different row scopes without saying so. A variable’s proportion is \(M_j/n\) for the units in scope. If the analysis uses a subset, calculate and label the audit for that subset rather than presenting its rate as if it covered the full data set.
- Including identifiers as ordinary data fields. An ID column serves to identify units and is not usually part of the set of substantive variables summarized by row. State which fields are included.
- Treating a high count as evidence of a data error. Missingness flags a value for review; it does not explain why the value is absent. Check the source before deciding what the omission means.
- Assuming matching totals validate every entry. Equal row and column totals are a useful check, but they cannot rule out misclassification or other data errors.
Check Your Understanding
Use the definitions and calculations in this tutorial to answer the following questions.
- A data set has 20 observational units, and a selected variable has 5 missing values. What are its missing proportion and percentage?
- A row has 3 missing values among 12 selected fields. Calculate its missing proportion and percentage.
- Why should an identifier column usually be excluded from the selected fields used to calculate row missing proportions?
- Variable-level counts total 7, while the row-level counts total 8. What does this tell you, and what should you check?
- A unit has several missing fields. What can the summary establish, and what would require checking the original source?