Two Ways Survey Information Can Be Missing
A survey may begin with a selected group of people, yet the final data set may contain less information than planned. Some selected people do not take part at all. Others participate but leave one or more questions unanswered. These situations are both forms of nonresponse, but they occur at different levels and should not be confused.
The distinction matters because it changes what the survey has actually observed. When a selected person does not participate, the survey receives no information from that person. When a participant skips a question, the survey may still have that person's answers to other questions; only the skipped item lacks a response. A survey can contain both patterns at once.
These definitions distinguish the level at which information is missing. Unit nonresponse concerns the selected person’s participation in the survey as a whole. Item nonresponse concerns the availability of a response to a particular question from someone who did participate. For example, a selected person who never returns a questionnaire is a unit nonrespondent. A person who returns it but leaves the question about weekly exercise blank is an item nonrespondent for that question.
The classification depends on what information was provided, not on why it is missing. A person might be unreachable, decline, or stop responding for many reasons; those reasons do not change the basic distinction. Likewise, a skipped question is item nonresponse whether the participant skipped it intentionally or overlooked it.
Classifying Missing Information
For each selected unit, first ask whether the person provided any usable information. If not, classify that case as unit nonresponse. If the person did participate, examine each question separately: an unanswered question is item nonresponse for that item, while answered questions have observed responses.
“Usable information” is interpreted according to how the survey defines participation and what counts as a response. A questionnaire with no answers may be treated as no participation. If a participant answers some questions and then stops, the completed questions still contain information; the unanswered questions have item nonresponse. Some surveys also record this as a partial interview or breakoff. The practical lesson is not to label every incomplete questionnaire as wholly missing: identify which items have answers and which do not.
Worked Example: No Survey Returned Versus One Blank Question
A hypothetical campus survey selects 50 students. Forty-two return a questionnaire with at least one answer. Of those 42 participants, 37 answer the question about weekly study hours, while 5 leave that question blank. The remaining 8 selected students provide no survey information.
The 8 students who provide no information are unit nonrespondents. The 5 participants who leave the study-hours question blank are item nonrespondents for that question. The other 37 participants provide an observed value for study hours.
The survey therefore has 42 participants, but only 37 observed answers to the study-hours item. The item-specific count is not 42 because five participants skipped that question, and it is not 50 because eight selected students did not participate at all.
As a simple accounting check, the selected students divide into 8 with no survey information and 42 who participated: \(8+42=50\). Among the 42 participants, the study-hours item divides into 5 missing answers and 37 observed answers: \(5+37=42\). These counts describe different stages of the information available.
The distinction can also be expressed with counts. Let \(n\) be the number of selected eligible units, \(r\) the number who participate, and \(m_j\) the number of participants who answer item \(j\). In a survey with no other eligibility or data-processing complications, the number of unit nonrespondents is \(n-r\), and the number of item nonrespondents for item \(j\) is \(r-m_j\). The item count \(m_j\) can differ from question to question.
The second count uses participants as its starting group because item nonresponse refers to unanswered questions among participants. A selected person who supplied no information is not counted as someone who skipped one particular item in this simple accounting; that person is counted at the unit level. Survey reports may use more detailed categories for partial responses, but the basic distinction remains whether information is missing for the person as a whole or for a specific item.
Worked Example: Finding the Denominator for an Item
Suppose 120 people are selected for a hypothetical survey. Of these, 90 provide at least one answer. On the question about commute time, 81 of those 90 participants provide a value.
First calculate unit nonresponse: \(120-90=30\) selected people provided no survey information. Next calculate item nonresponse for commute time: \(90-81=9\) participants did not answer that question.
If the survey reports the fraction of selected people who participated, the denominator is 120: \(90/120=0.75\), or 75%. If it reports the fraction of participants who answered the commute-time item, the denominator is 90: \(81/90=0.90\), or 90%. The 81 observed commute-time values are the available responses to that item.
These percentages answer different questions. The first describes participation among selected people; the second describes completion of a particular item among participants. Using 120 as the denominator for the item-completion percentage would describe something else: the fraction of everyone selected who supplied a commute-time answer, \(81/120=0.675\), or 67.5%. A report should make clear which group its denominator represents.
One Survey Can Have Both Forms
A survey’s missing information is not always one uniform block. Some selected people may provide nothing, some participants may answer every question, and others may skip only certain items. As a result, the sample size used for one variable can be smaller than the number of survey participants.
For instance, a questionnaire might ask about age, travel time, and transportation mode. A participant could report age and mode but omit travel time. That person contributes observed values for age and transportation mode, but is an item nonrespondent for travel time. It would be inaccurate to discard all of that person’s answers merely because one variable is missing, unless a particular analysis requires a complete set of answers.
The following accounting separates selected units, participants, and observed answers for a specific item. It is a short way to verify that the two kinds of missing information are not being combined.
There are \(n=100\) eligible people selected for a hypothetical survey.
If \(r=76\) people provide at least some information, then \(100-76=24\) are unit nonrespondents.
If 65 participants answer item \(j\), then \(76-65=11\) are item nonrespondents for that item.
The 100 selected units comprise 24 with no information and 76 participants. Those 76 participants comprise 65 answers and 11 blanks for item \(j\). Thus \(24+76=100\) and \(65+11=76\).
This accounting is useful because it prevents a common denominator error. There are 24 unit nonrespondents, but there are 11 item nonrespondents for item \(j\) among the participants. The 35 selected people without an observed answer to item \(j\) are the sum \(24+11\), but that combined number includes two different situations. It should not be described as 35 participants skipping the item: 24 did not participate in the survey at all.
Worked Example: Classifying a Partially Completed Questionnaire
A hypothetical survey selects 60 patients and asks each about age, usual sleep duration, and satisfaction with a clinic visit. Fifty people submit at least some answers. Among them, 50 answer the age item, 44 answer sleep duration, and 47 answer satisfaction. The other 10 selected people provide no information.
The 10 people with no information are unit nonrespondents. All 50 participants provide an age response, so there is no item nonresponse for age among these participants. Six participants leave sleep duration blank, because \(50-44=6\). Three leave satisfaction blank, because \(50-47=3\).
The available sample size is therefore different for each item: 50 for age, 44 for sleep duration, and 47 for satisfaction. The six people missing sleep-duration answers are not unit nonrespondents; they participated and answered other questions. Likewise, the three people missing satisfaction answers may still contribute their age and sleep-duration information.
This example also shows why “the survey had 50 responses” can be too vague. It may mean 50 people participated, but it does not mean that 50 answers are available for every variable.
Why the Distinction Matters
Unit and item nonresponse can each affect what a survey describes. If people who do not participate differ in a relevant way from those who do, the participants may not represent the selected group well for that characteristic. If participants who skip an item differ from those who answer it, the observed answers to that item may not represent all participants. These are possible sources of nonresponse bias, not automatic consequences of missing data.
The type of nonresponse also guides what can be calculated. A summary of one item can use only the observed values for that item. If some participants omit it, the calculation’s actual data count is the number who answered it, not the total number of participants or selected people. For a comparison involving several items, the analyst must also know which records contain the needed answers.
The distinction does not by itself determine whether a survey estimate is biased. Missing answers may or may not systematically differ from observed answers. Classification tells us where information is missing; assessing its consequences requires considering how the missingness relates to the survey question and the people or answers observed.
Common Mistakes
- Calling every blank an instance of unit nonresponse. A participant who answers most of a questionnaire but leaves one question blank has item nonresponse for that question, not unit nonresponse for the entire survey.
- Calling a person who never participates an item nonrespondent for every question. In the basic classification, that person is a unit nonrespondent because no survey information was provided.
- Using the participant count as the available count for every variable. The number of observed answers can differ by item. Check the number who answered the particular question being analyzed.
- Assuming nonresponse necessarily creates bias. Nonresponse creates missing information; bias depends on whether the missing cases differ in relevant ways from those observed.
- Combining distinct counts without describing them. The number with no answer to an item may include both unit nonrespondents and participants who skipped that item. State the two groups separately when their distinction matters.
Check Your Understanding
For each situation, identify the level at which information is missing and explain which group supplies the relevant denominator.
- A survey selects 80 people. Twelve never provide any information; of the 68 participants, 60 answer a question about exercise. How many unit nonrespondents and how many item nonrespondents are there for the exercise question?
- A participant answers questions about age and travel mode but leaves the income question blank. Classify the missing income response.
- Why is the number of participants not necessarily the number of observed answers for a particular survey variable?
- In a survey of 100 selected people, 70 participate and 63 answer a particular item. Calculate the number of unit nonrespondents and item nonrespondents for that item.
- Does finding item nonresponse establish that a survey estimate is biased? Explain briefly.