Why encode missingness patterns explicitly?
Longitudinal analyses need more than a column of outcomes with blank cells. Downstream code often needs to know which visits were planned, which outcomes were observed, whether a participant returned after a missed visit, and when follow-up ended. A reliable encoding makes those distinctions explicit instead of relying on software-specific interpretations of blanks.
The central object is a participant’s ordered sequence of outcome-observation indicators. When observed visits form an initial uninterrupted run followed only by missing visits, the sequence is monotone and can be summarized by a dropout time. If an outcome is missing and then observed at a later visit, the pattern is intermittent, not monotone. Calling the first missed visit “dropout” in that situation changes the recorded history.
This convention treats the follow-up visits as the ordered sequence being encoded. If baseline is part of the dataset, it can be retained as a separate visit (often indexed \(j=0\)); it should not be confused with the post-baseline dropout time. The convention also presumes that the visit sequence is defined for the participant. Visits not planned under the trial design should be marked as not scheduled, rather than silently represented as participant-level missing outcomes.
From wide records to visit-level records
A wide file has one row per participant and separate outcome columns for each visit. A long, or visit-level, file has one row per participant per planned visit, with fields such as participant identifier, visit number, outcome, observation indicator, and—when available—scheduled status and reason for missingness. The long form makes the time order visible and lets code check each participant’s history in the same way.
Keep the outcome and its observation indicator consistent: if \(R_{ij}=1\), \(Y_{ij}\) must contain an observed value; if \(R_{ij}=0\), the outcome should be stored as missing, not as a numeric placeholder such as zero. Zero may be a valid measured outcome. A separate reason field can record, for example, a missed appointment, withdrawal, or an administrative circumstance. The reason field adds context; it does not replace the observation indicator.
Worked Example: Reshaping Two Participants’ Records
Consider a hypothetical trial with three planned follow-up visits at weeks 4, 8, and 12. A wide file contains the following values: participant A has outcomes 8, 7, and missing; participant B has missing, 6, and 5. Their baseline outcomes, recorded separately, are 10 and 9. Reshape the follow-up data and encode observation.
For participant A, the week-4 and week-8 outcomes are observed and the week-12 outcome is missing. Thus the visit-level records carry \(R_{A1}=1\), \(R_{A2}=1\), and \(R_{A3}=0\). For participant B, the sequence is \(R_{B1}=0\), \(R_{B2}=1\), and \(R_{B3}=1\).
| Participant | Visit | Outcome | \(R_{ij}\) |
|---|---|---|---|
| A | Week 4 | 8 | 1 |
| A | Week 8 | 7 | 1 |
| A | Week 12 | Missing | 0 |
| B | Week 4 | Missing | 0 |
| B | Week 8 | 6 | 1 |
| B | Week 12 | 5 | 1 |
A’s pattern is \((1,1,0)\), which is monotone, and \(D_A=2\): the last observed follow-up is week 8. B’s pattern is \((0,1,1)\), which is not monotone. The later observed outcomes show that the missed week-4 outcome was not the start of permanent dropout. Assigning B a dropout time of zero from this pattern would be incorrect.
Deriving and checking dropout time
For a monotone pattern, the observed entries form a prefix of ones followed by zeros. If that prefix has length \(d\), the last observed visit is \(d\), so \(D_i=d\). This gives a convenient calculation from the indicator sequence—but only after monotonicity has been checked.
For \(D_i=0\), all follow-up indicators are zero. For \(D_i=J\), all planned follow-up outcomes are observed; this means follow-up is complete, not that the participant necessarily dropped out. A useful equivalent calculation for a monotone pattern is \(D_i=\sum_{j=1}^{J}R_{ij}\), since the number of ones equals the length of the prefix.
The justification is direct. If the pattern is monotone, every observed visit before or at the last observed visit must be observed: a zero before that visit would, by monotonicity, force all later indicators to be zero. Every visit after the last observed visit is zero by the definition of “last.” Therefore the sequence has exactly \(D_i\) ones, followed by zeros, and summing its indicators returns \(D_i\). Without monotonicity, the sum counts observed visits but need not identify the last one.
Worked Example: Identifying Valid and Invalid Patterns
Suppose five hypothetical participants have four planned follow-up visits. Their observation patterns are A: \((1,1,1,1)\), B: \((1,1,1,0)\), C: \((1,1,0,0)\), D: \((1,0,0,0)\), and E: \((1,0,1,0)\). Determine which dropout times can be assigned under the monotone convention.
A through D are monotone. Their dropout times are \(D_A=4\), \(D_B=3\), \(D_C=2\), and \(D_D=1\), respectively. The calculation by summing indicators agrees: for C, \(D_C=1+1+0+0=2\).
E is nonmonotone because \(R_{E2}=0\) but \(R_{E3}=1\). Its indicator sum is \(1+0+1+0=2\), while its last observed visit is 3. Thus using the sum as a dropout-time calculation would return 2 even though the final observed visit is visit 3. The proper result is to flag E as intermittent missingness; do not assign a monotone dropout time to this sequence without an explicit, justified data-handling rule.
When the final observed visit is not the dropout event
A dropout time in visit units is a summary of observed follow-up, not necessarily the exact time at which withdrawal occurred. If a participant has a final observed outcome at visit 2 and withdraws sometime before visit 3, the available visit-level data may establish only that follow-up ended between those visits. Record the visit-based summary and any known withdrawal timing or reason in separate fields.
Likewise, a participant observed at every planned visit has \(D_i=J\), but may simply have completed the study. If downstream code needs to distinguish a withdrawal before the end from administrative completion, retain a separate status variable. For example, define \(G_i=1\) when there is documented dropout before planned follow-up ends and \(G_i=0\) when follow-up is complete; this status cannot safely be inferred from \(D_i\) alone. The coding should follow the protocol’s definitions of completion and withdrawal.
Worked Example: Completion, No Follow-Up, and Dropout Status
In a hypothetical study with four planned follow-up visits, participant P has pattern \((1,1,0,0)\) and is documented as withdrawing after visit 2. Participant Q has pattern \((1,1,1,1)\) and completes all visits. Participant R has pattern \((0,0,0,0)\) after an observed baseline; the record confirms withdrawal before the first follow-up.
P has \(D_P=2\) and documented pre-completion dropout, so \(G_P=1\). Q has \(D_Q=4\), but completed follow-up, so \(G_Q=0\). R has \(D_R=0\) because no follow-up outcomes are observed; given the documented withdrawal, \(G_R=1\). The same \(D=J\) pattern could not by itself tell us whether a participant completed, and \(D=0\) would not by itself establish the reason for no observed follow-up. The documented status supplies that distinction.
A practical encoding and validation sequence
A safe workflow constructs the full planned participant-visit grid first, then attaches outcomes and observation information. This prevents a missing row from being mistaken for an observed outcome or an unplanned visit. The following sequence is suitable before any downstream analysis code uses the patterns.
Specify the ordered follow-up visits and which participants were eligible or scheduled at each one. Keep baseline distinct if it is not part of the follow-up sequence.
Reshape the outcomes to long form. Check that each participant-visit pair appears once and that visit labels map to the intended order.
Set \(R_{ij}=1\) for an observed scheduled outcome and \(R_{ij}=0\) for a missing scheduled outcome. Preserve schedule status and known missingness reason separately.
Confirm that observed indicators have nonmissing outcomes and missing indicators do not contain placeholder outcomes. Order each participant’s indicators and flag every zero followed by a later one.
For monotone sequences, calculate \(D_i\) as the number of leading ones. Keep intermittent patterns flagged rather than forcing them into a dropout-time field.
Diagnostics, alternatives, and limitations
Useful checks include confirming the expected number of planned rows per participant, detecting duplicate participant-visit pairs, checking that visit order is consistent, and tabulating pattern frequencies. A simple transition check is especially informative: any pair of visits \(j<k\) with \(R_{ij}=0\) and \(R_{ik}=1\) proves that participant \(i\)’s observed pattern is not monotone. Also compare the count of observed outcomes in the long file with the count in the source file. Disagreement can reveal dropped records or duplicated rows during reshaping.
Several plausible shortcuts fail. Treating every blank as dropout loses the distinction between an isolated missed visit and permanent loss to follow-up. Replacing missing outcomes with zero corrupts the outcome values. Computing \(D_i\) as the number of observed visits before checking the sequence fails for intermittent patterns. Finally, defining all participants with \(D_i=J\) as “non-dropouts” may be appropriate for a particular status definition, but it must be supported by the follow-up record rather than assumed from the visit pattern alone.
Long format is convenient for visit-level checks and most longitudinal software; wide format can be convenient for fixed-width data entry or some model routines. Either format can represent the same information if visit order and missingness are preserved. A pattern string, such as “1100,” is a compact alternative to storing separate indicators, but is less transparent for row-level joins and validation unless its visit order is documented. In either representation, retain the underlying visit-level records.
This encoding describes what was observed and when; it does not explain why outcomes are missing or establish assumptions about the missingness mechanism. Intermittent missingness can occur alongside later return, and monotone patterns can arise from different reasons. Pattern coding is therefore a data-organization step for downstream analysis, not a substitute for choosing or justifying an analysis model. When a workflow requires monotone dropout, report how many records are nonmonotone and investigate their source rather than silently recoding them.
Check Your Understanding
Answer each question using the definitions in this tutorial. Solutions follow the questions.
- With five planned follow-up visits, what is \(D_i\) for the pattern \((1,1,0,0,0)\), and is it monotone?
- Is \((1,0,1,0)\) monotone? Why is its indicator sum not a valid dropout time?
- A participant has observed outcomes at all four planned visits. What is the visit-based dropout time, and what additional information distinguishes completion from dropout?
- Why should an unobserved outcome not be replaced with the number zero during reshaping?
Solutions. (1) The pattern is monotone because no zero is followed by a one; \(D_i=2\). (2) No. The zero at visit 2 is followed by an observed outcome at visit 3. Its indicator sum is 2, but its last observed visit is 3, so the sum does not encode a valid monotone dropout time. (3) \(D_i=4\). A separate documented completion or dropout status is needed, because the observed pattern alone does not establish why follow-up ended. (4) Zero can be a valid outcome value, so using it as a missingness placeholder would make an unobserved outcome indistinguishable from a measured zero and would corrupt analyses of the outcome.