Why Classify Missing Covariates First?
A blank covariate cell is not an explanation. It may mean that the covariate has no defined value for that record, that the study never planned to collect it, that an intended measurement could not be obtained, or that a recorded value disappeared during data handling. Those situations have different meanings for dataset construction. If they receive the same generic missing code, downstream users cannot tell whether a value is structurally absent, absent by design, unavailable at the source, or missing only from the analysis extract.
The classification should be made for a particular record-variable pair, using the applicable study plan and source data. A participant may have an observed baseline value but an unavailable value at a later visit; a covariate can therefore have different statuses across records. The aim here is to identify and document why a value is absent, not to choose a value to fill it in.
These are operational classifications, not conclusions about the statistical mechanism that produced missing data. A classification is only as reliable as the protocol, data dictionary, source records, and transfer history used to support it. If the available evidence does not identify the cause, record the status as unresolved and investigate; do not infer a reason from the blank cell alone.
A Record-Level Decision Process
Start with the meaning of the covariate and the record’s scope. Then check whether collection was planned, whether the source contains a value, and whether that value reached the analysis dataset. This order matters: it avoids calling a value “unavailable” when it was never intended to be collected, and avoids calling it “not applicable” merely because it was not collected.
If no, classify it as not applicable. Confirm that the definition and record context support this conclusion; do not use a numeric zero to represent a value that has no meaning.
If the covariate is defined but collection was not planned, classify it as not collected. Use the protocol, schedule, or documented data-collection plan as evidence.
If collection was planned but the source has no obtainable value, classify the covariate as unavailable when the reason is supported by source documentation, such as a missed assessment or failed measurement.
If the source contains the value but the analysis record does not, investigate the data path and classify it as lost in transfer only when the discrepancy is confirmed.
Keep the record unresolved, preserve the original data, and query the responsible source or data-processing step. Do not force it into one of the supported categories.
The logic can be summarized with four checks: defined for this record, planned for collection, present in source, and present in analysis. For example, “defined and planned, absent from source” points toward unavailability; “present in source, absent from analysis” points toward transfer loss. These checks are not a substitute for documentation. A blank source field alone cannot establish whether an assessment was missed, refused, omitted from the plan, or lost before the source file was created.
Worked Examples
Worked Example: A Covariate Collected Only in a Substudy
Suppose a hypothetical study enrolls 80 participants. Estimated glomerular filtration rate (eGFR) is meaningful for all 80 participants, but the data-collection plan calls for it only in a 30-participant substudy. The source records show eGFR for 27 substudy participants; the other 3 had a planned assessment that was missed. The remaining 50 participants were not in the substudy.
For the 50 participants outside the substudy, eGFR is still defined and meaningful, but collection was not planned. Their records are therefore not collected, not not applicable. For the 3 substudy participants with missed assessments, collection was planned and the covariate applies, but no value was obtained; classify those records as unavailable. The 27 recorded values are observed.
The categories account for all participant records: \(27 + 3 + 50 = 80\). As a check, the proportions are \(27/80 \times 100 = 33.75\%\) observed, \(3/80 \times 100 = 3.75\%\) unavailable, and \(50/80 \times 100 = 62.5\%\) not collected. These percentages describe the status of records, not the proportion of participants with a particular eGFR.
The classification depends on the stated definition: eGFR applies to everyone, while its collection was restricted by design. If, instead, the covariate had no defined meaning for a particular record, that record would be not applicable. A substudy boundary alone does not make a meaningful covariate inapplicable.
Worked Example: Event-Specific Covariate Values
In a hypothetical dataset, infusion interruption duration is recorded for each documented interruption event. The covariate is defined only when an interruption event occurs. There are 60 participant-visit records: 12 contain an interruption event and 48 do not. Of the 12 event records, 9 have a duration in both source and analysis files, 2 have no duration documented at source after a planned assessment, and 1 has a duration in the source worksheet but a blank in the analysis extract.
The 48 records without an interruption have no interruption duration to report, so the field is not applicable on those records. The 2 event records with no source value are unavailable, assuming the source review confirms the value was not obtained. The event record with a source value missing from the analysis extract is lost in transfer once the discrepancy is verified. The remaining 9 values are observed.
The record counts reconcile: \(48 + 2 + 1 + 9 = 60\). Among the 12 interruption events specifically, \(9/12 \times 100 = 75.0\%\) have observed values, \(2/12 \times 100 = 16.67\%\) are unavailable, and \(1/12 \times 100 = 8.333\%\) is lost in transfer. The denominator is 12 for event-level percentages because only those records have a defined interruption duration.
This example illustrates why the unit of classification must be explicit. If the analysis table has rows for all visits, a no-interruption visit can carry a not-applicable status. It should not be converted to a duration of zero unless the variable definition explicitly treats “no interruption” as a measured duration of zero.
Worked Example: Finding Values Missing From an Analysis Extract
Consider a hypothetical study with 40 participants for whom baseline serum creatinine was planned and applicable. The reviewed source file contains values for 38 participants and no values for 2. The analysis extract contains values for 36 participants. A row-by-row reconciliation confirms that 2 of the 38 source values are absent from the extract; the other 2 participants have no source measurement documented.
Classify the 36 values present in the analysis extract as observed. Classify the 2 participants with source values missing from the extract as lost in transfer, provided the reconciliation confirms a processing or transfer discrepancy. Classify the 2 participants with no source value as unavailable if the assessment was planned but could not be obtained. No participant is not collected or not applicable in this example because the covariate applies to all 40 and collection was planned for all 40.
The full accounting is \(36 + 2 + 2 = 40\). Relative to all 40 participants, the observed share is \(36/40 \times 100 = 90.0\%\), the transfer-loss share is \(2/40 \times 100 = 5.0\%\), and the unavailable share is \(2/40 \times 100 = 5.0\%\). The two causes require different follow-up: retrieve or correct the transferred values where possible, and investigate the missing source assessments separately.
If the extract is blank but the source has not yet been checked, the status is not yet known. Calling every such blank “unavailable” would conceal a possible pipeline defect. The correct immediate action is reconciliation, not assumption.
Documenting the Classification and Checking the Dataset
A useful preparation record keeps the value, its status, and the evidence for the status distinguishable. Preserve source values and source identifiers; add a reason code or status field rather than overwriting the original covariate with an arbitrary substitute. Where feasible, retain the collection-plan reference, source-file location, query outcome, and date or version of the extract used in the reconciliation.
| Status | Evidence to seek | Preparation implication |
|---|---|---|
| Not applicable | Variable definition and record context show that no value is defined | Keep the status distinct from a measured zero or an unobserved value |
| Not collected | Protocol or collection plan excludes the record, although the covariate is meaningful | Document the planned scope; do not label it unavailable merely because a cell is blank |
| Unavailable | Collection was planned and applicable, but the source has no value; source notes may give the reason | Retain the reason when known and distinguish it from transfer problems |
| Lost in transfer | Source contains a value that is absent from the analysis extract | Investigate the data path and correct the extract when justified |
Basic diagnostics can reveal a faulty classification. Compare counts by status with the number of eligible records and confirm that every record-variable pair has one supported status. Check for impossible combinations, such as a not-applicable status alongside a recorded value, or a transfer-loss status when no source value has been located. Compare not-collected counts against the documented collection plan. Review unavailable records against source notes and visit completion information. Check whether transfer losses cluster by file, site, variable, or extract version, which may point to a systematic processing issue rather than isolated omissions.
A short justification follows directly from the decision order. Not applicable is determined by whether the covariate is defined; collection status is considered only after that question is answered. For a defined covariate, planned collection separates not collected from cases that should have a source value. For planned, applicable records, source-versus-analysis comparison separates a source-level absence from a transfer discrepancy. Thus the sequence uses different evidence at each branch and avoids assigning two explanations to one record. When evidence at a branch is missing, the classification remains unresolved rather than being guessed.
Common Pitfalls and Alternatives
A single blank code is easy to create but discards information needed to understand and prepare the dataset. At the other extreme, inventing a detailed reason code for every blank without supporting evidence creates false certainty. A practical approach is to use a small set of well-defined statuses, retain an unresolved option, and document how each status is assigned.
- Confusing not collected with not applicable. “Not in the substudy” is not enough to make a meaningful covariate inapplicable. If it applies but collection was not planned, classify it as not collected.
- Treating a blank as proof of unavailability. A blank in the analysis file does not reveal whether the source was blank or the value was lost later. Reconcile against source data.
- Using zero as a missingness code. Zero may be a valid value. It does not identify why a value is absent and can change the interpretation of a covariate.
- Forcing an uncertain cause into a known category. Keep the status unresolved until source records or processing logs provide evidence.
- Classifying only at participant level. A participant can have different statuses at different visits or events. Use the record level at which the covariate is defined.
A binary present/missing flag is useful for a quick inventory, but it cannot distinguish these causes. A reason-coded classification is more informative when the evidence supports it. A fully documented decision tree is especially helpful when multiple teams prepare or transfer data. Whichever representation is used, its definitions should be stable, and the original data should remain traceable.
This classification is a preparation step, not a rule for filling values. In particular, it does not determine which covariates should later be imputed or how a longitudinal value should be carried across records. It establishes which records are genuinely observed, which kinds of absence are supported by evidence, and which discrepancies need resolution before those decisions are considered.
Check Your Understanding
For each question, decide what the evidence supports before reading the solution.
- A covariate is meaningful for every participant, but the protocol planned collection only for a specified substudy. How should a non-substudy participant’s blank be classified?
- A covariate is defined only for records with a documented infusion interruption. A visit has no interruption. What is the status of the interruption-duration field?
- A planned, applicable laboratory value is blank in the analysis extract. The source file has not been reviewed. Can it yet be classified as unavailable or lost in transfer?
- A planned baseline value is present in the source file but missing from the analysis dataset. What evidence supports a transfer-loss classification?
Solutions. (1) Not collected: the covariate is meaningful, but collection was not planned for that participant. (2) Not applicable: no interruption-duration value is defined for a record without an interruption. (3) No. Reconcile the source first; until the cause is supported, mark the status unresolved. (4) Confirm the source value belongs to that participant and record, then trace the discrepancy through extraction or processing. If confirmed, classify it as lost in transfer.