Why Missing Values Need Plot-Specific Rules
An automated diagnostic plot is only interpretable if readers know which records could contribute to it and which records did contribute. A missing prediction may prevent a record from appearing in a residual-versus-prediction plot, while a missing covariate may prevent it from appearing in a residual-versus-covariate plot but leave it usable in the first plot. A missing subject identifier can prevent assignment to an individual panel without preventing the record from appearing in an ungrouped scatter plot.
The practical rule is to define eligibility for each plot from the quantities that plot actually needs. Do not remove a record from every plot because one field is missing, and do not let plotting software silently drop rows without reporting the consequence. The earlier material on diagnostic counts and coverage provides context for reporting denominators; here, the focus is on making those denominators explicit when required quantities are unavailable.
Specify What Each Plot Requires
Before drawing a plot, list its required fields and state the rules for deciding whether each field is usable. For a residual-versus-prediction scatter plot, the record generally needs an observed value and a prediction to calculate the residual, and a finite prediction and residual to position the point. If the plotted residual has additional requirements, such as a valid scale estimate for standardization, those requirements must also be included. A residual-versus-covariate plot additionally requires the covariate value. An individual profile may additionally require an identifier to assign observations to individuals.
A blank or coded missing value is not the only possible problem. A value may be present but invalid for the intended calculation: for example, an infinite prediction cannot be used as an ordinary finite plotting coordinate. A quantity may also be undefined by its mathematical definition. These cases should be distinguished in the preparation rules and, where relevant, in the exclusion report. Do not silently treat all of them as the same kind of missingness.
For a plot with \(N\) candidate records, let \(E\) be the set of eligible records. The number plotted is \(n_{\mathrm{plot}}=|E|\), and the number excluded is \(N-n_{\mathrm{plot}}\). The plotted fraction is \(n_{\mathrm{plot}}/N\), where \(N\) must be stated as the candidate-record denominator. If exclusions have several causes, report counts so that a reader can see how the total was obtained.
When several required fields can be unavailable on the same record, counting each field's missing values and adding those counts may double-count records. Let \(A\) be the set missing a prediction and \(B\) the set missing an observation. The number missing at least one of these fields is the size of their union, not generally the sum of their sizes.
A robust report can provide both field-level counts and a mutually exclusive accounting of plot exclusions. One clear method is to assign each excluded record a primary reason using a documented priority order, while also listing any additional missing fields. Another is to report overlapping reason counts explicitly and give the unique number of excluded records separately.
Worked Examples
Worked Example: Residuals and Predictions Are Not Both Available
Suppose a hypothetical diagnostic data set contains eight observation records. Residuals are defined here as observed value minus prediction. The observed values and predictions, in arbitrary concentration units, are: record 1, 10 and 9; record 2, 12 and 13; record 3, missing and 11; record 4, 15 and 14; record 5, 9 and 10; record 6, 11 and missing; record 7, 13 and 12; record 8, 8 and positive infinity. Determine which records can appear in a residual-versus-prediction plot.
The plot requires a valid observed value and prediction to calculate a residual, and finite values for both plotted coordinates. The residuals for records 1, 2, 4, 5, and 7 are, respectively, \(10-9=1\), \(12-13=-1\), \(15-14=1\), \(9-10=-1\), and \(13-12=1\) concentration units. Each has a finite prediction and residual, so these five records are eligible.
Record 3 lacks the observed value needed to calculate its residual. Record 6 lacks a prediction, so neither its residual nor its prediction coordinate is available. Record 8 has an infinite prediction; the residual calculation does not give a finite plotting coordinate, so it is invalid for this ordinary scatter plot. Thus, \(N=8\), \(n_{\mathrm{plot}}=5\), and \(n_{\mathrm{excluded}}=8-5=3\). The plotted fraction is \(5/8=0.625\), or 62.5%.
A concise report would say that 5 of 8 records were plotted and 3 were excluded: one because the observation was missing, one because the prediction was missing, and one because the prediction was nonfinite. This states what the denominator is and makes clear that the three records are distinct.
Worked Example: A Missing Covariate Affects Only One Plot
In a hypothetical data set of 12 records, 10 have valid observations and predictions, and 2 do not. Of those 10 otherwise usable records, 2 have a missing body-weight covariate. Compare the record counts for a residual-versus-prediction plot and a residual-versus-body-weight plot.
Both plots require a valid observation and prediction to calculate the residual. The residual-versus-prediction plot therefore has 10 eligible records: \(12-2=10\). Its plotted fraction is \(10/12=0.833\), or 83.3%. The two records with missing body weight remain usable for this plot because body weight is not one of its required fields.
The residual-versus-body-weight plot also requires body weight. The two missing body weights occur among the 10 records with valid residuals, so 8 records are eligible. Its plotted fraction is \(8/12=0.667\), or 66.7%. In this example the exclusions are disjoint: 2 records lack the observed value or prediction needed for either plot, and 2 additional records lack body weight. Hence \(12=8+2+2\) for the covariate plot.
The plot-specific exclusion report should not say simply that 4 records had missing data. It should explain that 2 of 12 were unavailable for residual calculation and 2 more were unavailable for the covariate plot. The first plot retains all 10 records with valid residuals.
Worked Example: Missing Identifiers in Individual Profiles
A hypothetical data set has nine records with valid times and residuals. Seven records have subject identifiers, belonging to subjects A, B, and C; two records have a missing subject identifier. Compare an ungrouped residual-versus-time plot with a plot that assigns observations to individual profiles.
The ungrouped plot requires time and residual, both valid for all nine records. It can therefore show all 9 records, or \(9/9=100\%\). The missing identifiers do not prevent these points from being placed on the axes.
The individual-profile plot requires an identifier to assign each record to a subject. Only 7 records can be assigned, so \(9-7=2\) are excluded from that plot, and the plotted fraction is \(7/9=0.778\), or 77.8%. The report should state that two records were omitted from subject-specific grouping because their identifiers were missing. It should not assign both records to a single invented “unknown subject,” since that would imply they belong to the same individual without evidence.
These two plots use the same source records but have different eligibility rules. Missing identifiers reduce coverage for the grouped plot, not necessarily for every diagnostic plot.
Justifying an Exclusion Count
The accounting identity \(N=n_{\mathrm{plot}}+n_{\mathrm{excluded}}\) follows by dividing the candidate records into two disjoint sets: records that meet every plot requirement and records that fail at least one requirement. Every candidate record belongs to exactly one of these sets. The identity is useful as a validation check in an automated workflow: if the counts do not add to \(N\), records may have been lost, duplicated, or omitted from the accounting.
For several exclusion causes, the report should make the relationship between counts explicit. Suppose 3 records lack predictions and 2 lack observations, with 1 record in both groups. The number of distinct records with at least one of those problems is \(3+2-1=4\), not 5. A report that gives only field-level counts can still be useful, but it must not present their sum as the unique number excluded unless the categories are known to be disjoint.
Diagnostics, Alternatives, and Limitations
A plot-generation routine should validate its accounting as well as its plotted values. For every plot, compare the candidate count with the eligible and uniquely excluded counts. Check that every plotted record has finite values for all required axes, and that each exclusion reason is consistent with the stated rules. If a plot uses identifiers or covariates for grouping, check eligibility for that grouping separately from eligibility for the underlying coordinates.
The main alternatives are not interchangeable. A plot-specific complete-case rule retains every record that has the fields needed for one plot. A global complete-case rule first removes any record missing any field used anywhere in a report; it is simpler to implement, but can unnecessarily reduce the sample for plots that do not need those fields. Imputation replaces unavailable values with estimates; it may be appropriate for some analyses under defensible assumptions, but it changes what is being plotted and should not be introduced merely to make a diagnostic figure look complete. If used, the imputation method and its implications must be made explicit.
Missingness can also affect how representative a diagnostic plot is of the candidate data. If records with unavailable predictions are concentrated at high concentrations, for example, the plotted points may not show that part of the data. The exclusion count reveals the amount of missing coverage, but does not by itself establish why values are missing or whether the plotted subset is representative. Report the pattern when it is known, and avoid interpreting an incomplete plot as if every candidate record were observed.
An important distinction is between a value that is unavailable and a value that is legitimately undefined. A standardized residual may be undefined when the scale used in its denominator is zero. Such a record cannot contribute that standardized residual, but the reason differs from a blank input field. Record the distinction rather than manufacturing a value. Likewise, do not replace missing predictions or covariates with zero unless zero is genuinely observed and valid for the quantity.
These rules do not determine a cause for missingness, correct a model, or guarantee that a plotted subset supports every scientific conclusion. Their purpose is narrower and essential: to make plot inclusion reproducible, retain usable records for plots that do not require the missing field, and expose the coverage that remains.
Check Your Understanding
Answer each question using the plot's required fields, then compare your reasoning with the solutions.
- A residual-versus-covariate plot has 20 candidate records. Three lack a prediction, and among the remaining records two lack the covariate. Assuming the sets do not overlap, how many records are plotted and what is the plotted fraction?
- Four records lack predictions, three lack observations, and two records are in both groups. How many distinct records have at least one of these problems?
- All coordinates for a scatter plot are valid, but one record lacks a subject identifier. Must it be excluded from the ungrouped scatter plot? Explain.
- Why can a global complete-case rule show fewer records than plot-specific eligibility rules?
Solutions. (1) The first three records cannot supply a residual. Of the remaining \(20-3=17\), two lack the covariate, leaving \(17-2=15\) plotted. The fraction is \(15/20=0.75\), or 75%. (2) Use the union count: \(4+3-2=5\) distinct records. (3) No. If the plot requires only the valid coordinates, the record can be shown; the missing identifier matters only for a plot that needs subject assignment. (4) Global complete-case removal excludes records for missing fields even when a particular plot does not use those fields, whereas plot-specific rules require only the fields needed for that plot.