Observed-Data Residual Checks Across MNAR Candidates

Why Check Residuals Across MNAR Candidates?

Competing missing-not-at-random (MNAR) models can imply different distributions for missing outcomes while fitting the observed data similarly. The earlier material on observed-data fit and fit agreement versus extrapolation agreement makes this distinction central: residual checks provide a way to inspect the observed outcomes for structure that a candidate has not captured. They can reveal a problem shared by several candidates, or a discrepancy that is more pronounced for one candidate.

The diagnostic target is the model-implied distribution of outcomes that were actually observed, conditional on the relevant covariates and response pattern. A residual check does not require this distribution to be correct. Rather, the model must specify such a distribution so its implications can be compared with the observed values; whether those implications are adequate is what the diagnostic helps assess. A good-looking residual plot is not proof that the MNAR assumptions for unobserved outcomes are correct.

Constructing Residuals From the Observed-Data Model

Write the observed outcome vector for subject \(i\) as \(\mathbf{Y}_{i,\mathrm{obs}}\). A candidate model specifies an observed-data distribution such as \(p_k(\mathbf{Y}_{i,\mathrm{obs}}\mid R_i,\mathbf{x}_i;\theta_k)\), where \(\theta_k\) denotes its parameters. This distribution is implied by the candidate MNAR model, including its assumptions about missing outcomes; it is not simply the outcome distribution among completers. In practice, the distribution or its relevant conditional components are evaluated at fitted parameter estimates.

For a continuous outcome, one useful residual at occasion \(j\) is the standardized conditional residual \(z_{ij}^{(k)}=(Y_{ij}-\widehat{\mu}_{ij}^{(k)})/\widehat{\sigma}_{ij}^{(k)}\). Here \(\widehat{\mu}_{ij}^{(k)}\) and \(\widehat{\sigma}_{ij}^{(k)}\) are the candidate’s fitted conditional mean and standard deviation for this observed outcome, with the conditioning made explicit. In a longitudinal setting, the joint observed-data distribution can be factored into successive conditional distributions. A residual for an observed occasion can then condition on the response pattern, covariates, and the subject’s earlier observed outcomes. This avoids treating a whole trajectory as if its components were unrelated.

When the conditional distribution is not adequately summarized by a mean and variance, a quantile residual is often more informative. For a continuous outcome, let \(\widehat{F}_{ij}^{(k)}\) be the candidate’s fitted conditional cumulative distribution function evaluated at the observed value. Then \(q_{ij}^{(k)}=\Phi^{-1}(\widehat{F}_{ij}^{(k)}(Y_{ij}))\), where \(\Phi^{-1}\) is the standard normal quantile function. If the specified conditional distribution is correct and its parameters are known, the probability integral transform is uniform, so these quantile residuals have a standard normal distribution. With estimated parameters and dependent repeated outcomes, that reference is approximate, not an exact independent-sample rule.

Worked Checks by Pattern and Candidate

Reading Residual Structure

Residuals are most useful when plotted or summarized against quantities that could reveal a specific failure. Include the response pattern in the display or facet the plot by pattern, particularly when the candidate’s conditional distribution depends on pattern. In longitudinal data, examine residuals by occasion and, where appropriate, against earlier observed outcomes. Against covariates, look for systematic location changes, curvature, changing spread, or clusters. A residual-versus-fitted plot can expose a mean-variance relationship that a single overall residual summary hides.

A sequential residual plot needs careful interpretation. If a residual at occasion \(j\) is conditional on earlier observed outcomes, a trend against one of those outcomes can indicate that the candidate’s conditional mean or variance is inadequate. Residuals from the same subject are not automatically independent, however. Nor should residuals from different response patterns be pooled without checking whether their conditional distributions and scales are comparable. Summaries such as the mean, standard deviation, and proportion with \(|z|>2\) are descriptive aids, not universal pass-fail cutoffs.

Justification and Assumptions

The rationale for standardized residuals follows directly from conditional expectation and variance. For a specified conditional distribution under candidate \(M_k\), suppose the conditional mean and variance at an observed occasion are \(\mu_{ij}^{(k)}\) and \((\sigma_{ij}^{(k)})^2\). If that conditional distribution describes the data-generating distribution, then \(E(Y_{ij}\mid\text{conditioning})=\mu_{ij}^{(k)}\) and \(\operatorname{Var}(Y_{ij}\mid\text{conditioning})=(\sigma_{ij}^{(k)})^2\). Consequently, for a residual standardized using the true quantities,

This follows by subtracting the conditional mean and dividing the conditional variance by \((\sigma_{ij}^{(k)})^2\). It justifies checking residual location and scale, but does not imply normality unless the conditional distribution is normal. In actual diagnostics, fitted parameters replace the true quantities, so residuals may have altered variability and dependence. A small or highly parameterized sample can also make plots appear deceptively orderly.

The practical assumptions are therefore modest but important: the candidate must specify a usable observed-data conditional distribution for the outcomes being checked; the response-pattern definition, covariates, and conditioning history must be represented correctly in the calculation; and the observed values and fitted distributions must be aligned on the same scale and occasion. These are requirements for constructing a meaningful diagnostic, not claims that the distribution is already correct. If a model fails to provide the needed conditional distribution or it is computed under the wrong pattern or covariate values, the resulting residuals do not check the intended model.

What These Checks Can and Cannot Establish

Compare candidates by asking whether they show the same systematic residual features and whether any feature is concentrated in a particular pattern, occasion, or covariate range. A shared discrepancy suggests that changing only the MNAR extrapolation assumption may not solve the observed-data fit problem. A discrepancy limited to one candidate can help explain why its observed-data implications differ. Conversely, similar residual displays mean only that these diagnostics have not exposed a difference in observed-data fit; they do not establish that the candidates have the same missing-outcome distributions.

Residual checks complement, rather than replace, other comparisons. Comparing observed-data fit can provide a broader likelihood-based assessment, while comparisons of implied missing-outcome distributions and treatment effects address different targets. Residual plots are especially useful for locating the form and context of a discrepancy, but they do not by themselves rank candidates using a calibrated overall criterion. They are also not posterior predictive checks: here the focus is on residuals computed from the observed-data conditional distributions, not simulated replicated data.

Several failure modes deserve attention. A pooled plot can hide opposite residual patterns in different response groups. A few influential observations can dominate a summary such as the largest absolute residual. Sparse patterns make residual summaries unstable, while estimated conditional variances near zero can create very large standardized residuals. Finally, all candidates can look acceptable on observed outcomes yet differ substantially in extrapolation. That last situation is a limitation of what observed-data residuals identify, not evidence that the diagnostics have failed.

Common Mistake