Missing Outcomes In GLMM And GEE Analyses

Why Missing Repeated Outcomes Matter

Repeated-outcome analyses use information from several planned measurements on the same cluster or participant. If some measurements are missing, the observed records may no longer represent the intended outcome distribution. The key issue is not simply how many values are absent: it is what the probability of being observed depends on. That distinction affects how a likelihood-based GLMM and an ordinary GEE can use the remaining outcomes.

Here \(i=1,\ldots,m\) indexes clusters and \(j=1,\ldots,n_i\) indexes planned measurements within cluster \(i\). Let \(Y_{ij}\) be the outcome, \(X_{ij}\) the observed covariates, and \(R_{ij}=1\) if \(Y_{ij}\) is observed and \(R_{ij}=0\) otherwise. Write \(Y_i\) for the full outcome vector and \(Y_{i,\mathrm{obs}}\) for the observed part. This tutorial concerns missing outcomes; it assumes the covariates used in the analysis are observed.

Definition: Outcomes are missing completely at random (MCAR) if the observation pattern \(R_i\) is independent of the full outcomes and covariates. They are missing at random (MAR) if, conditional on the observed outcomes and covariates, the observation pattern does not depend on the missing outcomes. They are missing not at random (MNAR) if the probability of the observation pattern still depends on missing outcomes after conditioning on the observed information.

MCAR is a strong special case of MAR. Under MAR, for example, a later visit could be more likely to be missed by participants whose earlier recorded score was high; that dependence is on observed history. Under MNAR, a participant's unrecorded current score could itself affect whether the score is observed, even after accounting for recorded history and covariates. These definitions concern the data and variables in the analysis: adding relevant observed predictors can change whether a proposed mechanism is regarded as MAR.

How a GLMM Uses the Observed Outcomes

A GLMM specifies a conditional outcome distribution given covariates and random effects. Let \(b_i\) denote the cluster-specific random effects, with density \(f(b_i;\theta)\), and let \(f(Y_i\mid X_i,b_i;\theta)\) be the conditional joint outcome model. The parameter vector \(\theta\) includes the outcome-model parameters. With complete outcomes, a cluster's marginal likelihood contribution integrates out its random effects. With incomplete outcomes, it also accounts for the unobserved outcomes by integrating them out.

L_i(\theta) = \int \left[ \int f(Y_i\mid X_i,b_i;\theta)\,dY_{i,\mathrm{mis}} \right] f(b_i;\theta)\,db_i

For discrete outcomes, the inner integral means a sum over possible missing values. Equivalently, the inner expression is the model's marginal distribution for the observed outcomes given \(X_i\) and \(b_i\). Missing outcomes are not filled in with guessed values; their possible values are accounted for through this marginalization. The random effects are then integrated out as in the complete-data likelihood.

This observed-data likelihood supports valid likelihood inference under standard conditions: the outcome model for the observed data is correctly specified; the missingness mechanism is MAR relative to the observed outcomes and covariates; and the parameters of the missingness process are distinct from the outcome-model parameters. The last two conditions make the missingness mechanism ignorable for likelihood inference. If missingness depends on an unobserved outcome after conditioning on what is observed, the observed-data likelihood alone generally does not identify the needed missingness dependence.

Worked Example: A GLMM Likelihood Contribution With One Missing Visit

Suppose, hypothetically, a Gaussian random-intercept GLMM has conditional model \(Y_{ij}\mid b_i\sim N(\beta+b_i,\sigma^2)\), with \(b_i\sim N(0,\tau^2)\). For one participant, the first outcome is observed as 12 units and the second outcome is missing. Let \(\beta=10\), \(\sigma^2=4\), and \(\tau^2=1\). Find the observed-data density contribution for the first outcome.

The missing second outcome is integrated out. Since it contributes no observed value, the contribution conditional on \(b_i\) is just the density of the first outcome. Integrating over the random intercept gives \(Y_{i1}\sim N(\beta,\sigma^2+\tau^2)=N(10,5)\). Thus the observed-data density at 12 is

f(12) = \frac{1}{\sqrt{2\pi(5)}} \exp\left\{-\frac{(12-10)^2}{2(5)}\right\} = \frac{1}{\sqrt{10\pi}}\exp(-0.4) \approx 0.1196

This is a density, not a probability that the outcome equals exactly 12. The calculation illustrates that the missing visit does not require discarding the observed one. In an actual analysis, \(\beta\), \(\sigma^2\), and \(\tau^2\) would be estimated from all observed-data contributions, and validity would still require the stated outcome-model and missingness assumptions.

What Ordinary GEE Requires

A standard GEE specifies a marginal mean model, such as \(g\{E(Y_{ij}\mid X_{ij})\}=X_{ij}^{T}\beta\), and uses a working covariance to construct estimating equations. With complete outcomes, the cluster contribution has the familiar form \(D_i^T V_i^{-1}(Y_i-\mu_i)\), where \(\mu_i\) is the modeled mean vector, \(D_i\) is its derivative with respect to \(\beta\), and \(V_i\) is the working covariance matrix.

When only some components are observed, an ordinary available-outcome GEE uses the corresponding observed components and their covariance submatrix. Its estimating equation is not generally protected from selection merely because it uses all recorded values. Under MCAR, the observation pattern is independent of the outcomes and covariates, so the observed estimating contributions have mean zero at the true mean-model parameter. Under general MAR, the observed components may be selected according to earlier outcomes; the ordinary equation can then have nonzero expectation and converge to a value other than the target.

The robust sandwich variance addresses uncertainty from estimating the GEE under a working covariance; it does not, by itself, correct bias caused by outcome-dependent observation. A response-probability-weighted GEE can sometimes address MAR by weighting observed contributions using a correctly specified model for observation probabilities. That approach requires its own conditions, including positive observation probabilities for relevant histories. It is distinct from simply using the ordinary GEE on whatever outcomes happen to be present.

Worked Example: MAR Selection Changes the Observed Follow-Up Mean

Consider a hypothetical binary two-visit outcome. Half the participants have \(Y_{i0}=1\) and half have \(Y_{i0}=0\). At follow-up, \(P(Y_{i1}=1\mid Y_{i0}=1)=0.8\) and \(P(Y_{i1}=1\mid Y_{i0}=0)=0.2\). Follow-up observation depends on the recorded baseline outcome: the observation probability is 0.9 when \(Y_{i0}=1\) and 0.1 when \(Y_{i0}=0\). Calculate the full-population follow-up mean and the mean among those observed.

The full-population probability of a follow-up success is

P(Y_{i1}=1) = 0.5(0.8)+0.5(0.2) = 0.50.

Observation depends on the observed baseline value, not on the missing follow-up value after conditioning on baseline. This is MAR in this example. The probability of both being observed and having a follow-up success is

P(Y_{i1}=1,R_{i1}=1) = 0.5(0.8)(0.9)+0.5(0.2)(0.1) = 0.37.

The probability of being observed is \(0.5(0.9)+0.5(0.1)=0.50\). Therefore the observed follow-up success proportion is

P(Y_{i1}=1\mid R_{i1}=1) = \frac{0.37}{0.50} = 0.74.

The observed proportion, 0.74, differs from the target full-population mean, 0.50. An ordinary available-outcome analysis of follow-up alone would therefore not recover that population mean in this example. A correctly specified likelihood-based analysis could use the joint outcome model and observed baseline history under MAR; alternatively, an appropriate response-probability weighting approach could be considered. Neither method is valid merely because it is more elaborate: its model and assumptions must be defensible.

Worked Example: Why MCAR Can Protect an Available-Outcome Mean

Suppose a hypothetical study has a binary follow-up outcome with population mean \(p=0.40\), and each participant's follow-up observation indicator is independent of that outcome, with the same observation probability for everyone. In one realized sample, the four observed outcomes are \(1,0,1,0\). Calculate the available-outcome estimate and explain what MCAR guarantees.

The observed proportion is

\hat{p}_{\mathrm{obs}} = \frac{1+0+1+0}{4} = 0.50.

This realized estimate is 0.10 above the population mean. MCAR does not force any particular sample proportion to equal \(p\). Rather, because observation is independent of the outcome, the observed outcomes have the same success probability as the full population. Across repeated samples with observed outcomes, the available-outcome proportion is unbiased for \(p\) under this simple setup. Thus a difference between 0.50 and 0.40 in one sample is ordinary sampling variation, not evidence that MCAR guarantees exact agreement.

Justification for Ignoring an MAR Missingness Process

The likelihood argument clarifies why MAR and parameter distinctness matter. Let \(g(R_i\mid Y_i,X_i;\psi)\) describe the observation process, with parameter \(\psi\). The joint model for outcomes and observation pattern is the product of the outcome model and this observation model. Under MAR, the latter depends on the observed outcomes and covariates, but not on the missing outcome values. If \(\theta\) and \(\psi\) are distinct, the observed-data likelihood factors into an outcome part and a missingness part.

\begin{aligned} L_i(\theta,\psi) &= \int f(Y_i\mid X_i;\theta)\, g(R_i\mid Y_i,X_i;\psi)\,dY_{i,\mathrm{mis}}\\ &= g(R_i\mid Y_{i,\mathrm{obs}},X_i;\psi) \int f(Y_i\mid X_i;\theta)\,dY_{i,\mathrm{mis}}. \end{aligned}

The first factor involves only \(\psi\), so it does not change the likelihood-based inference for \(\theta\). For a GLMM, the outcome part also integrates over \(b_i\), as in the earlier likelihood expression. This factorization is the justification for omitting the missingness model under ignorability; it is not a claim that missingness has no effect on the data observed or on the precision of estimates.

For ordinary GEE, the analogous protection under MCAR follows from the mean-zero estimating contribution. At the true parameter, the complete-data residual vector has conditional mean zero given covariates. When the observation pattern is independent of the outcomes, selecting an observed subset does not select residuals by their values; the corresponding observed-component contributions remain centered at zero. Under MAR, that independence need not hold, as the preceding numerical example demonstrates.

Diagnostics, Alternatives, and Limits

The missingness mechanism cannot generally be proven from the observed data alone, because the values that would distinguish MAR from MNAR are precisely the missing outcomes. Diagnostics are therefore checks for model and data problems, not definitive tests of MAR. Useful steps include:

Tabulate missingness by visit, treatment group, and fully observed baseline covariates; inspect whether missingness patterns are concentrated in particular groups.
Compare observed histories for participants with and without later outcomes. Associations with recorded history make MCAR less plausible, though they do not by themselves establish MNAR.
Check whether the fitted GLMM adequately represents the observed outcome distribution and within-cluster dependence; an ignorable missingness mechanism cannot rescue a misspecified outcome model.
Check whether observation probabilities are effectively zero for some covariate histories before relying on a weighting alternative; such histories provide little or no observed information.

The main contrast is therefore between assumptions and target equations, not between a universally safe method and an unsafe one. A correctly specified likelihood-based GLMM can use incomplete outcome vectors under ignorable MAR, while ordinary GEE is generally protected under MCAR but not under unrestricted MAR. Weighted GEE may be appropriate under MAR when its observation model and positivity conditions are credible. Complete-case analysis is another option, but it can waste observed outcomes and can be biased unless its selection conditions justify it.

No standard likelihood or ordinary GEE automatically resolves MNAR. Addressing MNAR requires additional assumptions about how missing outcomes relate to observation, often through an explicit joint model or a sensitivity analysis. Those assumptions cannot be verified from observed outcomes alone. Conclusions should therefore state the missingness assumption and, when plausible alternatives could materially change the result, examine sensitivity to them.

Common pitfall: “The GLMM uses maximum likelihood, so missing outcomes cannot bias it.” This is incorrect. Likelihood inference with incomplete outcomes relies on an appropriate outcome model and an ignorable missingness mechanism, commonly MAR with distinct missingness and outcome parameters. If the observation probability depends on unobserved outcomes, fitting the ordinary observed-data GLMM does not automatically correct the selection.

Check Your Understanding

Answer each question, then compare with the brief solutions below.

A later outcome is more likely to be missing when the participant's recorded baseline outcome is high. Is this necessarily MNAR?
In one sentence, explain what happens to a missing outcome in the observed-data GLMM likelihood.
A follow-up outcome has population success probability 0.30, but under MAR the observed follow-up success proportion is 0.45. Does the difference alone disprove MAR?
State one key condition under which the ordinary available-outcome GEE is generally valid, and name one assumption needed for likelihood-based GLMM inference under incomplete outcomes.

Solutions. (1) No. If missingness depends on the recorded baseline outcome and not on the missing follow-up outcome after conditioning on observed information, the mechanism is MAR. (2) The likelihood sums or integrates over its possible values, retaining the contribution of the observed outcomes. (3) No. A MAR mechanism can make the observed outcomes unrepresentative, and sampling variation can also contribute; the difference alone does not identify the mechanism. (4) MCAR generally protects the ordinary available-outcome GEE; likelihood-based GLMM inference requires, among other conditions, an appropriate outcome model and an ignorable MAR mechanism with distinct parameters.