Aligning Candidate Models on Shared Observed-Data Components

Why Alignment Matters

Two MNAR models can yield different treatment-effect estimates because they make different assumptions about unobserved outcomes. But a difference is informative about those assumptions only when other relevant parts of the comparison are held steady. If one candidate also uses a different analysis sample, covariate set, or outcome covariance structure, the resulting contrast mixes several changes together.

The goal here is not to make every candidate model identical. It is to specify the components that should be shared for the scientific comparison at hand, then vary the MNAR assumptions of interest. This requires care: a shared model for the full outcome vector does not automatically imply identical distributions among observed outcomes, particularly in selection models where observation depends on outcomes.

Definition: Candidate MNAR models are aligned on shared observed-data components when they use the same eligible analysis units, outcome definitions and measurement times, covariates and their coding, and the same stated model for relevant observed-outcome mean and covariance features. Their intended difference is then specified in the assumptions governing outcomes or missingness that are not identified from observed data alone. Alignment is a design choice for a comparison; it does not itself establish that the models fit well or that their shared assumptions are correct.

Specify What Is Shared Before Fitting

For subject \(i\), write the outcome vector as \(Y_i\), and let \(R_i\) indicate which components are observed. Let \(X_i\) denote covariates in the outcome model and \(Z_i\) covariates in the missingness model. For a particular realized pattern \(r_i\), let \(O_i\) index the observed components and \(M_i\) the missing components. The observed record contains \(Y_{i,O_i}\), not \(Y_{i,M_i}\).

A selection model factors the full-data distribution into an outcome model and a missingness model. For candidate \(k\), one possible specification is \(f(Y_i\mid X_i;\theta)\,P_k(R_i=r_i\mid Y_i,Z_i;\psi_k)\). The observed-data contribution is obtained by integrating over missing outcomes:

L_{ik} = \int f(Y_{i,O_i},Y_{i,M_i}\mid X_i;\theta) P_k(R_i=r_i\mid Y_{i,O_i},Y_{i,M_i},Z_i;\psi_k) \,dY_{i,M_i}.

Holding the outcome model \(f(Y_i\mid X_i;\theta)\) fixed is a meaningful way to align selection-model candidates, but it does not generally make their observed-data likelihood contributions equal. Changing the selection model changes the integrand, including its dependence on unobserved outcomes. Thus the likelihood may change even though the full-data outcome mean and covariance specifications are shared.

A pattern-mixture factorization instead describes the pattern and outcomes conditional on that pattern, for example \(P(R_i=r_i\mid Z_i)\,f(Y_{i,O_i},Y_{i,M_i}\mid R_i=r_i,Z_i)\). It can be useful to preserve the distribution of the pattern and the conditional distribution of observed outcomes within that pattern, while varying the conditional distribution assigned to missing outcomes. Such specifications must define a coherent joint distribution; independently chosen pieces are not automatically compatible.

Important distinction: A shared full-data mean and covariance model is not the same statement as a shared observed-data mean and covariance. When observation depends on outcomes, the distribution among observed cases can be selected. State explicitly which model-implied quantities are being matched and for which observed patterns.

A Short Justification: What Can Stay Fixed in a Pattern-Mixture Comparison?

Suppose a pattern-mixture model holds \(P(R=r\mid X=x)\) and \(f(Y_O=y_O\mid R=r,X=x)\) fixed, while changing the conditional distribution of the missing part. The observed-data density for that pattern is found by integrating out \(Y_M\):

\begin{aligned} f(y_O,r\mid x) &=P(R=r\mid x) \int f(y_O,y_M\mid R=r,x)\,d y_M\\ &=P(R=r\mid x) f(y_O\mid R=r,x) \int f(y_M\mid y_O,R=r,x)\,d y_M\\ &=P(R=r\mid x) f(y_O\mid R=r,x). \end{aligned}

The final equality follows because a proper conditional density integrates to one. Therefore, changing \(f(Y_M\mid Y_O,R,X)\) leaves this observed-data density unchanged, provided the pattern probability and observed-outcome conditional distribution really are held fixed and the new conditional density is proper. This justifies the sense in which some pattern-mixture candidates can share observed components while differing in extrapolation. It does not show that every proposed combination of conditional distributions is scientifically plausible.

Worked Example: Keeping the Observed Component Fixed

Suppose, hypothetically, that 60% of subjects have a week-12 outcome observed and 40% do not. Among the observed-pattern group, suppose the week-12 outcomes have a normal distribution with mean 10 points and variance 4 points squared. Consider two pattern-mixture candidates that keep those quantities and the pattern proportions fixed, but assign the missing-pattern outcomes normal distributions with variance 4 and means of 7 and 11 points, respectively.

The observed-data distribution for a subject with a week-12 outcome recorded is the same under both candidates: it is the same pattern probability, 0.60, multiplied by the same conditional density for the observed score. Outcomes in the missing pattern are not observed, so their different means do not change that observed-data component.

The implied overall means differ. Under the first candidate, \(0.60(10)+0.40(7)=6+2.8=8.8\) points. Under the second, \(0.60(10)+0.40(11)=6+4.4=10.4\) points. The 1.6-point difference is due to the missing-pattern assumption in this illustration, not a change to the specified observed component. These are model-implied population means, not sample estimates or evidence that either missing-pattern mean is correct.

The assumptions used are explicit: the two pattern proportions are fixed, the observed-pattern distribution is fixed, and each missing-pattern distribution is a proper normal distribution. Those conditions make the calculation coherent. In an application, the missing-pattern means are not learned from those observed week-12 outcomes alone.

Align the Mean, Covariance, Covariates, and Sample

For repeated outcomes, an outcome model might specify a mean vector \(E(Y_i\mid X_i)=X_i\beta\) and covariance matrix \(\operatorname{Var}(Y_i\mid X_i)=\Sigma\). A comparison should identify whether both the mean and covariance specifications are to be shared. Matching marginal means while allowing different covariance structures can still yield different likelihoods, uncertainty estimates, and predictions.

The covariate specification includes more than a list of variable names. It includes the coding of categories, transformations, interactions, time effects, and the way covariates enter the mean and missingness models. The analysis sample likewise means the same eligibility criteria and the same included subjects or records, not merely the same nominal sample size. For incomplete outcomes, the relevant data are the observed records and missingness patterns; restricting analysis to complete cases changes the analysis and can discard information.

Worked Example: Matching the Analysis Sample

In a hypothetical illustration, six eligible subjects have week-12 scores as follows: subjects 1–4 have observed scores 10, 12, 14, and 16; subject 5 has an observed score of 18; subject 6 has a missing week-12 score. Suppose candidate A includes subjects 1–6, while candidate B mistakenly excludes subject 5 because of a different covariate-availability rule. To show the effect on a simple observed-score summary, calculate the mean among those with a recorded week-12 score.

Candidate A has five observed scores, so its observed-score mean is \((10+12+14+16+18)/5=70/5=14\) points. Candidate B has four, giving \((10+12+14+16)/4=52/4=13\) points. The one-point difference cannot be attributed to the MNAR assumptions: the models were applied to different analysis samples.

If subject 5 is eligible and the intended comparison includes that subject, both candidates must use the same six eligible subjects and account for the same observed and missing records. The shared sample then contains five observed week-12 scores and one missing week-12 score. These calculations are a sample-alignment check, not a proposed MNAR estimator. They also show why “same \(n\)” is insufficient: two samples of equal size can contain different subjects or missingness patterns.

Worked Example: Matching a Covariance Structure

Suppose a hypothetical two-time-point outcome model has a shared mean structure and the following covariance matrix, with scores measured in points:

\Sigma= \begin{pmatrix} 16 & 8\\ 8 & 25 \end{pmatrix}.

The baseline variance is 16 points squared, the follow-up variance is 25 points squared, and the covariance is 8 points squared. The implied correlation is \(8/(\sqrt{16}\sqrt{25})=8/(4\cdot5)=0.400\). A candidate using the same variances but covariance 4 would imply correlation \(4/(4\cdot5)=0.200\), so it would not share this covariance structure. A comparison of that candidate with one using covariance 8 would change both the missingness assumptions and the outcome dependence over time.

To isolate a missingness-assumption contrast, impose the same covariance structure—here, the displayed matrix—on both candidates, along with the same mean model, covariates, and analysis sample. Then specify the intended MNAR difference. This check assumes the covariance matrices are valid: the displayed matrix has positive diagonal entries and determinant \(16(25)-8^2=400-64=336>0\), so it is positive definite. This is a model-structure check; it does not establish that the matrix fits any actual data.

Practical Alignment and Diagnostics

Before interpreting a contrast, write down a compact alignment specification. It should identify the target population and eligibility rules, outcome scale and measurement occasions, covariates and coding, mean structure, covariance structure, and treatment of incomplete records. Then record which MNAR component is intentionally changed. This makes it possible to tell whether a disagreement answers the intended question.

Fix the data definition.
Use the same units, outcome timing, eligibility rules, and subjects or records. Tabulate missingness patterns under the common rules.

Fix the observed-outcome model.
State which mean and covariance features are shared, and distinguish model-implied observed-pattern quantities from full-data quantities.

Fix covariates and coding.
Check the included variables, reference categories, transformations, interactions, and how each covariate enters each relevant model.

State the intended contrast.
Describe precisely which MNAR assumption changes. Do not describe candidates as differing “only in MNAR” if their sample or outcome model also differs.

Check fit and computation.
Inspect convergence, parameter estimates, boundary behavior, and observed-data fit. Alignment does not excuse poor fit or unstable estimation.

Warning signs include unexpectedly different sample counts, a changed design matrix, different time-specific variances, incompatible outcome scales, or a covariance matrix that is not valid. Numerical warnings and large changes in estimates after seemingly minor adjustments also merit investigation. In selection models, compare the actual observed-data likelihood or observed-data summaries rather than inferring equality from a shared full-data outcome model. As emphasized in the earlier material on observed-data fit and fit agreement versus extrapolation agreement, agreement on observed records does not establish agreement about missing outcomes.

Alternatives, Limits, and a Common Mistake

One alternative is to compare each candidate using its own best-fitting outcome model and sample. That can be appropriate when the purpose is to compare complete modeling strategies, but the resulting contrast is not isolated to the MNAR assumption. Another is to hold selected model components fixed and vary the missingness or missing-outcome specification. That makes the contrast easier to interpret, but can make the comparison sensitive to the chosen shared structure. Neither strategy removes the need to assess observed-data fit and substantive plausibility.

A shared mean and covariance structure is not proof that candidates fit equally well. In selection models, the missingness component can affect the observed-data likelihood through integration over missing outcomes. Even in a pattern-mixture comparison that preserves a specified observed component, model fit may be inadequate, covariates may be misspecified, or the extrapolation assumptions may be implausible. Alignment organizes the comparison; it does not identify the missing outcomes or validate an MNAR assumption.

Wrong reasoning: “Both selection models use the same normal outcome mean and covariance, so they must have the same observed-data fit.”
Correction: Their selection mechanisms enter the observed-data integral. Different mechanisms can imply different observed-data likelihoods even with a shared full-data outcome model.
Wrong reasoning: “The candidates have the same sample size, so the analysis samples are aligned.”
Correction: Check the actual subjects, eligibility rules, observed records, and missingness patterns, not only the count.

Key takeaway: A fair MNAR comparison begins by stating what is held constant and what is varied. Align the data, covariates, and intended observed-outcome structure; then interpret differences with awareness that selection mechanisms may also change observed-data fit.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions.

Two selection models share \(f(Y\mid X)\) but use different \(P(R\mid Y,Z)\). Must their observed-data likelihoods be equal? Why?
A two-time-point outcome has standard deviations 3 and 4 points. What covariance corresponds to correlation 0.50?
In a pattern-mixture comparison, the probability of a missing pattern is 0.30 and the observed-pattern conditional mean is 20 units. Candidate A assigns missing-pattern mean 16; candidate B assigns 22. Find each implied overall mean, assuming the pattern probability and observed-pattern mean are fixed.
What should be checked if two candidates both report \(n=100\) but give different treatment-effect estimates?

Solutions. (1) No. The selection probability is inside the observed-data integral, so changing it can change the integrated contribution even when the full-data outcome model is shared. (2) The covariance is \(0.50(3)(4)=6\) points squared. (3) The observed-pattern probability is \(1-0.30=0.70\). Candidate A gives \(0.70(20)+0.30(16)=14+4.8=18.8\) units; candidate B gives \(0.70(20)+0.30(22)=14+6.6=20.6\) units. (4) Check that the same subjects and eligibility rules are used, along with the same outcome definitions, covariates and coding, mean structure, covariance structure, and handling of missing records. Equal sample counts alone do not establish alignment.