Worked Comparison of Complete-Case, Weighted, and Imputed Estimates

One Cohort, Three Ways to Handle Missing Modalities

When two omics modalities are measured on different subsets of a cohort, an integrated quantity may be observed only for subjects with both modalities available. An estimate based only on those subjects is easy to calculate, but it may describe a selected subset rather than the cohort of interest. Inverse-probability weighting and multiple imputation also use assumptions about the missingness process; they do not automatically repair selection or make the missing values known.

This tutorial compares the three approaches for one clearly hypothetical cohort. For each subject, let \(A\) and \(B\) denote modality-specific scores measured on the same scale, and define the integrated score \(Y=(A+B)/2\). The target is the mean integrated score that would be obtained if both modalities were available for every subject in the cohort. The numerical examples assume independent subjects and use the usual sample-variance estimate for the mean; they are intended to show the calculations, not to support substantive conclusions.

Definition: Let \(R=1\) when both modalities are observed and \(R=0\) otherwise. A complete case has \(R=1\), so \(Y\) is directly observed. A partially observed subject has at least one modality missing, so the integrated score is not directly observed. The full-cohort target is \(\mu_Y=E(Y)\), including subjects with \(R=0\).

The methods answer the same target question only when their respective assumptions are adequate. In particular, their estimates need not agree in a finite sample. Their standard errors also represent different uncertainty calculations.

Hypothetical Cohort and Complete-Case Estimate

The cohort has ten subjects, five in each of two observed groups, L and H. The group is available for everyone and is used to model modality availability. One subject in L and three subjects in H have both modalities observed. The observed integrated scores are 2 in L and 6, 8, and 10 in H. The other subjects have one modality observed, so their integrated scores require imputation if they are to enter a completed-data analysis.

The complete-case mean is the arithmetic average among subjects with both modalities measured. It estimates the full-cohort mean without adjustment when completion is independent of \(Y\), as in a missing-completely-at-random setting. More generally, selection may depend on observed variables, but the unadjusted complete-case mean is not automatically representative of the full cohort.

Worked Example: Complete-Case Estimate

There are four complete cases, with integrated scores \(2,6,8,10\). The complete-case estimate is

\bar{Y}_{CC}=\frac{2+6+8+10}{4}=\frac{26}{4}=6.5.

The sample variance among the complete cases is

s^2_{CC}=\frac{(2-6.5)^2+(6-6.5)^2+(8-6.5)^2+(10-6.5)^2}{4-1} =\frac{35}{3}=11.67.

If these four subjects were treated as an independent representative sample, the usual standard error would be \(SE=s_{CC}/\sqrt{4}=\sqrt{11.67/4}=1.708\). That calculation does not account for selection into the complete cases. Here, the observed complete-case group composition is one L and three H, even though the full cohort is evenly split between L and H. Thus the estimate 6.5 should not be interpreted as an assured estimate of the full-cohort mean.

Inverse-Probability Weighting

Inverse-probability weighting (IPW) gives more influence to complete cases who represent subjects less likely to have both modalities observed. Let \(\pi(X)=P(R=1\mid X)\) be the probability of observing both modalities given fully observed predictors \(X\). For a cohort with estimated availability probabilities \(\hat{\pi}_i\), a common estimator is the Hájek weighted mean:

\hat{\mu}_{IPW} = \frac{\sum_{i=1}^{n} R_iY_i/\hat{\pi}_i} {\sum_{i=1}^{n} R_i/\hat{\pi}_i}.

The denominator normalizes the weights, so the estimate is a weighted average of observed outcomes. The key assumptions are conditional exchangeability, \(Y\mathrel{\perp}R\mid X\), and positivity, \(P(R=1\mid X=x)>0\) for covariate patterns represented in the target cohort. The availability model must also be correctly specified, or the probabilities must otherwise be consistently estimated. These conditions make the weighted complete cases representative of the full cohort with respect to \(Y\), after conditioning on \(X\).

Justification: Under conditional exchangeability, \(E(Y\mid R=1,X)=E(Y\mid X)\). If \(\pi(X)=P(R=1\mid X)>0\), then \(E\{RY/\pi(X)\mid X,Y\}=Y\), because \(E(R\mid X,Y)=\pi(X)\). Thus weighting corrects the selection in expectation. The ratio estimator uses the corresponding weighted outcome total divided by the weighted representation of the cohort; in a finite sample it is not generally exactly unbiased.

Worked Example: IPW by Availability Group

In the hypothetical cohort, the observed availability fractions are \(\hat{\pi}(L)=1/5=0.2\) and \(\hat{\pi}(H)=3/5=0.6\). For this calculation, treat these as the fitted probabilities from a model using group. The L complete case receives weight \(1/0.2=5\); each H complete case receives weight \(1/0.6=1.667\). Substituting into the Hájek estimator gives

\hat{\mu}_{IPW} = \frac{5(2)+(1.667)(6+8+10)} {5+3(1.667)} \approx\frac{50}{10}=5.0.

The weighted denominator assigns total weight 5 to L and total weight 5 to H, matching the two groups’ equal representation in the full cohort. This estimate is lower than the unweighted complete-case estimate of 6.5 because the complete cases over-represent H, whose observed scores are higher. That interpretation depends on the model: group must adequately account for differences in availability related to the outcome, and every relevant group must have a positive chance of having both modalities observed.

A rough uncertainty illustration uses the Kish effective sample size, \(n_{\mathrm{eff}}=(\sum w_i)^2/\sum w_i^2\). Here it is \(10^2/[5^2+3(1.667^2)]\approx3.0\). The weighted mean’s normalized weighted squared deviation is approximately \([5(2-5)^2+1.667\{(6-5)^2+(8-5)^2+(10-5)^2\}]/10=10.33\). Dividing by \(n_{\mathrm{eff}}\) gives an approximate variance of \(3.444\), hence an approximate \(SE=\sqrt{3.444}=1.856\). This is a diagnostic approximation, not a universal IPW variance formula; formal uncertainty calculations should account for how the weights were estimated and use an appropriate robust variance estimator or bootstrap.

Multiple Imputation and Pooled Uncertainty

Multiple imputation (MI) replaces each missing modality block with draws from an imputation model, producing \(M\) completed data sets. The integrated analysis is then performed in each data set. For an estimate \(Q_m\) with estimated variance \(U_m\) in imputation \(m\), the pooled estimate and variance are

\bar{Q}=\frac{1}{M}\sum_{m=1}^{M}Q_m,\qquad \bar{U}=\frac{1}{M}\sum_{m=1}^{M}U_m,\qquad B=\frac{1}{M-1}\sum_{m=1}^{M}(Q_m-\bar{Q})^2,

T=\bar{U}+\left(1+\frac{1}{M}\right)B,\qquad SE(\bar{Q})=\sqrt{T}.

Here \(\bar{U}\) represents average within-imputation uncertainty, while \(B\) represents between-imputation variation. The additional factor \(1+1/M\) accounts for estimating the missing-data uncertainty from a finite number of imputations. MI relies on an appropriate imputation model and, for the usual analysis, a missing-at-random assumption conditional on variables included in that model. The model should preserve the cross-modality relationships relevant to the integrated score. The earlier material on pooling integrated estimates across imputations gives the general pooling framework; the example here applies it to a cohort mean.

Worked Example: Imputed Integrated Scores and a Pooled Interval

The table shows two hypothetical completed data sets. “A only” means modality A was observed and B was imputed; “B only” means B was observed and A was imputed. The displayed integrated scores are the resulting values of \(Y=(A+B)/2\). The four originally complete scores stay fixed across imputations.

Group	Availability pattern	Observed complete-case \(Y\)	Imputation 1 \(Y\)	Imputation 2 \(Y\)
L	Both	2	2	2
L	A only	—	3	4
L	A only	—	4	3
L	B only	—	5	6
L	B only	—	6	5
H	Both	6	6	6
H	Both	8	8	8
H	Both	10	10	10
H	A only	—	7	8
H	B only	—	9	10

The first completed-data mean is \((2+3+4+5+6+6+8+10+7+9)/10=6.0\). The second is \((2+4+3+6+5+6+8+10+8+10)/10=6.2\). Therefore \(\bar{Q}=(6.0+6.2)/2=6.1\).

For a mean based on ten independent completed observations, use \(U_m=s_m^2/10\). In imputation 1 the sum of squared deviations from 6.0 is 60, so \(s_1^2=60/9=6.667\) and \(U_1=0.6667\). In imputation 2 the sum of squared deviations from 6.2 is 69.6, so \(s_2^2=69.6/9=7.733\) and \(U_2=0.7733\). Hence \(\bar{U}=(0.6667+0.7733)/2=0.7200\), and \(B=[(6.0-6.1)^2+(6.2-6.1)^2]/(2-1)=0.0200\).

The pooled variance and standard error are

T=0.7200+\left(1+\frac{1}{2}\right)(0.0200)=0.7500,\qquad SE=\sqrt{0.7500}=0.8660.

Using the usual Rubin degrees-of-freedom approximation, \(df=(M-1)\{1+\bar{U}/[(1+1/M)B]\}^2=625\). The approximate 95% interval is \(6.1\pm1.964(0.8660)\), or \((4.399,7.801)\). Under the imputation and sampling assumptions, this procedure gives an interval for the full-cohort mean; it does not mean there is a 95% probability that the fixed mean lies in this particular interval. Two imputations are far too few for routine analysis, so this small \(M\) is only for transparent arithmetic.

Comparing the Estimates and Checking the Assumptions

The three point estimates in this example are 6.5 for complete cases, 5.0 for IPW, and 6.1 for MI. The methods do not produce a ranking of truth: the estimates differ because their assumptions and use of observed information differ. Complete-case analysis discards all partially observed subjects. IPW adjusts the observed complete cases using an availability model. MI uses imputed modality values for incomplete subjects and carries their variation into the pooled uncertainty.

Method	Uses incomplete subjects?	Central assumption for the full-cohort mean	Important diagnostic
Complete case	No	Completion does not select on the outcome in a way that changes the mean, or the analysis otherwise accounts for selection	Compare availability patterns and covariates with the complete cases
IPW	Indirectly, through weights	Conditional exchangeability, positivity, and an adequate availability model	Inspect fitted probabilities, weight distribution, and effective sample size
Multiple imputation	Yes, through completed data sets	Conditional missing at random and adequate, compatible imputation models	Check imputed distributions and cross-modality relationships against observed data

The introductory material on inverse-probability weighting for modality availability covers the method’s general setup; in practice, diagnostics are essential. For IPW, very small estimated probabilities produce very large weights. Such weights can make the estimate unstable and reduce the effective sample size; truncation may reduce variance but changes the estimator and can introduce bias. A stratum with no complete cases violates positivity, and weighting cannot recover information from it. For MI, compare imputed and observed values in the relevant groups and inspect whether important dependence between modalities is retained. Plausible-looking marginal distributions alone do not establish that the joint imputation model is adequate.

Both methods can fail when missingness depends on an unobserved modality value even after conditioning on measured variables. Such a missing-not-at-random mechanism is not resolved merely by fitting a more elaborate availability or imputation model to the observed data. It requires additional information or explicit sensitivity assumptions. Also, if the two modalities are measured on incomparable scales, defining their average as \(Y\) may not be scientifically meaningful; the target and scale must be specified before comparing missing-data methods.

Common Mistake

Pitfall: “The IPW estimate is corrected because the weights are inverse probabilities, so it must be unbiased.” The correction works only when the relevant predictors of availability and outcome are included, the availability model is adequate, and positivity holds. If an important selection factor is omitted or some subjects have essentially zero probability of complete observation, inverse weights do not guarantee a representative estimate.

Another error is to pool only the point estimates from imputed data sets and report the average of their standard errors. That omits between-imputation uncertainty. Use \(T=\bar{U}+(1+1/M)B\), and report the pooled estimate, standard error or interval, and the assumptions that support its interpretation.

Check Your Understanding

For each question, work out the answer before reading the short solution.

In the hypothetical cohort, why is the unweighted complete-case estimate 6.5 not automatically an estimate of the full-cohort mean? Solution: Complete cases are one L and three H, while the full cohort has five in each group. If group relates to \(Y\), this selection changes the represented group mix; the unweighted mean does not correct for it.
If a group has estimated complete-modality probability 0.25, what IPW weight does a complete case in that group receive? Solution: \(1/0.25=4\). It represents four cohort subjects in the weighted calculation, subject to the IPW assumptions.
Why does positivity matter for IPW? Solution: The weight \(1/\pi(X)\) is defined only when \(\pi(X)>0\). If a covariate group has no chance of being complete, observed complete cases cannot represent that group through weighting.
In the MI example, what are \(\bar{U}\) and \(B\), and which represents disagreement between imputations? Solution: \(\bar{U}=0.7200\) is the average within-imputation variance; \(B=0.0200\) is the between-imputation variance and represents disagreement between the two estimates.
Would agreement between complete-case, IPW, and MI estimates prove that missingness is ignorable? Solution: No. Similar estimates can occur despite incorrect assumptions, and the observed data alone generally cannot rule out missing-not-at-random mechanisms. Agreement is a useful comparison, not proof of validity.