From Missing Outcomes to a Marginal Contrast
A clustered study may have a clear scientific question—how much does treatment change the average probability of an outcome?—even when some outcomes are unobserved. The challenge is to estimate the same population contrast despite that missingness, and to account for the fact that outcomes from the same cluster may be dependent. This tutorial works through those tasks for a binary outcome, including a small-sample robust test.
Let \(Y_{ij}\) be the binary outcome for individual \(j\) in cluster \(i\), where \(Y_{ij}=1\) indicates the event. Let \(A_{ij}\) indicate treatment, and let \(R_{ij}=1\) when the outcome is observed and \(R_{ij}=0\) otherwise. A marginal risk difference compares population-average risks under treatment and control. It is a difference on the probability scale, not a conditional odds ratio.
A treatment coefficient is not automatically the desired marginal contrast. With an identity-link model, the saturated mean specification \(\mu_{ij}=\beta_0+\beta_1 A_{ij}\) has \(\beta_0=p_0\) and \(\beta_1=\Delta\), provided the model is appropriate for the target population. For binary outcomes, fitted means must be probabilities; the identity-link specification is most straightforward when its fitted values remain between zero and one.
Worked Example: Standardizing Stratum-Specific Risks
Suppose the target population is 75% in stratum L and 25% in stratum H. The treatment and control risks within those strata are \(0.30,0.80\) and \(0.20,0.60\), respectively. Calculate the marginal risks and risk difference for this target population.
The population weights are \(0.75\) for L and \(0.25\) for H. The treatment risk is \(p_1=0.75(0.30)+0.25(0.80)=0.225+0.200=0.425\).
The control risk is \(p_0=0.75(0.20)+0.25(0.60)=0.150+0.150=0.300\). Thus \(\Delta=0.425-0.300=0.125\), a 12.5 percentage-point marginal increase. The calculation averages risks using the target population’s stratum proportions; it does not average stratum-specific odds ratios.
What Complete-Case and Weighted GEE Require
A complete-case analysis uses only observed outcomes. For a marginal mean parameter, this approach retains the target without selection adjustment when observation is independent of the outcome and relevant covariates under the analysis design—most simply, under missing completely at random (MCAR). If response depends on a covariate associated with the outcome, the observed cases can represent a different covariate mix from the target population, even when missingness does not depend on the unobserved outcome after conditioning on that covariate.
Inverse-probability weighting can address a broader condition. Write \(Z_{ij}\) for observed predictors of response and outcome, and let \(\pi_{ij}=P(R_{ij}=1\mid Z_{ij},A_{ij})\). Under missing at random (MAR), the response probability can depend on these observed variables but not further on the unobserved outcome, conditional on them. If the probabilities are correctly specified and positive, a responding individual receives weight \(1/\pi_{ij}\). Positivity means \(\pi_{ij}>0\) throughout the target population and treatment groups of interest.
This identity explains why weighted estimating equations can recover a population marginal mean: the response indicator, divided by its conditional response probability, restores the contribution that would have been present without missingness. It does not make a misspecified response model correct, remove the need for positivity, or justify MAR when response depends on the unobserved outcome after conditioning on \(Z_{ij}\) and \(A_{ij}\).
Worked Example: Complete-Case and Weighted Risks
A target population has six individuals in stratum A and four in stratum B. In A, three outcomes are observed, two of which are events; the response probability is \(0.5\). All four outcomes in B are observed, and three are events; the response probability is \(1\). Calculate the complete-case and inverse-probability-weighted risks.
There are five observed events among seven observed outcomes, so the complete-case risk is \(5/7\approx0.714\).
The weighted event total is \(2(2)+1(3)=7\): each observed A event receives weight \(1/0.5=2\), and B events receive weight 1. The total weight is \(3(2)+4(1)=10\). The weighted risk is \(7/10=0.700\). These are not expected to be identical in a finite sample. The weighted estimate targets the full population under the stated MAR model and known probabilities; the complete-case estimate can reflect the observed stratum mix instead.
A Six-Cluster Study: Estimates and Robust Inference
Consider six independent clusters, each containing one control individual and one treated individual. Outcomes within a cluster may be correlated. All control outcomes are observed. For treated individuals, response depends on an observed stratum: the response probability is \(0.5\) in stratum A and \(1\) in stratum B. Assume these probabilities are known, MAR holds conditional on stratum and treatment, and positivity holds. The table gives the observed data; “—” means the treated outcome was not observed.
| Cluster | Stratum | Control outcome | Treated response probability | Treated response \(R\) | Treated outcome if observed |
|---|---|---|---|---|---|
| 1 | A | 0 | 0.5 | 1 | 1 |
| 2 | B | 1 | 1 | 1 | 1 |
| 3 | A | 0 | 0.5 | 1 | 0 |
| 4 | B | 1 | 1 | 1 | 1 |
| 5 | A | 0 | 0.5 | 0 | — |
| 6 | B | 1 | 1 | 1 | 0 |
For this calculation use an identity-link marginal mean with separate means \(\gamma_0\) and \(\gamma_1\) for control and treatment. The contrast is \(\Delta=\gamma_1-\gamma_0\). With working independence and constant working variance, the estimating equations reduce to weighted group means. This special case makes the cluster-robust calculation transparent; the sandwich method extends to more general GEE estimating equations.
For each group, set the sum of observed, response-weighted residuals to zero. The control weights are 1; the treated weights are \(1/\pi_i\).
Use the resulting group means to form the treatment-minus-control risk difference.
Form one score contribution per independent cluster so the sandwich meat allows arbitrary within-cluster dependence.
Use the finite-sample covariance multiplier and cluster-based \(t\) reference distribution specified below.
Worked Example: Complete-Case and IPW Estimates
For control, the outcomes are \(0,1,0,1,0,1\), so \(\hat\gamma_0=3/6=0.5\). Among the five observed treated outcomes, the outcomes are \(1,1,0,1,0\), so \(\hat\gamma_{1,\mathrm{CC}}=3/5=0.6\). The complete-case contrast is \(\hat\Delta_{\mathrm{CC}}=0.6-0.5=0.10\).
For the weighted treated mean, the observed response weights are \(2,1,2,1,1\) in clusters 1, 2, 3, 4, and 6. Their weighted event total is \(2(1)+1(1)+2(0)+1(1)+1(0)=4\), and their total weight is \(2+1+2+1+1=7\). Thus \(\hat\gamma_{1,\mathrm{IPW}}=4/7\approx0.5714\), and \(\hat\Delta_{\mathrm{IPW}}=4/7-1/2=1/14\approx0.0714\).
The point estimates differ because the complete cases do not have the same stratum composition as the target population: the unobserved treated individual is in stratum A, where response is less likely. Under the stated response model, the IPW estimate is the analysis aimed at the full-population marginal contrast. Neither estimate alone establishes that the contrast is causal; that interpretation additionally requires the treatment-assignment and causal assumptions.
Cluster-Robust Wald Test With Few Clusters
For a transparent derivation, let \(W_{ia}\) be the weight for observed individual \(a\) in group \(i\), and let \(x_{ia}\) identify its group-specific mean. In this example the two means are estimated separately. The weighted estimating equation for a group mean is \(\sum W_{ia}(Y_{ia}-\gamma)=0\), so its solution is the weighted mean. The derivative of the estimating equation with respect to \(\gamma\) is \(-\sum W_{ia}\), the negative “bread.” The cluster score is the sum of weighted residuals within that cluster. Linearizing the estimating equation around the fitted value gives the sandwich covariance: inverse bread, multiplied by the sum of outer products of cluster scores, multiplied by inverse bread again. This construction treats clusters, not individuals, as independent units.
Worked Example: Small-Sample Robust Test of the Risk Difference
Test \(H_0:\Delta=0\) using the IPW estimate. Work first with \((\gamma_0,\gamma_1)\). The bread is diagonal with entries \(A_0=6\) and \(A_1=7\), since the control weights sum to 6 and the treated weights sum to 7. The fitted control residuals are \((-1/2,1/2,-1/2,1/2,-1/2,1/2)\).
The treated cluster scores \(U_{i1}=R_i(Y_i-\hat\gamma_1)/\pi_i\) are \(6/7,3/7,-8/7,3/7,0,-4/7\). The zero in cluster 5 is its unobserved treated contribution. The sum of their squares is \((36+9+64+9+0+16)/49=134/49\). The sum of squared control scores is \(6(1/2)^2=3/2\). The sum of cross-products between control and treated cluster scores is \((-3/7)+(3/14)+(4/7)+(3/14)+0-(2/7)=2/7\).
Therefore the uncorrected sandwich variances of the two means are \((3/2)/6^2=1/24\) and \((134/49)/7^2=134/2401\), and their covariance is \((2/7)/(6\cdot7)=1/147\). For the difference, the uncorrected variance is \(1/24+134/2401-2/147\approx0.08387\). There are \(K=6\) clusters and \(p=2\) mean parameters, so the chosen multiplier is \(6/(6-2)=1.5\). The corrected variance is approximately \(1.5(0.08387)=0.12581\), giving standard error \(\sqrt{0.12581}\approx0.3547\).
The test statistic is \(t=0.0714/0.3547\approx0.201\), with \(6-2=4\) degrees of freedom. The two-sided \(t_4\) p-value is approximately \(0.85\). At a 5% level, this test does not reject \(H_0\). The large uncertainty is important: a small estimated difference is not evidence that the risks are equal. With only six clusters, the reference distribution and correction are approximate, and inference should be interpreted cautiously.
Interpretation and Common Pitfalls
The robust covariance addresses correlation within clusters by aggregating residual contributions at the cluster level; it does not repair an incorrect estimand, response model, or missingness assumption. In particular, the IPW point estimate depends on the response probabilities, while its uncertainty must reflect the weighted cluster scores. If response probabilities are estimated rather than known, their estimation can also affect the variance; the calculation above deliberately treats them as known.
- Using complete cases as if selection were harmless. Complete-case analysis does not generally preserve the target population when response varies with outcome-related observed covariates. State the missingness assumption that justifies it.
- Calling MAR the same as MCAR. MAR permits response to depend on observed variables. IPW can use that information; MCAR is a stronger condition.
- Using individual residuals as independent contributions. The independent sampling units here are clusters. The robust “meat” must sum cluster score outer products.
- Treating the corrected test as exact. A finite-sample multiplier and cluster-based degrees of freedom can improve small-sample calibration, but do not make a six-cluster test exact or eliminate sensitivity to influential clusters.
- Interpreting a non-significant result as proof of no effect. The estimated contrast and its uncertainty describe what the data support; failure to reject zero is not evidence that the true contrast equals zero.
Check Your Understanding
Use the definitions and calculations above to answer these questions.
- What does the marginal risk difference compare, and when can it be interpreted causally?
- In the six-cluster study, why do the complete-case and IPW treatment means differ?
- State the MAR and positivity conditions needed to justify inverse-probability weighting.
- Why are the sandwich score contributions summed by cluster rather than by individual?
- For \(K=6\) clusters and \(p=2\) parameters, what covariance multiplier and \(t\) degrees of freedom were used?
- Why does the non-significant robust test not establish that the population risk difference is zero?