Mean Misspecification Versus Correlation Misspecification

Two Different Ways a GEE Model Can Be Wrong

A generalized estimating equation uses a specified marginal mean and a working covariance to combine information across outcomes within each cluster. These two ingredients play different roles. If the marginal mean is correct, a working covariance that gets the within-cluster correlations wrong can still yield a centered estimating equation. If the mean is wrong, however, the estimating equation will generally target a different parameter. A robust sandwich covariance can address uncertainty from the first kind of misspecification; it cannot turn the second kind into a correct mean model.

The distinction is about more than standard errors. It concerns what the estimator converges to. Throughout, index independent clusters by \(i=1,\ldots,n\). Write \(Y_i\) for the response vector in cluster \(i\), \(\mu_i(\beta)\) for the fitted marginal mean vector, and \(D_i(\beta)=\partial\mu_i(\beta)/\partial\beta^T\). Let \(V_i(\beta,\alpha)\) be the working covariance matrix, where \(\alpha\) denotes any working correlation or other nuisance parameters. The cluster estimating function is

\psi_i(\beta,\alpha) = D_i(\beta)^T V_i(\beta,\alpha)^{-1} \{Y_i-\mu_i(\beta)\}, \qquad \Psi_n(\beta,\alpha) = \frac{1}{n}\sum_{i=1}^n\psi_i(\beta,\alpha).

A GEE estimate solves \(\Psi_n(\hat\beta,\hat\alpha)=0\), or approximately solves it in numerical computation. The discussion below concerns the consequences of misspecification, not numerical solution of the equation.

Why a Wrong Working Correlation Need Not Bias the Estimate

Definition: The marginal mean model is correct at \(\beta_0\) if \(E[Y_i\mid X_i]=\mu_i(\beta_0)\) for the covariates and cluster design \(X_i\) under study. The working covariance is correct if it equals the true conditional covariance of \(Y_i\) given \(X_i\). Working-correlation misspecification means the latter equality fails, while the mean equality may still hold.

Suppose the mean model is correct, and the working covariance is a function of the covariates and fixed or consistently estimated nuisance quantities, rather than of the unobserved response residuals in a way that changes the conditional expectation. If \(V_i\) is invertible, then

E[\psi_i(\beta_0,\alpha)\mid X_i] = D_i(\beta_0)^T V_i(\beta_0,\alpha)^{-1} E[Y_i-\mu_i(\beta_0)\mid X_i] =0.

Thus the expected estimating function is zero at the true mean parameter for any such working covariance, not only the true covariance. This is the key reason that a wrong working correlation need not move the probability limit of the GEE estimate. It can still affect efficiency and the model-based standard error.

Result: If the marginal mean is correct, the working covariance is appropriately specified as a weight matrix, and the population estimating equation has a unique root at \(\beta_0\), then working-correlation misspecification preserves the target \(\beta_0\). This conclusion does not say that every finite-sample estimate is unbiased, or that every covariance estimate is valid.

Worked Example: Correct Mean, Incorrect Independence Working Correlation

Suppose each of \(n\) independent clusters contains two measurements. Conditional on the design, both have mean \(\mu\), variance \(\sigma^2\), and correlation \(\rho=0.6\). Fit a common-mean model using the independence working covariance \(V_i=\sigma^2 I_2\).

For this intercept-only mean, \(D_i=(1,1)^T\). The cluster estimating function is

\psi_i(\mu) = \frac{1}{\sigma^2} \{(Y_{i1}-\mu)+(Y_{i2}-\mu)\}.

Setting the sum of these contributions to zero gives \(\hat\mu=(2n)^{-1}\sum_i(Y_{i1}+Y_{i2})\). Since each measurement has expectation \(\mu\), \(E[\hat\mu]=\mu\). The correlation of 0.6 does not change this point-estimation target.

But the variance is not what the independence working model says. The true variance of a cluster sum is \[ \operatorname{Var}(Y_{i1}+Y_{i2}) = \sigma^2+\sigma^2+2(0.6)\sigma^2 =3.2\sigma^2. \] Therefore \[ \operatorname{Var}(\hat\mu) =\frac{n(3.2\sigma^2)}{(2n)^2} =\frac{0.8\sigma^2}{n}. \] The independence model-based variance would be \(2n\sigma^2/(2n)^2=0.5\sigma^2/n\). It understates the variance. A cluster-robust sandwich estimate uses the observed cluster contributions to estimate their variability and, under the usual independent-cluster and large-\(n\) conditions, estimates the variance around the same target \(\mu\).

What Changes When the Marginal Mean Is Wrong?

If the proposed \(\mu_i(\beta)\) does not equal the true conditional mean for any \(\beta\), the conditional residual generally has nonzero expectation. The previous zero-expectation calculation no longer applies. Instead, the estimator may converge to a value \(\beta^*\) that solves the population estimating equation

E[\psi_i(\beta^*,\alpha)]=0.

Such a root is often called a pseudo-true parameter: it is the value selected by the specified estimating equation under the actual data-generating distribution. Its value can depend on the working covariance, because the covariance weights residuals differently. It need not describe the true marginal means, and it need not equal a scientific parameter defined through those means. A root need not exist or be unique; conclusions about convergence require that the relevant root exist and be identified.

Worked Example: A Linear Mean Fitted to a Curved Relationship

Consider independent observations at three covariate values \(x=-1,0,2\), with the same number of observations at each value. The true conditional mean is \(E[Y\mid x]=x^2\). Fit the misspecified linear mean \(E[Y\mid x]=\beta_0+\beta_1x\) using an identity-link Gaussian estimating equation with equal working variances. In this setting, the population estimating equation gives the ordinary least-squares projection of the true means onto a line.

The three true means are \(1,0,4\). Their covariate and response averages across the design points are \(\bar x=1/3\) and \(\bar\mu=5/3\). The slope of the projection is

\beta_1^* = \frac{\sum (x-\bar x)(x^2-\bar\mu)} {\sum (x-\bar x)^2} = \frac{\sum x^3-3\bar x\bar\mu} {\sum x^2-3\bar x^2} = \frac{7-5/3}{5-1/3} = \frac{16/3}{14/3} = \frac{8}{7}.

The intercept is \(\beta_0^*=\bar\mu-\beta_1^*\bar x=5/3-8/21=9/7\). The fitted means at \(x=-1,0,2\) are therefore \(1/7,9/7,25/7\), not the true means \(1,0,4\). The coefficient \(8/7\) describes a linear projection for this design and weighting; it does not make the linear mean correct. A sandwich covariance can quantify sampling variation around this projection under appropriate conditions, but cannot make those fitted means equal the true ones.

The example also shows why a robust standard error is not a general correction for model misspecification. Robustness refers to estimating the sampling covariance of an estimator under specified regularity conditions. It does not generally change the estimator’s probability limit, replace an incorrect mean function, or remove the discrepancy between \(\mu_i(\beta^*)\) and the true marginal mean.

Large-Sample Limit and the Sandwich Under Misspecification

The following result makes the distinction precise. It is stated for a fixed working-covariance specification; the same reasoning can apply with estimated nuisance quantities when their estimation satisfies suitable regularity conditions and the resulting estimating functions have the stated limits. Let \(\psi_i(\beta)\) denote the resulting cluster contribution, and define

A=-E\left[\frac{\partial\psi_i(\beta^*)}{\partial\beta^T}\right], \qquad B=E[\psi_i(\beta^*)\psi_i(\beta^*)^T].

Assume clusters are independent and identically distributed, \(\beta^*\) is an interior unique root of \(E[\psi_i(\beta)]=0\), \(A\) is nonsingular, and \(E[\|\psi_i(\beta^*)\|^2]\) is finite. Also assume the estimator is consistent for \(\beta^*\), a uniform law of large numbers applies to the Jacobian in a neighborhood of \(\beta^*\), and the central limit theorem applies to \(\psi_i(\beta^*)\). In particular, assume the averaged Jacobian along the segment from \(\beta^*\) to \(\hat\beta\) converges in probability to \(-A\). These conditions make explicit the local smoothness and convergence needed for the expansion.

Theorem: Under the stated conditions, a consistent root \(\hat\beta\) of the estimating equation satisfies \[ \sqrt n(\hat\beta-\beta^*) \xrightarrow{d} N\!\left(0,A^{-1}B A^{-T}\right). \] This is the sandwich covariance for the estimator’s sampling distribution around its population estimating-equation root.

Proof. The fundamental theorem of calculus applied along the line segment between \(\beta^*\) and \(\hat\beta\) gives the exact vector expansion

0=\Psi_n(\hat\beta) =\Psi_n(\beta^*)+\bar J_n(\hat\beta-\beta^*), \quad \bar J_n = \int_0^1 \frac{1}{n}\sum_{i=1}^n \frac{\partial\psi_i\{\beta^*+t(\hat\beta-\beta^*)\}} {\partial\beta^T}\,dt.

This integrated-Jacobian expression does not assert that there is a single intermediate parameter value whose Jacobian gives the multivariate expansion. By the assumed convergence, \(\bar J_n\xrightarrow{p}-A\), so \(\bar J_n\) is invertible with probability approaching one and \(-\bar J_n^{-1}\xrightarrow{p}A^{-1}\). Rearranging the exact expansion yields

\sqrt n(\hat\beta-\beta^*) = -\bar J_n^{-1}\sqrt n\,\Psi_n(\beta^*).

Because \(E[\psi_i(\beta^*)]=0\), the independent-cluster central limit theorem gives \(\sqrt n\,\Psi_n(\beta^*)\xrightarrow{d}N(0,B)\). Slutsky’s theorem then gives the stated limit, since multiplication by matrices converging in probability to \(A^{-1}\) transforms the limiting covariance to \(A^{-1}B A^{-T}\). \(\square\)

In practice, the empirical sandwich replaces \(A\) by the negative averaged empirical Jacobian and \(B\) by the average outer product of the fitted cluster contributions. Under the theorem’s conditions these estimate the two components of the sampling covariance. If the mean is correct, \(\beta^*=\beta_0\) under the stated identification condition. If the mean is wrong, the theorem instead concerns \(\beta^*\). Valid uncertainty quantification about that root is not the same as valid inference about a different scientific target.

Worked Example: A Robust Variance Does Not Remove Mean Bias

Return to the curved-mean example and suppose the true means are observed without noise at the three design points, with \(m\) independent replicates at each point. The fitted linear slope converges to \(8/7\), while the true mean change from \(x=0\) to \(x=2\), expressed per unit of \(x\), is \((4-0)/2=2\). Their difference is

\frac{8}{7}-2=-\frac{6}{7}.

If independent mean-zero errors are added, a sandwich estimate may consistently estimate the sampling variance of the fitted slope around \(8/7\), assuming its regularity conditions hold. Increasing \(m\) makes that sampling uncertainty shrink, but the slope still approaches \(8/7\), not \(2\). In the noiseless version, the sandwich meat is zero because the fitted estimating contributions have no random variation; this does not make the fitted means correct. With noise, nonzero robust standard errors quantify randomness, not the deterministic mismatch of the mean function.

Why the Distinction Matters in Practice

A correlation model is often a working device for weighting repeated outcomes. When the marginal mean is right, choosing a convenient but imperfect working correlation can be reasonable, provided inference uses a suitable cluster-level sandwich and the number and behavior of independent clusters support its approximation. A wrong mean is more fundamental: it can distort fitted marginal means, contrasts, and the parameter being estimated. Changing only the standard-error formula does not fix those problems.

There is also an important qualification: mean misspecification does not guarantee that the numerical value of every coefficient changes under every weighting scheme. Special designs, symmetries, or accidental cancellations can leave a particular estimating-equation root unchanged. The general warning is that the root is no longer protected by the correct-mean zero-expectation argument and may depend on the working weights. Its interpretation must be justified rather than assumed.

When a fitted marginal mean is in doubt, examine whether the chosen mean structure represents the scientific question and the observed outcome patterns. Cluster summaries can help reveal systematic discrepancies, but a robust covariance is not a diagnostic or a repair for those discrepancies. Conversely, when the mean is credible but correlation is uncertain, the robust sandwich is designed to protect large-sample variance estimation against working-covariance misspecification under its assumptions.

Common Mistakes

Calling both errors “just covariance misspecification.” A wrong working correlation changes weights; a wrong mean changes the residual expectation and can change the estimating-equation root.
Assuming robust standard errors correct the coefficient. The sandwich changes estimated uncertainty, not the estimating equation’s probability limit.
Interpreting the pseudo-true coefficient as a true marginal effect. Under a wrong mean, it may be only a design- and weighting-dependent projection.
Claiming that any wrong correlation leaves every estimate unchanged. The unbiasedness argument requires a correct mean and appropriate working weights, and conclusions about the target also require identification and regularity conditions.
Using a single intermediate Jacobian for a vector expansion. The exact expansion uses the integrated Jacobian along the path between the two parameter vectors.

Key takeaway: With a correct marginal mean, a misspecified working correlation can leave the GEE target intact while making model-based standard errors unreliable. With a misspecified marginal mean, the GEE generally targets a pseudo-true root; a robust sandwich can estimate uncertainty around that root but does not repair the mean model or remove bias relative to another target.

Check Your Understanding

For each question, distinguish the point-estimation target from its estimated sampling uncertainty.

Why does a correct marginal mean make the expected GEE contribution zero for an incorrect but valid working covariance?
In the two-measurement example, what was the true variance of the common-mean estimator, and why did the independence model-based variance differ?
What does the pseudo-true parameter represent when the fitted marginal mean is wrong?
In the curved-mean example, calculate the fitted intercept and explain why the sandwich covariance cannot make the fitted means equal the true means.
Why is the integrated-Jacobian expansion preferable to claiming a single intermediate point for a vector-valued estimating equation?