When GLMM And GEE Estimates Are Similar

Why Similar Estimates Need Careful Interpretation

A GLMM and a GEE can produce similar coefficient estimates for some clustered data, but similarity is a numerical observation, not evidence that the methods estimate the same quantity. A GLMM coefficient describes a covariate effect conditional on the random effects in the model. A GEE coefficient describes a population-averaged effect through a marginal mean model. The distinction matters even when the fitted numbers are close.

Similarity is most plausible when averaging over cluster effects changes the mean relationship little: for example, when cluster heterogeneity is weak, the mean function is close to linear over the relevant range, or a particular link function preserves slopes under averaging. The conditions differ by outcome and link. The following results explain when closeness has a clear model-based reason, and why it should not be assumed by default.

Conditional and Marginal Mean Models

Let \(Y_{ij}\) denote the outcome for observation \(i\) in cluster \(j\), and let \(x_{ij}\) be a covariate. For a random-intercept GLMM, write the conditional mean as

E(Y_{ij}\mid b_j,x_{ij})=h(\alpha+\beta x_{ij}+b_j),

where \(b_j\) is a cluster-specific random intercept and \(h=g^{-1}\) is the inverse link. The population-averaged mean is obtained by averaging over the random-intercept distribution:

m(x)=E_{b_j}\{h(\alpha+\beta x+b_j)\}.

A GEE specifies and estimates a marginal mean relationship such as \(g\{m(x)\}=\alpha_G+\beta_G x\). Its coefficient \(\beta_G\) is a population-averaged coefficient for that marginal model. It need not equal the GLMM’s conditional coefficient \(\beta\). A direct numerical comparison is meaningful only after checking that the outcome, covariates, link, and target contrast correspond.

Key distinction: Similarity means that two estimates are close in value for the fitted data and models. Interchangeability would require the methods to target the same estimand and to have compatible assumptions. Similar numbers alone establish neither.

Cases Where Averaging Preserves or Barely Changes Slopes

With an identity link and a mean-zero random intercept, averaging gives \(m(x)=\alpha+\beta x\): the marginal intercept and slope match the conditional ones. For nonlinear links, averaging generally changes the mean curve. However, a log link with a random intercept has a special property: under a common random-intercept distribution independent of the covariates, the marginal mean is a constant multiple of the conditional mean. That changes the intercept on the log scale but not the slope.

Worked Example: Exact Slope Agreement Under a Log Link

Suppose a hypothetical count GLMM has conditional mean \(E(Y_{ij}\mid b_j,x_{ij})=\exp(-0.4+0.3x_{ij}+b_j)\), with \(b_j\sim N(0,0.2^2)\), independent of \(x_{ij}\). Assume the marginal GEE uses the correct log-link mean model.

For a normal random intercept, \(E\{\exp(b_j)\}=\exp(0.2^2/2)=\exp(0.02)\). Therefore,

E(Y_{ij}\mid x_{ij}) =\exp(-0.4+0.3x_{ij})\exp(0.02) =\exp(-0.38+0.3x_{ij}).

At \(x=0\), the marginal mean is \(\exp(-0.38)\approx0.684\). At \(x=1\), it is \(\exp(-0.08)\approx0.923\). Their ratio is \(0.923/0.684\approx1.35=\exp(0.3)\). The conditional rate ratio for a one-unit increase in \(x\) is also \(\exp(0.3)\approx1.35\). Thus, in this model, the marginal and conditional log-link slopes are exactly equal, although their intercepts differ by \(0.02\).

This exact result depends on the stated structure: the random intercept has a common distribution independent of \(x\), and the marginal mean follows the log-link form. It does not imply that a log-link GLMM and GEE will always give equal estimated slopes in finite samples or under different model specifications.

For a nonlinear mean function, weak random-effect variation can also make the marginal curve close to the conditional curve. The approximation must be justified using how the random effect varies as its variance shrinks; its mean and variance alone are not enough to determine the second-order term.

Regular small-variance expansion: Suppose \(U=\tau Z\), where \(E(Z)=0\), \(E(Z^2)=1\), and the distribution of \(Z\) is fixed as \(\tau\) decreases. Suppose \(E|Z|^3<\infty\), and \(h\) has bounded third derivative on the relevant range. Then, as \(\tau\to0\),

E\{h(\eta+\tau Z)\} =h(\eta)+\frac{\tau^2}{2}h''(\eta)+O(\tau^3).

If \(h\) has a bounded fourth derivative as well, applying the same argument to \(h'\) gives \( \frac{d}{d\eta}E\{h(\eta+\tau Z)\}=h'(\eta)+\frac{\tau^2}{2}h'''(\eta)+O(\tau^3) \).

Why the Small-Variance Expansion Holds

Taylor’s theorem with a third-derivative remainder gives

h(\eta+\tau Z) =h(\eta)+\tau Z h'(\eta) +\frac{\tau^2Z^2}{2}h''(\eta)+R, \qquad |R|\leq \frac{M}{6}|\tau Z|^3,

where \(M\) bounds \(|h'''|\) on the range in question. Taking expectations, the linear term vanishes because \(E(Z)=0\), and the quadratic term becomes \(\tau^2h''(\eta)/2\) because \(E(Z^2)=1\). Also \(E|R|\leq M|\tau|^3 E|Z|^3/6\), so the expected remainder is \(O(\tau^3)\). The fixed-distribution condition \(U=\tau Z\) is essential: it controls how the entire random-effect distribution changes as its variance shrinks. For instance, a distribution with rare, fixed-size jumps can have the same small variance but a different second-order coefficient. The derivative expansion follows by applying the same argument to \(h'\) under the stated smoothness condition.

Worked Example: A Small Random Intercept on the Logistic Scale

Consider a hypothetical binary-outcome random-intercept GLMM with \(h(\eta)=1/(1+\exp(-\eta))\), conditional linear predictor \(\eta=\alpha+\beta x+b_j\), \(\beta=0.8\), and \(b_j=0.3Z\), where \(Z\sim N(0,1)\). At \(\eta=0\), \(h(0)=0.5\), \(h'(0)=0.25\), and \(h'''(0)=-0.125\). The normal \(Z\) has a fixed distribution, finite moments, and the logistic function has bounded derivatives, so the small-variance expansion applies.

At \(\eta=0\), the marginal probability is exactly \(0.5\): symmetry gives \(h(u)+h(-u)=1\). The derivative of the marginal probability with respect to \(\eta\), using the second-order expansion, is approximately

0.25+\frac{0.3^2}{2}(-0.125) =0.25-0.005625 =0.244375.

The derivative of the marginal log odds is the derivative of the probability divided by \(m(1-m)\). Here \(m=0.5\), so the local marginal logit slope with respect to \(x\) is approximately

0.8\frac{0.244375}{0.5(1-0.5)} =0.8\frac{0.244375}{0.25} \approx0.782.

This second-order approximation is about \(0.018\) below the conditional slope \(0.8\), a difference of roughly \(2.25\%\). It is a local approximation at \(\eta=0\), not a claim that the marginal logit mean is exactly linear in \(x\) or that a GEE fitted across a broad covariate range must estimate \(0.782\).

When Closeness Breaks Down

A nonlinear link can make averaging materially alter the coefficient, particularly when the random-intercept distribution is broad or the mean curve is strongly curved over the observed range. The following hypothetical binary example calculates the marginal contrast directly rather than relying on a small-variance approximation.

Worked Example: A Larger Logistic Random-Intercept Spread

Suppose the conditional logit is \(-1+0.8x+b_j\), and each cluster intercept is \(+1.2\) or \(-1.2\) with probability \(1/2\). This is a mean-zero random-intercept distribution with substantial spread. For a binary covariate \(x\), a marginal logistic model with an intercept and \(x\) can represent the two population-averaged probabilities exactly.

At \(x=0\), the two conditional probabilities are \(h(-2.2)\approx0.09975\) and \(h(0.2)\approx0.54983\). Their average is \(m(0)\approx0.32479\), with logit \(\log\{0.32479/(1-0.32479)\}\approx-0.732\).

At \(x=1\), the probabilities are \(h(-1.4)\approx0.19782\) and \(h(1)\approx0.73106\). Their average is \(m(1)\approx0.46444\), with logit \(\log\{0.46444/(1-0.46444)\}\approx-0.142\). The marginal logit contrast is therefore

-0.142-(-0.732)\approx0.590.

The conditional logit coefficient is \(0.8\), whereas the marginal contrast is approximately \(0.590\). The discrepancy is about \(0.210\), or 26% of the conditional coefficient. Both calculations are valid for their respective targets; neither is a computational error. The example shows why results from a strongly heterogeneous logistic GLMM and a marginal GEE can differ substantially.

Diagnostics, Comparisons, and Limitations

To assess whether observed similarity has a credible explanation, check the fitted mean relationship and the random-effects structure rather than comparing coefficients in isolation. Useful checks include whether the GLMM’s random-effect variance is small relative to the scale on which the inverse link curves, whether marginal predicted means track the fitted GEE mean across the covariate range, and whether the models use the same covariates and comparable populations. For logistic models, compare predicted probabilities and marginal contrasts as well as logit coefficients. A small random-effect variance estimate is not by itself proof of equality: sampling uncertainty, covariate patterns, and model misspecification still matter.

The main alternative is not a different trick for making coefficients agree, but a model chosen for the inferential target. A GLMM is appropriate for conditional questions about covariate effects given cluster effects and for modelling cluster-specific variation. A GEE is appropriate for population-averaged mean effects when its marginal mean model is suitable. As covered in the earlier material on GEE efficiency, a working correlation choice affects efficiency; it does not turn a marginal target into a conditional one. Likewise, the earlier material on misspecified mean models is relevant: a wrong mean structure can change the fitted coefficients even if one method’s estimates happen to resemble the other’s.

Closeness is most informative when it follows from a transparent property, such as the log-link random-intercept result, or from a justified local approximation under weak heterogeneity. It can be misleading when estimates agree only for one sample, when the relationship is curved outside a narrow covariate range, or when cluster size or random effects relate to covariates in ways excluded by the comparison. It also does not establish that standard errors, predictions, or conclusions about clusters agree. Similar point estimates are not a substitute for stating what each coefficient means.

Common Mistake

Pitfall: “The GLMM and GEE coefficients are close, so the two methods are interchangeable and estimate the same effect.”

Correction: The coefficients may be close because heterogeneity is weak or the chosen link preserves slopes under averaging. But the GLMM coefficient is conditional and the GEE coefficient is marginal. State both targets, verify the model conditions that explain the closeness, and compare marginal predictions or contrasts when those are the scientific quantities of interest.

Check Your Understanding

Answer each question, then compare with the solutions below.

For a random-intercept log-link model with \(b_j\) independent of \(x\) and \(E\{\exp(b_j)\}<\infty\), why can the marginal and conditional slopes agree?
Why does knowing only that a random effect has mean zero and variance \(\tau^2\) not always justify the stated second-order small-variance expansion?
In the logistic worked example with \(\beta=0.8\) and \(\tau=0.3\), what is the second-order approximation to the local marginal logit slope at \(\eta=0\)?
In the larger-spread logistic example, what are the conditional and marginal logit contrasts for a one-unit increase in the binary covariate, and what is the main interpretation?

Solutions. (1) The marginal mean factors as \(\exp(\alpha+\beta x)E\{\exp(b_j)\}\); taking logs adds a constant to the intercept but leaves the coefficient of \(x\) equal to \(\beta\). (2) The distribution could change with \(\tau\) in a way that retains rare, fixed-size jumps, so higher-order contributions need not be controlled by a fixed scaled distribution \(U=\tau Z\). (3) The approximation is \(0.8[0.244375/0.25]\approx0.782\). (4) The conditional contrast is \(0.8\); the marginal contrast is approximately \(0.590\). They differ because the conditional and population-averaged logit effects are distinct quantities under random-intercept variation.