Marginalization Under Covariate-Dependent Cluster Effects

Why the Random-Effects Distribution Matters

A GLMM describes responses conditionally on cluster effects, but a marginal mean averages over a distribution of those effects. The familiar integration against one common random-effects distribution is appropriate only when that distribution is the right one for the covariates being held fixed. If clusters with different covariates tend to have different random effects, the relevant distribution changes with those covariates, and so can the population-averaged response.

This is not a minor technical adjustment. A conditional coefficient describes the response model given the random effect; a marginal prediction also reflects which clusters are represented at a given covariate value. Conditioning on the wrong random-effects distribution can therefore produce a different prediction even when the conditional response model is unchanged.

The Conditional Integration Formula

Suppose the conditional mean follows a GLMM with link function \(g\): \(E(Y_{ij}\mid b_j,x_{ij},C_j)=g^{-1}(x_{ij}^{T}\beta+z_{ij}^{T}b_j)\), where \(z_{ij}\) is the random-effects design vector. By the law of total expectation, the marginal mean at specified covariates is

The integration is on the response scale: apply the inverse link for each value of \(b\), then average those conditional means. The distribution in the integral is \(f(b\mid C_j=c)\), not automatically the unconditional distribution \(f(b)\). If \(b_j\) is independent of \(C_j\), then \(f(b\mid C_j=c)=f(b)\) and the usual common-distribution formula follows. If that independence assumption is not credible, the common-distribution formula does not generally give the stated conditional marginal mean.

A common specification is a normal random-effects distribution whose mean depends on cluster covariates:

where \(D\) describes how the conditional mean of the cluster effects changes with \(c\), and \(\Sigma\) is the conditional covariance matrix. A more general model can allow the covariance, or the distributional shape, to depend on \(c\) as well. The conditional response model and the conditional distribution of \(b_j\) are distinct parts of the specification: dependence in the latter does not, by itself, change the meaning of the former’s coefficients.

Derivation: Why Conditional Integration Is Required

Write \(m(x,b)=g^{-1}(x^T\beta+z^Tb)\). For clusters with \(C_j=c\), the conditional response mean at a fixed random effect is \(m(x,b)\). Averaging over the random effects actually represented among those clusters gives

When the random-effects distribution depends on cluster covariates \(C_j\), and its conditional form does not otherwise depend on the member-specific \(x_{ij}\), this reduces to the formula using \(f(b\mid C_j=c)\). The reduction relies on that modeling condition. If the distribution also depends on the individual covariate, or on a larger cluster covariate history, the conditioning information in the density must be expanded accordingly. In particular, it is not enough to condition on one member’s covariate if the model says cluster effects depend on the cluster’s full covariate pattern.

This argument is simply iterated expectation, but it identifies exactly where the independence assumption enters: it is used to replace the conditional density of \(b_j\) by a common density. It is not a consequence of the inverse-link function or of the response family.

Nonlinear Links: The Distribution Shape Also Matters

With a nonlinear inverse link, substituting the conditional mean of \(b_j\) into the linear predictor generally does not produce the marginal mean. One must average the inverse-link response over the full conditional distribution. The following probit model gives a closed-form illustration.

For a logistic link, the same conditional integration formula applies, but a normal random intercept generally does not yield this probit closed form. Numerical integration or other appropriate computation may be needed. The next example uses a discrete conditional distribution so that the averaging can be seen directly.

Assumptions, Diagnostics, and Alternatives

A covariate-dependent integration is meaningful only in relation to a clearly stated model and target. In particular, check the following:

• Correct conditioning information. Specify which covariates determine the distribution of cluster effects. A cluster-level covariate, a cluster summary of member-level covariates, and a single member’s covariate are not interchangeable.
• Conditional response model. State the response distribution, link, fixed-effects design, and random-effects design. The integration formula uses that conditional mean and does not repair misspecification of it.
• Conditional random-effects model. Specify \(f(b_j\mid C_j)\), including its mean and any modeled covariance or shape dependence. A normal distribution is an assumption, not a consequence of clustering.
• Target population and weights. A mean at fixed \(C_j=c\) is a stratum-specific mean. For an overall mean, average these conditional means using the target population’s distribution of \(C_j\); changing that distribution changes the estimand.
• Data support and identification. The data need enough information across covariate values to distinguish conditional response effects from changes in the distribution of cluster effects. Sparse covariate strata or few clusters can make this separation unstable.

Useful checks include examining whether cluster-level covariates are associated with estimated cluster deviations under a preliminary model, inspecting the covariate distribution and cluster counts, and checking whether the assumed conditional distribution fits the observed between-cluster pattern. Estimated random effects are shrunken estimates, so a plot of them against covariates is a diagnostic prompt rather than a definitive test of independence. Sensitivity analyses can compare plausible conditional distributions or mean structures. A lack of an obvious trend in a diagnostic plot does not prove independence.

The main alternatives answer different questions. A conventional random-effects model uses one common distribution, often a mean-zero normal distribution, and is appropriate when that assumption fits the intended conditioning and sampling structure. A covariate-dependent random-effects model changes the distribution used for integration and directly represents the dependence specified here. Empirical standardization can then average predictions over an explicitly chosen covariate distribution, but it does not remove the need to define the conditional random-effects model. Generalized estimating equations (GEE) provide a population-averaged modeling approach with a different specification strategy; they are not simply the same GLMM integration with another name.

One common specification, often called a correlated random-effects or within-between formulation, allows the random-effect distribution’s mean to depend on cluster-level covariate summaries. Such a formulation can represent a particular form of dependence, but it does not guarantee that all dependence has been captured. Conversely, simply adding a cluster covariate to the conditional response predictor does not necessarily model dependence between that covariate and the random effects: those are different model components.

Common Pitfalls and Limits

• Using the unconditional distribution at every covariate value. This silently assumes that the random-effects distribution is unchanged by those covariates. Use the conditional distribution when the model says it varies.
• Averaging the linear predictor instead of the response mean. For nonlinear links, in general \(E\{g^{-1}(\eta)\}\neq g^{-1}\{E(\eta)\}\). Integrate the inverse-link mean over the specified random-effects distribution.
• Confusing a response-model covariate effect with a distributional effect. A covariate can enter the conditional predictor, the distribution of \(b_j\), or both. State which role it plays; do not interpret one as the other.
• Calling a conditional stratum mean a universal population mean. A prediction at fixed \(C_j=c\) applies to that stratum. An overall mean requires a further average over a stated target distribution of \(C_j\).

The method’s flexibility has costs. Conditional random-effects distributions may be hard to identify separately from response-model effects, particularly with few clusters or limited covariate overlap. Results can also be sensitive to distributional assumptions and to whether the target covariates describe the clusters adequately. Marginalization makes an estimand explicit; it does not make it assumption-free or causal.