Effect Modification And Population Averaging

Why Modification and Averaging Must Be Considered Together

A treatment effect can vary across clusters, across measured groups, or both. Even when a GLMM specifies one treatment coefficient, the corresponding effect on the probability scale need not be the same in every cluster: in a logistic model, the baseline risk changes the probability difference associated with a fixed log-odds contrast. Conversely, an explicit treatment interaction says that the treatment coefficient changes with a modifier on the model’s link scale, but it does not by itself give the treatment contrast averaged over a population.

The key distinction is between a conditional effect, defined with cluster random effects held fixed, and a population-averaged contrast, calculated after predicting outcomes under each treatment and averaging over a stated population distribution. These answer different questions. Neither is automatically the uniquely correct treatment effect; the choice depends on the target of the analysis.

Conditional Interaction and Population-Averaged Risk Difference

For a binary outcome, consider a logistic random-intercept and random-slope model with a binary modifier \(Z_{ij}\):

Here \(\operatorname{logit}(p)=\log\{p/(1-p)\}\). The random intercept \(b_{0j}\) represents cluster-specific baseline variation, while \(b_{1j}\) allows the treatment coefficient to vary across clusters. A common assumption is that the random-effects vector has a specified distribution, often a mean-zero multivariate normal distribution, and that outcomes are independent conditional on the random effects and covariates. The chosen random-effects distribution and mean structure must be adequate for the intended inference.

At fixed \(Z_{ij}=z\) and fixed random effects, the conditional treatment log odds ratio is \(\beta_A+\beta_{AZ}z+b_{1j}\). Thus \(\beta_{AZ}\) is modification of the treatment log odds ratio by \(Z\), conditional on cluster effects. If \(b_{1j}=0\), the conditional odds ratio is \(\exp(\beta_A+\beta_{AZ}z)\). This is not generally a risk difference, and it is not automatically a population-average odds ratio.

Let \(p_{ij}(a,z,b_{0j},b_{1j})\) be the predicted probability after setting treatment to \(a\). A conditional risk difference is \(p_{ij}(1,z,b_{0j},b_{1j})-p_{ij}(0,z,b_{0j},b_{1j})\). To obtain an average treatment contrast for a target population, average the two predicted probabilities over that population before subtracting. For a finite target population with weights \(w_k\), where \(w_k\geq 0\) and \(\sum_k w_k=1\), the average risk difference is:

For a model-based target over random effects, the analogous operation is integration with respect to their distribution, and possibly averaging over covariates as well. The target weights matter: equal weight per cluster, weight per individual, and a prespecified external population distribution can lead to different averages.

Modification by a Measured Cluster or Individual Characteristic

A treatment-by-modifier interaction can be present even when the model has no random treatment slope. In that case, the treatment coefficient differs by the measured modifier, while remaining constant across clusters conditional on the random intercept. The following example calculates both the conditional contrasts and a weighted population contrast.

Why Averaging and Prediction Do Not Commute

The source of the distinction can be seen directly. Write \(\operatorname{expit}(x)=1/(1+\exp(-x))\), and let \(\eta(a,b)\) be the linear predictor under treatment \(a\) and random effect \(b\). For a logistic GLMM, the population risk under treatment \(a\) is \(E_b[\operatorname{expit}\{\eta(a,b)\}]\). In general, this is not \(\operatorname{expit}\{E_b[\eta(a,b)]\}\). Consequently, setting random effects to their mean, often zero, does not generally calculate a population-averaged risk.

A short derivation makes the issue explicit. With a random intercept \(b\) and treatment log odds increment \(\beta_A\), the conditional risk difference at linear predictor \(\eta\) is \(\operatorname{expit}(\eta+\beta_A)-\operatorname{expit}(\eta)\). Its population average is:

The equality follows from linearity of expectation. It does not allow either expectation to be replaced by the inverse-logit evaluated at the mean linear predictor. Since the inverse-logit is nonlinear, heterogeneity in \(b\) can change both averaged risks and their difference. On the odds-ratio scale, an additional distinction arises: a conditional odds ratio generally differs from an odds ratio computed from marginal risks, even without treatment-effect modification. This is the non-collapsibility of the odds ratio, not evidence by itself of confounding or a faulty model.

Choosing and Reporting the Contrast

A useful analysis begins by naming the population and contrast before interpreting an interaction coefficient. For a target population, specify which covariates and cluster effects are averaged over and how they are weighted. Then calculate the predicted outcome under treatment and control for each target unit or target stratum, average each set of predictions, and form the desired contrast. If the goal is a risk difference, subtract the average risks; if it is a risk ratio, divide them. These calculations need not agree with a coefficient or exponentiated coefficient from the conditional model.

A marginal model such as GEE directly parameterizes a population-average mean relationship, whereas a GLMM specifies a conditional model given random effects and obtains population averages by integrating or standardizing its predictions. They can target related quantities, but their regression coefficients are not interchangeable by default. A GLMM can still be used to estimate an average contrast by standardization, provided its conditional mean and random-effects assumptions are sufficiently credible for that purpose. The introductory material on GLMM and GEE distinguishes their broad conditional and marginal targets; here the key point is how modification and averaging affect the contrast itself.

A treatment interaction is scale-dependent. A zero interaction on the log-odds scale means the conditional log odds ratio does not vary with the modifier; it does not imply equal risk differences across modifier values. Likewise, unequal risk differences do not alone establish interaction on every scale. State the scale and the conditioning set whenever reporting effect modification.

Diagnostics, Alternatives, and Limitations

Check that the interaction coding matches the scientific question, that treatment and modifier combinations have adequate data support, and that predicted probabilities are within the possible range. For a random-slope model, inspect whether the data support the added cluster-specific treatment variation and whether conclusions are sensitive to plausible random-effects distributions. Compare fitted and observed outcome patterns across treatment, modifier, and cluster strata where feasible. Sparse combinations, near separation, or a few influential clusters can make interaction estimates unstable.

The main alternative is to fit a marginal mean model and estimate a population-average treatment relationship directly, commonly using GEE. This avoids specifying a random-effects distribution for the mean interpretation, but still requires a suitable marginal mean model and valid inference for the cluster structure. A GLMM is especially natural when cluster-specific prediction or conditional effects are central; a marginal approach is natural when the population mean is the primary target. Either approach can be misinterpreted if the coefficient scale is mistaken for a different contrast.

Neither averaging nor interaction analysis removes design requirements for causal interpretation. A treatment contrast is causal only under conditions appropriate to the study, such as randomized treatment assignment or defensible control of confounding, along with a well-defined intervention and adequate support. Model-based averages also depend on the target population and model assumptions. An average can conceal meaningful heterogeneity, so reporting only one number may be inadequate when subgroup contrasts are important.

Common Pitfalls

• Reading a conditional coefficient as a population effect. The treatment coefficient in a logistic GLMM is conditional on random effects. It is not generally the marginal risk difference or marginal odds ratio.
• Assuming no interaction means equal risk differences. A constant conditional log odds ratio can coexist with different probability differences when baseline risks vary.
• Calling the interaction coefficient a risk-difference interaction. In a logistic model, \(\beta_{AZ}\) is a difference in conditional log odds ratios across \(Z\), not a difference in risk differences.
• Evaluating at random effects equal to zero and calling it population-average. A zero random effect gives a cluster at the reference random-effect value; population averaging requires averaging predicted probabilities over the specified distribution.
• Leaving the population weights unstated. Equal cluster weights and equal individual weights need not define the same average when cluster sizes differ.