Confidence Intervals For Marginal Effects

Why the Interval Must Match the Estimand

A GLMM can produce estimates on several scales. A conditional coefficient describes a relationship on the model’s link scale, while a marginal mean averages predicted responses over a specified target population and the random-effects distribution. A confidence interval for one of these quantities is not automatically an interval for another. In particular, transforming a coefficient interval does not generally produce a confidence interval for a standardized marginal effect.

This tutorial concerns intervals for three response-scale quantities: a marginal mean, a difference between marginal means, and a ratio of marginal means. For binary outcomes, these include a marginal probability, a risk difference, and a risk ratio. For count outcomes, a ratio may compare marginal expected counts or rates. The estimand, target population, and scale should be settled before calculating an interval.

For example, empirical standardization may calculate each condition’s predicted response for the same observed covariate records, then average those predictions. The resulting interval is conditional on treating those records as the target population unless the procedure also accounts for uncertainty in how that population was sampled. Marginalization over the random-effects distribution is part of the estimand; substituting estimated cluster effects for that integration would instead target a different quantity.

Delta-Method Intervals for Marginal Effects

Write the estimated marginal quantity as \(T(\hat{\theta})\), where \(\hat{\theta}\) contains the estimated fixed effects and any estimated covariance parameters needed for marginalization. If the target covariate records are treated as fixed, they enter the function \(T\), but are not random in the uncertainty calculation. Suppose \(\hat{\theta}\) is approximately multivariate normal with estimated covariance matrix \(\widehat V\), and \(T\) is differentiable near the true parameter. The delta method uses the gradient of \(T\) to approximate its variance.

For a vector of marginal means, let \(\hat{\boldsymbol{\mu}}\) be the estimates and let \(\widehat\Sigma\) be their estimated covariance matrix. A difference uses the covariance between the two means, not merely their separate standard errors:

A basic Wald interval for an unrestricted response-scale estimand is the estimate plus or minus \(z_{1-\alpha/2}\) times its estimated standard error. Here \(z_{1-\alpha/2}\) is the corresponding standard normal quantile; for a 95% interval it is approximately 1.96. This construction is useful when the sampling distribution is adequately approximated by a normal distribution and the interval’s endpoints are meaningful on the estimand’s scale.

For an estimand restricted to a range, a direct Wald interval may extend beyond that range. A common remedy for a marginal probability is to form an interval on the logit scale and transform its endpoints back. For a positive ratio, a log-scale interval is natural because the logarithm of the ratio is unrestricted. These transformations preserve the estimand while making the interval’s endpoints conform to the parameter space. They do not, however, guarantee good coverage when the estimate is unstable or the normal approximation is poor.

Differences: Preserve the Dependence Between Predictions

A risk difference is measured in probability points; a count-mean difference is measured in expected counts, or in rate units if the estimand is a rate. Its interval should use the uncertainty in both marginal means and their covariance. The covariance can be substantial because both predictions are calculated from the same fitted model and often the same target covariate records. Ignoring it can give an incorrect standard error in either direction.

This interval happens to stay within the possible range \([-1,1]\) for a difference between probabilities. Wald intervals are not constrained to do so, however. If a difference interval has implausible endpoints, or if the estimated contrast is near a boundary with a skewed sampling distribution, use a method that better reflects the sampling distribution, such as a suitable simulation-based interval. Do not apply a logit transformation directly to a risk difference: the difference can be zero or negative.

Ratios: Work on the Log Scale

A ratio of positive marginal means is positive, while its logarithm can range over the real line. For \(R=\mu_1/\mu_0\), the log ratio is \(\log(R)=\log(\mu_1)-\log(\mu_0)\). The delta-method gradient with respect to the two means is \((-1/\mu_0,1/\mu_1)\). Applying this gradient to their joint covariance matrix yields an estimated standard error for the log ratio. A confidence interval is then calculated on the log scale and exponentiated at both endpoints.

Simulation-Based Intervals and Assumptions

The delta method is not the only way to quantify uncertainty. A simulation-based procedure can propagate uncertainty through the entire calculation of the marginal estimand. For example, one may draw parameter vectors from an approximate sampling distribution based on \(\hat{\theta}\) and \(\widehat V\), calculate the marginal means for each draw, and then calculate the desired difference or ratio for that same draw. Alternatively, a parametric bootstrap simulates data under the fitted model and refits the model in each replicate. Percentiles of the resulting estimand values can form an interval. The simulated quantity must be the marginal estimand itself, not just a conditional coefficient.

These approaches still require assumptions. The GLMM must adequately describe the outcome distribution, link, random-effects distribution, and relevant dependence; the marginalization and target-population definition must match the scientific question; and the fitted model must be estimable. A parametric bootstrap relies on the fitted model being a useful generator for the data. Simulation from an asymptotic parameter distribution relies on that approximation being reasonable. Neither method repairs a misspecified model or an unclear estimand.

When the target covariates represent a sample from a wider population rather than a fixed standardization set, there can be additional uncertainty from sampling that covariate distribution. The uncertainty procedure should reflect that target. For example, resampling independent clusters may be appropriate when clusters are the sampling units and the target includes variation across clusters. Resampling individual rows as if they were independent can be inappropriate when within-cluster dependence matters. The choice follows the sampling design and estimand, not just computational convenience.

Diagnostics, Alternatives, and Limitations

Inspect whether the fitted model converged, whether variance components are near boundaries, and whether the covariance matrix used in the calculation is numerically stable. Sparse outcomes or few clusters can make normal approximations unreliable, especially for probabilities near zero or one and ratios with a small denominator. For simulation-based intervals, inspect the distribution of simulated marginal effects: strong skewness, failed fits, or extreme ratios signal that a symmetric Wald interval may be a poor summary and that even simulation may be unstable.

The main choices have different strengths. A direct Wald interval is simple and can be effective for a well-estimated, approximately symmetric unrestricted contrast. A transformed interval respects the range of a probability or the positivity of a ratio, but remains dependent on an approximate normal distribution on the transformed scale. Simulation-based intervals can capture asymmetry and nonlinear transformations more directly, but cost more computation and depend on the simulation scheme and model. No interval method removes uncertainty about model specification, target-population choice, or the interpretation of the estimand.