AIC And QIC For GLMM-GEE Comparisons

Why These Criteria Require Care

AIC and QIC are often used to compare candidate models for clustered outcomes, but they are not interchangeable scores. A generalized linear mixed model (GLMM) specifies a conditional outcome model given random effects and has a likelihood when its probability model is specified. Generalized estimating equations (GEE) specify a marginal mean model and use a quasi-likelihood criterion rather than a full joint likelihood. AIC is designed for the former setting; QIC is a GEE criterion.

The distinction matters especially when the models target different estimands. A GLMM coefficient describes a relationship conditional on the random effects, while a GEE coefficient describes a population-averaged relationship. A lower numerical value from one criterion does not establish that its model is better than a model scored by the other criterion. First decide what quantity the analysis is meant to estimate; then compare candidate models using a criterion appropriate to that target.

AIC for Likelihood-Based GLMMs

For clusters indexed by \(i=1,\ldots,m\), let \(Y_i\) denote the vector of outcomes in cluster \(i\), and let \(b_i\) denote its random effects. A GLMM specifies a conditional distribution for \(Y_i\) given \(b_i\), along with a distribution for \(b_i\). Its likelihood for the observed outcomes is the marginal likelihood

where \(\theta\) contains the model parameters. The integral averages over the random-effects distribution; it is not the likelihood obtained by treating estimated cluster effects as known constants. Let \(\ell(\hat{\theta})\) be the maximized log marginal likelihood and \(K\) the number of freely estimated model parameters.

AIC balances fit against complexity: the first term rewards a larger maximized likelihood, while the second penalizes the number of estimated parameters. Within a candidate set fitted to the same outcome data, using the same likelihood convention, lower AIC indicates better estimated predictive fit relative to complexity. AIC is not a test of whether a model is true, and a small difference does not establish that the lower-AIC model is substantively superior.

For fixed-effect comparisons in a GLMM, likelihoods should ordinarily be fitted by maximum likelihood rather than restricted maximum likelihood (REML), because REML likelihoods are not generally comparable when the fixed-effects design changes. Comparisons also require the same response observations and a consistent likelihood definition. If one model omits observations used by another, their AIC values do not constitute a fair comparison of fit to the same data.

QIC for GEE

GEE estimates a marginal mean model and accounts for within-cluster dependence through a working correlation structure. In general, it does not specify a full joint probability distribution for all outcomes in a cluster. Consequently, the likelihood-based AIC formula is not directly available. QIC uses a quasi-likelihood evaluated under working independence, with a penalty that accounts for the difference between model-based and robust covariance estimates of the regression coefficients.

Write \(Q(\hat{\beta};I)\) for the independence quasi-likelihood evaluated at the fitted coefficient vector \(\hat{\beta}\). Let \(H_I\) be the inverse of the model-based covariance matrix for \(\hat{\beta}\) calculated under working independence, and let \(V_R\) be the robust sandwich covariance matrix. In this notation, the QIC is

The trace term measures the adjustment for uncertainty using the robust covariance rather than relying only on the model-based covariance. Some references and software express the same criterion using different symbols for the inverse covariance or sensitivity matrix, so the precise convention should be checked before comparing reported scores. A commonly reported related criterion, QICu, uses a simpler parameter-count penalty:

where \(p\) is the number of regression coefficients in the mean model. QICu is often used to compare mean structures; QIC is commonly used when comparing working correlation structures. Neither is a likelihood-based AIC, and their values should only be compared when the same criterion definition, outcome data, and analysis conventions are used.

Why the Criteria Cannot Be Put on One Scale

AIC and QIC both include a fit component and a complexity adjustment, but their fit components are different quantities: AIC uses a maximized log likelihood, whereas QIC uses an independence quasi-likelihood. Their penalties also arise from different calculations. Thus, an AIC of 372 and a QIC of 194 are not scores that can be ranked against one another. The numerical difference has no interpretation as evidence for one model over the other.

There is a second, substantive reason not to use the two criteria as a cross-model scoreboard. A GLMM and a GEE may describe different estimands: the GLMM models conditional effects given random effects, while the GEE models population-averaged effects. The question of which target is relevant is scientific and design-based, not something resolved by comparing criterion values. Even if the same records are used, the two fits need not be competing approximations to the same inferential target.

Justification and Practical Diagnostics

The \(2K\) in AIC is an estimate of optimism: a model’s fit to the data used to estimate its parameters is typically better than its fit to new data. In regular likelihood settings, estimating \(K\) parameters produces an approximate expected log-likelihood optimism of \(K\). Since deviance is on the \(-2\) log-likelihood scale, the corresponding correction is \(2K\). This motivates AIC as an estimate of relative out-of-sample predictive discrepancy, rather than as a test statistic.

For GEE, the robust covariance can differ from the model-based covariance because the working covariance need not be correct. The QIC trace penalty adjusts for this mismatch. In the special case where the robust and model-based covariance matrices coincide, the penalty reduces to \(2p\), since the trace of a \(p\)-dimensional identity matrix is \(p\). This explains the simpler QICu form, though QICu and QIC can rank candidates differently when the covariance adjustment matters.

Before interpreting either criterion, check the conditions that make the comparison meaningful. For GLMM AIC, confirm that fits use the same observations, outcome definition, and likelihood convention, and that the stated parameter count includes estimated variance and dispersion parameters. For GEE QIC, confirm that the same records, outcome and criterion definition are used; also verify that the fitting procedure converged and that the mean model and working covariance choices are stated.

Warning signs include a nonconverged GLMM, unstable variance estimates, a GEE that failed to converge, or scores computed on different subsets because of missing outcomes. Such problems undermine the numerical comparison. A criterion cannot repair a misspecified mean model, an inappropriate estimand, or a poorly supported random-effects distribution. Review the substantive model assumptions and the fitted estimates alongside the score. When candidate scores are close, uncertainty in the ranking and scientific interpretability deserve more attention than a small numerical advantage.

Common Mistakes and Alternatives

• Comparing AIC directly with QIC. They use different fit terms and penalties. Compare AIC with compatible AIC values, or QIC with values computed under the same QIC convention.
• Choosing a model solely because its score is smallest across model families. First identify the estimand required by the scientific question. A conditional and a population-averaged effect are not made interchangeable by a criterion.
• Counting random-effect estimates as GLMM parameters. The AIC parameter count includes estimated distributional parameters, such as random-effects variances, not each cluster’s conditional mode as a separate model parameter.
• Ignoring different analysis samples. A lower score from a fit using fewer observed outcomes is not a straightforward improvement over a fit to a larger or different data set.

AIC and QIC are not the only ways to assess candidates. Likelihood-ratio procedures may be available for certain nested likelihood-based models when their regularity conditions hold; they are not a general replacement for QIC in GEE. Predictive validation can assess performance on held-out clusters or observations, provided the validation scheme matches the intended prediction task. These approaches answer different questions, so the analysis should state whether the priority is prediction, a particular estimand, or a parsimonious representation of the data.