Testing Fixed Effects While Holding Random Structure Fixed

Why Hold the Random Structure Fixed?

A GLMM separates systematic differences in the conditional mean from variation associated with clusters or other grouping units. When the scientific question concerns a fixed effect, the comparison should change the fixed-effects specification while leaving the random-effects structure unchanged. Otherwise, the difference in fit can reflect either change, and it is not isolated evidence about the fixed-effect hypothesis.

For example, suppose patient outcomes are clustered within clinics. A comparison of a model with treatment and clinic random intercepts against a model without treatment but with the same clinic random intercepts addresses whether treatment contributes to the fixed-effects specification, conditional on that random-effects model. It does not test whether the random intercept is needed, nor does it establish that the chosen random-effects model is correct.

For cluster \(i\), a common representation is \(g\{E(Y_{ij}\mid b_i)\}=x_{ij}^{T}\beta+z_{ij}^{T}b_i+o_{ij}\), where \(g\) is the link, \(x_{ij}\) and \(z_{ij}\) are the fixed- and random-effect design vectors, \(b_i\) is a random effect, and \(o_{ij}\) is an offset if present. Holding the random structure fixed means retaining the same specification for \(z_{ij}\) and the distribution and covariance structure of \(b_i\), while changing the columns or restrictions in the fixed-effects design.

Conditions for a Fixed-Effect Comparison

The usual likelihood-ratio comparison is most straightforward when the models are nested: the reduced model is obtained by imposing \(q\) restrictions on the fixed effects of the full model. For example, dropping one coefficient imposes one restriction, while removing a categorical predictor represented by three coefficients imposes three restrictions jointly. The two models must use the same outcome observations and the same conditional response distribution, link, offset, and random-effects structure.

For GLMMs, the likelihood used for this test is the likelihood of the observed data after integrating over the random effects. The nuisance parameters, including random-effect covariance parameters, are estimated separately under each fixed-effects specification. Thus, “holding the random structure fixed” means holding its form fixed, not forcing its estimated variance components to be numerically identical.

Regularity matters. The fitted models should be identifiable, the likelihood should be sufficiently smooth near the null value, and the estimates should not be constrained by a boundary or otherwise be nonregular. Sparse binary outcomes, separation, very few independent clusters, or poorly identified covariance parameters can make the chi-square approximation unreliable. Holding random effects constant across the comparison does not remove those problems.

Likelihood-Ratio Statistic and Its Justification

Write \(\ell_0\) and \(\ell_1\) for the maximized log likelihoods under the reduced and full models, respectively. Because every parameter value allowed by the reduced model is also allowed by the full model, maximization gives \(\ell_1\geq\ell_0\). The likelihood-ratio statistic is therefore nonnegative:

Under the null hypothesis, the full model contains \(q\) additional fixed-effect directions that do not improve the population fit beyond random sampling variation. Locally, the log likelihood is approximated by a quadratic function of the parameter displacement. After scaling by its information, the \(q\) unrestricted directions contribute the sum of \(q\) squared, approximately standard-normal quantities. That sum has a \(\chi^2_q\) distribution, which gives the reference law in the theorem. This argument requires regularity; it is not a guarantee for small or irregular samples.

The p-value is the probability, assuming \(H_0\) and the reference approximation, of observing a statistic at least as large as the one calculated. A small p-value is evidence against the restrictions in \(M_0\), not a probability that \(H_0\) is true. The degrees of freedom are the number of independent fixed-effect restrictions, not the number of clusters or observations.

What Goes Wrong When the Random Structure Changes?

A likelihood difference is not a clean fixed-effect test if one fit has a random intercept and the other has both a random intercept and a random slope. The larger model has changed the fixed-effects question and the modeled dependence structure at once. Even when the specifications can be arranged as nested models in a broader sense, the resulting comparison does not isolate the fixed-effect hypothesis, and the usual degrees of freedom and reference distribution may not apply.

Diagnostics, Alternatives, and Limits

Before interpreting a result, verify that both fits converged, that the reported likelihoods correspond to the intended models, and that the same observations were used. Check for warnings, extreme estimates, large standard errors, separation in binary outcomes, and evidence that the random-effects structure is weakly identified. A singular or near-singular covariance estimate is a warning that the assumed structure may be difficult to estimate; it does not by itself prescribe a different fixed-effect test. If the response is sparse or the number of independent clusters is small, the asymptotic p-value may be poorly calibrated.

The likelihood-ratio test is one approach, not the only one. A Wald test evaluates an estimate relative to its estimated standard error and can be convenient for a single coefficient, but it may be sensitive to small-sample approximations. A joint Wald test can assess multiple coefficients. Comparing information criteria addresses a model-selection objective rather than a null-hypothesis test. These alternatives answer related but not identical questions; none makes a changed random structure an isolated fixed-effect comparison.

The key limitation is conditionality on the model choices. A significant result does not validate the link, the random-effects distribution, or the selected covariance structure. A nonsignificant result may reflect limited information rather than a negligible effect. Report the fixed-effect hypothesis, the shared random structure, the likelihood-ratio statistic, degrees of freedom, p-value, and the practical effect estimate with an appropriate interval when the goal includes effect size.