Worked Comparison of Fixed-Effect Tests in a Binary GLMM

One Model, Three Fixed-Effect Tests

A fixed-effect question in a generalized linear mixed model (GLMM) can be tested in more than one way. A Wald test uses the estimated coefficient and its estimated uncertainty; a likelihood-ratio test compares the maximized likelihoods of nested models. These procedures often lead to similar conclusions in well-behaved, sufficiently large samples, but they are not identical calculations. Their differences matter when estimates are imprecise, the likelihood is asymmetric, or the number of independent clusters is modest.

This tutorial follows one hypothetical binary-outcome example through a coefficient Wald test, a joint Wald test, and a nested-model likelihood-ratio test. The comparisons concern fixed effects only. In every model comparison, the random-effects structure and analysis data are held fixed.

Model and assumptions: Let $Y_{ij}$ indicate remission (1 for remission, 0 otherwise) for participant $i$ in clinic $j$. For this example, assume $$ Y_{ij}\mid b_j \sim \operatorname{Bernoulli}(\pi_{ij}),\qquad \operatorname{logit}(\pi_{ij}) = \beta_0+\beta_T T_j+\beta_S S_{ij}+\beta_{TS}T_jS_{ij}+b_j, \qquad b_j\sim N(0,\tau^2). $$ Here $T_j=1$ for clinics assigned to treatment and 0 for control. $S_{ij}$ is baseline severity, centered at its sample mean and expressed in units of 10 severity points. The random intercepts are independent across clinics; conditional on them, outcomes are independent Bernoulli observations. The fixed-effects tests below use the same random-intercept structure, the same observations, and a correctly specified marginal likelihood. Wald reference distributions and the likelihood-ratio reference distribution are large-sample approximations under regularity conditions, including identifiable fixed effects and a nonsingular estimated covariance matrix.

Hypothetical Data and Fitted Models

Suppose a simulated dataset contains 40 clinics, with 10 participants per clinic. Twenty clinics are assigned to treatment and 20 to control. There are 124 remissions among the 200 treatment participants and 100 among the 200 control participants. These counts give useful context, but they do not by themselves account for baseline severity or within-clinic dependence. The tests use the participant-level data and the GLMM, not a test that treats all 400 outcomes as independent.

Assume the full model fit gives the estimates and uncertainty shown below. These are hypothetical fitted results, specified to illustrate the arithmetic; they are not claims about an actual study. The estimated random-intercept variance is $\hat{\tau}^2=0.36$. The conditional logit coefficient for treatment is evaluated at average baseline severity because $S=0$ represents the centered mean.

Fixed effect	Estimate	Standard error
Intercept, $\hat{\beta}_0$	0.20	0.18
Treatment, $\hat{\beta}_T$	0.55	0.25
Severity, $\hat{\beta}_S$	0.40	0.20
Treatment by severity, $\hat{\beta}_{TS}$	-0.15	0.15

Coefficient Wald Test

A coefficient Wald test asks whether one fixed-effect parameter equals a specified value, usually zero. For the treatment coefficient, the hypotheses are $H_0:\beta_T=0$ and $H_a:\beta_T\ne 0$. The estimate is divided by its standard error to obtain a Wald $z$ statistic. Squaring it gives a statistic with an approximate chi-square reference distribution with one degree of freedom.

Worked Example: Wald Test of the Treatment Coefficient

Test whether the treatment coefficient is zero in the full model, using a two-sided test at $\alpha=0.05$.

Calculate the statistic. The fitted treatment coefficient is 0.55 and its standard error is 0.25: $$ z=\frac{\hat{\beta}_T-0}{SE(\hat{\beta}_T)} =\frac{0.55}{0.25}=2.20, \qquad W=z^2=2.20^2=4.84. $$ Under $H_0$, the approximate reference distribution is $\chi^2_1$. Equivalently, the two-sided normal-reference p-value is about 0.0278.

Conclusion and interpretation. Since 0.0278 is less than 0.05, this test provides evidence against $H_0$ at the 5% level. At $S=0$, the estimated treatment contrast in conditional log odds is 0.55. Exponentiating gives a conditional odds ratio of $\exp(0.55)\approx1.73$: for participants at average baseline severity, the fitted remission odds under treatment are about 1.73 times those under control, conditional on the clinic random intercept. This is an association implied by the fitted model, not evidence by itself of a causal effect.

For context, a Wald 95% interval on the log-odds scale is $0.55\pm1.96(0.25)=[0.060,1.040]$. Exponentiating the endpoints gives an approximate conditional odds-ratio interval of $[1.06,2.83]$. The interval and the two-sided Wald test agree here because zero is excluded on the coefficient scale.

Joint Wald Test for a Block of Coefficients

A joint Wald test evaluates several restrictions at once. Here, test $H_0:\beta_S=\beta_{TS}=0$, while retaining the treatment and intercept terms. The null says that the model has neither a severity association nor a treatment-by-severity interaction on the conditional logit scale. The joint test must use the covariance between the estimates as well as their individual standard errors.

Worked Example: Joint Wald Test of Severity and Interaction

Suppose the estimated covariance matrix for $(\hat{\beta}_S,\hat{\beta}_{TS})^\mathsf{T}$ is $$ \widehat V= \begin{pmatrix} 0.0400 & 0.0060\\ 0.0060 & 0.0225 \end{pmatrix}. $$ Its diagonal entries are the squared standard errors, $0.20^2$ and $0.15^2$. The estimate vector is $\hat{\boldsymbol{\beta}}=(0.40,-0.15)^\mathsf{T}$.

Calculate the statistic. For the null vector $\boldsymbol{0}$, the joint Wald statistic is $\hat{\boldsymbol{\beta}}^\mathsf{T}\widehat V^{-1}\hat{\boldsymbol{\beta}}$. The determinant is $0.0400(0.0225)-0.0060^2=0.000864$, so $$ \widehat V^{-1} =\frac{1}{0.000864} \begin{pmatrix} 0.0225 & -0.0060\\ -0.0060 & 0.0400 \end{pmatrix}. $$ Consequently, $$ W= \frac{0.0225(0.40)^2-2(0.0060)(0.40)(-0.15)+0.0400(-0.15)^2} {0.000864} =\frac{0.005220}{0.000864} =6.0417. $$ There are two restrictions, so the reference distribution is approximately $\chi^2_2$. Its upper-tail p-value is $\exp(-6.0417/2)\approx0.0487$.

Conclusion. The joint test provides evidence against the two restrictions at the 5% level, although the p-value is close to that threshold. This is a single test of the block; it does not establish that each coefficient is individually nonzero. In particular, the separate estimates and standard errors answer different questions from the joint statistic, which accounts for their covariance.

Nested-Model Likelihood-Ratio Test

The same two restrictions can be tested by comparing the full model with a reduced model that removes severity and the interaction but retains the intercept, treatment effect, and random intercept. The models are nested: the reduced model is obtained from the full model by setting $\beta_S=\beta_{TS}=0$. Fit both models by maximum likelihood to the same observations, using the same link and random-effects structure. For a binary GLMM, the likelihood being compared is the likelihood with the random effects integrated out, not a likelihood that treats estimated random effects as known.

Let $\ell_F$ and $\ell_R$ be the maximized log likelihoods for the full and reduced models. The likelihood-ratio statistic is $G^2=2(\ell_F-\ell_R)$. Under the null and regularity conditions for a fixed-effect comparison, it is approximately $\chi^2_q$, where $q$ is the number of independent restrictions.

Worked Example: Likelihood-Ratio Test of the Same Block

Suppose maximum-likelihood fitting gives $\ell_F=-128.40$ for the full model and $\ell_R=-132.05$ for the reduced model. The estimated random-intercept variance is allowed to be re-estimated in each fit; suppose the fitted values are 0.36 and 0.38, respectively.

Calculate the statistic. $$ G^2=2\{\ell_F-\ell_R\} =2\{-128.40-(-132.05)\} =2(3.65)=7.30. $$ The models differ by two fixed-effect restrictions, so the reference distribution is $\chi^2_2$. Its upper-tail probability is $\exp(-7.30/2)\approx0.0260$.

Conclusion. The likelihood-ratio test provides evidence against the reduced model at the 5% level. It indicates that allowing the severity and interaction terms improves the fitted marginal likelihood enough to be inconsistent with the two restrictions under the approximate null reference distribution. It does not say which term is responsible for the improvement.

Why the Results Can Differ

The joint Wald test and likelihood-ratio test address the same two restrictions in this example, but they use different information. The Wald statistic measures the estimated distance from the null in units of the estimated local covariance matrix. The likelihood-ratio statistic measures the drop in maximized log likelihood when the restrictions are imposed. Here the Wald p-value is about 0.0487 and the likelihood-ratio p-value is about 0.0260. Both reject at 5%, but they do not give the same numerical evidence.

A short justification shows why agreement is expected only approximately. Near the unrestricted maximum, a sufficiently regular log likelihood can be approximated by a quadratic function. If $\hat{\boldsymbol{\beta}}$ is close to the null value and the curvature is well estimated, the loss in log likelihood from imposing the null is approximately half the covariance-standardized squared distance to it. Thus twice that loss is approximately the Wald quadratic form. When the likelihood is noticeably non-quadratic, or the estimates and covariance are unstable, this local approximation can be poor. This explains why the tests may differ without either calculation being arithmetically wrong.

Comparison: A coefficient Wald test is convenient for one parameter and uses its estimate and standard error. A joint Wald test is convenient for a block and uses the full covariance submatrix. A likelihood-ratio test compares nested fits and can reflect likelihood curvature beyond the local quadratic approximation. These tests rely on large-sample reference distributions; none automatically resolves small-sample or model-specification problems.

Diagnostics, Pitfalls, and Limits

Before interpreting any of the three tests, check that the fits are trustworthy. Confirm convergence, inspect whether the estimated covariance matrix is positive definite, and look for unusually large standard errors or coefficients that may signal weak identification or sparse-data problems. Check that both likelihood fits used the same observations and the same random-effects structure. The full model should not have a lower maximized likelihood than its nested reduced model, apart from negligible numerical optimization error. If it does, investigate the fitting procedure before calculating the likelihood-ratio statistic.

The chi-square and normal reference distributions are approximations, not guarantees. A modest number of independent clinics, sparse outcomes in parts of the predictor space, separation-like patterns, or a poorly specified random-effects distribution can make reported p-values unreliable. Repeated observations within a clinic do not provide the same information about between-clinic variation as an equal number of independent clinics. Interpret the coefficient on the scale of the model: with an interaction, $\beta_T$ is the treatment contrast only at $S=0$, not a constant treatment contrast at every severity.

Wrong: “The treatment p-value is below 0.05, so treatment has a constant effect at every severity.” Correction: The treatment coefficient is the conditional log-odds contrast at centered severity $S=0$; the interaction allows that contrast to vary with severity.
Wrong: “The joint test is significant, so both severity terms are significant.” Correction: A joint rejection says that the block is inconsistent with both coefficients being zero; it does not establish that each coefficient separately differs from zero.
Wrong: “The likelihood-ratio test is significant, so the reduced model is impossible.” Correction: The result is evidence against the restrictions under the specified model and reference approximation; it does not prove that one model is true.
Wrong: “The two p-values differ, so one procedure must be incorrect.” Correction: Wald and likelihood-ratio statistics use different summaries of the fitted likelihood and can differ, especially outside ideal large-sample conditions.

These tests answer prespecified fixed-effect questions. If many coefficients or blocks are tested as a family, interpret individual p-values in light of the planned multiplicity adjustment. Also distinguish statistical evidence from practical importance: a small p-value does not by itself establish a clinically meaningful change in remission odds.

Check Your Understanding

Try each question, then compare your reasoning with the solutions.

A coefficient estimate is 0.30 with standard error 0.20. Calculate its two-sided Wald statistic and approximate p-value using a one-degree-of-freedom chi-square reference.
How many degrees of freedom are used for a joint Wald test of three independent coefficient restrictions?
Two nested models have maximized log likelihoods $-50.2$ and $-53.0$, with the first value belonging to the larger model. Calculate the likelihood-ratio statistic and name its null reference degrees of freedom if the models differ by two fixed effects.
Why must the likelihood-ratio comparison use the same observations and random-effects structure in both models?
In the model above, what does the treatment coefficient represent when the treatment-by-severity interaction is included?

Solutions. (1) $z=0.30/0.20=1.50$, so $W=1.50^2=2.25$. The two-sided normal-reference p-value is about 0.134, so the test does not reject at 0.05. (2) The reference distribution is $\chi^2_3$, because there are three restrictions. (3) $G^2=2\{-50.2-(-53.0)\}=5.6$, with an approximate $\chi^2_2$ reference distribution. (4) Changing the observations or random-effects structure changes what likelihoods are being compared, so their difference no longer isolates the stated fixed-effect restrictions. (5) It is the conditional treatment log-odds contrast at $S=0$, that is, at average centered baseline severity, holding the clinic random intercept fixed.

Fixed effect	Estimate	Standard error
Intercept, \(\hat{\beta}_0\)	0.20	0.18
Treatment, \(\hat{\beta}_T\)	0.55	0.25
Severity, \(\hat{\beta}_S\)	0.40	0.20
Treatment by severity, \(\hat{\beta}_{TS}\)	-0.15	0.15