Why the Test Depends on the Null Hypothesis
A GLMM test is not chosen by asking only whether the model was fitted by likelihood or whether the sample is large. The key question is what parameter restriction defines the null hypothesis? A fixed-effect coefficient equal to zero is ordinarily an interior restriction. A variance component equal to zero lies at the edge of its allowed parameter space. Those different geometries can produce different reference distributions, even when both tests use a likelihood-ratio statistic.
A useful general representation is \(\eta_{ij}=x_{ij}^{T}\beta+z_{ij}^{T}b_j\), where \(Y_{ij}\), conditional on the random effects \(b_j\), has a distribution from an exponential-family model with link \(g\), and \(g\{E(Y_{ij}\mid b_j)\}=\eta_{ij}\). The random effects are typically modeled as independent across clusters, with \(b_j\sim N(0,D)\). Inference uses the likelihood after accounting for the random effects. The exact distribution and link are determined by the GLMM being fitted.
The usual Wald and LRT reference distributions require regularity conditions. In particular, the null value should be an interior point for the parameter being tested, nuisance parameters should be identifiable under the null, and the likelihood should be sufficiently well behaved for its local quadratic approximation to apply. These are asymptotic conditions, not guarantees for a small or weakly informative data set.
A Decision Framework
Is it a fixed-effect contrast, a variance component, a covariance or correlation, or a block of random-effect parameters? Record the alternative as well.
A fixed-effect coefficient can usually vary on either side of zero. A variance cannot be negative. Also ask whether parameters in the alternative remain identifiable when the null holds.
For an LRT, the null model must be nested in the alternative and both likelihoods must be fitted to the same observations using compatible likelihood calculations. For a Wald test, the estimate and its estimated covariance matrix must be sufficiently reliable.
Use the regular Wald or chi-square LRT approximation only when its conditions are defensible. For boundary or poorly calibrated cases, use a justified mixture, finite-sample calibration, or parametric bootstrap.
The framework does not make one procedure universally best. A Wald test is convenient for a prespecified contrast when its standard error is trustworthy. An LRT directly measures the loss of fit under a restriction, but its usual chi-square calibration can fail at a boundary. Boundary calibration is not a general correction that can be applied to every random-effect test: its form depends on the null and the model structure.
Fixed-Effect Tests Under Regular Conditions
Suppose the target is \(H_0:c^T\beta=r\), where \(c\) is a specified contrast and \(r\) is a hypothesized value. If the estimate \(\hat\beta\) is approximately normal and its estimated covariance matrix is \(V\), then the Wald statistic for one restriction is \(z=(c^T\hat\beta-r)/\sqrt{c^TVc}\). Under \(H_0\), it is approximately standard normal. For \(q\) restrictions, the quadratic Wald statistic is approximately chi-square with \(q\) degrees of freedom.
An LRT for the same fixed-effect question compares a null model and a larger nested alternative. With maximized log likelihoods \(\ell_0\) and \(\ell_1\), respectively, the statistic is \(G^2=2(\ell_1-\ell_0)\). If the restrictions are regular and number \(q\), \(G^2\) is approximately chi-square with \(q\) degrees of freedom under the null. For fixed-effect comparisons in a GLMM, fit the models by maximum likelihood on the same data; changing the fixed-effect structure changes the likelihood being compared.
Worked Example: Wald Test for One Fixed-Effect Contrast
In a hypothetical logistic GLMM, a treatment coefficient estimate is \(\hat\beta=0.60\) on the log-odds scale, with \(SE(\hat\beta)=0.25\). Test \(H_0:\beta=0\) against a two-sided alternative at \(\alpha=0.05\). Assume the fitted model is identifiable, the estimate is not at a boundary, and the estimated standard error is adequate for the normal approximation.
The Wald statistic is \(z=(0.60-0)/0.25=2.40\). Under \(H_0\), the reference distribution is approximately standard normal, so the two-sided p-value is \(2P(Z\geq 2.40)\approx 2(0.00820)=0.0164\), where \(Z\sim N(0,1)\).
Because \(0.0164<0.05\), reject \(H_0\) at the 5% significance level. The data provide evidence that this fixed-effect coefficient differs from zero on the model's log-odds scale. This test does not establish a causal treatment effect; that would require a design and assumptions supporting a causal interpretation.
Worked Example: LRT for a Two-Parameter Fixed-Effect Block
In a hypothetical GLMM, researchers compare a reduced model omitting two prespecified fixed-effect terms with a full model containing them. Both models use the same observations, the same random-effect structure, and maximum likelihood. Suppose the maximized log likelihoods are \(\ell_0=-216.1\) and \(\ell_1=-212.4\). Assume the reduced model is nested in the full model and the two restrictions are regular, identifiable fixed-effect restrictions.
The LRT statistic is \(G^2=2\{\ell_1-\ell_0\}=2\{-212.4-(-216.1)\}=2(3.7)=7.40\). There are \(q=2\) restrictions, so the regular reference distribution is chi-square with 2 degrees of freedom. Its upper-tail probability is \(P(\chi^2_2\geq7.40)=e^{-7.40/2}=e^{-3.70}\approx0.0247\).
Since \(0.0247<0.05\), reject the joint null that both terms are zero. This is evidence that the two-term block improves fit under the specified model and regular approximation; it does not by itself identify which term accounts for the improvement.
Boundary Nulls and the Need for Calibration
For a random-effect variance \(\tau^2\), the parameter space requires \(\tau^2\geq0\). Thus \(H_0:\tau^2=0\) is a boundary null, not an interior null. Applying the ordinary chi-square LRT reference distribution automatically can give a miscalibrated p-value.
A common special case has one tested variance component, with other parameters identifiable and regular under the null. In that case, the asymptotic LRT distribution can be a 50:50 mixture of a point mass at zero and a chi-square distribution with 1 degree of freedom. The mixture is specific to those conditions; more complicated random-effect structures may require different calibration.
Worked Example: LRT for a Random-Intercept Variance
A hypothetical analysis compares a GLMM with a random intercept to the corresponding model with the random-intercept variance fixed at zero. The models use the same fixed effects and observations. Suppose the LRT statistic is \(G^2=2.40\). Assume the only tested boundary parameter is this one variance component, nuisance parameters are identifiable under the null, and the asymptotic 50:50 mixture is appropriate.
For a positive observed statistic, the mixture upper-tail probability is one half the chi-square-1 upper tail: \(p=0.5P(\chi^2_1\geq2.40)\). Because a chi-square-1 variable is the square of a standard normal variable, \(P(\chi^2_1\geq2.40)=2\{1-\Phi(\sqrt{2.40})\}\approx0.121\). Thus \(p\approx0.5(0.121)=0.0607\).
At \(\alpha=0.05\), do not reject the zero-variance null using this calibration. The result is not proof that the variance is exactly zero; it means this test does not provide sufficient evidence against zero at the chosen significance level. If the mixture conditions or large-sample approximation are doubtful, a finite-sample calibration or parametric bootstrap may be more appropriate.
Comparing Procedures and Diagnosing Trouble
| Procedure | Best fit to the question | Key checks and limitations |
|---|---|---|
| Wald | A specified fixed-effect contrast with a reliable estimate and standard error | Can be sensitive to poor standard-error estimation, weak information, or an estimate far from the region where a normal approximation is reasonable. |
| Regular LRT | Nested models differing by regular, identifiable restrictions | Requires comparable likelihoods and the correct degrees of freedom; the ordinary chi-square reference is not automatic for boundary nulls. |
| Boundary-calibrated LRT | A variance-component null with a justified boundary reference distribution | The mixture weights and degrees of freedom depend on the tested structure and regularity conditions; there is no universal mixture for all random-effect tests. |
| Finite-sample calibration or parametric bootstrap | Situations where a simple asymptotic reference is questionable but a suitable null model can be fitted | Requires a defensible null model and enough simulated data sets for stable tail probabilities; the calibration inherits assumptions of that model. |
Before reporting a result, inspect whether the models converged, whether the likelihood comparison is genuinely nested, and whether the estimated covariance structure indicates a singular or weakly identified fit. A variance estimate near zero, a large standard error relative to a fixed-effect estimate, or unstable results across reasonable fitting choices are warnings that a routine asymptotic approximation may not describe the test statistic well. These warnings call for diagnosis and calibration, not an automatic change to a different test.
A particularly common mistake is to use a chi-square reference with one degree of freedom for every one-parameter LRT. The number of restrictions alone does not settle the reference distribution. If the restriction sets a variance to zero, the null is on the boundary, so the ordinary regular chi-square argument does not apply. Conversely, a fixed coefficient equal to zero is not automatically a boundary test merely because the model contains random effects.
Another pitfall is comparing fixed-effect models using likelihoods that are not comparable, for example because they were fitted to different observations or with incompatible likelihood calculations. Also, a small p-value is not a measure of effect size or practical importance. Report the parameter or block tested, the statistic, the reference distribution, the p-value, and the model conditions that justify that reference.
Check Your Understanding
For each item, identify the appropriate testing approach and explain the key condition. Solutions follow each question.
- A single fixed-effect coefficient is tested against zero. The estimate is 0.30 and its standard error is 0.12. What is the Wald statistic and the approximate two-sided p-value? Solution: \(z=0.30/0.12=2.50\). The standard-normal two-sided p-value is approximately \(2(0.00621)=0.0124\). This supports rejecting the zero-coefficient null at \(\alpha=0.05\), assuming the normal approximation and standard error are adequate.
- Two nested, maximum-likelihood GLMMs differ by three regular fixed-effect restrictions. What is the usual LRT reference distribution? Solution: Use a chi-square distribution with 3 degrees of freedom, provided both models use the same observations and comparable likelihoods, the restrictions are identifiable, and the regularity conditions hold.
- A random-intercept variance is tested against zero. Why is an ordinary chi-square-1 reference not automatically justified? Solution: A variance is constrained to be nonnegative, so zero is on the boundary. Under the special one-component conditions described above, the asymptotic reference is the 50:50 point-mass/chi-square-1 mixture; other structures may need another calibration.
- A fixed-effect model comparison produces \(G^2=4.00\) for one regular restriction. Compute the approximate p-value and state the decision at \(\alpha=0.05\). Solution: \(P(\chi^2_1\geq4.00)\approx0.0455\). Since \(0.0455<0.05\), reject the null at the 5% level, subject to the regular LRT assumptions.