Choosing a Test for a GLM Restriction
A test result depends not only on the data but also on the restriction being tested, the model fit used to construct the statistic, and the approximation used for its p-value. In a generalized linear model (GLM), the Wald, score, and likelihood-ratio tests often agree in large samples, yet can differ noticeably with sparse data, weak information, or estimates near a parameter-space boundary. A defensible analysis chooses a test for a stated reason, rather than selecting the one that gives the smallest p-value.
Let \(\boldsymbol\beta\) denote the model parameter vector and \(\ell(\boldsymbol\beta)\) its log-likelihood. Consider testing \(H_0:r(\boldsymbol\beta)=\boldsymbol 0\), where the restriction imposes \(q\) independent constraints. For a linear restriction, \(r(\boldsymbol\beta)=A\boldsymbol\beta-\boldsymbol a\), with \(A\) of rank \(q\). The unrestricted model allows all parameter values in its parameter space; the restricted model allows only values satisfying \(H_0\).
For a regular model and a fixed number \(q\) of restrictions, each statistic is commonly compared with a \(\chi^2_q\) distribution under the null. This is an asymptotic calibration, not a guarantee of accurate small-sample p-values. The tests also require that the null be a regular, interior restriction and that the relevant information matrices are nonsingular. For the likelihood-ratio result, the restricted model must be nested in the unrestricted one.
What Each Test Requires
| Test | Fit needed | Statistic for a regular \(q\)-restriction | Practical consideration |
|---|---|---|---|
| Wald | Unrestricted fit and its covariance estimate | Quadratic distance from the null restriction; reference \(\chi^2_q\) | Convenient after a full fit, but can depend strongly on parameterization and behave poorly when estimates are unstable. |
| Score | Restricted fit and score and information there | Efficient score quadratic form; reference \(\chi^2_q\) | Useful when fitting the unrestricted model is difficult; it assesses departure from the null locally. |
| Likelihood-ratio | Restricted and unrestricted fits | Twice the log-likelihood gain; reference \(\chi^2_q\) | Uses the change in fit across the restriction, but requires both fits and can still be inaccurate in small or sparse samples. |
For a linear restriction \(A\boldsymbol\beta=\boldsymbol a\), write \(\widehat{\operatorname{Var}}(\hat{\boldsymbol\beta})\) for the estimated covariance matrix of the unrestricted estimator. The Wald statistic is
The score test uses the efficient score and information for the tested parameters after accounting for nuisance parameters. Denote these by \(\boldsymbol U^*\) and \(I^*\), evaluated at the restricted fit. Then \(S=(\boldsymbol U^*)^\mathsf{T}(I^*)^{-1}\boldsymbol U^*\). The likelihood-ratio statistic is \(G^2=2\{\ell(\hat{\boldsymbol\beta})-\ell(\tilde{\boldsymbol\beta})\}\), where the tilded estimate maximizes the log-likelihood subject to the null. In all three cases, the reported degrees of freedom are the number \(q\) of independent restrictions, not the number of observations or the number of coefficients in the entire model.
Why the Wald Reference Distribution Is Chi-Square
The chi-square calibration for the Wald statistic follows directly from asymptotic normality. Suppose the model is correctly specified and identifiable, the true parameter is an interior point, the information is nonsingular, and the usual conditions for asymptotic normality and consistent covariance estimation hold. Let \(V\) be the asymptotic covariance matrix, so that \(\sqrt n(\hat{\boldsymbol\beta}-\boldsymbol\beta)\xrightarrow{d}N(\boldsymbol 0,V)\). Suppose \(A\) has rank \(q\), and the null \(A\boldsymbol\beta=\boldsymbol a\) is true.
To see this, let \(C=AVA^\mathsf{T}\). Since \(V\) is positive definite and \(A\) has rank \(q\), \(C\) is positive definite. Under the null,
Consistency of the covariance estimator gives \(nA\widehat{\operatorname{Var}}(\hat{\boldsymbol\beta})A^\mathsf{T}\xrightarrow{p}C\). Rewriting \(W\) using \(\boldsymbol Z_n\), then applying Slutsky's theorem, shows that its limit is the quadratic form \(\boldsymbol Z^\mathsf{T}C^{-1}\boldsymbol Z\), where \(\boldsymbol Z\sim N(\boldsymbol 0,C)\). The vector \(C^{-1/2}\boldsymbol Z\) has \(q\) independent standard normal components. Its squared length is therefore \(\chi^2_q\), proving the result. \(\square\)
This argument explains why the test uses \(q\) degrees of freedom: the restriction has \(q\) independent directions of departure. The corresponding large-sample calibrations for score and likelihood-ratio tests are standard consequences of regular likelihood theory. They do not imply that all three tests give identical finite-sample answers.
Choosing and Reporting in Practice
First specify the scientific restriction and count its independent constraints. Then choose a test based on the fitting task and the model's regularity, not on the p-value each method happens to produce. A Wald test is natural when the unrestricted estimate and its covariance are reliable. A score test can be useful when the restricted model is easy to fit but the full model is difficult. A likelihood-ratio test is a natural comparison when both nested models can be fit reliably. If sparsity, separation, boundary estimates, or weak identification undermine the usual approximations, say so and consider an appropriate method rather than treating a routine chi-square p-value as exact.
A complete report names the null restriction, the test and statistic, degrees of freedom, p-value, and effect estimate in interpretable units. For a single log-link coefficient, for example, \(\exp(\hat\beta_j)\) is the estimated multiplicative change in the mean for a one-unit increase in that predictor, with other model terms held fixed. For a logit link, the corresponding exponentiated coefficient is an odds ratio. For a joint test, report the relevant coefficient estimates or transformed effects as well as the joint test: one p-value cannot show which effects are large, in what direction they point, or whether they matter practically.
Worked Example: Reporting a Wald Test for a Poisson Rate Ratio
A Poisson regression with a log link estimates an exposure coefficient as \(\hat\beta=0.35\), with estimated standard error \(0.14\). Test \(H_0:\beta=0\) using a two-sided Wald test, and report the estimated rate ratio.
There is one restriction, so the asymptotic reference distribution is \(\chi^2_1\). The Wald statistic is
Equivalently, the signed normal statistic is \(z=2.5\). Thus the two-sided p-value is approximately \(2\{1-\Phi(2.5)\}=0.0124\), where \(\Phi\) is the standard normal distribution function. The rate-ratio estimate is \(\exp(0.35)=1.419\), or about a 42% higher rate per one-unit increase in exposure. A Wald 95% interval for the coefficient is \(0.35\pm1.96(0.14)=[0.0756,0.6244]\); exponentiating its endpoints gives an approximate rate-ratio interval \([1.079,1.867]\).
A suitable report is: “The estimated rate ratio was 1.42 per exposure unit (Wald 95% CI 1.08–1.87); the Wald test of \(H_0:\beta=0\) gave \(\chi^2_1=6.25\), \(p=0.012\).” The estimate conveys direction and size; the test alone would not.
Worked Example: Using a Score Test for Two Restrictions
Suppose a restricted GLM fit tests two coefficients jointly equal to zero. At that fit, the efficient score is \(\boldsymbol U^*=(3,2)^\mathsf{T}\), and the efficient information is \(I^*=\begin{pmatrix}4&0\\0&1\end{pmatrix}\). Calculate and report the score test.
The information matrix is invertible, with \((I^*)^{-1}=\begin{pmatrix}1/4&0\\0&1\end{pmatrix}\). Therefore,
There are two independent restrictions, so the reference distribution is \(\chi^2_2\). For two degrees of freedom, the upper-tail probability is \(\exp(-S/2)\); hence \(p=\exp(-3.125)\approx0.0439\). The score test rejects at the 5% level using this asymptotic calibration. It does not itself provide unrestricted coefficient estimates: report those estimates separately if the full model can be fit, and describe the score test as evidence about the joint null rather than evidence that each coefficient is individually nonzero.
Worked Example: A Joint Likelihood-Ratio Test
A Poisson regression compares a model without two predictors to a nested model that includes both. The restricted log-likelihood is \(-106.1\), and the unrestricted log-likelihood is \(-103.1\). The estimated coefficients in the unrestricted model are \(0.20\) and \(-0.15\). Calculate the likelihood-ratio test and explain what the estimates add.
The unrestricted model has two additional freely estimated coefficients, so the null imposes \(q=2\) restrictions. The likelihood-ratio statistic is
Its asymptotic reference distribution is \(\chi^2_2\), giving \(p=\exp(-6/2)=\exp(-3)\approx0.0498\). The test is just below the 5% threshold. Under the log link, the two estimated multiplicative effects are \(\exp(0.20)\approx1.221\) and \(\exp(-0.15)\approx0.861\), holding other terms fixed. The joint test assesses whether both coefficients are zero; it does not show that both effects are individually distinguishable from zero. Their sizes and directions should be reported without turning the joint p-value into two separate conclusions.
Common Mistakes
- Choosing whichever test is significant. Different p-values do not justify reporting only the preferred one. State a principled choice in advance or report the relevant tests transparently, with a reason for emphasizing one.
- Using the wrong degrees of freedom. Count independent restrictions. A joint test of two independent coefficients has two degrees of freedom, even if the model has many more parameters.
- Reporting only “significant” or “not significant.” Give the test statistic, degrees of freedom, p-value, and the estimated effect in context. Do not equate a threshold crossing with practical importance.
- Treating asymptotic p-values as exact in sparse data. The chi-square approximation can be poor when the regularity conditions are doubtful. Flag that limitation and use a method suited to the data and model.
Write the null clearly and determine the number of independent constraints.
Use the unrestricted fit for Wald, the restricted fit for score, and both nested fits for likelihood-ratio.
Assess whether regularity and adequate information support the chi-square reference distribution.
Give the estimate in interpretable units along with the statistic, degrees of freedom, and p-value.
Check Your Understanding
For each question, explain your reasoning rather than giving only a test name or number.
- A model's unrestricted fit is stable, but fitting the restricted model is computationally difficult. Which test might be convenient, and what does it require?
- A null hypothesis imposes three independent linear restrictions. What degrees of freedom are used for the usual asymptotic chi-square calibration?
- Explain why a small joint-test p-value does not establish that every coefficient in the tested set is individually important.
- For a log-link GLM, a coefficient estimate is \(0.4\). What multiplicative effect does its exponentiated value describe?
- Why is it not sound practice to calculate all three tests and report only the one with the smallest p-value?