Wald and Score Tests for Candidate Terms

Why Test a Candidate Term?

A GLM can include a term because it is plausible, because it improves fit, or because the analysis needs to assess whether it contributes evidence after accounting for other predictors. A test of a candidate term addresses a specific question: under the assumed model, are the data inconsistent with setting that term's coefficient or coefficients to zero?

Wald, score, and likelihood-ratio tests address this same null hypothesis in different ways. The Wald test evaluates the estimated term in the unrestricted model. The score test evaluates the evidence for moving away from the restricted model, without requiring the unrestricted fit. The likelihood-ratio test compares the maximized likelihoods of both fits. In large samples under regularity conditions, their test statistics have the same limiting chi-square distribution, but their finite-sample values need not agree.

Definition: Suppose a GLM has parameter vector $\theta=(\gamma,\beta)$, where $\gamma$ contains nuisance or other model coefficients and $\beta$ contains the candidate term's $q$ coefficients. The term test has null hypothesis $H_0:\beta=0$ and alternative $H_1:\beta\ne 0$. If the model and data satisfy the regularity conditions for the test, each statistic is compared with a chi-square distribution with $q$ degrees of freedom.

Three Tests, Three Evaluation Points

Let $\ell(\theta)$ denote the log likelihood. The restricted fit maximizes it subject to $H_0$; the unrestricted fit maximizes it without that constraint. Assume the likelihood is correctly specified, the observations are independent, the null parameter is an interior point, and the relevant parameters are identifiable. The usual chi-square reference distributions are large-sample approximations, not exact guarantees for every data set.

Wald test: Use the unrestricted estimate $\hat\beta$ and its estimated covariance matrix $\widehat{\operatorname{Var}}(\hat\beta)$: $$ W=\hat\beta^\mathsf{T}\left[\widehat{\operatorname{Var}}(\hat\beta)\right]^{-1}\hat\beta. $$ For one coefficient, this reduces to $W=(\hat\beta/\operatorname{SE}(\hat\beta))^2$. The statistic measures how far the unrestricted estimate is from zero in estimated standard-error units.

Score test: Evaluate the likelihood's score and information at the restricted fit. If $\gamma$ is estimated under the null, the efficient score and information for $\beta$ are $$ U_{\beta\cdot\gamma}=U_\beta-I_{\beta\gamma}I_{\gamma\gamma}^{-1}U_\gamma,\qquad I_{\beta\cdot\gamma}=I_{\beta\beta}-I_{\beta\gamma}I_{\gamma\gamma}^{-1}I_{\gamma\beta}. $$ The score statistic is $S=U_{\beta\cdot\gamma}^{\mathsf T}I_{\beta\cdot\gamma}^{-1}U_{\beta\cdot\gamma}$. At a restricted maximum-likelihood fit, $U_\gamma=0$, but the adjustment to the information still accounts for estimating nuisance parameters.

Likelihood-ratio test: Compare the maximized log likelihoods: $$ G^2=2\left\{\ell(\hat\theta_{\mathrm{unrestricted}})-\ell(\hat\theta_{\mathrm{restricted}})\right\}. $$ Because the unrestricted model includes the restricted model as the special case $\beta=0$, its maximized log likelihood cannot be smaller. Thus $G^2\geq 0$. A negative result usually signals a calculation or model-fitting error.

For all three tests, a small upper-tail p-value is evidence against $H_0$, conditional on the specified model and its assumptions. The tests do not establish that a term is causally important, that the model is well specified, or that it predicts well on new data. A test of a multi-coefficient factor or interaction uses the number of jointly tested coefficients as $q$; it is not generally correct to test just one coefficient and describe that as a test of the whole term.

Why the Tests Are Related

The relationship is clearest in a one-parameter test after accounting for nuisance coefficients. Let $\ell_p(\beta)$ be the log likelihood profiled over nuisance parameters, and suppose $H_0:\beta=0$. Near zero, a second-order Taylor approximation gives

\ell_p(\beta)\approx \ell_p(0)+U_{\beta\cdot\gamma}\beta-\frac{1}{2}I_{\beta\cdot\gamma}\beta^2.

Here the score and information are evaluated at the restricted fit. Maximizing this quadratic approximation gives $\hat\beta\approx U_{\beta\cdot\gamma}/I_{\beta\cdot\gamma}$. Substituting that value into twice the approximate log-likelihood increase gives

2\{\ell_p(\hat\beta)-\ell_p(0)\} \approx 2\left(U_{\beta\cdot\gamma}\frac{U_{\beta\cdot\gamma}}{I_{\beta\cdot\gamma}} -\frac{1}{2}I_{\beta\cdot\gamma}\frac{U_{\beta\cdot\gamma}^2}{I_{\beta\cdot\gamma}^2}\right) =\frac{U_{\beta\cdot\gamma}^2}{I_{\beta\cdot\gamma}}.

The right-hand side is the score statistic, while the left-hand side is the likelihood-ratio statistic under the approximation. Near the null, the Wald statistic is also approximately this quantity. This explains their large-sample agreement: the likelihood is locally close to a quadratic. When that local approximation is poor, or when estimates are imprecise or unstable, the three calculations can differ.

Worked Examples

Worked Example: Comparing Tests for a Poisson Group Effect

Method: Fit a Poisson model with a common rate under the null and separate group rates under the alternative. Suppose two hypothetical groups each have exposure 1, with observed counts 20 and 30. Test whether adding a group indicator improves the model. Assume independent Poisson counts, a log link, and no other predictors. Use a 5% significance level.

Under the null, the common rate estimate is $(20+30)/(1+1)=25$. The restricted expected count in each group is therefore 25. Under the alternative, the fitted counts are 20 and 30, so the estimated log rate ratio is $\hat\beta=\log(30/20)=\log(1.5)=0.405465$.

Wald calculation. For independent Poisson counts, the large-sample variance of the log rate ratio is approximately $1/20+1/30=1/12=0.083333$. Thus

W=\frac{(\log(1.5))^2}{1/12} =(0.405465)^2(12) \approx 1.9728.

The one-degree-of-freedom chi-square upper-tail p-value is approximately 0.1601. The Wald test does not reject $H_0$ at 5%.

Score calculation. At the restricted fit, the candidate-group count is 30 and its expected count is 25, giving a group score of $30-25=5$. Because the common rate is estimated, the efficient information for the group contrast is $25(25)/(25+25)=12.5$. Therefore

S=\frac{5^2}{12.5}=2.0000.

The one-degree-of-freedom upper-tail p-value is approximately 0.1573, so the score test also does not reject.

Likelihood-ratio calculation. The restricted and unrestricted maximized log likelihoods differ by the Poisson terms involving the fitted means; terms common to both fits cancel. Hence

G^2=2\left\{20\log(20/25)+30\log(30/25)\right\} =2\left\{20\log(0.8)+30\log(1.2)\right\} \approx 2.0136.

The one-degree-of-freedom upper-tail p-value is approximately 0.1559. The likelihood-ratio test also does not reject. All three tests give the same decision here, although their statistics and p-values differ.

Common mistake avoided: Using observed counts to calculate restricted information would give $20(30)/(20+30)=12$, not the information at the restricted fit. The score test evaluates information under the null: both expected counts are 25, so the correct calculation is $25(25)/50=12.5$.

Worked Example: Testing a Logistic Treatment Term

Method: Compare a common event probability with a model that has a treatment indicator. In a hypothetical randomized study, the control group has 40 events and 60 non-events; the treatment group has 60 events and 40 non-events. Test the treatment coefficient using all three methods. Assume independent Bernoulli outcomes and a logit link.

Under the null, both groups have event probability $100/200=0.5$. Under the alternative, the fitted probabilities are $0.4$ in control and $0.6$ in treatment. The estimated treatment log odds ratio is $\hat\beta=\log(60/40)-\log(40/60)=2\log(1.5)=0.810930$.

Wald calculation. The estimated variance of the log odds ratio is the sum of the four reciprocal cell counts: $1/60+1/40+1/40+1/60=1/12$. Therefore

W=\frac{(0.810930)^2}{1/12}\approx 7.8913.

The one-degree-of-freedom p-value is approximately 0.0050, so the Wald test rejects at 5%.

Score calculation. Under the null, the treatment group has 50 expected events. Its observed-minus-expected score is $60-50=10$. The efficient information for the group contrast, after estimating the common probability, is $25(25)/(25+25)=12.5$, since each group has size 100 and the null Bernoulli variance is $0.5(0.5)=0.25$. Thus

S=\frac{10^2}{12.5}=8.0000.

The one-degree-of-freedom p-value is approximately 0.0047, so the score test rejects.

Likelihood-ratio calculation. The alternative log likelihood is $80\log(0.4)+120\log(0.6)$, and the null log likelihood is $200\log(0.5)$. Their difference is approximately $4.0271$, giving

G^2=2\{80\log(0.4)+120\log(0.6)-200\log(0.5)\} \approx 8.0542.

The one-degree-of-freedom p-value is approximately 0.0045. The likelihood-ratio test rejects as well. This example shows that the three statistics can be numerically distinct even when their decisions agree.

Common mistake avoided: The score test does not use the unrestricted fitted probabilities 0.4 and 0.6 to form its score. It evaluates departure from the restricted probability 0.5; the treatment group's residual is $60-50=10$.

Worked Example: A Two-Coefficient Term Near the Decision Boundary

Method: Test a hypothetical two-coefficient term jointly, using model summaries from a fitted GLM. Suppose the unrestricted estimates are $\hat\beta=(0.20,0.20)^\mathsf{T}$, with covariance matrix having diagonal entries 0.01 and 0.02 and zero off-diagonal entries. Suppose the restricted fit gives efficient score vector $U=(3,4)^\mathsf{T}$ and efficient information matrix with diagonal entries 2 and 16 and zero off-diagonal entries. The unrestricted and restricted log likelihoods are $-198.10$ and $-201.00$, respectively. Use a 5% level.

Wald calculation. Inverting the diagonal covariance matrix gives diagonal entries 100 and 50. Therefore

W=\frac{0.20^2}{0.01}+\frac{0.20^2}{0.02}=4+2=6.0000.

With $q=2$, the p-value is $\exp(-6/2)\approx 0.0498$. The Wald test narrowly rejects at 5%.

Score calculation. Inverting the stated efficient information matrix gives diagonal entries $1/2$ and $1/16$. Thus

S=\frac{3^2}{2}+\frac{4^2}{16}=4.5+1=5.5000.

For two degrees of freedom, the p-value is $\exp(-5.5/2)\approx 0.0639$. The score test does not reject.

Likelihood-ratio calculation. The difference between the maximized log likelihoods is $-198.10-(-201.00)=2.90$, so

$$ G^2=2(2.90)=5.8000. $$

The two-degree-of-freedom p-value is $\exp(-5.8/2)\approx 0.0550$, so the likelihood-ratio test does not reject at 5%. The Wald test alone rejects in this hypothetical summary. The methods assess the same null but use different estimates and approximations; near a cutoff, modest numerical differences can change the decision.

Common mistake avoided: A two-coefficient term requires a joint test with two degrees of freedom. Comparing each coefficient separately with a one-degree-of-freedom cutoff would not answer the stated joint question.

Choosing and Interpreting a Test

The score test can be convenient when the restricted model is easy to fit but the unrestricted fit is difficult. The Wald test is readily obtained from the unrestricted estimate and covariance matrix, but it can be sensitive to parameterization and unreliable when estimates are unstable. The likelihood-ratio test requires both fits, but directly measures the change in maximized log likelihood. Under standard large-sample conditions none is universally superior for every practical data set.

The chi-square approximation deserves particular caution when data are sparse, a logistic model has separation or near-separation, or a coefficient is poorly identified. The earlier material on separation and unstable logistic coefficients and on sparse factor levels is relevant: in such settings, ordinary standard errors and large-sample reference distributions may be unreliable. Likewise, the earlier material on overdispersion matters when the assumed GLM variance is inadequate; a test based on a misspecified likelihood can give misleading evidence.

A careful report names the term, null hypothesis, test, degrees of freedom, statistic, p-value, and decision threshold. If tests disagree, report the disagreement rather than selecting whichever result supports a preferred conclusion. Also separate statistical evidence from practical relevance: a small p-value does not by itself quantify the size or predictive value of the effect.

Specify the candidate term.
State $H_0:\beta=0$ and the number $q$ of coefficients being tested jointly.

Identify the evaluation point.
Use the unrestricted estimate for Wald, the restricted fit for score, and both maximized fits for likelihood ratio.

Compare on the correct reference distribution.
Under the regularity conditions, compare each statistic with chi-square $q$, and state the approximate nature of the p-value.

Check stability and context.
Consider sparse data, convergence, separation, and variance assumptions before treating a nominal p-value as persuasive evidence.

Key takeaway: Wald, score, and likelihood-ratio tests assess the same candidate-term null from different points in the likelihood. Their asymptotic agreement does not require identical finite-sample results; the null hypothesis, degrees of freedom, model assumptions, and stability of the fit must all be checked.

Check Your Understanding

For each problem, calculate or interpret the requested test using the stated large-sample chi-square reference distribution.

A one-coefficient GLM term has estimate 0.30 and standard error 0.15. Calculate the Wald statistic, its one-degree-of-freedom p-value, and the decision at 5%. (Answer: $W=4.0000$, $p\approx0.0455$; reject at 5%.)
At a restricted fit, a candidate coefficient has efficient score 6 and efficient information 8. Calculate the score statistic and p-value for one degree of freedom. (Answer: $S=4.5000$, $p\approx0.0339$.)
A two-coefficient term has restricted log likelihood $-91.0$ and unrestricted log likelihood $-88.4$. Calculate the likelihood-ratio statistic, degrees of freedom, and p-value. (Answer: $G^2=5.2000$, 2 degrees of freedom, $p\approx0.0743$.)
For a two-coefficient term, the Wald, score, and likelihood-ratio statistics are 7.1, 6.4, and 6.8. Calculate their p-values and decisions at 5%. (Answer: p-values are approximately 0.0287, 0.0408, and 0.0334, respectively; all three reject at 5%.)
A candidate term's unrestricted GLM fit has a convergence warning and a very large standard error; the restricted fit is stable. Which test is directly based on the restricted fit, and what caution applies to a usual Wald conclusion? (Answer: The score test is based on the restricted fit; the unstable unrestricted estimate makes the Wald statistic and its usual approximation questionable.)

What You'll Learn