Wald Tests for Single GLM Coefficients

Why Test One Coefficient?

A GLM coefficient describes a component of the linear predictor, so a test about that coefficient asks whether the data are compatible with a specified value on that scale. For example, testing whether a logistic-regression slope is zero is equivalent to testing whether its conditional odds ratio is one. Testing a Poisson-regression slope against zero asks whether the corresponding rate ratio is one. The Wald test uses the estimated coefficient and its standard error to measure how far the estimate is from the null value in estimated standard-error units.

This tutorial treats a two-sided test of one coefficient. The test is an asymptotic procedure: its calibration comes from the large-sample distribution of the estimator, not from a claim that the statistic has an exact finite-sample distribution.

Definition and Test Statistic

For a maximum-likelihood fit, the standard error is typically the square root of the \(j\)th diagonal entry of an estimated covariance matrix for \(\hat{\beta}\). It must be the standard error appropriate to the fitted model and inferential assumptions. When a valid sandwich covariance estimate is used, its diagonal entry supplies the corresponding squared standard error.

The usual asymptotic justification requires a regular, identifiable model with the true parameter in the interior of the parameter space, information that grows with sample size, and an estimator with an asymptotically normal distribution. The estimated variance must consistently estimate the variance used in that normal approximation. Under those conditions, and when \(H_0\) is true, $$ Z_W\xrightarrow{d}N(0,1). $$ In a correctly specified regular GLM, this is the familiar large-sample likelihood result; under misspecification, the covariance estimate must account for the actual asymptotic variance. Boundary or other nonregular cases need not have this calibration.

For a level-\(\alpha\) two-sided test, where \(0<\alpha<1\), let \(z_{1-\alpha/2}\) denote the \(1-\alpha/2\) quantile of the standard normal distribution. Reject \(H_0\) when \(|Z_W|>z_{1-\alpha/2}\). Equivalently, the asymptotic p-value is \(2[1-\Phi(|Z_W|)]\), where \(\Phi\) is the standard normal distribution function.

Why the Squared Statistic Has a Chi-Square Form

Proof. Under \(H_0\), the standardized estimator satisfies \(Z_W\xrightarrow{d}Z\), where \(Z\sim N(0,1)\). The function \(g(z)=z^2\) is continuous, so the continuous mapping theorem gives \(Z_W^2\xrightarrow{d}Z^2\). By definition, the square of a standard normal random variable has a chi-square distribution with one degree of freedom. Since \(W=Z_W^2\), the stated limit follows. \(\square\)

The two-sided normal and upper-tail chi-square calculations agree because, for any \(c\ge 0\), $$ P(\chi^2_1\ge c^2)=P(Z^2\ge c^2)=P(|Z|\ge c)=2[1-\Phi(c)]. $$ In particular, the asymptotic p-value can also be written \(P(\chi^2_1\ge W)\). The sign of \(Z_W\) is useful for seeing the direction of the estimate’s departure from the null value; squaring removes that sign but preserves the two-sided evidence.

Coefficient-Scale Confidence Intervals

The two-sided Wald test is directly connected to an asymptotic confidence interval for \(\beta_j\). This connection is useful both for checking calculations and for interpreting which null values are compatible with the test at a given level.

Derivation. The test rejects when $$ \left|\frac{\hat{\beta}_j-\beta_{j,0}}{\widehat{\operatorname{se}}(\hat{\beta}_j)}\right|>z_{1-\alpha/2}. $$ The estimated standard error is positive, so multiplying by it preserves the inequality: $$ |\hat{\beta}_j-\beta_{j,0}|>z_{1-\alpha/2}\widehat{\operatorname{se}}(\hat{\beta}_j). $$ The absolute-value inequality holds precisely when $$ \hat{\beta}_j-\beta_{j,0}<-z_{1-\alpha/2}\widehat{\operatorname{se}}(\hat{\beta}_j) \quad\text{or}\quad \hat{\beta}_j-\beta_{j,0}>z_{1-\alpha/2}\widehat{\operatorname{se}}(\hat{\beta}_j). $$ Rearranging the first inequality gives \(\beta_{j,0}>\hat{\beta}_j+z_{1-\alpha/2}\widehat{\operatorname{se}}(\hat{\beta}_j)\); rearranging the second gives \(\beta_{j,0}<\hat{\beta}_j-z_{1-\alpha/2}\widehat{\operatorname{se}}(\hat{\beta}_j)\). Thus the null value is rejected exactly when it is strictly below the interval’s lower endpoint or strictly above its upper endpoint. Values inside the interval, including its endpoints under the strict rejection rule, are not rejected. \(\square\)

The interval and test use the same normal approximation and estimated standard error. They do not turn an asymptotic procedure into an exact one. Their agreement is an algebraic property of this particular Wald construction.

Reading the Result on the Coefficient Scale

The numerator of \(Z_W\) is a difference on the scale of the coefficient being tested, and its denominator expresses that difference in standard-error units. The test’s null value must therefore be stated on the coefficient scale. If a scientific question is naturally expressed as a rate ratio or odds ratio, the corresponding null value may be translated to a coefficient value before carrying out the test. For example, a null odds ratio of one corresponds to a logistic coefficient of zero.

The coefficient’s interpretation depends on the GLM and on how predictors are measured and coded. In a model with a logit link, a slope is a change in log odds per unit change in its predictor, conditional on the other terms in the model. In a Poisson model with a log link, it is a change in log mean per predictor unit. With an identity link, a slope is a change in the mean response per predictor unit. The Wald statistic itself always compares the coefficient estimate with its null coefficient value; substantive interpretation then uses the model’s link and predictor scale.

Common Mistakes and Practical Checks

• Using the estimate instead of its difference from the null. The numerator is \(\hat{\beta}_j-\beta_{j,0}\), not automatically \(\hat{\beta}_j\). The two coincide only when the null value is zero.
• Reporting \(Z_W\) as a chi-square statistic. \(Z_W\) is compared with standard normal critical values. Its square \(W\), not \(Z_W\), is compared with a \(\chi^2_1\) distribution.
• Reversing the rejection inequalities. For a two-sided test, rejection requires the difference to be below the negative critical distance or above the positive critical distance: $$ \hat{\beta}_j-\beta_{j,0}<-z_{1-\alpha/2}\widehat{\operatorname{se}}(\hat{\beta}_j) \quad\text{or}\quad \hat{\beta}_j-\beta_{j,0}>z_{1-\alpha/2}\widehat{\operatorname{se}}(\hat{\beta}_j). $$ A difference of zero cannot satisfy either condition.
• Interpreting non-rejection as confirmation of the null. A p-value above the chosen level means the test did not reject at that level; it does not establish that the coefficient equals the null value.
• Forgetting the approximation. The normal and chi-square reference distributions are justified by regular large-sample conditions. Check that the standard error corresponds to the estimator’s asymptotic variance and that the model is not in a boundary or other nonregular setting.

A clear report gives the coefficient estimate, its standard error, the null value, the test statistic and reference distribution, the p-value, and an interpretation in the model’s coefficient scale. When useful, a coefficient-scale Wald interval provides a compact account of the null values that the corresponding two-sided tests would not reject.