Joint Wald Tests For Multinomial Predictors

Why Test a Predictor Across All Logits at Once?

A baseline-category multinomial logit model has one logit for each nonbaseline response category. A predictor may therefore have several coefficients: one for each logit, or more when it is represented by several design columns. To ask whether the predictor contributes to the model as a whole, the relevant question is whether all coefficients in that block are zero jointly.

Testing each coefficient separately does not answer that question. It also discards information in the estimated covariance between coefficients, including covariance between coefficients from different logits. A joint Wald test uses the entire estimated coefficient vector and its covariance matrix to assess the specified block.

Hypothesis and Wald Statistic

Let the response have categories $0,1,\ldots,K-1$, with category $0$ as the baseline. The model’s baseline-category logits are

\log\left\{\frac{P(Y=k\mid x)}{P(Y=0\mid x)}\right\}=x^\mathsf{T}\beta_k,\qquad k=1,\ldots,K-1.

Suppose a predictor is represented by $d$ design columns. Write its $d$ coefficients in logit $k$ as $\beta_{k,S}$, where $S$ identifies the predictor block. Stack these coefficients across the $K-1$ logits into a vector $\beta_S$ of length $q=(K-1)d$. The order used for stacking must match the corresponding rows and columns of the covariance matrix.

Definition: The joint Wald test for the predictor block tests $H_0:\beta_S=0$ against the alternative that at least one coefficient in the block is nonzero. More generally, for a linear hypothesis $H_0:R\theta=r$, where $\theta$ is the full parameter vector and $R$ has $q$ linearly independent rows, the Wald statistic is $$ W=(R\hat{\theta}-r)^\mathsf{T}\left\{R\widehat{\operatorname{Cov}}(\hat{\theta})R^\mathsf{T}\right\}^{-1}(R\hat{\theta}-r). $$

For the zero-coefficient test, $R$ selects the $q$ coefficients in the predictor block and $r=0$. The matrix $R\widehat{\operatorname{Cov}}(\hat{\theta})R^\mathsf{T}$ is the estimated covariance matrix of that block. It is a submatrix of the covariance matrix for the full fitted model; it is not generally diagonal.

Under the null, the usual large-sample reference distribution is chi-square with $q$ degrees of freedom. For a predictor with $d$ columns and $K-1$ logits, $q=(K-1)d$, provided the block is identifiable and the constraints are linearly independent. The $p$-value is the upper-tail probability $P(\chi^2_q\geq W)$.

Worked Example: A Continuous Predictor in Three Categories

A fitted model has three response categories, with category 0 as baseline, and a continuous biomarker with one slope in each of the two logits. The estimated slopes, in logit order, are $\hat{\beta}_S=(0.4,-0.2)^\mathsf{T}$. Their estimated covariance matrix is

\widehat{\operatorname{Cov}}(\hat{\beta}_S)= \begin{pmatrix} 0.04&0.01\\ 0.01&0.09 \end{pmatrix}.

The omnibus null is $H_0:\beta_{1,S}=\beta_{2,S}=0$, so there are $q=2$ restrictions. The determinant of the covariance matrix is $0.04(0.09)-0.01^2=0.0035$, giving

\widehat{\operatorname{Cov}}(\hat{\beta}_S)^{-1} =\frac{1}{0.0035} \begin{pmatrix} 0.09&-0.01\\ -0.01&0.04 \end{pmatrix} \approx \begin{pmatrix} 25.714&-2.857\\ -2.857&11.429 \end{pmatrix}.

Multiplying this inverse by the estimate gives approximately $(10.857,-3.429)^\mathsf{T}$. Therefore,

W=(0.4,-0.2) \begin{pmatrix} 10.857\\-3.429 \end{pmatrix} \approx 5.029.

For two degrees of freedom, the upper-tail probability is $\exp(-W/2)$, which here is approximately $\exp(-2.5145)=0.081$. At the 0.05 level, this test does not reject the joint null. The result concerns the two slopes together; it does not assert that either individual slope is zero.

Why the Chi-Square Reference Distribution Applies

The reference distribution is asymptotic, not an exact finite-sample guarantee. A standard sufficient framework is a correctly specified, identifiable multinomial logit model with fixed parameter dimension, independent observations, a true parameter in the interior of the parameter space, and nonsingular Fisher information. Under the usual regularity conditions for maximum likelihood, the maximum-likelihood estimator is consistent and asymptotically normal, and its estimated covariance is consistent.

Theorem: Under those regularity conditions, if $H_0:R\theta_0=r$ is true and $R$ has $q$ linearly independent rows, then the Wald statistic satisfies $W\xrightarrow{d}\chi^2_q$, provided the covariance estimate used in $W$ is consistent and the covariance of $R\hat{\theta}$ is nonsingular.

Proof. Let $V$ be the asymptotic covariance matrix of $\hat{\theta}$, so that $\sqrt{n}(\hat{\theta}-\theta_0)\xrightarrow{d}N(0,V)$. Define $C=RVR^\mathsf{T}$, which is positive definite by the nonsingularity condition. Since $R\theta_0=r$, the limiting distribution of $\sqrt{n}(R\hat{\theta}-r)$ is $N(0,C)$. Consistency of the covariance estimate implies that the estimated covariance of $R\hat{\theta}$, multiplied by $n$, converges in probability to $C$. By Slutsky’s theorem, the Wald statistic converges in distribution to $Z^\mathsf{T}C^{-1}Z$, where $Z\sim N(0,C)$. The whitened vector $C^{-1/2}Z$ has $q$ independent standard normal components. Its squared length is therefore chi-square with $q$ degrees of freedom. $\square$

The statistic’s value does not depend on which full-rank set of equations is used to describe the same hypothesis. This matters, for example, when the same coefficient restrictions are written using different contrast rows.

Result: If $A$ is an invertible $q\times q$ matrix, replacing $R\theta=r$ with the equivalent constraints $AR\theta=Ar$ leaves the Wald statistic unchanged.

Proof. Write $C_R=R\widehat{\operatorname{Cov}}(\hat{\theta})R^\mathsf{T}$ and $u=R\hat{\theta}-r$. Under the transformed constraints, the residual is $Au$ and the covariance matrix is $AC_RA^\mathsf{T}$. Since $A$ is invertible, $(AC_RA^\mathsf{T})^{-1}=(A^\mathsf{T})^{-1}C_R^{-1}A^{-1}$. The transformed statistic is consequently

(Au)^\mathsf{T}(AC_RA^\mathsf{T})^{-1}(Au) =u^\mathsf{T}A^\mathsf{T}(A^\mathsf{T})^{-1}C_R^{-1}A^{-1}Au =u^\mathsf{T}C_R^{-1}u,

which is the original statistic. $\square$

Worked Applications

Worked Example: A Binary Treatment Across Two Logits

In another three-category response model, a binary treatment is represented by one indicator, with estimated slopes $0.4$ and $-0.3$ in the two nonbaseline logits. Suppose their estimated covariance matrix is diagonal with variances $0.04$ and $0.09$. The joint null sets both treatment slopes to zero, so $q=2$. The inverse covariance matrix is $\operatorname{diag}(25,11.111\ldots)$, and

W=\frac{0.4^2}{0.04}+\frac{(-0.3)^2}{0.09}=4+1=5.

The two-degree-of-freedom upper-tail probability is $\exp(-5/2)\approx0.082$. At the 0.05 level, the data do not provide sufficient evidence to reject the joint null. This is an omnibus assessment of the treatment’s two-logit coefficient block.

Worked Example: A Three-Level Predictor

A predictor with three levels is represented by two indicator columns. In a model with three response categories, it therefore has four coefficients across the two nonbaseline logits. Suppose the stacked estimate is $(0.2,-0.3,0.1,0.2)^\mathsf{T}$, and its covariance matrix is diagonal with entries $0.04,0.09,0.04,0.09$. Testing whether the predictor has no coefficients in either logit gives $q=2(2)=4$. The Wald calculation is

W=\frac{0.2^2}{0.04}+\frac{(-0.3)^2}{0.09} +\frac{0.1^2}{0.04}+\frac{0.2^2}{0.09} =1+1+0.25+0.444\ldots \approx2.694.

The upper-tail probability for a chi-square distribution with four degrees of freedom is approximately $0.61$. Thus the joint null is not rejected at the 0.05 level. The four restrictions arise from two design columns in each of two logits—not simply from the two levels of the response.

Interpretation and Common Pitfalls

For a predictor block consisting of all its relevant coefficients, the null says that the predictor has no effect through those terms in any of the modeled baseline-category logits. Rejecting the null is evidence that at least one coefficient in the block differs from zero; it does not identify which coefficient is responsible. The test is conditional on the other terms in the fitted model.

Ignoring cross-logit covariance. A common mistake is to test the logits separately or to compute a joint statistic using only the diagonal entries of the coefficient covariance matrix. The correction is to use the full covariance submatrix for the tested block, including cross-logit covariances when they are nonzero.
Using the wrong degrees of freedom. The degrees of freedom are the number of independent restrictions. For a predictor with $d$ columns and $K-1$ logits, that is $(K-1)d$ when all those coefficients are independently estimable.
Testing a block that does not match the question. With interactions, a predictor’s main-effect coefficients alone may not represent every term involving that predictor. Define the intended coefficient block before calculating the test; include interaction coefficients if the hypothesis concerns the predictor’s full contribution.
Reading non-rejection as proof of no effect. A large $p$-value means the test did not find sufficiently strong evidence against the joint null. It does not establish that the coefficients are exactly zero.

Key takeaway: A multinomial predictor’s omnibus Wald test is built from its full coefficient block and that block’s full covariance matrix. The chi-square degrees of freedom equal the number of independent zero restrictions being tested.

Check Your Understanding

Use the model’s baseline category and the stated coefficient block when answering.

With four response categories and a predictor represented by two design columns, how many restrictions are in the joint zero-coefficient test?
Why can separate Wald tests for each logit fail to answer the same question as the joint test?
What covariance matrix belongs in the Wald statistic when testing a selected predictor block?
State the regularity conditions that support the chi-square reference distribution.
If a model includes interactions involving a predictor, what should guide the choice of coefficients in its joint test?