Likelihood-Ratio Tests For Multinomial Effects

Why Compare Nested Multinomial Models?

A predictor in a baseline-category logit model can have a different coefficient for each nonbaseline response category. To ask whether the predictor contributes to the model, the relevant null hypothesis is therefore usually a joint one: all its category-specific coefficients are zero. A likelihood-ratio test compares the best fit allowed by that null hypothesis with the best fit of the larger model.

The comparison is useful because it assesses the predictor’s contribution to the model as a whole, on the likelihood scale. The test does not ask whether each separate coefficient is significant, and its degrees of freedom depend on both the number of response categories and the number of predictor parameters being removed.

Models, Likelihoods, and the Test Statistic

Let the response have \(J\) categories, with category \(J\) as the baseline. For observation \(i\), a baseline-category logit model specifies \(\log\{P(Y_i=j)/P(Y_i=J)\}=x_i^\mathsf{T}\beta_j\) for \(j=1,\ldots,J-1\). The resulting category probabilities are positive and sum to one. For grouped data with counts \(y_{ij}\) and total \(m_i\), the likelihood is the product of the multinomial probabilities across independent groups; for individual observations, it is the corresponding product of category probabilities. In either case, let \(\ell\) denote the log-likelihood.

Because every parameter value allowed by the reduced model is allowed by the full model, maximizing over the full parameter space cannot give a smaller likelihood. Thus \(G^2\geq 0\). A large value means that the reduced model fits substantially worse relative to the full model, after accounting for the difference in their numbers of parameters through the reference distribution.

Why the Chi-Square Reference Distribution Applies

The chi-square calibration is asymptotic: it is justified by regular likelihood theory as the amount of independent information grows. A standard form of Wilks’ theorem says that, under the null hypothesis, twice the difference between the full and reduced maximized log-likelihoods converges in distribution to a chi-square random variable whose degrees of freedom equal the number of independent restrictions.

Here is the likelihood argument behind the result. Write the full parameter as \(\theta\in\mathbb R^d\), let \(\theta_0\) be its true null value, and let the reduced model’s local parameter directions form a linear subspace \(S\) of dimension \(d-r\). Under the stated smoothness and information conditions, the local log-likelihood has the quadratic expansion $$ \ell(\theta_0+h/\sqrt n)-\ell(\theta_0) =h^\mathsf{T}Z_n-\tfrac12h^\mathsf{T}Ih+o_p(1), $$ uniformly for bounded \(h\), where \(Z_n\) converges in distribution to \(N(0,I)\) and \(I\) is the positive-definite Fisher information per unit of information. Consistency and root-\(n\) behavior ensure that the relevant maximizers lie in this local region. Maximizing the quadratic over all \(h\) gives \(\tfrac12 Z_n^\mathsf{T}I^{-1}Z_n\); maximizing it over \(h\in S\) gives the same quadratic maximum restricted to that subspace. Their difference, multiplied by two, is the squared length of the component of a standard normal vector orthogonal to the whitened subspace \(I^{1/2}S\). That orthogonal complement has dimension \(r\), so the squared length converges to \(\chi^2_r\). This yields the stated reference distribution. \(\square\)

For multinomial logit regression, regularity requires more than having a numerical optimizer return estimates. The model must be identifiable, the information for the tested parameters must not be singular, and the true probabilities must be in the interior rather than on a boundary. Separation or severe sparsity can prevent ordinary maximum-likelihood estimates from behaving regularly. In such cases, the chi-square approximation may be unreliable.

Making a Valid Comparison

A likelihood-ratio test answers a precise question only when the reduced model is genuinely nested in the full model and both likelihoods refer to the same data and response specification. When removing a predictor, retain the other terms specified by the scientific comparison. If a factor has \(q\) independent contrast columns, count those columns in every nonbaseline logit. For more general linear restrictions, the degrees of freedom are the rank of the restriction matrix.

The comparison uses ordinary maximized log-likelihoods. Replacing these with penalized likelihoods, or comparing models fitted to different observations, does not give this standard likelihood-ratio test. Also, if a parameter is on a boundary or ordinary identifiability fails, the usual chi-square result need not hold. A small p-value measures incompatibility with the tested restrictions under the model; it is not an estimate of effect size, nor proof that the model is scientifically correct.

Common Mistake

• Wrong: “The predictor has two columns, so the test has two degrees of freedom.” This counts the predictor parameters in only one logit.
• Correction: With category-specific coefficients, each predictor column contributes a coefficient to each of the \(J-1\) nonbaseline logits. Count the independent restrictions across all logits: \(r=(J-1)q\).
• Wrong: “The full model has more parameters, so its likelihood-ratio statistic must be significant.” The full model’s maximized likelihood cannot be smaller, but the chi-square reference distribution determines whether its improvement is unusually large.