Why the Chi-Square Reference Needs Conditions
A likelihood-ratio test can compare a smaller generalized linear model (GLM) with a larger one by measuring how much the fit improves when parameters are added. A deviance difference is often used to calculate that improvement. But a numerical difference alone does not guarantee that a chi-square reference distribution is appropriate. The comparison depends on the models being nested, on regularity conditions for likelihood inference, and on having enough information for a large-sample approximation to be credible.
The earlier material on likelihood-ratio tests from deviance differences gives the calculation. Here the focus is on when that calculation has its usual chi-square interpretation, and when the interpretation deserves caution. This distinction matters in risk modeling: an apparently small p-value can be misleading if the fitted models or the data do not support the approximation.
Suppose the smaller model imposes \(q\) independent restrictions on the larger model. Under the regularity and large-sample conditions described below, the likelihood-ratio statistic is approximately chi-square with \(q\) degrees of freedom when the null hypothesis is true. “Independent” is important: a list of \(q\) equations does not necessarily impose \(q\) independent restrictions.
Here \(\ell\) is the maximized log likelihood for the stated model, \(D\) denotes deviance, and \(q\) is the number of independent restrictions. The symbol \(\dot{\sim}\) indicates an approximate distribution, not an exact one. The reference is used to assess the statistic under the null model; it does not assert that the statistic is exactly chi-square for a finite sample.
Why the Deviance Difference Is a Likelihood-Ratio Statistic
For a likelihood-based model, deviance is defined relative to a saturated model, which fits the observed responses as closely as the likelihood formulation allows. If \(D_{\text{small}}\) and \(D_{\text{large}}\) use the same response likelihood and the same saturated-model reference, subtracting them cancels the saturated-model contribution.
For either fitted model, \(D=2\{\ell_{\text{sat}}-\ell_{\text{fit}}\}\), where \(\ell_{\text{sat}}\) is the saturated-model log likelihood.
\(D_{\text{small}}-D_{\text{large}}=2\{(\ell_{\text{sat}}-\ell_{\text{small}})-(\ell_{\text{sat}}-\ell_{\text{large}})\}\).
The two \(\ell_{\text{sat}}\) terms cancel, leaving \(2(\ell_{\text{large}}-\ell_{\text{small}})\), the likelihood-ratio statistic.
This cancellation explains why the comparison is meaningful only when the two deviances are defined on the same response data and with compatible likelihood and dispersion treatment. Comparing deviances from different response samples, incompatible likelihoods, or unrelated model families does not produce this nested-model likelihood-ratio test.
Conditions Behind the Chi-Square Approximation
The standard reference is an application of Wilks’ theorem. In a GLM setting, the key checks are as follows.
- The models must be nested. The null model must be obtainable from the larger model by imposing parameter restrictions. A smaller deviance is not enough to establish nesting.
- The restrictions must be identifiable. The difference in effective parameter dimensions, or equivalently the rank of the independent restrictions, gives \(q\). Redundant parameters or constraints can make a simple parameter-count difference incorrect.
- The likelihood must be regular near the null value. Typically, parameters are identifiable, the log likelihood is sufficiently smooth, and the information matrix is nonsingular. The null value should lie in the interior of the allowed parameter space, rather than on a boundary.
- The likelihood model and sampling assumptions must be appropriate. For example, the GLM likelihood should match the assumed response distribution and account for the modeled predictors, exposure, and dependence structure. Misspecification can undermine the standard reference.
- The sample must be large enough for the asymptotic approximation. A large total number of observations is not by itself sufficient if much of the information is concentrated in very few events or rare categories.
- The comparison must use a likelihood. A quasi-likelihood may support other inferential procedures, but without a specified likelihood it does not supply the ordinary likelihood-ratio statistic and its usual Wilks chi-square justification.
In GLMs, nuisance parameters such as a dispersion parameter require consistent treatment across the two fits. A likelihood-ratio comparison can include nuisance parameters when regularity conditions hold, but using an ad hoc dispersion adjustment or comparing quantities that are not the same likelihood does not automatically preserve the usual chi-square result. Check what the reported statistic actually represents before assigning degrees of freedom or interpreting a p-value.
Worked Example: Testing Two Added Poisson Predictors
A hypothetical insurer fits Poisson count models to the same policy records, with the same exposure offsets and likelihood. The smaller model has an intercept and one predictor; the larger model adds two predictors. Their maximized log likelihoods are \(-82.40\) and \(-78.95\), respectively. Assume the records are independent, the added coefficients are identifiable, fitted means are interior, and event counts are adequate for a large-sample approximation. Test the added predictors jointly at the 5% level.
Method and nesting. Setting both added coefficients to zero gives the smaller model, so the models are nested. There are two independent restrictions, hence \(q=2\).
Calculate the statistic. The larger model has the higher maximized log likelihood, as it should when it contains the smaller model. Thus \(T=2\{(-78.95)-(-82.40)\}=2(3.45)=6.90\). The same result would be obtained as the smaller deviance minus the larger deviance, provided both deviances use the same saturated reference.
Reference and decision. Under the stated conditions and the null hypothesis, use approximately \(\chi^2_2\). The 5% critical value is 5.991, and \(6.90>5.991\), so reject the joint null at the 5% level. For two degrees of freedom, the upper-tail probability is \(e^{-6.90/2}=e^{-3.45}\approx0.0317\), to four decimals.
Check and common mistake avoided. The statistic is nonnegative, consistent with the larger model fitting at least as well. The degrees of freedom are the two added independent coefficients, not the sample size minus a parameter count.
Cases That Can Make the Approximation Unreliable
The conditions are practical diagnostics, not formalities. In particular, boundary estimates and sparse observations can change the sampling distribution or prevent the usual large-sample behavior from being a good guide.
- Sparse counts or rare binary outcomes: a few events may determine the estimate of an added effect. The total sample can be large while the relevant information remains limited.
- Boundary estimates: if a parameter is constrained to be nonnegative and the null value is zero, the null may be on the boundary. The ordinary chi-square result need not apply; some boundary problems have mixture reference distributions, but the appropriate one depends on the model and hypothesis.
- Complete or quasi-complete separation in logistic regression: a coefficient may have no finite maximum-likelihood estimate. The usual smooth, finite interior estimate assumptions then fail.
- Dependence or misspecification: unmodeled clustering or an incorrect response distribution can make the nominal likelihood and its reference distribution inappropriate.
- Quasi-likelihood or separately adjusted dispersion: a deviance-like quantity from a quasi-likelihood is not automatically an ordinary likelihood-ratio statistic. Its interpretation needs a method designed for that setting.
Worked Example: Sparse Poisson Counts and a Boundary Estimate
Suppose two hypothetical, equally exposed groups have observed Poisson counts \(0\) and \(4\). Compare a null model with one common rate against an alternative with a separate rate for each group. The observations are independent, and each group has exposure 1. Calculate the nominal likelihood-ratio statistic, then explain why its chi-square p-value may be unreliable.
Method and nesting. The common-rate model is nested in the separate-rates model because the two rates can be constrained to be equal. Under the null, the maximum-likelihood rate is \((0+4)/(1+1)=2\) in each group. Under the alternative, the maximum-likelihood rates are \(0\) and \(4\).
Calculate the likelihood ratio. For a Poisson count \(y\) with mean \(\lambda\), the likelihood contribution is \(e^{-\lambda}\lambda^y/y!\). The alternative likelihood is \(e^0(0^0/0!)\times e^{-4}4^4/4!=e^{-4}256/24\), where the zero-count, zero-mean contribution is understood as its limiting value 1. The null likelihood is \(e^{-2}2^0/0!\times e^{-2}2^4/4!=e^{-4}16/24\). The likelihood ratio is \(16\), so \(T=2\ln(16)=8\ln(2)=5.5452\), to four decimals.
Nominal reference and caution. The models differ by one rate restriction, so the ordinary nominal reference would be \(\chi^2_1\); its upper-tail probability is approximately 0.0185. But the alternative estimate for the first group is zero, at the boundary of the nonnegative rate space. In a log-rate parameterization this corresponds to a coefficient tending to negative infinity, not a finite interior estimate. With only four total events, the large-sample approximation is also questionable. The nominal p-value should not be treated as dependable evidence on its own.
Check and common mistake avoided. Conditional on the total count of four and under equal rates, the first group’s count has a Binomial distribution with size 4 and probability \(1/2\). The probability of the two outcomes at least as extreme as \(0\) or \(4\) is \(2(1/2)^4=0.125\). This illustrates how sparse-data inference can differ substantially from the nominal chi-square result; it does not make a conditional test interchangeable with the likelihood-ratio test in every setting.
Degrees of Freedom and a Common Exam Trap
The chi-square degrees of freedom describe the number of independent restrictions tested, not the number of observations, the residual degrees of freedom of the larger model, or the value of the statistic. If the larger model has \(p_{\text{large}}\) identifiable parameters and the smaller has \(p_{\text{small}}\), then \(q=p_{\text{large}}-p_{\text{small}}\) when those parameter counts are valid and the restrictions are independent.
Worked Example: Three Independent Restrictions in a Binary GLM
A hypothetical health-benefit study fits two logistic-regression models to the same 600 independent records. The smaller model has maximized log likelihood \(-120.7342\). A nested larger model adds three prespecified, identifiable coefficients and has maximized log likelihood \(-116.2042\). Assume no separation, no boundary estimates, and sufficient information for the approximation. Test the three coefficients jointly at the 5% level.
Method and degrees of freedom. The smaller model is obtained by setting all three added coefficients to zero. The restrictions are independent, so \(q=3\).
Calculate the statistic. Using the displayed likelihoods, \(T=2\{(-116.2042)-(-120.7342)\}=2(4.5300)=9.0600\). The larger model has the greater maximized log likelihood, as required for a nested comparison.
Decision. The 5% critical value for \(\chi^2_3\) is approximately 7.815. Since \(9.0600>7.815\), reject the joint null at the 5% level, conditional on the stated assumptions.
Check and common mistake avoided. Use three degrees of freedom because three independent restrictions are tested. Using \(600-p_{\text{large}}\) would answer a different question; it is not the reference degrees of freedom for this likelihood-ratio comparison.
A related trap is to compare any two deviances and assume the larger-minus-smaller model interpretation. First verify that the models are nested and use the same data and likelihood basis. Then check the parameter restrictions, the dispersion treatment, and whether the fitted estimates and data support the regularity and large-sample assumptions. A correct calculation with an unjustified reference distribution is not a valid test.
Check Your Understanding
For each problem, decide what the deviance-difference test can establish and check the assumptions before relying on a chi-square reference.
- A Poisson model with an intercept and exposure offset is compared with a nested model adding one independent coefficient. Both use the same records and likelihood; the log likelihoods are \(-45.00\) and \(-43.20\). Find the statistic and degrees of freedom. (Answer: \(T=3.60\), \(q=1\).)
- A nested model adds four coefficients, but one is an exact linear combination of the other three. If the remaining restrictions are identifiable, how many degrees of freedom does the joint likelihood-ratio test use? (Answer: 3 degrees of freedom, because only three restrictions are independent.)
- A logistic model is compared with a nested model that adds a predictor perfectly separating all zero outcomes from all one outcomes. Why is the ordinary chi-square approximation suspect? (Answer: The added coefficient may have no finite maximum-likelihood estimate, violating the usual regularity conditions.)
- Two nested Poisson models are fitted to the same data, but one uses a likelihood with an exposure offset and the other omits the offset even though exposure varies. Is their deviance difference automatically a valid likelihood-ratio test for the added predictors? (Answer: No; the models do not represent a compatible comparison of the same specified likelihood structure, so the intended nested-model test is not established.)
- For a likelihood-based GLM comparison, the larger and smaller models have 9 and 6 identifiable parameters, respectively, and satisfy regularity conditions. The statistic is 8.00. State the reference degrees of freedom and whether it exceeds the 5% critical value. (Answer: \(q=3\); yes, since \(8.00>7.815\).)