Why Compare Nested Models?
A candidate ratemaking model may gain predictors in stages: a base model might include territory, while a larger model adds a vehicle characteristic. When the larger model contains all the terms in the smaller model, the models are nested. Their fit can be compared by asking whether the additional parameters improve the likelihood enough to be inconsistent with chance variation under the smaller model.
The likelihood-ratio deviance test answers a specific question: assuming the smaller model is adequate, is the observed improvement in likelihood unusually large? It is not a general method for comparing any two candidates, and it does not establish that the larger model will predict better on new data. The test is aligned with the CAS Exam 8 (Advanced Ratemaking) syllabus.
Deviance Difference and the Test
Let \(\ell_R\) and \(\ell_F\) be the maximized log likelihoods for the reduced and full models, respectively. Because the full model includes the reduced model as a special case, its maximized likelihood cannot be smaller: \(\ell_F\geq \ell_R\). For a model \(M\), its deviance is \(D_M=2(\ell_{\mathrm{sat}}-\ell_M)\), where \(\ell_{\mathrm{sat}}\) is the maximized log likelihood of a saturated model for the same data. Subtracting the two deviances cancels the saturated-model term.
Under the null hypothesis, and under the regularity conditions described below, this likelihood-ratio statistic has an approximate chi-square distribution with degrees of freedom equal to the number of independent restrictions imposed by the reduced model on the full model. In ordinary full-rank GLMs, this is the difference in the models’ numbers of estimated parameters.
Here \(q\) is the difference in parameter dimensions. It is not necessarily the number of named predictors. For example, a categorical predictor with four levels generally adds three coefficients when the model includes an intercept and uses a reference level. An interaction can add several coefficients as well. Use the actual rank or parameter count of the fitted models.
Why the Chi-Square Reference Distribution Appears
The likelihood-ratio result follows from a local approximation to the log likelihood. Under the null, the reduced model’s parameter values are also permissible in the full model. For a sufficiently large sample and regular, identifiable parameters, the increase in maximized log likelihood from estimating the \(q\) added parameters behaves approximately like one-half the sum of squares of \(q\) independent standard normal quantities.
Multiplying that increase by two gives a sum of \(q\) squared standard normal quantities, which has a chi-square distribution with \(q\) degrees of freedom. This explains both the factor of two and the degrees of freedom. The approximation relies on regularity: the null value should be an interior point of the parameter space, parameters should be identifiable, and the likelihood should have suitable smoothness. It is an approximation, not a guarantee for every data set or model.
Check that the reduced model is obtained by restricting the full model, and that both models use the same response observations, weights, likelihood family, and offset treatment.
Compute \(2(\ell_F-\ell_R)\), or equivalently subtract the full-model deviance from the reduced-model deviance.
Set \(q\) equal to the parameter-dimension difference, compare the statistic with \(\chi^2_q\), and state the conclusion in the context of the added terms.
Worked Applications
Worked Example: Adding a Territory Effect to a Poisson Model
A hypothetical insurer models claim counts in two equally exposed territories with independent Poisson observations. The observed counts are 1 and 3. The reduced model has a common fitted mean for both territories; the full model allows a separate mean for each. Test whether the territory effect improves fit using the likelihood-ratio deviance test.
Method and assumptions. For independent Poisson counts, the deviance contribution from an observation \(y\) with fitted mean \(\mu\) is \(2\{y\ln(y/\mu)-(y-\mu)\}\), with the zero-count term interpreted by continuity. The Poisson likelihood is fully specified, so no dispersion parameter is estimated. The models use the same two observations and are nested.
Under the reduced model, the maximum-likelihood common mean is the average count: \((1+3)/2=2\). Under the full model, the fitted means are 1 and 3, so the full model fits these two groups exactly and has deviance zero.
For the reduced model, the first contribution before multiplying by 2 is \(1\ln(1/2)-(1-2)=-0.6931+1=0.3069\). The second is \(3\ln(3/2)-(3-2)=1.2164-1=0.2164\). Thus \(D_R=2(0.3069+0.2164)=1.0465\), while \(D_F=0\).
The full model adds one parameter, so \(q=1\). The statistic is \(G^2=1.0465-0=1.0465\). Its chi-square upper-tail probability is approximately 0.3062. There is not strong evidence against the common-mean model at conventional significance levels. This conclusion concerns evidence of improved in-sample fit, not whether the territory distinction is operationally useful.
Check. The statistic is nonnegative, as required because the full model cannot have lower maximized likelihood. The common mistake this calculation avoids is comparing the reduced deviance directly with a chi-square distribution without accounting for the full model’s deviance.
Worked Example: Testing a Two-Parameter Rating Factor
A hypothetical claim-frequency analysis fits Poisson GLMs to the same policies, using the same exposure offset. A reduced model has maximized log likelihood \(-500.0\). A full model adds a three-level rating factor and has maximized log likelihood \(-494.8\). The factor is coded with a reference level. Test the added factor.
Method. A three-level factor represented by two indicator coefficients adds two parameters, provided the design matrix is full rank. Thus \(q=2\). The likelihoods are from the same Poisson response observations and the same offset specification, so their difference is comparable.
The log-likelihood gain is \(-494.8-(-500.0)=5.2\). Therefore \(G^2=2(5.2)=10.4\). Equivalently, the reduced model’s deviance exceeds the full model’s deviance by 10.4.
For two degrees of freedom, the chi-square upper-tail probability is \(\exp(-10.4/2)=\exp(-5.2)\approx0.0055\). Under the test assumptions, this is evidence against the null that both added coefficients are zero. It does not by itself show that the factor yields stable, well-calibrated rates or better out-of-sample predictions.
Check. The parameter count is two, not three: one level is the reference. Counting all levels as added coefficients would use the wrong reference distribution. The likelihood gain is positive and the test statistic is twice that gain.
Worked Example: Comparing Nested Binomial Frequency Models
A hypothetical insurer models claim occurrence as a binomial response for policy groups. A reduced logistic GLM has deviance 28.70, and a nested full model adding two independent interaction coefficients has deviance 21.10. The fits use the same policy groups, denominators, and binomial likelihood. Assess the added terms.
Method and assumptions. For a binomial GLM with the usual likelihood and a full-rank design, subtracting deviances gives the likelihood-ratio statistic. The two added coefficients give \(q=2\). The chi-square approximation requires independent observations or groups under the specified binomial sampling model, an adequate sample for the approximation, and no non-identifiability.
The statistic is \(G^2=28.70-21.10=7.60\). With two degrees of freedom, the upper-tail probability is \(\exp(-7.60/2)=\exp(-3.80)\approx0.0224\). The test therefore provides evidence against the joint null that both interaction coefficients are zero at the 5% level.
The conclusion is joint: it does not establish that each coefficient is individually significant, nor does it show that the interaction is justified after considering stability, credibility, or validation performance. This calculation avoids treating the reduced model’s deviance of 28.70 as the test statistic; the relevant quantity is the difference, 7.60.
Assumptions, Limits, and Exam Traps
A routine chi-square likelihood-ratio test is most straightforward for likelihood-based Poisson or binomial GLMs with a correctly specified response distribution and a common set of observations. A rate model’s exposure offset must be handled consistently in both fits. If one candidate drops observations because of missing predictors, the likelihood difference combines a model change with a data change and does not have the stated interpretation.
For a Gaussian model with an unknown variance, or a quasi-likelihood fit with an estimated dispersion, do not automatically treat a raw deviance difference as \(\chi^2_q\). The reference distribution and scaling depend on the model and dispersion treatment; in normal linear models, for example, a partial \(F\)-test is commonly used. Similarly, clustered or dependent observations may require a method that accounts for their dependence. Sparse data, separation in logistic models, parameters on boundaries, or a rank-deficient design can also make the usual chi-square approximation unreliable.
The test evaluates a specified nested comparison. It does not correct for repeatedly trying many terms and reporting only the most favorable result. Nor does statistical significance settle a ratemaking decision: predictive validation, practical stability, and implementation constraints address different questions.
Check Your Understanding
For each problem, identify the likelihood-ratio statistic and degrees of freedom before interpreting the comparison. Use the chi-square approximation where the stated assumptions permit it.
- Two nested Poisson models use identical observations and offsets. Their maximized log likelihoods are \(-126.4\) and \(-123.2\), with the second model adding one coefficient. What are the statistic and degrees of freedom? (Answer: \(G^2=6.4\), \(q=1\), \(p\approx0.0114\).)
- A binomial GLM comparison has reduced deviance 54.2 and full deviance 48.9; the full model adds three independent parameters. Find the test statistic and degrees of freedom. (Answer: \(G^2=5.3\), \(q=3\), \(p\approx0.1511\).)
- A nested Poisson model’s log likelihood improves from \(-807.0\) to \(-803.4\) after adding a two-parameter factor. Calculate the test statistic and approximate upper-tail probability. (Answer: \(G^2=7.2\), \(q=2\), \(p\approx0.0273\).)
- A four-level rating factor is added to a full-rank Poisson model with an intercept. The reduced deviance is 31.6 and the full deviance is 22.0. Find the test statistic and degrees of freedom. (Answer: \(G^2=9.6\), \(q=3\), \(p\approx0.0223\).)
- A reduced model is fitted to 8,000 policies, while a nested full model is fitted only to the 7,600 policies with complete data. Their reported deviances differ by 12.0, and the full design adds two coefficients. Is 12.0 a valid likelihood-ratio statistic for this comparison? (Answer: No. The models use different observations, so the deviance difference does not have the stated nested-model likelihood-ratio interpretation.)