Deviance As A Likelihood-Ratio Comparison

Why Compare a Model With a Saturated Reference?

A fitted statistical model summarizes data using a limited number of parameters. A more flexible reference model can fit the observed data at least as closely, often by assigning a separate parameter to each observation or group. Comparing these fits provides a scale for describing how much likelihood is lost by imposing the simpler model’s structure.

The likelihood-ratio comparison with this maximally flexible reference is called deviance. It is useful for assessing how far a fitted model is from a saturated fit, and it also clarifies why the likelihood-ratio statistic is nonnegative. Deviance is a measure of relative fit, not by itself a probability or a proof that a model is adequate.

Definition: Let $L_{\mathrm{sat}}$ be the maximized likelihood under a saturated model and let $L_{\mathrm{fit}}$ be the maximized likelihood under the fitted model, using the same observed data and likelihood convention. The deviance of the fitted model relative to the saturated model is $$ D=2\{\log L_{\mathrm{sat}}-\log L_{\mathrm{fit}}\} =2\log\left(\frac{L_{\mathrm{sat}}}{L_{\mathrm{fit}}}\right). $$ The saturated model must include the fitted model among its possible distributions. Consequently, $L_{\mathrm{sat}}\geq L_{\mathrm{fit}}$ and $D\geq 0$.

What “Saturated” Means

A saturated model has enough flexibility to reproduce the observed data at the level being modeled. For individual observations, it may assign a separate parameter to each observation. For grouped data, it commonly assigns a separate probability or mean to each group or cell. “Saturated” therefore depends on how the observations have been grouped: changing the grouping can change the reference model and its deviance.

The likelihoods in the comparison must be based on the same data and the same probability model for those data. Terms that are constant with respect to the model parameters, such as binomial coefficients or Poisson factorials, can cancel when taking a likelihood ratio. They should not be selectively included in one likelihood and omitted from the other.

Why Deviance Is Twice a Log-Likelihood Difference

The likelihood-ratio statistic compares the maximized likelihood under a restricted model with the maximized likelihood under a more flexible model. Taking a logarithm turns a likelihood ratio into a difference of log likelihoods. Defining the statistic with the factor of 2 gives the usual likelihood-ratio scale:

-2\log\left(\frac{L_{\mathrm{fit}}}{L_{\mathrm{sat}}}\right) =-2\{\log L_{\mathrm{fit}}-\log L_{\mathrm{sat}}\} =2\{\log L_{\mathrm{sat}}-\log L_{\mathrm{fit}}\}.

This establishes the deviance formula directly: it is the generalized likelihood-ratio statistic for comparing the fitted model with the saturated reference. The factor of 2 is not an extra adjustment chosen to make a particular data set fit better; it is part of the standard likelihood-ratio statistic. Under suitable regularity conditions, a likelihood-ratio statistic can have an approximate chi-square distribution, with degrees of freedom determined by the difference in the numbers of independent parameters. That calibration is an asymptotic result, not a guarantee for every saturated comparison. Small samples, boundary parameters, or sparse cells can make a routine chi-square approximation unreliable.

Deviance for Grouped Binomial Data

Suppose groups are independent and group $j$ has $n_j$ Bernoulli trials, of which $x_j$ are successes. Under a fitted model, let the success probability in that group be $\widehat p_j$. The saturated binomial model estimates a separate probability $x_j/n_j$ in each group, provided $0<x_j<n_j$; at the endpoints the saturated estimate is 0 or 1.

The binomial coefficients are the same under both models and cancel. For groups with probabilities strictly between 0 and 1, the resulting deviance is

D=2\sum_j\left[ x_j\log\left(\frac{x_j}{n_j\widehat p_j}\right) +(n_j-x_j)\log\left(\frac{n_j-x_j}{n_j(1-\widehat p_j)}\right) \right].

A term with zero observed count is defined to contribute 0, consistent with the limit $0\log(0/a)=0$ for $a>0$. If the fitted probability is at a boundary that assigns zero probability to an outcome that occurred, the fitted likelihood is zero and the usual finite deviance formula does not apply: the likelihood-ratio statistic is infinite.

Worked Example: Two Groups Under a Common Success Probability

A hypothetical insurer reviews two independent groups of policies. In group 1, 1 of 4 policies has a specified claim outcome; in group 2, 3 of 6 policies have that outcome. Compare a model with one common success probability with the saturated model, which allows separate probabilities. Assume independent binomial observations within the groups.

Step 1: Fit the common-probability model. There are $1+3=4$ successes among $4+6=10$ trials, so the maximum-likelihood estimate under the common-probability model is $ \widehat p=4/10=0.4 $.

Step 2: Fit the saturated model. Its group probabilities are $1/4=0.25$ and $3/6=0.5$. The fitted common model has expected success counts $4(0.4)=1.6$ and $6(0.4)=2.4$, and expected failure counts $2.4$ and $3.6$.

Step 3: Calculate deviance contributions. For group 1, the success contribution is $1\log(1/1.6)=\log(0.625)\approx-0.470003629$. Its failure contribution is $3\log(3/2.4)=3\log(1.25)\approx0.669430654$. For group 2, the success contribution is $3\log(3/2.4)=3\log(1.25)\approx0.669430654$. Its failure contribution is $3\log(3/3.6)=3\log(5/6)\approx-0.546964670$.

Step 4: Add and double. The sum inside the brackets is $-0.470003629+0.669430654+0.669430654-0.546964670=0.321893009$. Thus $D=2(0.321893009)=0.643786018$, or 0.6438 to four decimal places.

The value is positive because allowing separate group probabilities improves the maximized likelihood. A common mistake is to miscalculate the last contribution as $-0.547965170$; the correct value is $3\log(5/6)\approx-0.546964670$. Using the incorrect term changes both the bracketed sum and the deviance.

Worked Example: A Common Probability Versus Separate Probabilities

Consider two independent hypothetical binomial groups, each with 5 trials. The observed numbers of successes are 1 and 4. Find the deviance of a model using one common success probability relative to a saturated model with separate group probabilities.

Method and estimates. The pooled estimate under the common-probability model is $\widehat p=(1+4)/(5+5)=0.5$. The saturated estimates are $1/5=0.2$ and $4/5=0.8$. Each group therefore has fitted expected counts of 2.5 successes and 2.5 failures.

Group 1 contributions. There is 1 success and 4 failures, giving $1\log(1/2.5)+4\log(4/2.5)$. Since $\log(1/2.5)=-0.916290731$ and $\log(4/2.5)=\log(1.6)=0.470003629$, the contribution is $-0.916290731+4(0.470003629)=0.963723785$.

Group 2 contributions. There are 4 successes and 1 failure, so the contribution is $4\log(4/2.5)+\log(1/2.5)=4(0.470003629)-0.916290731=0.963723785$.

The total inside the factor of 2 is $0.963723785+0.963723785=1.927447570$. Therefore, $D=2(1.927447570)=3.854895140$, or 3.8549 to four decimal places.

The method is a direct likelihood comparison, with the common probability estimated by pooling the successes and trials. A common mistake is to use the two saturated probabilities as though they were the common-model estimate, or to round intermediate logarithms too early. The correct common estimate is 0.5, and the final rounded deviance is 3.8549.

Poisson Deviance and Zero Counts

For independent Poisson counts $Y_j$ with fitted means $\widehat\mu_j>0$, the saturated model uses mean $y_j$ for each observed count $y_j$. Comparing Poisson log likelihoods gives

D=2\sum_j\left[y_j\log\left(\frac{y_j}{\widehat\mu_j}\right)-(y_j-\widehat\mu_j)\right],

where the logarithmic term is defined as 0 when $y_j=0$. The linear terms must not be dropped unless their sum is known to be zero. In particular, if the fitted and observed totals agree, then $\sum_j(y_j-\widehat\mu_j)=0$, and only then do those terms cancel in the total.

Worked Example: Counts Fitted by a Common Poisson Mean

Suppose four independent hypothetical claim-processing categories have observed counts $1,2,2,3$. Compare a common-mean Poisson model with a saturated model of four separate Poisson means. Assume each count has a Poisson distribution and the categories are independent.

Step 1: Estimate the common mean. The observed total is $1+2+2+3=8$. The maximum-likelihood estimate of the common mean is $8/4=2$, so each fitted mean is $\widehat\mu_j=2$.

Step 2: Identify the saturated fit. The saturated means equal the observed counts: $1,2,2,3$. The total observed count and total fitted count are both 8, so the linear terms in the Poisson deviance sum to zero.

Step 3: Calculate the log contributions. The four terms $y_j\log(y_j/2)$ are $1\log(1/2)=-0.693147181$, $2\log(2/2)=0$, $2\log(2/2)=0$, and $3\log(3/2)=3(0.405465108)=1.216395324$. Their sum is $0.523248143$.

Thus $D=2(0.523248143)=1.046496286$, or 1.0465 to four decimal places. The result is nonnegative, as required. For this common-mean Poisson fit, the observed and fitted totals happen to agree; dropping the linear terms without checking that equality would be a mistake in a different model.

Interpreting Deviance Carefully

A deviance of zero means the fitted model attains the same maximized likelihood as the saturated model for the data and grouping in question. A larger deviance means a larger log-likelihood loss relative to that reference. Its magnitude alone does not give a universal judgment of adequacy: interpretation depends on the data scale, sample size, model, and reference distribution used for inference.

When a fitted model is nested within a more flexible fitted model, their difference in deviances is twice the difference in maximized log likelihoods. This makes the difference a likelihood-ratio comparison between those two models. Comparing a model’s deviance with a saturated model is a particular use of that same likelihood-ratio construction; it does not mean that every deviance automatically has a reliable chi-square calibration.

Exam takeaway: State the fitted and saturated models, verify that they use the same observations, find their maximized likelihoods, and calculate $2(\log L_{\mathrm{sat}}-\log L_{\mathrm{fit}})$. For grouped binomial or Poisson data, use the model-specific deviance formula and handle zero counts by the appropriate limit. A chi-square interpretation requires suitable conditions and an appropriate degrees-of-freedom calculation.

Check Your Understanding

For each problem, use the saturated model specified in the question. Round numerical deviances to four decimal places.

Two independent binomial groups each have 5 trials, with 1 and 4 successes. A fitted model uses a common success probability. Find its deviance against separate group probabilities. (Answer: 3.8549)
Three independent Poisson counts are observed as $0,2,4$. Fit one common Poisson mean and compare with the saturated model. (Answer: 5.5452)
Three independent binomial groups each have 4 trials, with success counts $0,2,4$. Find the deviance for a common probability against separate group probabilities. (Answer: 11.0904)
Three observations are $3,5,8$ from independent normal distributions with a common mean and known variance $9$. Compare the common-mean model with a saturated model having a separate mean for each observation. (Answer: 1.4074)
A regular fitted model has 2 free parameters for 5 independent binomial groups, each with a separate saturated probability. Under the usual regularity and large-sample conditions, what is the nominal chi-square degrees of freedom for the deviance? (Answer: 3)

What You'll Learn