Deviance as an Absolute-Fit Measure

Why Compare a Model With a Saturated Model?

A goodness-of-fit measure should quantify how much the fitted model falls short of matching the observed data. For a generalized linear model, deviance makes this comparison using likelihood: it compares the fitted model with a saturated model, which gives each observation its own parameter and therefore fits the observations as closely as the data type allows.

Deviance is an absolute-fit measure: it summarizes the discrepancy between a fitted model and the observations. A larger value indicates a greater likelihood loss relative to the saturated model, but the number alone is not a universal verdict. Its interpretation depends on how the observations are represented and on the setting in which it is assessed.

Definition and Formulas

Definition: The deviance $D$ is twice the difference between the maximized log-likelihood of the saturated model and the log-likelihood of the fitted model: $$ D=2\{\ell_{\mathrm{sat}}-\ell(\hat{\theta})\} =-2\log\!\left(\frac{L(\hat{\theta})}{L_{\mathrm{sat}}}\right). $$ Here $L_{\mathrm{sat}}$ is the maximum likelihood in the saturated model, and $\hat{\theta}$ denotes the fitted model’s parameter estimate. The saturated model must use the same data representation and likelihood contributions as the fitted model.

For independent binomial observations, let $Y_i$ be the number of successes among $m_i$ trials, with fitted success probability $\hat p_i$, where $m_i>0$ and $0<\hat p_i<1$. The saturated model assigns observation $i$ its own probability $q_i=Y_i/m_i$. With the convention $0\log(0/a)=0$ for $a>0$, the binomial deviance is

D_{\mathrm{bin}} =2\sum_i \left[ Y_i\log\!\left(\frac{Y_i}{m_i\hat p_i}\right) +(m_i-Y_i)\log\!\left(\frac{m_i-Y_i}{m_i(1-\hat p_i)}\right) \right].

For independent Poisson observations $Y_i$ with fitted means $\hat\mu_i>0$, the saturated model sets the mean for observation $i$ to $Y_i$. The resulting formula is

D_{\mathrm{Pois}} =2\sum_i\left[ Y_i\log\!\left(\frac{Y_i}{\hat\mu_i}\right) -(Y_i-\hat\mu_i) \right],

where a term $Y_i\log(Y_i/\hat\mu_i)$ is defined to be zero when $Y_i=0$. In that case the observation contributes $2\hat\mu_i$ to the deviance. These expressions follow directly from the likelihood comparison, as shown below.

Theorem: For independent binomial observations with $0<\hat p_i<1$, and independent Poisson observations with $\hat\mu_i>0$, the displayed formulas are twice the log-likelihood difference between the saturated model and the fitted model.

Proof. For a binomial observation, its log-likelihood at probability $p$ is

\ell_i(p)=\log {m_i\choose Y_i} +Y_i\log p+(m_i-Y_i)\log(1-p).

The saturated value uses $q_i=Y_i/m_i$, including $q_i=0$ or $q_i=1$ at the boundary. Subtracting the fitted log-likelihood from the saturated log-likelihood cancels the binomial coefficient and gives

\ell_i(q_i)-\ell_i(\hat p_i) =Y_i\log\!\left(\frac{Y_i}{m_i\hat p_i}\right) +(m_i-Y_i)\log\!\left(\frac{m_i-Y_i}{m_i(1-\hat p_i)}\right).

Terms with zero counts are interpreted by continuity. Summing over observations and multiplying by two gives $D_{\mathrm{bin}}$. For a Poisson observation, the log-likelihood at mean $\mu$ is $Y_i\log\mu-\mu-\log(Y_i!)$. At the saturated mean $\mu=Y_i$, subtracting the fitted log-likelihood gives $Y_i\log(Y_i/\hat\mu_i)-(Y_i-\hat\mu_i)$ when $Y_i>0$. For $Y_i=0$, the saturated log-likelihood is zero apart from the common factorial term, and the difference is $\hat\mu_i$, matching the stated zero-count convention. Summation and multiplication by two give $D_{\mathrm{Pois}}$. $\square$

Deviance Is Nonnegative

Because the saturated model can reproduce every fitted-model distribution by choosing its observation-specific parameters, its maximized likelihood cannot be smaller. The likelihood comparison therefore suggests that deviance is nonnegative. The formulas also show this directly.

Result: Each binomial or Poisson observation contributes a nonnegative amount to deviance. For a binomial observation, its contribution is zero exactly when $\hat p_i=Y_i/m_i$. For a Poisson observation, its contribution is zero exactly when $\hat\mu_i=Y_i$.

Proof. For a binomial observation, set $q=Y_i/m_i$ and $p=\hat p_i$. Its contribution is $2m_i$ times

q\log(q/p)+(1-q)\log\!\left(\frac{1-q}{1-p}\right).

For $0<q<1$, apply $\log x\leq x-1$ to $p/q$ and $(1-p)/(1-q)$. The negative of the displayed expression is at most

$$ q(p/q-1)+(1-q)((1-p)/(1-q)-1)=0. $$

Thus the contribution is nonnegative, with equality only if $p=q$, since equality in $\log x\leq x-1$ requires $x=1$. The cases $q=0$ and $q=1$ follow directly from the formula, with equality only at $p=q$. For a Poisson observation with $Y_i>0$, write $t=Y_i/\hat\mu_i$. Its contribution is $2\hat\mu_i(t\log t-t+1)$. The function $t\log t-t+1$ has derivative $\log t$, decreases up to $t=1$, increases after $t=1$, and equals zero there. It is therefore nonnegative, with equality only at $t=1$. If $Y_i=0$, the contribution is $2\hat\mu_i>0$. $\square$

Worked Calculations

Worked Example: Grouped Binomial Data

Two groups each contain 10 individuals. The observed success counts are 2 and 8. Fit an intercept-only binomial model, so both groups share one success probability. Calculate the deviance.

There are 10 successes among 20 trials, so the maximum-likelihood estimate under the common-probability model is $\hat p=10/20=0.5$. In the saturated model the group probabilities are $2/10=0.2$ and $8/10=0.8$. Thus

\begin{aligned} D_{\mathrm{bin}} &=2\left[ 2\log(0.2/0.5)+8\log(0.8/0.5)\right.\\ &\qquad\left.+8\log(0.8/0.5)+2\log(0.2/0.5) \right]\\ &=4\left[2\log(0.4)+8\log(1.6)\right]\\ &=4\left[2(-0.916291)+8(0.470004)\right]\\ &\approx 7.710. \end{aligned}

The two groups’ observed proportions differ substantially, whereas the fitted model requires them to be equal. The positive deviance records the resulting loss in likelihood relative to a model that assigns a separate probability to each group.

Worked Example: Poisson Counts With a Common Mean

Suppose three independent Poisson counts are $0,2,6$, and fit a model with one common mean. Calculate the deviance.

The fitted common mean is the sample average, $\hat\mu=(0+2+6)/3=8/3$. The saturated means are $0,2,6$, respectively. The three contributions are

\begin{aligned} D_1&=2(8/3)=5.3333,\\ D_2&=2\left[2\log(2/(8/3))-(2-8/3)\right]\\ &=2\left[2\log(3/4)+2/3\right]\approx 0.1826,\\ D_3&=2\left[6\log(6/(8/3))-(6-8/3)\right]\\ &=2\left[6\log(9/4)-10/3\right]\approx 3.0645. \end{aligned}

Adding them gives $D_{\mathrm{Pois}}\approx5.3333+0.1826+3.0645=8.5804$. The zero count contributes $2\hat\mu$, not zero: the fitted model assigns positive expected count to an observation where none occurred.

Worked Example: Ungrouped Binary Outcomes

Four independent Bernoulli observations are $0,1,0,1$. Fit an intercept-only logistic model and calculate the deviance using one Bernoulli observation per row.

The fitted probability is the sample proportion, $\hat p=2/4=0.5$. The saturated model assigns each row its own probability: zero to rows with outcome 0 and one to rows with outcome 1. Each saturated Bernoulli likelihood is one, so the saturated log-likelihood is zero. The fitted log-likelihood is

\ell(\hat p)=2\log(0.5)+2\log(0.5)=4\log(0.5).

Therefore

D=-2\ell(\hat p)=-8\log(0.5)=8\log 2\approx5.545.

This is not the same deviance as representing the same outcomes as one grouped binomial observation with $m=4$, $Y=2$: that grouped model is itself saturated under an intercept-only fit and has deviance zero. The saturated model depends on the observational units and grouping used in the likelihood.

Interpreting Deviance and Avoiding a Common Error

Deviance measures how much worse the fitted model’s likelihood is than the saturated model’s likelihood. Within a fixed data representation, a larger deviance signals a greater discrepancy between fitted values and observations. It does not by itself identify which rows drive the discrepancy; the individual contributions can help locate them. Nor does a given numerical value have a universal interpretation across different sample sizes, groupings, or models. Reference distributions and degrees of freedom require additional conditions and are considered separately.

Common mistake: “The observations are binary, so the saturated model must have deviance zero, and any nonzero deviance means the calculation is wrong.” The first part confuses the saturated likelihood with the fitted model’s deviance. A row-level saturated Bernoulli model fits each observed zero or one perfectly, but the fitted model generally does not. The deviance is the likelihood gap between them and can be positive, as the ungrouped example shows. Grouping changes the saturated model and can change the deviance, so the data representation must be stated.

Key takeaway: Deviance is twice the log-likelihood loss relative to a saturated model. For binomial and Poisson GLMs, its contributions are nonnegative and vanish only when fitted probabilities or means match the corresponding saturated values. Interpret it in the context of the model and the observational units.

Check Your Understanding

Use the likelihood comparison and the distinction between grouped and ungrouped observations to answer these questions.

What parameter does the saturated model assign to each grouped binomial observation?
For a Poisson observation with $Y_i=0$, what is its deviance contribution, and why is it not zero when $\hat\mu_i>0$?
Why must the likelihood ratio be written with the saturated likelihood in the numerator when expressing deviance as $-2$ times a log ratio?
For a grouped binomial observation, under what condition is its deviance contribution zero?
Why can the same binary outcomes have different deviances when recorded as individual Bernoulli rows versus one grouped binomial count?