Gaussian and Gamma Deviance Residual Calculations

Why Gaussian and Gamma Deviance Residuals Differ

A deviance residual measures how much worse a fitted mean explains an observation than the saturated mean for that observation. The calculation depends on the response distribution: for a Gaussian model, the likelihood loss is a squared difference; for a Gamma model, it is a function of the observed-to-fitted ratio. The signed deviance residual preserves the direction of the discrepancy while its square equals the observation’s unit deviance.

Here, $y_i$ is the observed response and $\hat{\mu}_i$ is its fitted mean. We use the unit deviance $d(y_i,\hat{\mu}_i)$, which does not include the dispersion parameter, and define the signed deviance residual by attaching the sign of $y_i-\hat{\mu}_i$ to its square root. The model deviance is the sum of unit deviances. This convention keeps the residual calculation distinct from dispersion scaling.

Definition: For an observation with unit deviance $d(y_i,\hat{\mu}_i)\geq 0$, its signed deviance residual is $$ r_{D,i}=\operatorname{sign}(y_i-\hat{\mu}_i)\sqrt{d(y_i,\hat{\mu}_i)}, $$ where $\operatorname{sign}(0)=0$. Thus $r_{D,i}^2=d(y_i,\hat{\mu}_i)$, and the model deviance is $D=\sum_i r_{D,i}^2$.

Gaussian Deviance: Squared Differences

Suppose $Y_i$ has a Gaussian distribution with mean $\mu_i$ and common variance $\phi>0$. Its density at $y_i$ is proportional, as a function of $\mu_i$, to $\exp\{-(y_i-\mu_i)^2/(2\phi)\}$. The saturated mean for this observation is $y_i$, which makes its squared difference zero. Comparing the fitted mean with that saturated mean gives the Gaussian unit deviance.

Result: For a Gaussian response with fitted mean $\hat{\mu}_i$, $$ d(y_i,\hat{\mu}_i)=(y_i-\hat{\mu}_i)^2, \qquad r_{D,i}=\operatorname{sign}(y_i-\hat{\mu}_i)|y_i-\hat{\mu}_i| =y_i-\hat{\mu}_i. $$ The signed Gaussian deviance residual is exactly the response residual.

To verify this from the likelihood, the log-density for one observation is $\ell_i(\mu_i)=-\frac12\log(2\pi\phi)-\frac{(y_i-\mu_i)^2}{2\phi}$. At the saturated mean $\mu_i=y_i$, the squared-error term is zero. Subtracting the fitted log-likelihood from the saturated log-likelihood therefore gives

\ell_i(y_i)-\ell_i(\hat{\mu}_i) =\frac{(y_i-\hat{\mu}_i)^2}{2\phi} =\frac{d(y_i,\hat{\mu}_i)}{2\phi}.

This derivation identifies the unit deviance as the squared difference. It also shows where dispersion enters: with fixed $\phi$, the log-likelihood loss is the unit deviance divided by $2\phi$. The deviance residual itself is not divided by $\sqrt{\phi}$.

Worked Example: Gaussian Residuals and Total Deviance

Suppose three Gaussian observations are $y=(4,7,10)$, with fitted means $\hat{\mu}=(5,5,8)$. Calculate each unit deviance, each signed deviance residual, and the total deviance.

For the first observation, $y_1-\hat{\mu}_1=4-5=-1$. Thus $d_1=(-1)^2=1$ and $r_{D,1}=-\sqrt{1}=-1$.

For the second observation, $y_2-\hat{\mu}_2=7-5=2$. Thus $d_2=2^2=4$ and $r_{D,2}=+\sqrt{4}=2$.

For the third observation, $y_3-\hat{\mu}_3=10-8=2$. Thus $d_3=2^2=4$ and $r_{D,3}=+\sqrt{4}=2$.

The total deviance is $D=1+4+4=9$, and the residual squares confirm it: $(-1)^2+2^2+2^2=9$. If the common variance is $\phi=4$, the saturated-versus-fitted log-likelihood difference is $D/(2\phi)=9/8=1.125$. The residual values remain $(-1,2,2)$; the variance changes the likelihood scale, not these unit deviance residuals.

Gamma Deviance: Log-Ratio Differences

For a Gamma GLM, assume $y_i>0$, $\mu_i>0$, and $\phi>0$, with $\operatorname{Var}(Y_i)=\phi\mu_i^2$. Under the usual mean-dispersion parameterization, the density is $f(y_i;\mu_i,\phi)=y_i^{1/\phi-1}\exp\{-y_i/(\phi\mu_i)\}/[\Gamma(1/\phi)(\phi\mu_i)^{1/\phi}]$. In comparing means for a fixed observation and fixed dispersion, the terms involving $y_i$ alone and $\phi$ alone cancel. The part of the log-likelihood that depends on the mean is $-\{ \log(\mu_i)+y_i/\mu_i\}/\phi$.

Result: For a Gamma response with positive observed value $y_i$ and positive fitted mean $\hat{\mu}_i$, $$ d(y_i,\hat{\mu}_i) =2\left[\frac{y_i-\hat{\mu}_i}{\hat{\mu}_i} -\log\left(\frac{y_i}{\hat{\mu}_i}\right)\right] =2\left[\frac{y_i}{\hat{\mu}_i}-1 -\log\left(\frac{y_i}{\hat{\mu}_i}\right)\right], $$ and $$ r_{D,i} =\operatorname{sign}(y_i-\hat{\mu}_i) \sqrt{2\left[\frac{y_i}{\hat{\mu}_i}-1 -\log\left(\frac{y_i}{\hat{\mu}_i}\right)\right]}. $$

For the derivation, let $x=y_i/\mu_i$. The saturated mean is $\mu_i=y_i$, so its ratio $x$ is 1. Using the mean-dependent log-likelihood above, the saturated-minus-fitted log-likelihood is

\ell_i(y_i)-\ell_i(\mu_i) =\frac{1}{\phi}\left[\log\left(\frac{\mu_i}{y_i}\right) +\frac{y_i}{\mu_i}-1\right] =\frac{1}{\phi}\left[\frac{y_i}{\mu_i}-1 -\log\left(\frac{y_i}{\mu_i}\right)\right] =\frac{d(y_i,\mu_i)}{2\phi}.

This gives the stated unit deviance when $\mu_i=\hat{\mu}_i$. It also establishes that the formula is a likelihood loss: the Gamma contribution depends on the ratio $y_i/\hat{\mu}_i$, not simply on the absolute difference between response and fitted mean.

The expression under the square root is nonnegative. For $x>0$, define $g(x)=x-1-\log x$. Then $g(1)=0$ and $g'(x)=1-1/x$, which is negative for $0<x<1$ and positive for $x>1$. Therefore $g$ has its minimum at 1 and $g(x)\geq 0$, with equality only when $x=1$. Consequently, the Gamma unit deviance is nonnegative and is zero exactly when $y_i=\hat{\mu}_i$; the sign in the residual is positive above the fitted mean and negative below it.

Worked Example: A Gamma Observation Twice Its Fitted Mean

Let a Gamma observation be $y=20$, with fitted mean $\hat{\mu}=10$. Its ratio is $y/\hat{\mu}=2$, so

d(20,10)=2[2-1-\log(2)] =2[1-0.693147] \approx 0.613706.

Because $20-10>0$, the signed residual is positive: $r_D=+\sqrt{0.613706}\approx 0.7834$. Squaring the rounded residual gives approximately $0.6137$, agreeing with the unit deviance. If $\phi=0.5$, the log-likelihood loss relative to the saturated mean is $d/(2\phi)=0.613706/1=0.613706$. The unit deviance remains $0.613706$, regardless of that dispersion value.

Worked Example: Gamma Residuals Below and Above the Mean

Consider two Gamma observations, $y_1=5$ and $y_2=20$, each with fitted mean $\hat{\mu}=10$. Calculate the signed residuals and the total deviance.

For $y_1=5$, the ratio is $0.5$ and the response is below the fitted mean. Since $\log(0.5)=-0.693147$, $$ d_1=2[0.5-1-\log(0.5)] =2[-0.5+0.693147] \approx 0.386294. $$ Therefore $r_{D,1}=-\sqrt{0.386294}\approx -0.6215$.

For $y_2=20$, the ratio is $2$ and the response is above the fitted mean. As calculated directly from the formula, $d_2=2[2-1-\log(2)]\approx0.613706$, so $r_{D,2}=+\sqrt{0.613706}\approx0.7834$.

The total deviance is $D=d_1+d_2\approx0.386294+0.613706=1.000000$. The residual squares give the same total: approximately $0.6215^2+0.7834^2=0.3863+0.6137=1.0000$. The two observations are equally far from 10 in raw units, but their Gamma deviance contributions are not equal. The likelihood compares each response to its fitted mean on a ratio scale.

Worked Example: Equal Gamma Ratios at Different Scales

Compare $y=12,\hat{\mu}=10$ with $y=120,\hat{\mu}=100$. Both have ratio $y/\hat{\mu}=1.2$, although their raw differences are 2 and 20.

For either pair, $$ d=2[1.2-1-\log(1.2)] =2[0.2-0.182322] \approx 0.035357. $$ Both signed residuals are positive and equal to $\sqrt{0.035357}\approx0.1880$. Thus the Gamma deviance assigns the same contribution to the same proportional discrepancy at different scales. A squared-error calculation would instead give $2^2=4$ and $20^2=400$, illustrating why the Gaussian and Gamma residuals have different interpretations.

Reading the Two Residual Scales Correctly

The Gaussian deviance residual is measured in response units: its magnitude is the absolute difference between the observed and fitted values. The Gamma deviance residual is dimensionless and depends only on the positive ratio $y_i/\hat{\mu}_i$. For small relative discrepancies, the Gamma unit deviance is approximately a squared relative error. Indeed, writing $y_i/\hat{\mu}_i=1+\delta$, with $\delta$ near zero, the expansion $\log(1+\delta)=\delta-\delta^2/2+O(\delta^3)$ gives $d\approx\delta^2$. This approximation does not replace the exact formula, particularly for large discrepancies.

A frequent mistake is to use $(y_i-\hat{\mu}_i)^2$ as the Gamma deviance because it is the Gaussian formula. That ignores the Gamma likelihood: it makes the contribution depend strongly on response scale even when the proportional discrepancy is unchanged. The correction is to use $2[y_i/\hat{\mu}_i-1-\log(y_i/\hat{\mu}_i)]$, and then attach the sign of $y_i-\hat{\mu}_i$.

Another error is to insert the dispersion parameter inside the square root and call the result the deviance residual. Under the convention used here, the unit deviance and signed deviance residual omit $\phi$; for either family the likelihood loss for one observation is $d/(2\phi)$. If a dispersion-scaled residual is wanted for a separate purpose, it must be identified as such rather than substituted for the unit deviance residual.

Finally, the Gamma formula requires strictly positive observed responses and fitted means. A zero response cannot be inserted into this continuous Gamma likelihood formula, and a nonpositive fitted mean is outside the Gamma mean parameter space. Those are model-support requirements, not numerical inconveniences to resolve by taking an arbitrary logarithm.

Key takeaway: Gaussian deviance contributions are squared raw differences; Gamma contributions are twice a nonnegative log-ratio function. In both cases the signed residual squares to its observation-level unit deviance, while dispersion scales the corresponding likelihood loss.

Check Your Understanding

Work through these questions using the formulas and assumptions stated above.

For a Gaussian observation $y=3$ with fitted mean $\hat{\mu}=5$, calculate its unit deviance and signed deviance residual.
For a Gamma observation $y=15$ with fitted mean $\hat{\mu}=10$, write the exact unit-deviance expression and state the sign of its residual.
Why does the Gaussian likelihood loss equal $d/(2\phi)$, even though its unit deviance residual does not contain $\phi$?
Explain why two Gamma observations with the same positive response-to-fitted-mean ratio have the same unit deviance.
What support conditions must hold to calculate the Gamma deviance residual, and why is a zero observation not allowed in this formula?