Assessing Gamma Mean-Variance Fit

Why Assess the Gamma Mean-Variance Fit?

A Gamma regression specifies more than a positive conditional mean. Its variance is tied to that mean: observations with larger expected values should generally have greater absolute variability. A diagnostic check asks whether the observed residual spread is compatible with that relationship, rather than simply whether the fitted means look reasonable.

The check has two complementary parts. First, examine whether the response values are positive and whether their distributional pattern is plausibly right-skewed. Second, compare squared residuals with squared fitted means. A Gamma mean-variance specification predicts that the latter should track the former up to a constant dispersion factor. Neither check proves that the Gamma model is correct; both help identify patterns that deserve investigation.

Definition: In a Gamma response model, let $Y_i$ have conditional mean $\mu_i=E[Y_i\mid x_i]$, where $x_i$ denotes the covariates for observation $i$. The Gamma mean-variance specification is $$ \operatorname{Var}(Y_i\mid x_i)=\phi\mu_i^2,\qquad \mu_i>0,\quad \phi>0. $$ Here $\phi$ is the dispersion parameter. The model requires a positive response: a Gamma random variable assigns probability only to values greater than zero.

The coefficient of variation implied by this specification is constant across covariate values: $ \sqrt{\operatorname{Var}(Y_i\mid x_i)}/\mu_i=\sqrt{\phi} $. Thus, as the conditional mean increases, the typical absolute deviation increases in proportion to the mean. Squared deviations should consequently increase in proportion to $\mu_i^2$. This is the central comparison in the residual diagnostic.

What the Gamma Distribution Implies About Shape

A Gamma distribution is positive and, for finite positive shape, right-skewed. Right skew means that the distribution has a longer or more influential upper tail than lower tail; it does not mean every observed sample must display a dramatic tail. The extent of skewness is connected to the dispersion. In the usual shape-scale parameterization, shape $k=1/\phi$ and scale $\theta=\phi\mu$, giving mean $k\theta=\mu$ and variance $k\theta^2=\phi\mu^2$.

Result: If $Y$ has a Gamma distribution with shape $k>0$ and scale $\theta>0$, its skewness is $2/\sqrt{k}$. Under the mean-variance parameterization, this is $2\sqrt{\phi}$.

To verify the result, the raw moments are $E[Y]=k\theta$, $E[Y^2]=k(k+1)\theta^2$, and $E[Y^3]=k(k+1)(k+2)\theta^3$. Write $\mu=k\theta$. The third central moment is

\begin{aligned} E[(Y-\mu)^3] &=E[Y^3]-3\mu E[Y^2]+3\mu^2E[Y]-\mu^3\\ &=k(k+1)(k+2)\theta^3 -3k^2(k+1)\theta^3+3k^3\theta^3-k^3\theta^3\\ &=2k\theta^3. \end{aligned}

Since the variance is $k\theta^2$, the standard deviation is $\sqrt{k}\theta$. Dividing the third central moment by the cube of the standard deviation gives $2k\theta^3/(\sqrt{k}\theta)^3=2/\sqrt{k}$. Substituting $k=1/\phi$ gives $2\sqrt{\phi}$. Thus a right tail is consistent with a Gamma response, but right skew by itself does not establish that the conditional mean and variance are correctly specified. $\square$

Why Squared Residuals Should Track Fitted Means Squared

Let $\hat{\mu}_i$ be the fitted mean and let the raw residual be $r_i=y_i-\hat{\mu}_i$. If the true mean $\mu_i$ were known, the squared deviation from that mean would have conditional expectation $\phi\mu_i^2$. This provides the target for a plot of squared residuals against fitted means squared. In practice the true means are unknown, so fitted values and residuals are used as approximations.

Result: If $E[Y_i\mid x_i]=\mu_i$ and $\operatorname{Var}(Y_i\mid x_i)=\phi\mu_i^2$, then $$ E[(Y_i-\mu_i)^2\mid x_i]=\phi\mu_i^2. $$ Consequently, the conditional expected squared deviation is proportional to the squared conditional mean, with proportionality constant $\phi$.

Proof. By the definition of conditional variance,

\begin{aligned} E[(Y_i-\mu_i)^2\mid x_i] &=\operatorname{Var}(Y_i\mid x_i) +\{E[Y_i\mid x_i]-\mu_i\}^2\\ &=\phi\mu_i^2+( \mu_i-\mu_i)^2\\ &=\phi\mu_i^2. \end{aligned}

This identity holds observation by observation when the conditional mean and variance assumptions hold. It does not say that every squared residual equals $\phi\mu_i^2$; individual squared residuals can differ substantially from their expectation. $\square$

A useful diagnostic plot places $r_i^2$ on the vertical axis and $\hat{\mu}_i^2$ on the horizontal axis. The target is a roughly proportional increase in squared residuals, not a tight line through every point. The unknown dispersion determines the scale. A smooth trend that grows approximately linearly from near the origin is broadly compatible with the mean-variance relationship; a flat trend, strong curvature, or a pattern that changes systematically with fitted mean may suggest a mismatch.

Worked Example: Summarizing Squared Residuals in Two Groups

Suppose four observations have fitted mean $2$, with positive responses $1,3,1,4$, and four have fitted mean $5$, with positive responses $3,7,2,8$. Calculate the group averages of squared residuals and compare them with the fitted means squared.

For the first group, residuals are $1-2=-1$, $3-2=1$, $1-2=-1$, and $4-2=2$. Their squares are $1,1,1,4$, so the average squared residual is $ (1+1+1+4)/4=1.75 $. The squared fitted mean is $2^2=4$.

For the second group, residuals are $3-5=-2$, $7-5=2$, $2-5=-3$, and $8-5=3$. Their squares are $4,4,9,9$, so the average squared residual is $ (4+4+9+9)/4=6.5 $. The squared fitted mean is $5^2=25$.

The average squared residual rises from $1.75$ to $6.5$ as the squared fitted mean rises from $4$ to $25$, which is at least in the predicted direction. The ratios are $1.75/4=0.4375$ and $6.5/25=0.26$; they are not equal. With only four observations per group, this difference is not compelling evidence against the model. The calculation illustrates the diagnostic target, not a formal test or a precise estimate of dispersion.

Reading the Plot Without Overstating It

The plot is a visual check of a mean-variance pattern, not a standalone test of the entire model. A few large squared residuals can dominate a group average, especially with a right-skewed response. Look for a sustained pattern across the range of fitted means, and interpret it alongside the response scale, covariates, and fitted model.

For example, if squared residuals stay at about the same absolute level while fitted means squared increase substantially, the observed absolute spread may be more nearly constant than the Gamma specification predicts. Conversely, a roughly increasing spread is compatible with the specification, but does not prove it: an incorrect mean function can also produce residual patterns, and other features of the conditional distribution remain to be assessed.

Worked Example: Recognizing Nearly Constant Absolute Spread

Consider four positive responses with fitted mean $2$: $1.5, 2.5, 1, 3$. Consider another four with fitted mean $5$: $4.5, 5.5, 4, 6$. Compare their squared residual averages.

In the first group the residuals are $-0.5,0.5,-1,1$, so the squared residuals are $0.25,0.25,1,1$. Their average is $ (0.25+0.25+1+1)/4=0.625 $.

In the second group the residuals are also $-0.5,0.5,-1,1$; its squared residual average is therefore $0.625$ as well. Yet the fitted mean squared changes from $4$ to $25$. The ratios of average squared residual to fitted mean squared are $0.625/4=0.15625$ and $0.625/25=0.025$. If this pattern persisted across many observations, it would suggest that the absolute spread is not increasing as the Gamma mean-variance relationship predicts. These eight observations are only an illustration: their small size does not establish a population pattern.

Worked Example: Checking Positivity and a Right-Tail Pattern

Suppose ten comparable positive outcomes are $1,1.2,1.4,1.6,1.8,2,2.2,2.5,3,8$. The sum is $24.7$, so the sample mean is $24.7/10=2.47$. The median is $(1.8+2)/2=1.9$. Every value is positive, and the large upper observation makes the mean exceed the median, which is consistent with a right-tail pattern.

This sample therefore has two features compatible with Gamma outcomes: positive measurements and an upper tail. Those features do not verify the mean-variance specification. To assess that part, compare squared residuals with fitted means squared across observations or across suitable groups. If one recorded outcome in this example were actually zero, that value could not be an observation from an ordinary Gamma response model. It should prompt a check of the measurement and data definition, or consideration of a model that permits zero, rather than being treated as a valid Gamma observation.

A Practical Diagnostic Sequence

Check the response scale.
Confirm that the response values are positive and measured on a scale for which a Gamma model is plausible. Investigate zeros rather than silently including them as Gamma outcomes.

Inspect the distributional pattern.
Look for a plausible right tail, while remembering that skewness in a marginal sample is not by itself evidence about the conditional distribution after accounting for covariates.

Construct the variance diagnostic.
For each observation calculate $r_i=y_i-\hat{\mu}_i$, then plot $r_i^2$ against $\hat{\mu}_i^2$. Grouped averages or a smooth trend can make broad patterns easier to see.

Judge the trend cautiously.
Ask whether squared residuals increase roughly in proportion to squared fitted means. Treat isolated points and small-group differences as noisy evidence, not decisive failures.

Common Mistakes and Limitations

Expecting a straight, tight band. The relationship concerns a conditional expectation. It does not require each $r_i^2$ to lie near one line.
Confusing right skew with a successful variance check. A positive, right-skewed response may be compatible with a Gamma distribution while its residual spread still fails to scale with the fitted mean.
Treating fitted residuals as if the means were known. The identity $E[(Y_i-\mu_i)^2\mid x_i]=\phi\mu_i^2$ is exact for the true mean. In an analysis, $\hat{\mu}_i$ is estimated from the data, and residuals are affected by that fitting. The plotted relationship is therefore diagnostic, not an exact equality or an automatic hypothesis test.
Ignoring a zero because it is only one observation. An ordinary Gamma distribution has support on positive values only. A recorded zero requires a substantive explanation or a different response model.
Interpreting a rising pattern as proof. A rising squared-residual trend is consistent with the specified variance pattern, but it cannot establish the full Gamma distribution or rule out misspecification of the mean.

Key takeaway: For a Gamma response, the diagnostic target is $E[(Y_i-\mu_i)^2\mid x_i]=\phi\mu_i^2$. Positive, plausibly right-skewed outcomes and squared residuals that grow with fitted means squared support the specification, but neither feature alone confirms the model.

Check Your Understanding

Use the definitions and diagnostic target above to answer these questions.

Under a Gamma mean-variance specification, what is the expected squared deviation from the true conditional mean?
Why should a plot of squared raw residuals against fitted means squared show an increasing pattern if the variance specification is plausible?
What does constant absolute residual spread across a wide range of fitted means suggest, if the pattern persists in a sufficiently large data set?
Why is a right-skewed sample not enough to establish that the Gamma mean-variance relationship fits?
How should a recorded zero response be handled when assessing an ordinary Gamma model?