How Variance Assumptions Change Model Interpretation

Why the Variance Assumption Matters

Two regression models can assign the same fitted mean to an observation and still describe different response distributions. That difference matters: it changes the amount of response variability the model expects, and it can change the standard error of an estimated effect. In a generalized linear model, the mean structure and the variance function play distinct roles. Matching the fitted means does not make the models interchangeable.

This comparison focuses on Poisson and Gamma models with a log link. The Poisson model is used for counts and ties the conditional variance to the conditional mean. The Gamma model is used for positive continuous responses and makes the conditional variance proportional to the square of the mean. The models therefore respond differently to changes in fitted mean, and their coefficient uncertainties need not be ordered the same way at every mean and dispersion.

Definition: For an observation with predictors \(x_i\), write its conditional mean as \(\mu_i=E[Y_i\mid x_i]\). A Poisson model assumes \(Y_i\mid x_i\sim\operatorname{Poisson}(\mu_i)\), so \(\operatorname{Var}(Y_i\mid x_i)=\mu_i\). A Gamma model assumes a positive continuous response with \(E[Y_i\mid x_i]=\mu_i\) and \(\operatorname{Var}(Y_i\mid x_i)=\phi\mu_i^2\), where \(\phi>0\) is the dispersion parameter.

For the Gamma parameterization used here, the conditional distribution can be written with shape \(1/\phi\) and scale \(\phi\mu_i\). Its mean is \((1/\phi)(\phi\mu_i)=\mu_i\), and its variance is \((1/\phi)(\phi\mu_i)^2=\phi\mu_i^2\). These moment assumptions are the basis for the comparisons below. Observations are assumed independent conditional on the predictors, and the stated mean and variance relationships are assumed to be appropriate for the data.

Matched Means, Different Response Variability

At a given fitted mean, the Poisson variance is \(\mu\), while the Gamma variance is \(\phi\mu^2\). The associated standard deviations are \(\sqrt{\mu}\) and \(\sqrt{\phi}\mu\), respectively. The coefficients of variation—the standard deviation divided by the mean—are \(1/\sqrt{\mu}\) for Poisson and \(\sqrt{\phi}\) for Gamma. Thus, the Poisson coefficient of variation changes with the mean, whereas the Gamma coefficient of variation is constant when \(\phi\) is fixed.

Worked Example: Comparing Variability at the Same Fitted Mean

A hypothetical count model and a hypothetical positive-amount model both have fitted mean \(\mu=10\) for an observation. The Poisson model is assumed to have variance equal to its mean. The Gamma model has known dispersion \(\phi=0.25\). Compare their variances, standard deviations, and coefficients of variation.

Method. Apply each model’s variance function at the common mean, then divide each standard deviation by 10 to obtain its coefficient of variation.

Poisson model. The variance is \(\operatorname{Var}(Y\mid x)=\mu=10\), so the standard deviation is \(\sqrt{10}\approx3.1623\). The coefficient of variation is \(\sqrt{10}/10=1/\sqrt{10}\approx0.3162\).

Gamma model. The variance is \(\operatorname{Var}(Y\mid x)=\phi\mu^2=0.25(10^2)=25\). The standard deviation is \(\sqrt{25}=5\), and the coefficient of variation is \(5/10=0.5000\).

Check. Both models have mean 10, but the Gamma model assigns variance 25 rather than 10. Its standard deviation is larger, and its coefficient of variation is \(0.5000\) rather than \(0.3162\). The comparison concerns the models’ conditional variability; a count response and a positive continuous response remain different types of outcome.

Common mistake avoided. Equal fitted means do not imply equal fitted variances. The variance must be calculated from the model’s own variance function, including the Gamma dispersion parameter.

A larger variance means greater spread around the conditional mean under the assumed model; it does not by itself say that the fitted mean is biased or that the model fits well. In particular, the Gamma dispersion parameter matters. At the same mean, a Gamma model with small \(\phi\) can have less variance than a Poisson model, while one with large \(\phi\) can have more.

How the Variance Function Affects Coefficient Uncertainty

The variance function also determines how much information an observation provides about regression coefficients. Consider independent observations with a log-link mean model \(\log(\mu_i)=x_i^\mathsf{T}\beta\), where \(x_i\) is the predictor vector and \(\beta\) is the coefficient vector. Under the standard GLM information calculation, the weight for observation \(i\) is \((d\mu_i/d\eta_i)^2/\operatorname{Var}(Y_i\mid x_i)\), where \(\eta_i=\log(\mu_i)\). Because \(d\mu_i/d\eta_i=\mu_i\), the Poisson and Gamma weights differ as follows.

w_i^{\text{Poisson}}=\frac{\mu_i^2}{\mu_i}=\mu_i, \qquad w_i^{\text{Gamma}}=\frac{\mu_i^2}{\phi\mu_i^2}=\frac{1}{\phi}.

This calculation gives the key contrast. Under the Poisson model, an observation with a larger fitted mean has a larger information weight. Under the Gamma model with fixed dispersion, the information weight does not depend on the fitted mean. These are model-based information statements; they do not imply that every individual dataset will yield standard errors in a predictable direction.

Result: For independent observations in a log-link GLM, the information contribution for a coefficient direction is weighted by \(\mu_i\) under the Poisson variance assumption and by \(1/\phi\) under the Gamma variance assumption. Coefficient uncertainty depends on these weights and on the predictor design, not just on the fitted means.

A short derivation explains why. For a small change in the linear predictor \(\eta_i\), the corresponding change in the mean is approximately \((d\mu_i/d\eta_i)\) times that change. The squared change in mean, measured relative to the response variance, is therefore proportional to \((d\mu_i/d\eta_i)^2/\operatorname{Var}(Y_i\mid x_i)\). With a log link, the derivative is \(\mu_i\). Substituting the Poisson variance \(\mu_i\) gives weight \(\mu_i\); substituting the Gamma variance \(\phi\mu_i^2\) gives weight \(1/\phi\). Summing these weighted contributions across observations forms the model’s information matrix. Its inverse gives the large-sample covariance approximation for the coefficient estimates, when the model and regularity conditions are suitable.

A Two-Group Log-Mean Contrast

For a transparent comparison, suppose there are two independent groups, with \(n_0\) observations in group 0 and \(n_1\) in group 1. Let their fitted means be \(\mu_0\) and \(\mu_1\). With a log link and a group indicator, the group coefficient is \(\beta_1=\log(\mu_1)-\log(\mu_0)\), the log ratio of the fitted means.

Under the usual large-sample approximation, the variance of a group’s estimated log mean is \(1/(n\mu)\) for Poisson data and \(\phi/n\) for Gamma data with known dispersion. Since the groups are independent, the variance of their log-mean difference is the sum of the two group variances:

\operatorname{Var}(\widehat{\beta}_1)_{\text{Poisson}} \approx \frac{1}{n_1\mu_1}+\frac{1}{n_0\mu_0}, \qquad \operatorname{Var}(\widehat{\beta}_1)_{\text{Gamma}} \approx \frac{\phi}{n_1}+\frac{\phi}{n_0}.

These expressions assume independent observations, the stated variance functions, a correctly specified log-link mean structure, and adequate conditions for the large-sample approximation. In a Gamma analysis, dispersion is often estimated rather than known; using an estimated dispersion adds uncertainty to inference. The formula shown treats \(\phi\) as known, as specified in the examples.

Worked Example: Correctly Counting Gamma Information

A hypothetical two-group study has 8 independent observations in each group. A log-link model has fitted means 3 in group 0 and 12 in group 1. Calculate the approximate standard error of the group-indicator coefficient under (a) a Poisson model and (b) a Gamma model with known \(\phi=0.25\).

Method. The coefficient is the log ratio \(\log(12/3)\). Use the independent-group variance formulas and take the square root of each coefficient variance.

Poisson calculation. The group 1 contribution is \(1/(8\cdot12)=1/96\), and the group 0 contribution is \(1/(8\cdot3)=1/24=4/96\). Therefore,

\operatorname{Var}(\widehat{\beta}_1)_{\text{Poisson}} \approx \frac{1}{96}+\frac{1}{24} =\frac{5}{96} \approx0.0520833, \qquad \operatorname{SE}(\widehat{\beta}_1)_{\text{Poisson}} =\sqrt{\frac{5}{96}} \approx0.2282.

Gamma calculation. Each observation has information weight \(1/\phi=1/0.25=4\). With 8 observations in a group, the total group information is \(8\cdot4=32\), so the group’s log-mean variance contribution is \(1/32\). Both groups have the same number of observations and the same dispersion. Thus,

\operatorname{Var}(\widehat{\beta}_1)_{\text{Gamma}} \approx \frac{1}{32}+\frac{1}{32} =\frac{1}{16}=0.0625, \qquad \operatorname{SE}(\widehat{\beta}_1)_{\text{Gamma}} =\sqrt{\frac{1}{16}}=0.2500.

Check. The fitted mean ratio is \(12/3=4\), so the coefficient is \(\log(4)\); the standard errors describe uncertainty in estimating that log ratio. Here the Gamma standard error, \(0.2500\), is larger than the Poisson standard error, approximately \(0.2282\).

Common mistake avoided. A frequent error is to use total Gamma information 16 per group. But each of the 8 observations contributes 4 units, so the group total is 32, not 16. The correct group contribution to variance is \(1/32\), giving contrast variance \(1/16\) and standard error \(0.2500\).

Worked Example: The Standard-Error Ordering Can Reverse

A hypothetical two-group comparison has 5 independent observations in each group, with fitted means 4 and 9. Compare the approximate standard errors of the log-mean contrast for a Poisson model and a Gamma model with known \(\phi=0.10\). State which model gives the smaller standard error for this setup.

Method. Apply the two-group formulas to the same fitted means and sample sizes. The coefficient itself is \(\log(9/4)\), but its standard error depends on the variance assumption.

Poisson calculation. The two contributions are \(1/(5\cdot9)=1/45\) and \(1/(5\cdot4)=1/20\). Hence,

\operatorname{Var}(\widehat{\beta}_1)_{\text{Poisson}} \approx\frac{1}{45}+\frac{1}{20} =\frac{4}{180}+\frac{9}{180} =\frac{13}{180} \approx0.0722222, \qquad \operatorname{SE}\approx\sqrt{\frac{13}{180}}\approx0.2687.

Gamma calculation. Each group contributes \(\phi/n=0.10/5=0.02\). The contrast variance and standard error are therefore

\operatorname{Var}(\widehat{\beta}_1)_{\text{Gamma}} \approx0.02+0.02=0.04, \qquad \operatorname{SE}=\sqrt{0.04}=0.2000.

Check. For these assumptions, the Gamma standard error is smaller: \(0.2000<0.2687\). This is consistent with the calculations, not a general ranking of the models. A different Gamma dispersion, group size, or set of fitted means can change the comparison.

Common mistake avoided. Do not infer the coefficient standard-error ordering from one response variance alone. The Poisson and Gamma models use different information weights, which are accumulated over both groups and then combined for the contrast.

Interpretation and Exam Traps

The distinction between response variability and coefficient uncertainty is important. The response variance describes the spread of a new response around its conditional mean. A coefficient standard error describes sampling uncertainty in an estimated model parameter. The latter is affected by the response variance, but also by sample size, predictor values, and how the parameter is defined. A fitted mean comparison alone is not enough to determine the standard error.

For a Poisson log-link model, larger fitted means contribute more information about the log mean because the weight is \(\mu_i\). For a Gamma log-link model with fixed \(\phi\), changing the fitted mean does not change the per-observation weight \(1/\phi\). Increasing the number of independent observations increases total information in both models and generally reduces coefficient uncertainty. Increasing Gamma dispersion decreases its information weight and increases uncertainty.

Do not equate means with variances. The Poisson variance is \(\mu\); the Gamma variance is \(\phi\mu^2\).
Do not omit dispersion. A Gamma variance calculation without \(\phi\) is incomplete unless its value is explicitly 1.
Do not confuse an observation’s weight with a group’s information. Multiply the per-observation weight by the number of observations before taking its reciprocal for a group log-mean variance.
Do not assert a universal standard-error ranking. Compare the relevant formulas using the actual means, sample sizes, and dispersion.
Do not compare outcome distributions as if their supports were identical. Poisson responses are counts; Gamma responses are positive continuous quantities. Matching means is a focused comparison of model assumptions, not a claim that the response types are interchangeable.

Key takeaway: At matched fitted means, Poisson and Gamma models can imply different response variability and different coefficient uncertainty. Use the variance function to find response variance, then use the model’s information weights and the design to assess coefficient precision.

Check Your Understanding

For each problem, use the stated model assumptions; standard errors are approximate unless the problem says otherwise.

A Poisson model and a Gamma model both have fitted mean 16. The Gamma dispersion is \(\phi=0.0625\). Calculate both response variances and state whether the standard deviations and coefficients of variation match. (Answer: Both variances are 16, both standard deviations are 4, and both coefficients of variation are 0.2500.)
Two independent groups each have 6 observations in a Gamma log-link model with known \(\phi=0.36\). Find the approximate variance and standard error of the group-indicator coefficient. (Answer: Variance \(=0.36(1/6+1/6)=0.12\); standard error \(\approx0.3464\).)
A Poisson log-link model has 10 observations in each of two groups, with fitted means 2 and 8. Find the approximate variance and standard error of the log-mean contrast. (Answer: Variance \(=1/20+1/80=1/16=0.0625\); standard error \(=0.2500\).)
A Gamma log-link model has 4 observations in each of two groups, fitted means 5 and 20, and known \(\phi=0.20\). Find the group-indicator coefficient and its approximate standard error. (Answer: Coefficient \(=\log(20/5)=\log(4)\approx1.3863\); variance \(=0.20(1/4+1/4)=0.10\); standard error \(\approx0.3162\).)
For two groups with 10 independent observations each and fitted means 2 and 8, compare the contrast standard errors under Poisson and Gamma assumptions with known Gamma dispersion \(\phi=0.10\). (Answer: Poisson variance \(=1/20+1/80=0.0625\), standard error \(=0.2500\); Gamma variance \(=0.10(1/10+1/10)=0.0200\), standard error \(\approx0.1414\).)

What You'll Learn