Empirical Mean–Variance Patterns

Why Compare Means and Variances?

A continuous response can have much the same amount of scatter across its range, or its scatter can widen as its expected value increases. That distinction matters when assessing a generalized linear model family: a Gaussian model has a constant conditional variance, while gamma, inverse-Gaussian, and Tweedie models describe variance that grows with the mean in different ways. Grouped means and variances offer a direct, preliminary way to look for these patterns.

The summaries are evidence, not a family-selection rule. A variance pattern can be obscured by small groups, outliers, or mixing observations with different underlying means. It also does not by itself check the response support or establish that a particular distribution is appropriate. The aim here is to calculate the summaries correctly and interpret their pattern with those limitations in view.

Definition: For a group of $m$ response observations $Y_1,\ldots,Y_m$, the sample mean and unbiased sample variance are $$ \bar{Y}=\frac{1}{m}\sum_{i=1}^{m}Y_i,\qquad S^2=\frac{1}{m-1}\sum_{i=1}^{m}(Y_i-\bar{Y})^2, $$ provided $m\geq 2$. In a grouped mean–variance display, each group contributes a point with horizontal coordinate $\bar{Y}$ and vertical coordinate $S^2$. Groups are usually formed using covariates or an initial estimate of the conditional mean, rather than by sorting observations on their response values.

Patterns Implied by Common Variance Functions

In a generalized linear model, the conditional mean for observation $i$ is $\mu_i=E[Y_i\mid x_i]$. A variance function describes how the conditional variance depends on that mean. In the common notation $\operatorname{Var}(Y_i\mid x_i)=\phi V(\mu_i)$, the dispersion parameter $\phi$ sets the scale, while $V$ describes the shape of the mean–variance relationship.

For a Gaussian response with constant dispersion, $V(\mu)=1$, so the conditional variance is $\phi$, regardless of the mean.
For a gamma response, $V(\mu)=\mu^2$, so the conditional variance is $\phi\mu^2$.
For an inverse-Gaussian response, $V(\mu)=\mu^3$, so the conditional variance is $\phi\mu^3$.
For a Tweedie model with power $p$, $V(\mu)=\mu^p$, giving $\operatorname{Var}(Y\mid x)=\phi\mu^p$ where that power-family model is defined.

These equations suggest different broad shapes in a plot of group variance against group mean: a horizontal pattern for constant variance, a curve proportional to the square of the mean for gamma-like growth, and still faster growth for cubic or higher-power patterns. More generally, if the variance follows $\phi\mu^p$, doubling the mean multiplies the variance by $2^p$. This relationship is about the model’s conditional variance; a finite-group sample variance is only an estimate of it.

Worked Example: A Roughly Constant Variance Pattern

Suppose covariates divide observations into two groups. The response values are $8,10,12,10$ in the first group and $18,20,22,20$ in the second. Calculate each group’s mean and sample variance.

For the first group, the mean is $(8+10+12+10)/4=10$. The deviations from 10 are $-2,0,2,0$, so the sum of squared deviations is $4+0+4+0=8$. Thus $S^2=8/(4-1)=8/3$.

For the second group, the mean is $(18+20+22+20)/4=20$. The deviations from 20 are again $-2,0,2,0$, so $S^2=8/3$ as well.

The group means differ, but the sample variances are equal. This is consistent with roughly constant dispersion, as in a Gaussian model with constant variance. It does not prove that the response is Gaussian: two groups with four observations each provide limited evidence, and other features of the response distribution still matter.

Worked Example: A Quadratic Variance Pattern

Consider three groups with response values $1,1,3,3$; $2,2,6,6$; and $3,3,9,9$. Compute their means and sample variances, then compare the variance with the square of the mean.

The first group has mean $(1+1+3+3)/4=2$. Its deviations are $-1,-1,1,1$, with squared deviations summing to 4. Its sample variance is $4/3$, and $S^2/\bar{Y}^2=(4/3)/4=1/3$.

The second group has mean $(2+2+6+6)/4=4$. Its deviations are $-2,-2,2,2$, with squared deviations summing to 16. Its sample variance is $16/3$, and $S^2/\bar{Y}^2=(16/3)/16=1/3$.

The third group has mean $(3+3+9+9)/4=6$. Its deviations are $-3,-3,3,3$, with squared deviations summing to 36. Its sample variance is 12, and $S^2/\bar{Y}^2=12/36=1/3$.

The sample variance divided by the squared mean is the same in all three groups. These constructed summaries follow $S^2=\bar{Y}^2/3$, illustrating the quadratic pattern associated with a gamma variance function. In real data, estimated ratios will not usually match exactly; the relevant question is whether the pattern is plausible given sampling variability.

What Does a Grouped Sample Variance Estimate?

A grouped variance is most informative when observations in each group have similar conditional means. The following result makes this requirement precise. It also shows why pooling observations with meaningfully different means can create an apparent increase in variance even when the individual conditional variances are small.

Result: Suppose $Y_1,\ldots,Y_m$ are independent, have finite second moments, and satisfy $E[Y_i]=\mu$ and $\operatorname{Var}(Y_i)=\sigma^2$ for every $i$, where $m\geq2$. Then the sample variance $S^2$ is unbiased for $\sigma^2$: $E[S^2]=\sigma^2$.

Proof. The identity $\sum_{i=1}^{m}(Y_i-\bar{Y})^2=\sum_{i=1}^{m}(Y_i-\mu)^2-m(\bar{Y}-\mu)^2$ follows by writing $Y_i-\bar{Y}=(Y_i-\mu)-(\bar{Y}-\mu)$, expanding the squares, and using $\sum_{i=1}^{m}(Y_i-\mu)=m(\bar{Y}-\mu)$. Taking expectations, the first sum on the right has expectation $m\sigma^2$. Independence gives $\operatorname{Var}(\bar{Y})=\sigma^2/m$, and $E[(\bar{Y}-\mu)^2]=\sigma^2/m$. Therefore

E\!\left[\sum_{i=1}^{m}(Y_i-\bar{Y})^2\right] =m\sigma^2-m\frac{\sigma^2}{m} =(m-1)\sigma^2.

Dividing by $m-1$ proves $E[S^2]=\sigma^2$. $\square$

The equal-mean assumption is important. If observations in a group have different means, the expected sample variance includes both their conditional variances and the spread of their means.

Result: Suppose $Y_1,\ldots,Y_m$ are independent with finite second moments, means $\mu_i$, and variances $\sigma_i^2$. Write $\bar{\mu}=m^{-1}\sum_{i=1}^{m}\mu_i$. Then $$ E[S^2]=\frac{1}{m-1}\left\{\sum_{i=1}^{m}\sigma_i^2+\sum_{i=1}^{m}(\mu_i-\bar{\mu})^2\right\}. $$

Proof. Use the identity $\sum_{i=1}^{m}(Y_i-\bar{Y})^2=\sum_{i=1}^{m}Y_i^2-m\bar{Y}^2$. Independence implies $E[\bar{Y}^2]=\operatorname{Var}(\bar{Y})+(E[\bar{Y}])^2=m^{-2}\sum_i\sigma_i^2+\bar{\mu}^2$. Also $E[Y_i^2]=\sigma_i^2+\mu_i^2$. Consequently,

E\!\left[\sum_{i=1}^{m}(Y_i-\bar{Y})^2\right] =\left(1-\frac{1}{m}\right)\sum_{i=1}^{m}\sigma_i^2 +\sum_{i=1}^{m}\mu_i^2-m\bar{\mu}^2.

Since $\sum_i(\mu_i-\bar{\mu})^2=\sum_i\mu_i^2-m\bar{\mu}^2$, dividing the displayed expression by $m-1$ gives the stated formula. $\square$

Thus, even if the conditional variances $\sigma_i^2$ are identical, the second term can make the grouped sample variance large when the conditional means vary within the group. Groups based on broad ranges of fitted means can therefore blur the pattern one is trying to assess.

Worked Example: Interpreting Cubic Growth

Suppose a grouped summary gives mean–variance pairs $(1,0.5)$, $(2,4)$, and $(4,32)$. Assess the power pattern using the first and second pair, and then check it with the second and third.

From mean 1 to mean 2, the mean doubles and the variance changes from 0.5 to 4, a factor of $4/0.5=8$. A power relationship $V(\mu)\propto\mu^p$ would multiply the variance by $2^p$ when the mean doubles. Since $2^3=8$, this pair is consistent with $p=3$.

From mean 2 to mean 4, the mean again doubles and the variance changes from 4 to 32, also a factor of 8. The same cubic pattern accounts for both comparisons. Indeed, all three points satisfy $S^2=0.5\bar{Y}^3$: the predicted variances are $0.5$, $0.5(2^3)=4$, and $0.5(4^3)=32$.

This is an inverse-Gaussian-like empirical variance pattern. It is not a distributional identification: a cubic trend in grouped estimates does not establish the full inverse-Gaussian model, and estimation noise or group composition may affect the observed values.

Reading the Pattern Without Overclaiming

A useful first comparison is whether the group variances stay roughly level as the means change, or rise systematically. If they rise, compare the rate of increase with a power pattern rather than relying on the visual steepness alone. For a variance proportional to $\mu^p$, two mean levels $\mu_a,\mu_b>0$ imply a variance ratio $(\mu_b/\mu_a)^p$. For example, doubling the mean predicts a factor of 4 under quadratic growth and a factor of 8 under cubic growth. A Tweedie power between these values produces an intermediate factor; for example, $p=1.5$ predicts $2^{1.5}=2\sqrt{2}$, approximately 2.83.

Grouped estimates are noisy, especially when group sizes are small. A single extreme observation can strongly affect $S^2$, and groups of unequal size have estimates with differing precision. When possible, use enough observations per group to make the summaries informative, show the group sizes, and check whether conclusions depend on one group. If some groups have only one observation, their sample variance is undefined.

There is also a design issue: groups should reflect comparable covariate or fitted-mean levels. If the responses themselves are used to create low- and high-response groups, the resulting comparison is not a clean assessment of conditional dispersion. If covariate-based groups contain wide ranges of true means, the within-group mean spread contributes to $S^2$, as the result above shows.

Common Mistake

Common mistake: “The sample variance increases from one group to the next, so the response must follow an inverse-Gaussian distribution.”

Correction: Increasing variance is evidence about a mean–variance relationship, not proof of an entire response distribution. First check whether group means are comparable within groups, whether sample sizes make the estimates credible, and whether the trend is roughly constant, quadratic, cubic, or another power pattern. Distributional support and other aspects of the data also matter.

Key takeaway: Grouped sample variances provide a preliminary view of conditional dispersion. A roughly horizontal pattern is consistent with constant Gaussian variance; quadratic and cubic growth are consistent with gamma and inverse-Gaussian variance functions, while a general power trend can suggest a Tweedie variance function. Treat these patterns as evidence to investigate, not as conclusive identification.

Check Your Understanding

Answer each question using the definitions and results in this tutorial.

Why does the sample variance use $m-1$ in its denominator, and under what assumptions is it unbiased for the common variance?
If the mean doubles and the variance quadruples, which power $p$ is consistent with a variance proportional to $\mu^p$?
Why can a grouped sample variance be large even when the observations’ conditional variances are all equal?
What broad mean–variance pattern is consistent with a Gaussian model with constant dispersion?
Why does a cubic pattern in grouped means and variances not by itself prove that the response is inverse-Gaussian?