Mean, Variance, and Dispersion in Gaussian GLMs

Why Separate the Mean from the Spread?

A Gaussian GLM describes a response with a conditional mean and a conditional variance. The mean describes the model’s target at a given covariate setting; the variance describes how much individual responses fluctuate around that target. Separating these roles is useful: two settings can have different means but the same response spread, and two models with the same mean structure can imply different amounts of response variability.

In this tutorial, the conditional response at observation $i$ is Gaussian with mean $\mu_i$ and common variance $\sigma^2$, where $\sigma^2>0$. The notation $N(\mu,\sigma^2)$ uses its second argument for the variance, not the standard deviation. We will express the variance in GLM form, derive its variance function, and quantify how the dispersion affects the precision of an estimate of a conditional mean.

Definition: In a GLM variance specification, write the conditional variance as $$ \operatorname{Var}(Y_i\mid x_i)=\phi V(\mu_i). $$ Here $\phi>0$ is the dispersion parameter, and $V(\mu)$ is the variance function. The variance function describes how variance changes with the mean, apart from the overall scale $\phi$.

For the Gaussian model, $\phi=\sigma^2$ and $V(\mu)=1$. Thus the variance is constant as the conditional mean changes. This does not mean that all Gaussian GLMs must have the same variance: the common value is set by $\phi$, which can differ between models or applications.

Deriving the Gaussian Variance Function

Suppose $Y_i\mid x_i\sim N(\mu_i,\sigma^2)$. A normal density with mean $\mu_i$ and variance $\sigma^2$ is

f(y_i\mid x_i)=\frac{1}{\sqrt{2\pi\sigma^2}} \exp\left\{-\frac{(y_i-\mu_i)^2}{2\sigma^2}\right\}.

Set $\phi=\sigma^2$. Expanding the square in the exponent gives

-\frac{(y_i-\mu_i)^2}{2\phi} = \frac{y_i\mu_i-\mu_i^2/2}{\phi} -\frac{y_i^2}{2\phi}.

Consequently the density can be written in the exponential-dispersion form

f(y_i\mid x_i)= \exp\left\{ \frac{y_i\theta_i-b(\theta_i)}{a(\phi)} +c(y_i,\phi) \right\},

with $\theta_i=\mu_i$, $b(\theta_i)=\theta_i^2/2$, $a(\phi)=\phi$, and $c(y_i,\phi)=-y_i^2/(2\phi)-\tfrac12\log(2\pi\phi)$. In this representation, the mean is $b'(\theta_i)$, and the variance is $a(\phi)b''(\theta_i)$. Since $b'(\theta_i)=\theta_i$ and $b''(\theta_i)=1$, this gives mean $\mu_i$ and variance $\phi$.

Result: For a Gaussian GLM written as an exponential-dispersion model, $$ \theta_i=\mu_i,\qquad b(\theta_i)=\frac{\theta_i^2}{2},\qquad a(\phi)=\phi, $$ so that $$ E[Y_i\mid x_i]=\mu_i,\qquad \operatorname{Var}(Y_i\mid x_i)=\phi,\qquad V(\mu_i)=1. $$ The Gaussian variance function is constant; the dispersion is the conditional variance itself.

The same conclusion follows directly from the normal model. By definition, $Y_i\mid x_i\sim N(\mu_i,\sigma^2)$ has conditional expectation $\mu_i$ and conditional variance $\sigma^2$. Matching this with $\phi V(\mu_i)$, with $\phi=\sigma^2$, requires $V(\mu_i)=1$ for every possible $\mu_i$. The exponential-dispersion representation makes that separation explicit rather than changing the result.

Worked Example: Reading the Gaussian Parameters

At a specified covariate setting, suppose the response follows $Y\mid x\sim N(12,9)$. Identify the mean, variance, standard deviation, dispersion, and variance function value.

The conditional mean is $\mu=12$. Because the second argument of $N(12,9)$ is the variance, $\operatorname{Var}(Y\mid x)=9$, and the standard deviation is $\sigma=\sqrt{9}=3$.

For the Gaussian GLM parameterization, $\phi=\sigma^2=9$ and $V(12)=1$. Therefore $\phi V(12)=9(1)=9$, as required. The value 9 is both the dispersion and the response variance here; it is not the standard deviation.

How Dispersion Controls Response Variability

For a single response, the dispersion determines the scale of deviations from the conditional mean. Standardizing the response gives

Z=\frac{Y_i-\mu_i}{\sqrt{\phi}}\sim N(0,1).

This equality shows that $\sqrt{\phi}$ is the conditional standard deviation. For any $c>0$, the probability of being within $c$ standard deviations of the mean is $P(|Y_i-\mu_i|\leq c\sqrt{\phi}\mid x_i)=P(|Z|\leq c)$. The probability depends on $c$, not on $\mu_i$ or $\phi$. In contrast, the interval’s width in the response’s original units grows with $\sqrt{\phi}$.

Worked Example: Probability Within One Standard Deviation

Suppose $Y\mid x\sim N(12,9)$, so that the standard deviation is 3. Find the probability that the response lies between 9 and 15.

Standardize each endpoint using $Z=(Y-12)/3$. The lower endpoint becomes $(9-12)/3=-1$, and the upper endpoint becomes $(15-12)/3=1$. Thus

P(9\leq Y\leq 15\mid x) =P(-1\leq Z\leq 1) =\Phi(1)-\Phi(-1) =0.68268949\ldots\approx 0.6827.

The probability is approximately 0.6827 when rounded to four decimal places. The interval has width $2\sqrt{\phi}=6$, exactly two standard deviations. If the dispersion were changed, the probability within one standard deviation of the new mean would remain approximately 0.6827, but the interval measured in response units would change.

Worked Example: Same Mean, Different Dispersion

Compare two conditional Gaussian responses with mean 20: $Y_A\mid x\sim N(20,4)$ and $Y_B\mid x\sim N(20,25)$. Calculate their standard deviations and the probabilities of falling within 4 units of the mean.

For $Y_A$, the variance is 4 and the standard deviation is $\sqrt{4}=2$. The interval from 16 to 24 extends 2 standard deviations on either side, so $P(16\leq Y_A\leq24\mid x)=P(-2\leq Z\leq2)=\Phi(2)-\Phi(-2)\approx0.9545$.

For $Y_B$, the variance is 25 and the standard deviation is $\sqrt{25}=5$. The same interval, from 16 to 24, extends only $4/5=0.8$ standard deviations on either side. Hence $P(16\leq Y_B\leq24\mid x)=P(-0.8\leq Z\leq0.8)=\Phi(0.8)-\Phi(-0.8)\approx0.5763$.

Both responses have the same conditional mean, but $Y_B$ has the larger dispersion and is more variable. A fixed-width interval around the mean is therefore less likely to contain $Y_B$. The variance function is 1 for both models; their different spreads come from their different values of $\phi$.

Dispersion and the Precision of a Conditional-Mean Estimate

A conditional mean is often estimated using several independent responses observed under the same conditions. The following result quantifies how the number of replicates and the dispersion affect the precision of their average. Here, “precision” refers to sampling variance or standard error: a smaller value means less sampling fluctuation.

Result: Suppose $Y_1,\ldots,Y_m$ are independent and each has distribution $N(\mu,\phi)$, with the same unknown or fixed mean $\mu$ and known $\phi>0$. Their sample average $\bar Y=m^{-1}\sum_{j=1}^{m}Y_j$ has $$ E[\bar Y]=\mu,\qquad \operatorname{Var}(\bar Y)=\frac{\phi}{m},\qquad \bar Y\sim N\left(\mu,\frac{\phi}{m}\right). $$

To prove the expectation and variance statements, linearity of expectation gives

E[\bar Y] =\frac{1}{m}\sum_{j=1}^m E[Y_j] =\frac{1}{m}\sum_{j=1}^m\mu =\mu.

Independence makes each covariance between distinct responses zero. Therefore

\operatorname{Var}(\bar Y) =\frac{1}{m^2}\operatorname{Var}\left(\sum_{j=1}^mY_j\right) =\frac{1}{m^2}\sum_{j=1}^m\operatorname{Var}(Y_j) =\frac{m\phi}{m^2} =\frac{\phi}{m}.

Finally, a linear combination of independent Gaussian random variables is Gaussian. Since $\bar Y$ has mean $\mu$ and variance $\phi/m$, its distribution is $N(\mu,\phi/m)$. Its standard error, when $\phi$ is known, is thus $\sqrt{\phi/m}$. This proves all three claims. $\square$

Worked Example: Replicates and Mean Precision

Suppose 25 independent responses at the same covariate setting each have distribution $N(\mu,16)$. Calculate the variance and standard error of their average. Then find the number of independent replicates needed for the standard error to be at most 0.5, assuming the same dispersion.

Here $m=25$ and $\phi=16$. Therefore $\operatorname{Var}(\bar Y)=16/25=0.64$, and the standard error is $\sqrt{0.64}=0.8$. The individual response standard deviation is $\sqrt{16}=4$, so averaging 25 responses reduces the standard error from 4 to $4/\sqrt{25}=0.8$.

For a target standard error no greater than 0.5, solve $\sqrt{16/m}\leq0.5$. Squaring both nonnegative sides gives $16/m\leq0.25$, so $m\geq64$. Thus 64 independent replicates achieve standard error $0.5$; fewer than 64 do not meet the stated bound.

The result separates two influences on precision. Holding the number of independent replicates fixed, multiplying $\phi$ by a factor $k$ multiplies the variance of the average by $k$, and its standard error by $\sqrt{k}$. Holding dispersion fixed, multiplying the number of replicates by $k$ divides the variance by $k$, and the standard error by $\sqrt{k}$. These statements concern the average of independent responses sharing one mean. They do not by themselves describe the uncertainty of every possible fitted mean in a model.

Interpretation and a Common Pitfall

The Gaussian variance function $V(\mu)=1$ says that response variance does not increase or decrease with the conditional mean under this model. The dispersion $\phi$ sets the variance level, and $\sqrt{\phi}$ sets the standard-deviation scale. A larger dispersion means more response noise around the same conditional mean; it does not, by itself, change that mean.

Confusing variance and standard deviation. If $\phi=9$, then the variance is 9 and the standard deviation is 3, not 9.
Calling the dispersion the variance function. The Gaussian variance function is $V(\mu)=1$; the variance is $\phi V(\mu)=\phi$.
Assuming a larger mean forces a larger variance. That may occur with some other response families, but the Gaussian variance function is constant in the mean.
Assuming more observations change individual response variability. Replication reduces the sampling variance of an average, $\phi/m$, while each individual response still has variance $\phi$.

These distinctions matter when interpreting a Gaussian GLM: the mean describes the response level, the variance function describes how variance depends on the mean, and dispersion sets the overall conditional variability. For repeated independent observations sharing a mean, dispersion and replicate count together determine the precision of the sample average.

Check Your Understanding

Use the definitions and results above to answer these questions.

If $Y\mid x\sim N(7,36)$, what are the dispersion, variance, standard deviation, and Gaussian variance function value?
Why is the Gaussian variance function constant even if the conditional mean changes across covariate settings?
For 16 independent responses with common distribution $N(\mu,9)$, calculate the variance and standard error of their average.
With dispersion fixed, what happens to the standard error of an average if the number of independent replicates is multiplied by four? Explain using its formula.
Explain why two Gaussian responses with the same conditional mean can have different probabilities of lying within a fixed-width interval around that mean.