Why GLM Variance Functions Matter
A GLM specifies a relationship between predictors and the response mean, but modeling the mean is only part of describing the response. The variance also matters: it tells us how much variability the model allows at a given mean. In a GLM, this relationship is often expressed compactly as a variance function, derived from the response distribution’s exponential-family form.
This tutorial derives that relationship and applies it to Bernoulli, Poisson, and Gaussian responses. The central distinction is between the variance function itself and the response variance: the latter may include a separate dispersion factor. Confusing those quantities is a common source of incorrect calculations.
From the Cumulant Function to the Variance
Assume the response has the exponential-family form above, with \(a(\phi)>0\). For the derivation, take \(\phi\) as fixed, assume the support of \(Y\) does not depend on \(\theta\), and assume the cumulant function is twice differentiable in the interior of its natural-parameter space. Also assume the moment-generating function exists for \(t\) in a neighborhood of zero. These conditions allow us to differentiate the cumulant-generating function at zero.
The moment-generating function can be found by replacing \(y\theta\) in the density or probability mass function with \(y(\theta+a(\phi)t)\). The resulting expression has the same normalizing structure as the original exponential-family form, but with natural parameter \(\theta+a(\phi)t\). Dividing by the normalizer at \(\theta\) gives:
The cumulant-generating function is the logarithm of this moment-generating function. Differentiate it once to obtain the mean, and twice to obtain the variance:
Because the first derivative of a cumulant-generating function at zero is the mean and its second derivative at zero is the variance, this establishes \(E[Y]=b'(\theta)\) and \(\operatorname{Var}(Y)=a(\phi)b''(\theta)\). If \(b'(\theta)\) can be inverted to express \(\theta\) as a function of \(\mu\), then \(b''(\theta)\) can be written as a function of \(\mu\). That function is the variance function.
Bernoulli Responses
Suppose \(Y\) is Bernoulli with success probability \(p\), so \(Y\) takes values 0 and 1 and \(P(Y=1)=p\). Its probability mass function can be written in exponential-family form by setting \(a(\phi)=1\), \(\theta=\log(p/(1-p))\), and \(b(\theta)=\log(1+e^\theta)\). There is no separately estimated dispersion factor in this Bernoulli representation.
Differentiate the cumulant function: \(b'(\theta)=e^\theta/(1+e^\theta)=p\). A second derivative gives \(b''(\theta)=e^\theta/(1+e^\theta)^2=p(1-p)\). Since the mean is \(\mu=p\), the variance function is \(V(\mu)=\mu(1-\mu)\), and the full response variance equals that function under the Bernoulli convention \(a(\phi)=1\).
Worked Example: Variability of Hypothetical Claim Indicators
A hypothetical portfolio model treats each policy’s claim indicator \(Y\) as Bernoulli with success probability \(p=0.25\). Find the variance for one policy. Then find the mean and variance of the total number of claims across 40 independent policies, each with the same probability. Assume the indicators are mutually independent and use the Bernoulli GLM representation with \(a(\phi)=1\).
Method. For one policy, the mean is \(\mu=p=0.25\). The Bernoulli variance function is \(V(\mu)=\mu(1-\mu)\), so:
Thus the variance of one indicator is \(0.1875\). Let \(T\) be the total number of claims. By additivity of expectations, \(E[T]=40(0.25)=10\). By independence, variances add, so \(\operatorname{Var}(T)=40(0.1875)=7.5\). The standard deviation of the total is \(\sqrt{7.5}\approx2.7386\).
Check and common mistake. The total’s mean of 10 is consistent with 40 policies each having claim probability 0.25. Do not use \(40^2(0.1875)\) for its variance: that would incorrectly treat the 40 indicators as perfectly dependent rather than independent.
The Bernoulli variance is largest at \(\mu=0.5\), where \(V(0.5)=0.25\), and decreases as the mean approaches either 0 or 1. This reflects the fact that an indicator is less variable when one outcome is much more likely than the other.
Poisson Responses
Suppose \(Y\) is Poisson with rate \(\lambda>0\). Its probability mass function has exponential-family form with \(a(\phi)=1\), \(\theta=\log(\lambda)\), and \(b(\theta)=e^\theta\). The mean is \(b'(\theta)=e^\theta=\lambda\), and the second derivative is also \(b''(\theta)=e^\theta=\lambda\). Therefore \(V(\mu)=\mu\), and the Poisson response variance equals its mean under this representation.
This is a distributional statement about a Poisson response: at a specified mean, its variance is the same numerical value. It does not say that every count response has variance equal to its mean, nor does it establish that a Poisson model is suitable for a particular data set.
Worked Example: A Hypothetical Event Count
For a hypothetical unit of exposure, assume the event count \(Y\) follows a Poisson distribution with rate \(\lambda=3.6\). Find its mean, variance, and standard deviation. Then find the variance of the sum of counts from five independent units with the same rate. Use the Poisson GLM representation with \(a(\phi)=1\).
Method. The Poisson mean is \(\mu=\lambda=3.6\). The variance function is \(V(\mu)=\mu\), so the variance for one unit is \(V(3.6)=3.6\). Its standard deviation is \(\sqrt{3.6}\approx1.8974\).
Let \(T\) be the sum across the five independent units. Its mean is \(E[T]=5(3.6)=18\). Its variance is \(\operatorname{Var}(T)=5(3.6)=18\), because the variances of independent counts add. The standard deviation of \(T\) is \(\sqrt{18}\approx4.2426\).
Check and common mistake. The aggregate mean and variance are both 18, as expected for the sum of independent Poisson counts with the same rate. Do not square the rate to obtain the variance: \(3.6^2=12.96\) is not the Poisson variance.
Gaussian Responses and the Dispersion Factor
Suppose \(Y\) has a Gaussian distribution \(N(\mu,\sigma^2)\), where the second parameter is the variance. In exponential-family form, take \(\theta=\mu\), \(b(\theta)=\theta^2/2\), and \(a(\phi)=\phi=\sigma^2\). The remaining term \(c(y,\phi)\) supplies the parts of the normal density that do not involve \(\theta\). Differentiating gives \(b'(\theta)=\theta=\mu\) and \(b''(\theta)=1\). Thus the Gaussian variance function is \(V(\mu)=1\), while the full response variance is \(a(\phi)V(\mu)=\phi=\sigma^2\).
The constant variance function does not mean that a Gaussian response has variance 1. It means that the mean-dependent part of its variance is constant; the dispersion factor sets the actual variance. A Gaussian model with a different \(\sigma^2\) has the same \(V(\mu)=1\) but a different response variance.
Worked Example: Separating Gaussian Variance From Dispersion
A hypothetical continuous response is modeled as \(Y\sim N(120,36)\), with mean 120 and variance 36. Assume independent observations and the Gaussian exponential-family representation \(a(\phi)=\phi\). Find the variance of one response and the variance of the average of nine such observations.
Method. The Gaussian variance function is \(V(\mu)=1\), and the dispersion is \(\phi=36\). Therefore the variance of one response is \(\phi V(\mu)=36(1)=36\), and its standard deviation is \(\sqrt{36}=6\).
Let \(\bar{Y}\) be the average of the nine independent observations. The variance of a sum of independent observations is the sum of their variances, and dividing that sum by \(9\) to form an average divides its variance by \(9^2\). Hence:
The average therefore has standard deviation \(\sqrt{4}=2\). Check and common mistake: its variance is less than the variance of one observation, as averaging independent observations should be. Using \(V(\mu)=1\) as the full variance would incorrectly give \(1\) for each observation and \(1/9\) for the average; the dispersion factor 36 must be included.
Comparison and Exam Traps
The three variance functions differ because their response distributions encode different relationships between mean and variability. Bernoulli variability depends on how close the mean is to 0.5; Poisson variability grows in direct proportion to the mean; and Gaussian variability is constant with respect to the mean, with its level set by dispersion.
| Response distribution | Mean in terms of the natural parameter | Variance function | Full response variance |
|---|---|---|---|
| Bernoulli | \(\mu=p\) | \(V(\mu)=\mu(1-\mu)\) | \(\mu(1-\mu)\) |
| Poisson | \(\mu=e^\theta\) | \(V(\mu)=\mu\) | \(\mu\) |
| Gaussian | \(\mu=\theta\) | \(V(\mu)=1\) | \(\phi=\sigma^2\) |
A reliable calculation follows a short sequence: identify \(b(\theta)\) and \(a(\phi)\), differentiate \(b\) to obtain \(b'\) and \(b''\), use \(\mu=b'(\theta)\) to rewrite \(b''\) in terms of \(\mu\), and finally multiply by \(a(\phi)\) to obtain the full variance. This separates the distribution’s mean-dependent variance pattern from its dispersion scale.
- Do not stop at the second derivative. \(b''(\theta)\) is the variance function expressed in terms of \(\theta\); express it in terms of \(\mu\) to identify \(V(\mu)\).
- Do not omit dispersion. The response variance is \(a(\phi)V(\mu)\), not always \(V(\mu)\). The Gaussian case makes this especially visible.
- Do not confuse variance with standard deviation. The variance function gives a variance component; take a square root only when a standard deviation is requested.
- Do not assume one variance pattern fits every family. A variance function follows from the selected response distribution and its assumptions.
Check Your Understanding
For each problem, identify whether the requested quantity is the variance function or the full response variance, and state any independence assumption used.
- A Bernoulli response has success probability \(0.30\) and uses \(a(\phi)=1\). Find its mean and variance. (Answer: mean \(0.30\); variance \(0.2100\))
- A Poisson response has mean \(6.2\) and uses \(a(\phi)=1\). Find its variance and standard deviation, rounding the standard deviation to four decimal places. (Answer: variance \(6.2\); standard deviation \(2.4900\))
- A Gaussian response has mean \(10\) and dispersion \(\phi=25\). Find \(V(\mu)\), the full response variance, and the standard deviation. (Answer: \(V(\mu)=1\); variance \(25\); standard deviation \(5\))
- An exponential-family response has \(a(\phi)=1\) and \(b(\theta)=e^\theta\). At \(\theta=\log(3)\), use derivatives of \(b\) to find the mean and variance. (Answer: mean \(3\); variance \(3\))
- Independent Gaussian observations each have variance \(4\). Find the variance of their average when there are two observations. (Answer: \(2\))