Why the Mean Does Not Determine the Uncertainty
A response distribution is not characterized by its mean alone. Two models can assign the same expected outcome at a given predictor value but imply different variability around that expectation. For a generalized linear model (GLM), the exponential-family variance function makes that distinction explicit: it describes how the response variance changes with its mean.
This relationship helps distinguish common response families and exposes a consequential modeling error. If the mean model is reasonable but the variance model is not, fitted means can still be useful, yet standard errors, tests, and intervals based on the assumed variance may be unreliable. The calculations here are aligned with the SOA Exam SRM (Statistics for Risk Modeling) syllabus.
The variance function \(V(\mu)\) describes how variance changes with the mean before multiplying by the dispersion. For independent observations with the same family and dispersion, a higher mean can therefore imply higher, lower, or unchanged variance depending on the family. If observations have different known weights or numbers of trials, those features also enter their variances; they must not be silently ignored.
Deriving the Mean-Variance Relationship
The relationship follows from the exponential-family form rather than being an extra assumption about a particular data set. The derivation below shows why the first two derivatives of \(b\) determine the mean and variance.
For one observation, the log probability is \(\ell=(y\theta-b(\theta))/\phi+c(y,\phi)\). Its score for \(\theta\) is \(\partial\ell/\partial\theta=(y-b'(\theta))/\phi\).
Under a regular model, the expected score is zero. Thus \(E[Y]=b'(\theta)\), so \(\mu=b'(\theta)\).
The score has variance \(\operatorname{Var}(Y)/\phi^2\). The expected negative second derivative of the log probability is \(b''(\theta)/\phi\). The information identity equates these quantities, giving \(\operatorname{Var}(Y)/\phi^2=b''(\theta)/\phi\).
Therefore \(\operatorname{Var}(Y)=\phi b''(\theta)\). Since \(\mu=b'(\theta)\), express \(b''(\theta)\) as a function of \(\mu\) and call it \(V(\mu)\).
This derivation assumes a regular exponential-family model for which differentiation and the information identity apply. It does not say that every data-generating process is exponential-family, nor that a chosen family will fit a particular data set. The resulting variance function is a model implication to assess, not a fact guaranteed by the observed mean.
Four Common Mean-Variance Patterns
The principal patterns differ in how variance scales with the mean. For a binomial response, be especially clear about whether the response is a single Bernoulli outcome, a count of successes, or a proportion.
| Response family | Mean and variance | Variance pattern |
|---|---|---|
| Gaussian (normal) | \(Y\sim N(\mu,\phi)\), so \(\operatorname{Var}(Y)=\phi\) | Constant variance when \(\phi\) is constant |
| Binomial count | \(Y\sim\operatorname{Binomial}(m,p)\), \(\mu=mp\), and \(\operatorname{Var}(Y)=mp(1-p)\) | Depends on both the success probability and the number of trials |
| Poisson | \(Y\sim\operatorname{Poisson}(\mu)\), so \(\operatorname{Var}(Y)=\mu\) | Variance equals the mean under the Poisson model |
| Gamma | \(E[Y]=\mu\), \(\operatorname{Var}(Y)=\phi\mu^2\) | Variance grows with the square of the mean |
For a single Bernoulli response, \(m=1\), so \(\operatorname{Var}(Y)=p(1-p)\). For a grouped binomial count, the variance is \(mp(1-p)\), not just \(p(1-p)\). For the corresponding observed proportion \(Y/m\), the variance is \(p(1-p)/m\). Those are three different scales for related outcomes.
For a Gamma response in the stated convention, \(\phi\) is the dispersion and the coefficient of variation is \(\sqrt{\operatorname{Var}(Y)}/\mu=\sqrt{\phi}\). This constant coefficient of variation is a direct consequence of variance proportional to \(\mu^2\). A Gamma model is appropriate only for a response with positive support; its variance pattern alone does not establish that its distribution is suitable.
Worked Example: Comparing Four Variances at Specified Means
A hypothetical analyst is comparing four candidate response models at their fitted means. The Gaussian candidate has mean 5 and dispersion \(\phi=4\); the binomial candidate is one Bernoulli outcome with \(p=0.30\); the Poisson candidate has mean 3.5; and the Gamma candidate has mean 8 and dispersion \(\phi=0.25\). Calculate each variance and identify its pattern. Treat the stated dispersions as known for this calculation.
Method. Apply the variance function for each specified family, keeping the outcome scale fixed. The Gaussian response is measured on its original continuous scale; the binomial response is one indicator, not a grouped count.
Gaussian. Constant variance gives \(\operatorname{Var}(Y)=\phi=4\). The mean of 5 does not enter this variance.
Bernoulli. Here \(m=1\), so \(\operatorname{Var}(Y)=p(1-p)=0.30(0.70)=0.21\).
Poisson. The variance equals the mean, so \(\operatorname{Var}(Y)=3.5\).
Gamma. The variance is \(\phi\mu^2=0.25(8^2)=0.25(64)=16\).
The variances are therefore 4, 0.21, 3.5, and 16, respectively. The calculations are consistent with the family patterns: the Gaussian variance is constant in its mean, the Bernoulli variance uses \(p(1-p)\), the Poisson variance equals its mean, and the Gamma variance is quadratic in its mean. A common mistake would be to use \(mp(1-p)\) for the Bernoulli candidate without noticing that its trial count is \(m=1\); if it instead represented a count over \(m\) trials, both its mean and variance would need to be stated on that count scale.
Grouped Binomial Counts and the Poisson Approximation
A grouped binomial count can have a variance near, but not equal to, its mean when the event probability is small. The comparison is useful, but it does not make the two variance assumptions interchangeable. It is important to calculate on the same outcome scale before comparing them.
Worked Example: Comparing a Grouped Binomial Count With a Poisson Count
In a hypothetical portfolio, each of 12 independent risk groups contains 8 exposure units. Each unit has probability \(p=0.15\) of producing one event during the period, independently of the others. For one group, let \(Y\) be its event count and let \(\hat p=Y/8\) be its observed event proportion. Calculate the binomial mean and variance for \(Y\) and for \(\hat p\), then compare the count variance with a Poisson model having the same mean.
Method. Use the grouped binomial formulas first, then apply the Poisson variance rule to a count with the same expected value. The 12 groups do not change the variance calculation for one group.
For the group count, \(Y\sim\operatorname{Binomial}(8,0.15)\). Its mean is \(E[Y]=mp=8(0.15)=1.2\), and its variance is \(\operatorname{Var}(Y)=mp(1-p)=8(0.15)(0.85)=1.02\).
For the proportion, \(E[\hat p]=p=0.15\). Since \(\hat p=Y/8\), its variance is \(\operatorname{Var}(\hat p)=\operatorname{Var}(Y)/8^2=1.02/64=0.0159375\). Equivalently, \(p(1-p)/m=(0.15)(0.85)/8=0.1275/8=0.0159375\).
A Poisson count with the same mean \(1.2\) has variance \(1.2\). Its variance exceeds the exact binomial count variance by \(1.2-1.02=0.18\), or \(0.18/1.02\approx0.1765\), which is about 17.65% of the binomial variance. Thus, even at a relatively small event probability, the Poisson variance is not identical here. The check is that the count variance and count mean are both on the count scale; comparing the Poisson variance with the proportion variance would mix scales. A common mistake is to omit the factor \(m\) for the count or to divide by \(m\), rather than \(m^2\), when converting a count variance to a proportion variance.
What a Variance Mismatch Does to Inference
Model-based standard errors depend on the assumed response variance. When that variance is too small, the resulting standard errors tend to be too small as well, making test statistics too large in magnitude and confidence intervals too narrow. When it is too large, the reverse can occur. The exact effect on a regression coefficient’s standard error depends on the design and fitted model; the direction should not be asserted without considering those details.
A variance mismatch does not automatically imply that the fitted mean is wrong. If the conditional mean model is correct, some estimators of mean-model coefficients can remain consistent under weaker distributional assumptions. However, their usual model-based uncertainty calculations rely on the assumed variance structure. If the mean model is also wrong, the coefficient estimates themselves may be affected. Robust variance estimation can sometimes address uncertainty under a correct mean model, but it does not repair a misspecified mean relationship or make a poor response-family choice appropriate.
The following calculation isolates the effect on uncertainty for a sample mean. It assumes independent observations and compares the model-based variance with a stipulated larger actual variance; it is not a claim that every GLM coefficient standard error changes by the same factor.
Worked Example: Understated Poisson Variance Narrows an Interval
Suppose 100 independent hypothetical groups each produce a count with common mean 4. An analyst uses a Poisson model, which assumes variance 4, but the actual count variance is 9. The observed sample mean is 4. Calculate a nominal 95% normal-approximation interval for the common mean using the model variance, then calculate the interval using the actual variance. Use 1.96 as the normal critical value and round endpoints to three decimals.
Method. For independent observations with common variance \(\sigma^2\), the variance of the sample mean is \(\sigma^2/n\), so its standard error is \(\sigma/\sqrt{n}\). Compare the two standard errors before forming the intervals.
Under the assumed Poisson variance, the standard error is \(\sqrt{4/100}=\sqrt{0.04}=0.2\). The half-width is calculated from the unrounded standard error: \(1.96\sqrt{4/100}=1.96(0.2)=0.392\). The model-based interval is \(4\pm0.392\), or \((3.608,4.392)\).
Under the stated actual variance of 9, the standard error is \(\sqrt{9/100}=\sqrt{0.09}=0.3\). The half-width is \(1.96\sqrt{9/100}=1.96(0.3)=0.588\). The interval using the actual variance is \(4\pm0.588\), or \((3.412,4.588)\).
The model-based standard error is \(0.2/0.3=2/3\) of the standard error using the actual variance, so it understates this uncertainty by one-third relative to the latter. The narrower interval is not justified by the stipulated data-generating variance. The arithmetic check is that each half-width is 1.96 times its own standard error. A common mistake is to treat the Poisson variance rule as a consequence of the observed mean rather than an assumption of the Poisson model.
Practical Checks Before Using a Variance Function
A variance function is useful only when it is applied to the response as actually modeled. Before using it in a calculation or interpreting a fitted model, check the following:
- Identify the response scale. A count, a binary indicator, and a proportion may refer to related outcomes but have different variances.
- State the dispersion convention. For the form used here, variance is \(\phi V(\mu)\). Do not insert a free dispersion parameter into the standard Poisson variance without stating that the model has been extended.
- Account for trials or exposure. For a grouped binomial count, the number of trials changes the variance; for a proportion, dividing the count by the trial count changes its variance by the square of that divisor.
- Separate mean fit from variance fit. A plausible fitted mean is not evidence by itself that the assumed spread around that mean is correct.
A useful diagnostic question is: if the mean doubled while other stated quantities stayed fixed, what would this family imply about variance? The Gaussian pattern leaves variance unchanged; the Poisson pattern doubles it; the Gamma pattern multiplies it by four. For binomial responses, the answer depends on the probability and trial count, because \(mp(1-p)\) is not simply proportional to the mean across all possible values of \(p\).
Check Your Understanding
Work each problem on the response scale stated, then compare your result with the answer.
- A Gaussian response has mean 12 and dispersion \(\phi=2.5\). A second Gaussian response has mean 30 with the same dispersion. What is each variance, and what variance pattern is illustrated? (Answer: Both variances are 2.5; the variance is constant in the mean.)
- A hypothetical group has 10 independent units, each with event probability 0.20. Find the mean and variance of its event count, and the variance of its observed event proportion. (Answer: Count mean \(2\), count variance \(10(0.20)(0.80)=1.6\), proportion variance \(1.6/100=0.016\).)
- A Poisson response has mean 6.5. A Gamma response also has mean 6.5 and dispersion \(\phi=0.16\). Find both variances and their ratio, Gamma variance divided by Poisson variance. (Answer: Poisson variance \(6.5\); Gamma variance \(0.16(6.5^2)=6.76\); ratio \(6.76/6.5=1.04\).)
- For 64 independent observations, the assumed common variance is 9 but the actual common variance is 16. Find the model-based and actual standard errors of the sample mean. (Answer: Model-based \(\sqrt{9/64}=0.375\); actual \(\sqrt{16/64}=0.5\).)
- A grouped binomial count has \(m=20\) and \(p=0.05\). A Poisson model is assigned the same mean. Calculate both count variances and the Poisson variance’s percentage excess relative to the binomial variance. (Answer: Mean \(=1\); binomial variance \(=20(0.05)(0.95)=0.95\); Poisson variance \(=1\); excess \(=(1-0.95)/0.95\approx5.26\%\).)