Two Different Choices in a GLM
A generalized linear model makes two distinct commitments about a response. It specifies a response family, which describes the conditional distribution of the response, and a link, which relates the conditional mean to the predictors. These choices work together, but they answer different questions. The family concerns what values and variation the response can have; the link concerns how its mean changes with the linear predictor.
Keeping the choices separate matters when a model fits poorly. A different link may describe the mean more suitably, but it does not change which response values the family permits or its variance function at a given mean. Conversely, changing the family changes distributional assumptions even if the same mean structure is retained.
Here the family specification includes more than a variance formula: it includes the distribution and its support. The variance function summarizes how conditional variance depends on the mean within that family. A link is a function applied to the mean, not a transformation of the observed response. The model is defined only where the mean is in the domain of \(g\), and where the inverse mapping from \(\eta_i\) gives an admissible mean for the selected family.
What the Family Fixes
For a given family and dispersion, the mean parameter determines a conditional distribution. That distribution determines the values the response can take, its variance at that mean, and other distributional features such as skewness and tail behavior. A link does not replace any of these features. It specifies how the model assigns a mean to each predictor value.
For example, a Gaussian response family has support on the whole real line and constant variance \(\phi\), while a Gamma response family has positive support and variance \(\phi\mu^2\). No choice of link turns the Gaussian distribution into a positive-only distribution, or changes its constant variance into a mean-squared variance. To make either change, one must change the response family, not merely the link.
This does not mean links are irrelevant to variance predictions. Under a particular link and regression coefficients, different predictor values lead to different fitted means, and therefore to different variances through \(V(\mu_i)\). The key distinction is that the link affects which means are assigned to predictor values, while the family fixes the variance associated with each mean.
Worked Example: Positive Mean Does Not Give Positive Support
Suppose a response has a Gaussian conditional distribution with mean \(10\) and variance \(25\). A researcher uses a log link and sets the linear predictor to \(\eta=\log(10)\), so the model mean is \(\mu=\exp(\eta)=10\). Does this link ensure that the response is positive?
The conditional distribution is \(Y\sim N(10,25)\), whose standard deviation is \(5\). Thus \[ P(Y<0)=P\left(\frac{Y-10}{5}<\frac{0-10}{5}\right)=\Phi(-2)\approx 0.0228. \] There is about a 2.28% probability of a negative response.
The log link has made the mean positive. It has not changed the Gaussian support, which remains the whole real line. By contrast, a Gamma family has positive support, so a Gamma model does not assign probability to negative responses. Choosing a positive-valued mean is not the same as choosing a positive-valued response distribution.
Result: A Link Reparameterizes the Mean, Not the Family
The following statement makes the separation precise. Assume the conditional response family is specified by distributions \(F_{\mu,\phi}\), with a common support for the admissible values of \(\mu\), and with variance \(\phi V(\mu)\). Fix a predictor value \(x\) and dispersion \(\phi\). Suppose two admissible links \(g_1\) and \(g_2\), with corresponding linear predictors, both give the same conditional mean \(\mu\).
The response-family specification assigns the distribution \(F_{\mu,\phi}\) to the response when its mean is \(\mu\) and dispersion is \(\phi\).
The first model represents the mean using \(\eta_1=g_1(\mu)\), and the second using \(\eta_2=g_2(\mu)\). These are two representations of the same mean.
Both models still assign \(F_{\mu,\phi}\) to the response. Its support is that of this distribution, and its variance is \(\phi V(\mu)\). Neither depends on which link represented \(\mu\).
Thus a link change can alter a model’s distributional predictions when it changes the fitted mean, but it cannot alter the family’s distribution at a fixed mean. This condition is important: comparing models at different fitted means does not isolate the link’s role from the change in mean.
Variance Functions Belong to Families
The variance function is another way to see why family choice cannot be delegated to link choice. Consider two continuous response families with variance functions \(V(\mu)=\mu^2\) and \(V(\mu)=\mu^3\), respectively. For a common dispersion value \(\phi=0.1\), they predict different variances even at the same conditional mean. A link can change the mean assigned at a particular predictor value; it cannot make the first variance function become the second.
Worked Example: Different Variance Patterns at the Same Means
Compare a Gamma family, with variance \(\phi\mu^2\), and an inverse-Gaussian family, with variance \(\phi\mu^3\). Let \(\phi=0.1\), and evaluate both families at means \(1\), \(2\), and \(3\).
For the Gamma family, the variances are \[ 0.1(1^2)=0.1,\qquad 0.1(2^2)=0.4,\qquad 0.1(3^2)=0.9. \] For the inverse-Gaussian family, they are \[ 0.1(1^3)=0.1,\qquad 0.1(2^3)=0.8,\qquad 0.1(3^3)=2.7. \] At mean \(1\), the variance predictions happen to agree. At mean \(2\), they differ by \(0.8-0.4=0.4\); at mean \(3\), they differ by \(2.7-0.9=1.8\).
Those differences follow from the selected variance functions, not from a link. If a proposed link leads a model to fitted means \(1,2,3\), the two families retain their respective variance predictions at those means. A link that gives a different set of means changes which points on a variance function are used; it does not change the function itself.
This result concerns the family’s conditional variance relationship. It does not say that every observation in a fitted model has the same variance: observations with different means generally have different variances when \(V\) depends on \(\mu\). Nor does it say that a link cannot change a variance pattern across predictor values. By changing the fitted means at those values, a link can change the model-implied pattern across predictors, while leaving the family’s mean–variance relationship intact.
Worked Example: Two Links, One Gamma Family
Worked Example: Two Links, One Gamma Family
Consider a Gamma response family with fixed \(\phi=0.25\), so that \(\operatorname{Var}(Y\mid x)=0.25\mu^2\). Compare two mean specifications over \(x=0,1,2\): a log-link model with \(\eta=\log(2)+x\log(2)\), and an identity-link model with \(\eta=2+2x\). For this calculation, both models give positive means at all three predictor values.
Under the log link, \(\mu=\exp(\eta)\). The three predictors are \(\log(2)\), \(\log(4)\), and \(\log(8)\), so the means are \(2,4,8\). The corresponding variances are \[ 0.25(2^2)=1,\qquad 0.25(4^2)=4,\qquad 0.25(8^2)=16. \]
Under the identity link, \(\mu=\eta\). The means at \(x=0,1,2\) are \(2,4,6\). The corresponding variances are \[ 0.25(2^2)=1,\qquad 0.25(4^2)=4,\qquad 0.25(6^2)=9. \]
Both models use the Gamma family, so they share its positive support and its variance function. They differ in the mean curve beyond the first two predictor values, which leads to a different fitted variance at \(x=2\). The difference \(16\) versus \(9\) is not evidence that the link changed the Gamma variance function: it is the result of evaluating that same function at means \(8\) and \(6\).
Choosing Which Part of the Model to Reconsider
When a model does not describe the data well, first identify the type of mismatch. A mean pattern that changes with predictors in a way the current link cannot capture suggests reconsidering the mean structure or link. Responses that fall outside the family’s support, or a conditional variance pattern inconsistent with its variance function, point instead to a family-level problem. A mean link cannot solve either structural family mismatch.
Support and variance are not the only distributional features worth considering. Two families can have similar support and variance functions yet differ in other aspects of their distributions. The same separation still applies: a link specifies the mean relationship, while the family specifies the conditional distribution at that mean. Assessing those distributional differences requires examining the response family itself.
In practice, observed data do not reveal conditional distributions and variances perfectly, particularly in small samples. A discrepancy is therefore evidence to investigate rather than automatic proof of a wrong family. The useful discipline is to state what is being changed: changing a link revises the predictor-to-mean relationship; changing a family revises the assumed response distribution.
Common Mistake
A related error is to see variances that increase with the fitted mean and conclude that changing the link will change the variance function. A new link can change the fitted means and therefore the variances predicted at different predictor values. But at any fixed mean, the variance remains the one prescribed by the family and dispersion.
Check Your Understanding
Use the distinction between distribution and mean specification to answer each question.
- A Gaussian GLM uses a log link. What does the link guarantee about the modeled mean, and what does it not guarantee about the response support?
- For a fixed family and dispersion, two observations have the same conditional mean but different predictor values. What does the variance function imply about their conditional variances?
- A family has variance function \(V(\mu)=\mu^2\). Does choosing a different link change this function? Explain how it could nevertheless change variances across predictor values.
- Why is a positive observed response not, by itself, enough to show that a Gaussian family is unsuitable? What additional distributional concern should be considered?
- Two models use the same family but predict different means at one predictor value. Explain why different predicted variances there do not show that the links changed the family’s variance function.