Why a Generalized Linear Model Has Three Components
A regression model must describe both how a response varies and how its expected value relates to explanatory variables. In a generalized linear model (GLM), these roles are separated: a probability model describes the response, a linear predictor combines covariates, and a link connects that predictor to the response’s conditional mean. Keeping the three pieces distinct is essential when the response is binary, a count, or another quantity for which ordinary linear regression may not be suitable.
A GLM does not claim that the response itself is a linear function of the predictors. Rather, it models a chosen function of the response’s conditional mean as linear in the predictors. The linear predictor can take any real value, while the conditional mean may be restricted to a range such as probabilities between zero and one or positive count means.
What Each Component Specifies
The random component identifies the probability distribution of the response conditional on its covariates. For example, a binary outcome may be modeled as Bernoulli with success probability \(\mu_i\); a count may be modeled as Poisson with mean \(\mu_i\); and a continuous response may be modeled as normal with mean \(\mu_i\) and a specified variance. The distribution is a modeling assumption about responses at given covariate values, not a statement that the covariates themselves are random or normally distributed.
The systematic component specifies how the covariates enter the model. In \(\eta_i=\beta_0+\sum_{j=1}^p\beta_jx_{ij}\), the \(\beta_j\) values are coefficients, and \(x_{ij}\) is the value of predictor \(j\) for observation \(i\). The predictor is linear in these coefficients and covariates as written. A model can include transformed covariates or indicator variables, but once those terms are specified, they enter the predictor through this linear combination.
The link function determines the scale on which the mean is modeled as linear. Its inverse translates a calculated predictor back to a conditional mean. The link must be appropriate for the mean’s possible values: a Bernoulli mean is a probability in \([0,1]\), while a Poisson mean is positive. A link whose inverse can produce values outside the required range may fail to define a valid model for some predictors.
These components answer different questions. The random component answers, “What distribution describes the response, conditional on the predictors?” The systematic component answers, “Which linear combination of predictors is used?” The link answers, “How is that linear combination related to the conditional mean?” Specifying only a distribution and a predictor is not enough; the link is also needed to complete the mean model.
| Component | Specifies | Example choice |
|---|---|---|
| Random | Conditional response distribution | Bernoulli response |
| Systematic | Linear predictor in covariates | \(\eta_i=\beta_0+\beta_1x_i\) |
| Link | Relationship between mean and predictor | \(\operatorname{logit}(\mu_i)=\eta_i\) |
Assembling a GLM
A reliable way to specify a GLM is to write its three components separately and then combine them. First identify the response and its conditional distribution. Next write the predictor, including its intercept and covariate terms. Finally state the link equation and, if a numerical mean is required, apply the inverse link. For a Bernoulli response, for example, the logit link gives \(\mu_i=\exp(\eta_i)/(1+\exp(\eta_i))\), which is a probability for every finite real value of \(\eta_i\).
State the distribution of \(Y_i\mid\mathbf{x}_i\), including its conditional mean and any other parameter needed to describe the distribution.
Define \(\eta_i\) as the intercept plus the specified coefficient-weighted covariates.
Write \(g(\mu_i)=\eta_i\), then use the inverse link to obtain \(\mu_i\) when needed.
Confirm that the implied mean is in the range allowed by the response distribution and that the stated conditional distribution uses that mean.
Why the Inverse Link Gives the Mean
The link equation is the defining connection between the systematic component and the mean. If \(g\) is invertible over the relevant mean range, applying \(g^{-1}\) to both sides recovers the mean. This simple step explains why the predictor is not generally itself the mean.
For the logit link, \(g(\mu)=\log\{\mu/(1-\mu)\}\). Solving \(g(\mu)=\eta\) gives \(\log\{\mu/(1-\mu)\}=\eta\), so \(\mu/(1-\mu)=\exp(\eta)\). Multiplying by \(1-\mu\) and collecting terms yields \(\mu=\exp(\eta)/(1+\exp(\eta))\). Thus every finite predictor maps to a mean strictly between zero and one, as required for a Bernoulli success probability.
Worked Example: A Bernoulli Response With a Logit Link
A hypothetical insurer models whether a submitted roadside-assistance claim is approved. For claim \(i\), let \(Y_i=1\) indicate approval and \(Y_i=0\) otherwise. Assume conditionally independent Bernoulli responses with success probability \(\mu_i\). The model uses the logit link and predictor \(\eta_i=-1+0.5x_{i1}+0.8x_{i2}\). For a claim with \(x_{i1}=2\) and \(x_{i2}=1\), find the predictor, approval probability, and conditional variance.
Method. Use the Bernoulli random component, calculate the linear predictor, and transform it using the inverse logit.
Step 1: Calculate the predictor. Substituting the covariates gives \(\eta_i=-1+0.5(2)+0.8(1)=-1+1+0.8=0.8\).
Step 2: Apply the inverse link. The approval probability is \(\mu_i=\exp(0.8)/(1+\exp(0.8))\). Since \(\exp(0.8)\approx2.22554\), \(\mu_i\approx2.22554/3.22554=0.68997\), or \(0.6900\) to four decimal places.
Step 3: Calculate the conditional variance. For a Bernoulli variable, \(\operatorname{Var}(Y_i\mid\mathbf{x}_i)=\mu_i(1-\mu_i)\). Therefore the variance is approximately \(0.68997(1-0.68997)=0.68997(0.31003)=0.2139\), to four decimal places.
Check and trap avoided. The probability lies between zero and one, as required. A common error is to report \(0.8\), the value of \(\eta_i\), as the approval probability. The predictor is on the log-odds scale; only its inverse-logit transformation gives the probability.
Worked Example: A Poisson Response With a Log Link
A hypothetical service team models the number \(Y_i\) of equipment-repair requests received from a facility in one week. Assume conditionally independent Poisson responses with conditional mean \(\mu_i\). Use the log link and predictor \(\eta_i=\log(3)+0.2x_{i1}-0.1x_{i2}\). For a facility with \(x_{i1}=2\) and \(x_{i2}=1\), calculate the predictor, expected count, and probability of no requests.
Method. The Poisson random component has mean \(\mu_i\), and the log link gives \(\log(\mu_i)=\eta_i\). Apply the exponential function to recover the mean.
Step 1: Calculate the predictor. Substitution gives \(\eta_i=\log(3)+0.2(2)-0.1(1)=\log(3)+0.4-0.1=\log(3)+0.3\). Using \(\log(3)\approx1.098612\), \(\eta_i\approx1.398612\).
Step 2: Calculate the conditional mean. The inverse log link is the exponential function, so \(\mu_i=\exp(\eta_i)=\exp(\log(3)+0.3)=3\exp(0.3)\). Since \(\exp(0.3)\approx1.349859\), \(\mu_i\approx3(1.349859)=4.049577\), or \(4.0496\) to four decimal places. This is the expected number of requests, not a requirement that the observed count equal a noninteger.
Step 3: Calculate the probability of no requests. For a Poisson variable with mean \(\mu_i\), \(P(Y_i=0\mid\mathbf{x}_i)=\exp(-\mu_i)\). Hence \(P(Y_i=0\mid\mathbf{x}_i)=\exp(-4.049577)\approx0.0174\), to four decimal places.
Check and trap avoided. The mean is positive, as a Poisson mean must be, and the no-request probability is small for a mean of about four. Do not use \(\eta_i\approx1.3986\) as the expected count: the log link means the expected count is \(\exp(\eta_i)\).
Worked Example: A Normal Response With an Identity Link
A hypothetical analyst models the repair duration \(Y_i\), in hours, for a class of machines. Assume conditionally independent normal responses with \(Y_i\mid\mathbf{x}_i\sim N(\mu_i,16)\), where the second parameter is the variance. Use the identity link and predictor \(\eta_i=40+3x_{i1}-2x_{i2}\). For \(x_{i1}=5\) and \(x_{i2}=4\), calculate the conditional mean and the probability that repair duration exceeds 55 hours.
Method. The identity link is \(g(\mu)=\mu\), so \(\mu_i=\eta_i\). Then standardize the response using the stated normal variance.
Step 1: Calculate the predictor and mean. \(\eta_i=40+3(5)-2(4)=40+15-8=47\). Under the identity link, \(\mu_i=47\) hours.
Step 2: Find the standard deviation. The specified variance is \(16\), so the standard deviation is \(\sqrt{16}=4\) hours.
Step 3: Standardize the threshold. The standardized value for 55 hours is \(z=(55-47)/4=8/4=2\). Therefore \(P(Y_i>55\mid\mathbf{x}_i)=P(Z>2)\approx0.0228\), where \(Z\sim N(0,1)\).
Check and trap avoided. The threshold is two standard deviations above the conditional mean, consistent with an upper-tail probability of \(0.0228\). The variance is \(16\), not the standard deviation; using 16 as the divisor would give the wrong standardized value.
Assumptions and Interpretation
A GLM specification should make clear what is assumed about the observations. A common setup assumes that responses are conditionally independent given their covariates and that each conditional response distribution belongs to the selected exponential-family model. It also specifies the conditional mean through the chosen link and predictor. Additional distribution parameters, such as the normal variance, may be fixed or modeled separately according to the particular model; they should not be confused with the mean link.
The conditional mean is an average over the response distribution at fixed covariate values. It is not necessarily an observed response, and it is not automatically a causal effect. In the Bernoulli example, \(\mu_i\) is a modeled approval probability. In the Poisson example, it is a modeled expected count. In the normal example, it is a modeled mean duration. The response distribution determines how individual outcomes vary around that mean.
The link scale also affects coefficient interpretation. With an identity link, a one-unit increase in a covariate changes the modeled mean by the corresponding coefficient, holding other covariates fixed. With a log link, a coefficient changes the log of the mean; with a logit link, it changes the log-odds. These interpretations depend on the chosen link and should not be transferred from one link to another.
Common Component and Compatibility Errors
- Confusing predictor and mean. In general, \(\eta_i\ne\mu_i\). They are equal for the identity link, but other links require transforming \(\eta_i\).
- Confusing the link with the distribution. A Bernoulli distribution describes the binary response; a logit link describes how its success probability relates to the predictors. These are different model components.
- Using an incompatible mean range. A probability must lie in \([0,1]\), and a Poisson mean must be positive. Check that the inverse link respects the relevant range.
- Ignoring the stated variance parameter. For a normal model, if the variance is \(\sigma^2\), standardization uses \(\sigma=\sqrt{\sigma^2}\), not \(\sigma^2\).
- Assuming “linear” means a straight-line response mean. The predictor is linear in the specified covariates; the mean follows after applying the inverse link and may be nonlinear in those covariates.
Check Your Understanding
For each problem, identify or calculate the requested GLM quantities. Probabilities are rounded to four decimal places where needed.
- A binary response has a Bernoulli distribution and a logit link. Its predictor is \(\eta=-0.5+0.4x\). At \(x=3\), find the predictor and conditional success probability. (Answer: \(\eta=0.7\), \(\mu\approx0.6682\))
- A count response has a Poisson distribution and a log link. Its predictor is \(\eta=\log(2)+0.1x_1+0.2x_2\). At \(x_1=4,x_2=1\), find the expected count. (Answer: \(\mu=2\exp(0.6)\approx3.6442\))
- A response satisfies \(Y\mid\mathbf{x}\sim N(\mu,9)\) with an identity link and predictor \(\eta=10+2x_1-0.5x_2\). At \(x_1=3,x_2=4\), find the mean and \(P(Y>17\mid\mathbf{x})\). (Answer: \(\mu=14\), \(P(Y>17\mid\mathbf{x})\approx0.1587\))
- A Bernoulli model uses a logit link and \(\eta=-1.2+0.3x_1+0.5x_2\). For \(x_1=2,x_2=1\), find the success probability. (Answer: \(\eta=-0.1\), \(\mu=\exp(-0.1)/(1+\exp(-0.1))\approx0.4750\))
- A proposed count model uses a Poisson random component, predictor \(\eta=\beta_0+\beta_1x\), and identity link \(g(\mu)=\mu\). What compatibility issue must be checked before using it for arbitrary real covariate values? (Answer: \(\mu=\eta\) must be positive for every covariate value in the model’s domain; an unrestricted linear predictor can be zero or negative, which is not a valid Poisson mean.)