The Identity Link and Gaussian Linear Regression

From a Gaussian GLM to a Linear Mean

In a Gaussian generalized linear model, the link connects the conditional mean of a response to a linear predictor. With the identity link, that connection is especially direct: the conditional mean itself equals the linear predictor. This is the mean structure of ordinary Gaussian linear regression.

Let \(Y_i\) be the response for observation \(i\), and let \(x_i\) be a \(p\)-dimensional column vector of predictors, possibly including a constant term. The coefficient vector is \(\beta\). For the ordinary, unweighted Gaussian model, assume the responses are conditionally independent and

where \(\phi\) is the common conditional variance. The identity link is \(g(\mu)=\mu\). The linear predictor is \(\eta_i=x_i^T\beta\), so the model specifies \(g(\mu_i)=\eta_i\).

The link determines the scale on which the predictors combine. For the identity link, each term in \(x_i^T\beta\) contributes additively to the expected response. This statement concerns the conditional mean, not the observed response: individual observations can lie above or below that mean because of their random Gaussian errors.

What the Coefficients Mean

In the familiar multiple-regression form, suppose the predictor vector contains an intercept and \(q\) measured predictors:

The intercept \(\beta_0\) is the conditional mean when all predictors equal zero. A slope \(\beta_j\) describes the change in conditional mean associated with a one-unit increase in \(x_{ij}\), with the other predictors held fixed. This is a statement about the model’s mean function; by itself, it does not establish that changing a predictor causes a response change.

The intercept depends on how predictors are represented. If zero is far outside the range of observed values, the intercept may have little practical meaning, even though it is needed to specify the line or plane. Centering a predictor can make the intercept describe the mean at a more useful reference value. Indicator variables also give slopes a comparison interpretation: with a reference category coded as zero, an indicator coefficient is the difference in conditional means from that reference, when other predictors are held fixed.

Fitted Means and Observed Responses

The coefficients in practice are usually estimated from data. Once coefficient estimates \(\hat{\beta}\) are available, the fitted mean for observation \(i\) is the model’s estimated conditional mean at that observation’s predictors:

The fitted mean is not the observed response \(y_i\). Their difference, \(y_i-\hat{\mu}_i\), records how far that particular observation lies from its fitted mean. The identity link makes this plug-in calculation simple: multiply each predictor value by its corresponding coefficient estimate and add the terms, including the intercept.

Two Useful Consequences of the Identity Link

Proof. By definition of the link in a GLM, \(g(\mu_i)=\eta_i\), where \(\mu_i=E[Y_i\mid x_i]\). For the identity link \(g(\mu)=\mu\), so \(\mu_i=\eta_i\). Since \(\eta_i=x_i^T\beta\), it follows that \(E[Y_i\mid x_i]=x_i^T\beta\). \(\square\)

Thus the linear predictor is not merely a convenient quantity that is transformed to obtain the mean. With the identity link, it is the mean itself. A second consequence gives a precise interpretation for any comparison between two predictor settings.

Proof. The identity-link mean function at \(x\) is \(\mu(x)=x^T\beta\), and at \(x^\ast\) it is \(\mu(x^\ast)=(x^\ast)^T\beta\). Subtracting gives \(\mu(x^\ast)-\mu(x)=(x^\ast)^T\beta-x^T\beta=(x^\ast-x)^T\beta\). This establishes the stated comparison. \(\square\)

When only predictor \(j\) changes by one unit and the others remain fixed, \(x^\ast-x\) has a one in position \(j\) and zeros elsewhere. The theorem then gives a mean difference of \(\beta_j\). If several predictors change, their coefficient-weighted changes add. This is the algebraic basis for interpreting slopes and for calculating fitted-mean contrasts.

Why the Identity Link Matters—and a Common Pitfall

The identity link preserves the response’s measurement scale. A slope can therefore be stated directly in response units per predictor unit, and the fitted mean is obtained by an ordinary linear combination. This makes the model a natural choice when additive changes in the expected response are scientifically meaningful and a Gaussian response model is appropriate.

A limitation follows from the same feature. A linear combination \(x_i^T\beta\) can be any real number, so the model does not constrain its mean predictions to be positive or to lie in a bounded interval. If the response is nonnegative or bounded, an identity-link fitted mean can fall outside its permissible range. That possibility should prompt a check of whether the model is suitable for the response and the predictor values being considered; it is not fixed merely by interpreting coefficients carefully.

• Common mistake: treating the intercept as the average response. The intercept is the model’s conditional mean when all predictors are zero. It equals an overall average only under particular designs or parameterizations; the definition alone does not make it an average over the observed sample.
• Correction: specify the predictor setting. Interpret the intercept at the stated zero values, or center predictors so that zero represents a meaningful reference setting.
• Common mistake: describing a fitted mean as an observed outcome. A fitted mean is \(x_i^T\hat{\beta}\), an estimated conditional expectation. The observed response can differ from it.
• Correction: retain the conditional language. Describe \(\hat{\mu}_i\) as the estimated mean response for observations with those predictor values, rather than as the response every such observation will have.