The Identity Link for Gaussian Means

Why Use the Identity Link?

For a Gaussian response, the conditional mean is often modeled directly as a linear function of covariates. The identity link is what makes this direct modeling possible: it leaves the mean unchanged when mapping it to the linear predictor. As a result, a prediction from the linear predictor is already a predicted mean in the original response units.

This is a natural choice when an additive change in a covariate corresponds to an additive change in the expected response, and the response can reasonably be treated as continuous and unbounded. For example, a model might describe how expected sensor voltage changes with an input setting. The model does not assert that every observed response lies exactly on its fitted mean; it describes the conditional average around which observations vary.

Gaussian Model and Assumptions

A basic identity-link Gaussian model specifies the distribution of each response conditional on its covariates:

The notation \(N(\mu_i,\sigma^2)\) uses \(\mu_i\) as the conditional mean and \(\sigma^2\) as the conditional variance. In this basic model, the observations are conditionally independent given their covariates, the variance is the same for each observation, and the conditional response distribution is Gaussian. The mean is assumed to follow the specified linear predictor over the covariate range being modeled.

These are modeling assumptions, not consequences of choosing an identity link. A Gaussian response distribution describes continuous outcomes on the real line; it is not automatically appropriate for every measurement recorded as a number. The mean structure must also be plausible: if the expected response curves with a covariate, a straight-line term alone may be inadequate. The data and scientific context must support both the distribution and the mean specification.

Why the Gaussian Canonical Link Is the Identity

The identity link is also the canonical link for the Gaussian family under its usual exponential-family parameterization. To see the connection, write the normal density for one observation with mean \(\mu\) and variance \(\sigma^2\) as:

In the exponential-family form, the term involving the response and the natural parameter is \(y\theta-b(\theta)\), divided by the dispersion. Comparing terms gives \(\theta=\mu\) and \(b(\theta)=\theta^2/2\). Differentiating the log-partition function gives the mean, \(b'(\theta)=\theta\), so the mean equals the natural parameter: \(\mu=\theta\). The canonical link maps the mean to that natural parameter, hence \(g(\mu)=\mu\).

For the model, the link equation is \(g(\mu_i)=\eta_i\). Substituting the identity link gives \(\mu_i=\eta_i\), and specifying \(\eta_i=\mathbf{x}_i^\mathsf{T}\boldsymbol{\beta}\) yields the stated Gaussian mean model. This derivation explains why the linear predictor is already on the response scale; there is no additional transformation to apply to a fitted value.

Fitted Values and Coefficients on the Response Scale

If the estimated coefficient vector is \(\hat{\boldsymbol{\beta}}\), the fitted value for observation \(i\) is:

Because \(g^{-1}(\eta)=\eta\), this calculation requires no inverse-link transformation. In a model with an intercept and one quantitative covariate, \(\mu_i=\beta_0+\beta_1x_i\). The intercept \(\beta_0\) is the modeled conditional mean when \(x_i=0\), and the slope \(\beta_1\) is the modeled change in conditional mean for a one-unit increase in \(x_i\). These interpretations are in response units and response units per covariate unit, respectively. The intercept may not have a useful practical interpretation if zero lies outside the relevant covariate range.

A categorical predictor can be represented with indicator variables. With a two-group predictor \(G\), define \(I(G=B)=1\) for group B and 0 otherwise. In a model with an intercept and this indicator, the group coefficient is the fitted difference in conditional means between B and the reference group A.

Justification: Why Least Squares Appears

For independent Gaussian observations with a common variance \(\sigma^2\), the log-likelihood for a fixed coefficient vector, apart from constants that do not depend on that vector, is:

Since \(\sigma^2>0\), maximizing this expression over \(\boldsymbol{\beta}\) is equivalent to minimizing the sum of squared residuals. The identity-link Gaussian mean model therefore connects directly to least squares under these assumptions. This equivalence does not establish that the model is suitable: the mean form, variance behavior, independence, and Gaussian approximation still require assessment.

Diagnostics, Alternatives, and Limitations

After fitting the model, inspect residuals \(y_i-\hat{\mu}_i\) against fitted values and important covariates. A curved pattern can indicate that the specified mean is not adequately linear. A fan-shaped spread suggests that the common-variance assumption may be questionable. A normal quantile plot can reveal substantial departures from the Gaussian residual assumption, while unusually large residuals may identify influential observations or data problems. Dependence between observations also matters: a pattern over time or within groups can conflict with the conditional independence assumption.

Predictions are most defensible within the covariate range that informed the fit. A straight-line mean that appears reasonable over observed values may behave implausibly when extended far beyond them. The identity link does not constrain the mean to any particular interval. That is suitable for outcomes whose plausible values span the real line, but it can be a poor modeling choice when the response has a strict range or when its variability changes substantially with its mean.

The main alternative is not a single universal model. If residuals indicate curvature, one can consider a more flexible mean structure, such as adding justified polynomial or other nonlinear terms. If the outcome’s distribution or support is poorly described by a Gaussian model, a different response distribution and link may be more appropriate. Transforming the response is another possibility, but it changes the modeled quantity and requires care when interpreting predictions. These alternatives should be selected based on the outcome and diagnostics rather than used simply to obtain a preferred-looking fit.