The Gaussian Identity-Link Model

Why the Gaussian Identity-Link Model Matters

For a quantitative response, a generalized linear model can use a normal distribution for the random component and the identity function for the link. This combination gives a familiar structure: the conditional mean is linear in the predictors, while the conditional variance is constant. It is the Gaussian model underlying ordinary linear regression, stated in the GLM framework aligned with the SOA Exam SRM (Statistics for Risk Modeling) syllabus.

The connection is useful, but it depends on the model assumptions. In particular, a linear conditional mean does not by itself imply normally distributed responses or constant variance. Those are separate parts of the Gaussian model.

Definition: For observations $Y_i$ and predictor vectors $x_i$, a Gaussian identity-link model with common variance $\sigma^2>0$ is $$ Y_i\mid x_i\sim N(\mu_i,\sigma^2),\qquad \mu_i=E[Y_i\mid x_i]=\eta_i=x_i^\mathsf{T}\beta. $$ Here $x_i^\mathsf{T}\beta$ is the linear predictor, including an intercept when the design vector has a component equal to 1. The identity link is $g(\mu_i)=\mu_i$, so setting $g(\mu_i)=\eta_i$ gives $\mu_i=\eta_i$. The variance $\sigma^2$ does not depend on the predictors.

The convention $N(\mu,\sigma^2)$ uses the second argument as the variance. Equivalently, each response can be written as $Y_i=x_i^\mathsf{T}\beta+\varepsilon_i$, where the errors have mean zero and variance $\sigma^2$. For the usual likelihood and regression interpretation, responses are conditionally independent given the predictors, and the errors are normally distributed. The predictor values are treated as fixed, or the model is understood conditionally on them.

In the common GLM notation $\operatorname{Var}(Y_i\mid x_i)=\phi V(\mu_i)$, the Gaussian identity-link model has $V(\mu_i)=1$ and $\phi=\sigma^2$. Thus the variance function is constant. This differs from a model in which the variance changes with the mean.

Interpreting the Mean and Variance

The coefficient vector controls the conditional mean, not the conditional variance. With one predictor and an intercept, $E[Y_i\mid x_i]=\beta_0+\beta_1x_i$. A one-unit increase in $x_i$ changes the conditional mean by $\beta_1$, while the model assigns every predictor value the same conditional variance $\sigma^2$. With several predictors, each coefficient describes the mean change for a one-unit increase in that predictor while the other predictors are held fixed.

The model’s normal distribution has support over the whole real line. That is suitable for many continuous measurements, but it can be inappropriate when the response must be nonnegative or lie within fixed bounds: a fitted mean may fall outside those bounds, and the normal distribution assigns positive probability to values outside them. The identity link is not a guarantee that fitted means obey a response-specific restriction.

Worked Example: Reading a Gaussian Mean Model

Suppose a hypothetical insurer models claim-handling expense $Y_i$, measured in thousands of dollars, using a centered workload measure $x_i$ and an indicator $z_i$ for a particular claim category. Assume independent conditional responses with $Y_i\mid x_i,z_i\sim N(\mu_i,0.49)$ and $\mu_i=1.8+0.4x_i-0.3z_i$. Find the conditional distribution when $x_i=2$ and $z_i=1$, and interpret the coefficients.

Method. Substitute the predictor values into the identity-link linear predictor: $1.8+(0.4)(2)-(0.3)(1)=1.8+0.8-0.3=2.3$. Thus the conditional mean is 2.3 thousand dollars. The specified conditional variance is 0.49, so the standard deviation is $\sqrt{0.49}=0.7$ thousand dollars.

The conditional distribution is therefore $N(2.3,0.49)$, in thousands of dollars. Holding $z_i$ fixed, a one-unit increase in $x_i$ raises the conditional mean by 0.4 thousand dollars. Changing $z_i$ from 0 to 1 lowers the conditional mean by 0.3 thousand dollars when $x_i$ is held fixed. Neither change alters the model’s variance.

Check. The mean calculation uses the identity link directly, and the variance remains 0.49 at this predictor value, as required by the model. A common mistake is to report 0.49 as the standard deviation; the standard deviation is 0.7.

Why Least Squares Appears

For observed responses $y_1,\ldots,y_n$, define the residual sum of squares for a coefficient vector $\beta$ as $\operatorname{RSS}(\beta)=\sum_{i=1}^n(y_i-x_i^\mathsf{T}\beta)^2$. Ordinary least squares chooses a coefficient vector that minimizes this quantity. The Gaussian likelihood leads to the same coefficient estimate when the variance is known, and also when it is unknown and the minimum residual sum of squares is positive.

Derivation: Under conditionally independent normal responses with common variance $\sigma^2>0$, the likelihood for $\beta$ is $$ L(\beta,\sigma^2) =(2\pi\sigma^2)^{-n/2} \exp\left\{-\frac{\operatorname{RSS}(\beta)}{2\sigma^2}\right\}. $$ Taking logarithms gives $$ \ell(\beta,\sigma^2) =-\frac n2\log(2\pi\sigma^2) -\frac{\operatorname{RSS}(\beta)}{2\sigma^2}. $$ If $\sigma^2$ is fixed, the first term is constant in $\beta$, and the log-likelihood is largest when the RSS is smallest. If $\sigma^2$ is unknown, for any fixed $\beta$ with positive RSS the maximizing variance is $\widehat{\sigma}^2(\beta)=\operatorname{RSS}(\beta)/n$. Substitution gives a profile log-likelihood that decreases as the RSS increases. Therefore, when the minimum RSS is positive, maximizing the Gaussian likelihood in $\beta$ also minimizes RSS. If some $\beta$ gives RSS equal to zero, the likelihood instead grows without bound as $\sigma^2$ tends to zero; there is no finite joint maximum likelihood estimate of the variance in that exact-fit case.

This derivation explains the connection with ordinary regression without making the models interchangeable under every circumstance. The same coefficient estimate does not mean that the assumptions are automatically satisfied. Normality, conditional independence, linearity of the conditional mean, and constant conditional variance remain model assumptions to assess.

Worked Example: Finding the Least-Squares Line

A hypothetical study records a continuous response $Y$ and predictor $x$ for four independent units. The data are $(0,1),(1,3),(2,4),(3,7)$. Under a Gaussian identity-link model with known variance $\sigma^2=0.25$, find the maximum-likelihood estimates of the intercept and slope, the fitted values, and the residual sum of squares.

Method. Because the variance is known and common, the Gaussian likelihood is maximized by minimizing RSS. For a line $\widehat{Y}=\widehat{\beta}_0+\widehat{\beta}_1x$, first calculate $\bar{x}=(0+1+2+3)/4=1.5$ and $\bar{y}=(1+3+4+7)/4=3.75$.

The centered sum of squares for $x$ is $S_{xx}=(-1.5)^2+(-0.5)^2+(0.5)^2+(1.5)^2=2.25+0.25+0.25+2.25=5$. The centered cross-product is $S_{xy}=(-1.5)(-2.75)+(-0.5)(-0.75)+(0.5)(0.25)+(1.5)(3.25)=4.125+0.375+0.125+4.875=9.5$. Consequently, $\widehat{\beta}_1=S_{xy}/S_{xx}=9.5/5=1.9$ and $\widehat{\beta}_0=\bar{y}-\widehat{\beta}_1\bar{x}=3.75-(1.9)(1.5)=0.9$.

The fitted values at $x=0,1,2,3$ are $0.9,2.8,4.7,6.6$. Subtract fitted values from observed values to obtain residuals $0.1,0.2,-0.7,0.4$. Therefore, $\operatorname{RSS}=0.1^2+0.2^2+(-0.7)^2+0.4^2=0.01+0.04+0.49+0.16=0.70$.

Check and trap. The fitted values average to $3.75=\bar y$, as a least-squares line with an intercept should. A tempting but incorrect shortcut is to choose a line that appears to pass near most points without minimizing the squared residuals. For example, the line $1+2x$ gives RSS $0+0+1+0=1$, which is greater than 0.70 and therefore has a lower likelihood under the stated Gaussian model.

Estimating an Unknown Variance

When the common variance is unknown, the coefficient estimate still minimizes RSS if the minimum RSS is positive. The maximum-likelihood estimate of the variance at the minimizing coefficient vector is $\operatorname{RSS}/n$. This is the maximum-likelihood estimate, not the residual variance estimate that uses a degrees-of-freedom adjustment in ordinary regression. They answer different estimation questions.

Worked Example: Profiling an Unknown Variance

Assume a Gaussian identity-link model with independent responses and data $(0,2),(1,3),(2,5),(3,6)$. Both the intercept and slope are estimated, and the common variance is unknown. Find the least-squares line, RSS, and maximum-likelihood estimate of the variance.

Method. The predictor mean is $\bar{x}=(0+1+2+3)/4=1.5$, and the response mean is $\bar{y}=(2+3+5+6)/4=4$. The centered predictor sum of squares is $S_{xx}=5$. The centered cross-product is $(-1.5)(-2)+(-0.5)(-1)+(0.5)(1)+(1.5)(2)=3+0.5+0.5+3=7$. Thus $\widehat{\beta}_1=7/5=1.4$ and $\widehat{\beta}_0=4-(1.4)(1.5)=1.9$.

The fitted values are $1.9,3.3,4.7,6.1$, giving residuals $0.1,-0.3,0.3,-0.1$. Hence $\operatorname{RSS}=0.01+0.09+0.09+0.01=0.20$, which is positive. The maximum-likelihood variance estimate is $\widehat{\sigma}^2=0.20/4=0.05$.

Check and trap. Since the minimum RSS is positive, the profile likelihood result applies. Dividing by $n=4$ gives the maximum-likelihood estimate requested; dividing by a residual degrees-of-freedom count would give a different estimator. If a coefficient vector instead fit every observation exactly, the likelihood would be unbounded as the variance approached zero, so the stated finite joint maximum-likelihood estimate would not exist.

Predictions and Model Limits

A fitted value from an identity-link model is a fitted conditional mean. If the model also specifies a Gaussian distribution and a common variance, that variance can be used to describe response variability around the mean. It should not be confused with uncertainty in the estimated coefficients; the examples below treat the coefficients and variance as given.

Worked Example: A Conditional Prediction Interval

Suppose hypothetical repair costs $Y$, measured in thousands of dollars, follow $Y\mid x\sim N(\mu,0.36)$, with $\mu=2+0.8x$. For a unit with $x=5$, find the conditional mean and a central 95% interval for an individual response, using 1.96 as the standard normal critical value.

Method. The identity link gives $\mu=2+(0.8)(5)=6$ thousand dollars. The conditional standard deviation is $\sqrt{0.36}=0.6$ thousand dollars. The interval is $6\pm1.96(0.6)=6\pm1.176$, or $(4.824,7.176)$ thousand dollars.

Check and trap. The interval is centered at the conditional mean, and its half-width is $1.96$ conditional standard deviations. It describes variation in an individual response under the specified distribution; it is not an interval for the unknown mean. Also, the normal model technically allows negative costs, despite this particular interval having positive endpoints. A mathematically convenient model should not be mistaken for a guarantee that every possible response is realistic.

Common Mistakes to Avoid

Confusing the link with the response distribution. The normal distribution specifies the random component; the identity link specifies how the conditional mean relates to the linear predictor.
Calling the variance a standard deviation. In $N(\mu,\sigma^2)$, the variance is $\sigma^2$, and the standard deviation is $\sigma$.
Assuming a linear mean implies constant variance. Constant variance is an additional Gaussian-model assumption, not a consequence of linearity alone.
Claiming a finite maximum likelihood estimate after an exact fit. With unknown common variance and zero minimum RSS, the likelihood is unbounded as the variance tends to zero.
Treating a fitted mean as an individual response. The fitted value is $E[Y_i\mid x_i]$; individual responses vary around it according to the model.

Key takeaway: In a Gaussian identity-link model, $E[Y_i\mid x_i]=x_i^\mathsf{T}\beta$ and $\operatorname{Var}(Y_i\mid x_i)=\sigma^2$. Under conditional independence and a common normal variance, likelihood estimation of the coefficients reduces to least squares, provided the minimum RSS is positive when the variance is unknown.

Check Your Understanding

For each problem, identify the model implication before calculating. Answers are given at the end of each item.

A hypothetical response satisfies $Y\mid x\sim N(\mu,4)$, with $\mu=10-0.5x$. At $x=6$, find the conditional mean, variance, and probability that $Y>9$, using $\Phi(1)=0.8413$. (Answer: mean $7$, variance $4$, and $P(Y>9)=0.1587$.)
Under a Gaussian identity-link model with a common known variance, the data are $(0,2),(1,4),(2,7)$. Find the least-squares intercept, slope, fitted values, and RSS. (Answer: $\widehat{\beta}_0=11/6$, $\widehat{\beta}_1=2.5$, fitted values $11/6,13/3,41/6$, and RSS $1/6$.)
A model specifies $E[Y\mid x,z]=3+0.2x-1.1z$ and $\operatorname{Var}(Y\mid x,z)=2.25$, with $Y\mid x,z$ normal. Find the conditional distribution at $x=10,z=1$, and the standard deviation. (Answer: $Y\mid x=10,z=1\sim N(3.9,2.25)$, with standard deviation $1.5$.)
Three independent responses are observed at $x=0,1,2$: $y=3,5,7$. An intercept and slope are fitted, and the common Gaussian variance is unknown. What happens to the joint likelihood? (Answer: The line $3+2x$ fits exactly, so the minimum RSS is zero and the likelihood is unbounded as $\sigma^2$ tends to zero; there is no finite joint maximum likelihood estimate of the variance.)
A Gaussian identity-link model has known variance $0.25$. For a proposed coefficient vector, the RSS is $1.2$; another proposed vector has RSS $0.8$. Which vector has the larger likelihood, and why? (Answer: The vector with RSS $0.8$, because with fixed variance the log-likelihood decreases as RSS increases.)

What You'll Learn