Why Use a Gaussian GLM for a Continuous Outcome?
A continuous target such as repair cost, processing time, or an account balance is often modeled through its conditional mean. A Gaussian generalized linear model (GLM) with the identity link is a direct specification: the mean is a linear function of the predictors. Under the usual unweighted setup, fitting this model gives the same coefficients and fitted means as ordinary least squares (OLS).
That equivalence is useful, but it is conditional on matching the response, observations, and predictor structure. It does not mean every Gaussian GLM and every OLS model must agree, nor does it establish that a linear mean or constant variance is appropriate for a particular application. The treatment here is aligned with the SOA Exam PA (Predictive Analytics) syllabus.
The predictor vector includes an intercept when the model has one. For example, with one numeric predictor \(x_i\) and an intercept, \(\mathbf{x}_i=(1,x_i)^\mathsf{T}\), so the conditional mean is \(\mu_i=\beta_0+\beta_1x_i\). With several predictors, the mean is the corresponding linear combination. “Linear” here refers to the coefficients; transformed predictors can be included if they are explicitly part of the model.
The Gaussian distribution allows any real-valued response, and the identity link allows any real-valued fitted mean. That flexibility is useful for many measurements, but it does not keep fitted means positive or within a fixed range. If the response has such restrictions, a Gaussian identity model can produce predictions that conflict with them.
From the Gaussian Likelihood to Least Squares
Suppose the observations are conditionally independent and have the Gaussian distributions in the definition, with the same variance \(\sigma^2\). Let \(X\) be the design matrix whose \(i\)th row is \(\mathbf{x}_i^\mathsf{T}\), let \(\mathbf{y}\) be the response vector, and let \(\boldsymbol{\beta}\) be the coefficient vector. Then the vector of fitted means is \(X\boldsymbol{\beta}\).
The log-likelihood, apart from no omitted terms depending on \(\boldsymbol{\beta}\), is
For fixed positive \(\sigma^2\), the first term does not depend on \(\boldsymbol{\beta}\), and the second term is largest when the squared residual sum is smallest. Thus maximizing the Gaussian likelihood over the coefficients is equivalent to minimizing the OLS objective.
The score equation gives a short justification. Differentiating the log-likelihood with respect to \(\boldsymbol{\beta}\) yields
At a maximum, this derivative is zero, so \(X^\mathsf{T}X\boldsymbol{\beta}=X^\mathsf{T}\mathbf{y}\). These are the OLS normal equations. When \(X^\mathsf{T}X\) is invertible, solving them gives the stated formula. If the design matrix is rank-deficient, coefficients may not be unique; nonetheless, least-squares fitted values are unique, and the Gaussian likelihood has the same fitted-mean criterion.
The equality concerns coefficient estimates and fitted means. It does not require the Gaussian distribution to calculate OLS coefficients: OLS can be computed for any observed responses. Gaussianity, independence, and the common-variance assumption give the likelihood interpretation and underpin conventional model-based uncertainty calculations. If errors are not Gaussian, the same algebraic OLS fit can still be calculated, but Gaussian likelihood-based inference may not be justified.
Worked Fits and Interpretation
Worked Example: One-Predictor Gaussian GLM
A hypothetical service team records a continuous resolution score \(y\) at three values of a workload index \(x\): \((0,8),(1,10),(2,15)\). Fit a Gaussian identity-link model with an intercept, calculate the fitted mean at \(x=3\), and check the residual sum of squares.
Method. Because this is an unweighted Gaussian identity model with the usual intercept and linear predictor, its coefficient estimates equal the OLS estimates. First calculate \(\bar{x}=(0+1+2)/3=1\) and \(\bar{y}=(8+10+15)/3=11\).
The centered sums are \(S_{xx}=\sum_i(x_i-\bar{x})^2=(-1)^2+0^2+1^2=2\) and \(S_{xy}=\sum_i(x_i-\bar{x})(y_i-\bar{y})=(-1)(-3)+0(-1)+1(4)=7\). Therefore, \(\hat{\beta}_1=S_{xy}/S_{xx}=7/2=3.5\), and \(\hat{\beta}_0=\bar{y}-\hat{\beta}_1\bar{x}=11-3.5(1)=7.5\).
The fitted mean is \(\hat{\mu}(x)=7.5+3.5x\). At \(x=3\), it is \(7.5+3.5(3)=18\) score units. For a fit check, the fitted values at \(x=0,1,2\) are \(7.5,11,14.5\). The residuals, observed minus fitted, are \(0.5,-1,0.5\), so the residual sum of squares is \(0.5^2+(-1)^2+0.5^2=1.5\). This verifies the fitted line’s residual calculation. The common mistake is to report a slope of \(2\) from an incomplete calculation; the centered cross-product is \(7\), not \(4\), so the correct slope is \(3.5\).
Worked Example: Two Predictors in a Balanced Design
A hypothetical operations study records a continuous outcome at four combinations of predictors \(x_1,x_2\), each coded as \(-1\) or \(1\). The observations are \((-1,-1,21),(-1,1,13),(1,-1,23),(1,1,23)\). Fit the identity-link mean \(\mu=\beta_0+\beta_1x_1+\beta_2x_2\), predict at \((1,1)\), and check the residuals.
Method. The four design rows are \((1,-1,-1),(1,-1,1),(1,1,-1),(1,1,1)\). Each predictor sums to zero, and their cross-product is \(\sum x_1x_2=1-1-1+1=0\); hence the intercept and predictor columns are mutually orthogonal. The response mean is \(\bar{y}=(21+13+23+23)/4=20\), giving \(\hat{\beta}_0=20\).
Because each predictor column has squared length \(4\), its coefficient is its response cross-product divided by \(4\). For \(x_1\), \(\sum x_1y=(-21-13+23+23)=12\), so \(\hat{\beta}_1=12/4=3\). For \(x_2\), \(\sum x_2y=(-21+13-23+23)=-8\), so \(\hat{\beta}_2=-8/4=-2\). The fitted mean is therefore \(\hat{\mu}=20+3x_1-2x_2\), and its prediction at \((1,1)\) is \(20+3-2=21\).
For the check, the fitted values in the listed order are \(19,15,25,21\). Subtracting these from the observations gives residuals \(2,-2,-2,2\); their sum is zero and their squared sum is \(4+4+4+4=16\). The last observation is \(23\), not the fitted mean \(21\), because a fitted conditional mean need not equal an individual observed response. The common mistake is to use the observed value at \((1,1)\) as the model prediction rather than evaluate the fitted mean.
Worked Example: Intercept-Only Fit and Dispersion
For a hypothetical continuous processing-time measure, four independent observations are \(12,15,9,14\) minutes. Assume \(Y_i\sim N(\mu,\sigma^2)\) with a common variance and fit an intercept-only Gaussian identity model. Find the fitted mean, residual sum of squares, Gaussian maximum-likelihood estimate of \(\sigma^2\), and the usual unbiased residual variance estimate.
Method. An intercept-only model has the same fitted mean for every observation. Its OLS estimate, and thus its Gaussian GLM estimate, is the sample mean: \(\hat{\mu}=(12+15+9+14)/4=50/4=12.5\) minutes.
The residuals are \(12-12.5=-0.5\), \(15-12.5=2.5\), \(9-12.5=-3.5\), and \(14-12.5=1.5\). Their squared sum is \(0.25+6.25+12.25+2.25=21\) minutes squared. There is one fitted coefficient, so the residual degrees of freedom are \(4-1=3\). The Gaussian maximum-likelihood estimate divides by \(n=4\): \(\hat{\sigma}^2_{\mathrm{ML}}=21/4=5.25\). The usual unbiased residual variance estimate divides by the residual degrees of freedom: \(21/3=7\).
Both variance calculations use the same fitted mean; they differ because their denominators answer different estimation questions. The common mistake is to call \(21/3\) the Gaussian maximum-likelihood estimate. Under this model the likelihood estimate is \(21/4\); the divisor \(3\) gives the unbiased residual variance estimate.
When the Equivalence Does—and Does Not—Hold
A Gaussian identity GLM and OLS produce equal fitted means when they use the same response values, rows, predictor columns, intercept choice, and unweighted least-squares objective. In R, the corresponding model specifications can be written as:
For equivalent results, the two fits must also treat missing observations consistently. A change in a factor’s reference coding can alter coefficient labels and values while leaving the underlying fitted means unchanged, provided the model spaces are the same. A genuinely different set of predictors or a different subset of observations generally changes the fit.
Other changes can break the equivalence with ordinary unweighted OLS. For example, prior weights lead to a weighted fitting criterion rather than the unweighted sum of squared residuals. A penalty, coefficient constraint, or a different mean structure also changes the estimation problem. A weighted Gaussian fit may still be compared with a weighted least-squares fit using the same weights, but it should not be described as the same fit as unweighted OLS.
Worked Example: Why Weights Change an Intercept-Only Fit
A hypothetical data set has two response values, \(10\) and \(16\), with prior weights \(1\) and \(3\). For an intercept-only Gaussian model using these weights, calculate the fitted mean and compare it with the unweighted OLS mean.
Method. The weighted least-squares objective for an intercept \(\mu\) is \(1(10-\mu)^2+3(16-\mu)^2\). Differentiating with respect to \(\mu\) gives \(-2(10-\mu)-6(16-\mu)=0\), or \(4\mu=10+48=58\). Thus the weighted fitted mean is \(58/4=14.5\).
If the weights are ignored, the ordinary mean is \((10+16)/2=13\). The two fits differ by \(14.5-13=1.5\). The high-weight observation contributes three times as much to the weighted objective, so the weighted fit is not the unweighted OLS fit. This is the trap in claiming that every Gaussian GLM automatically matches ordinary unweighted least squares.
Assumptions, Checks, and Practical Interpretation
A Gaussian identity model is not justified merely because the response is continuous. Its conditional mean must be adequately represented by the specified linear predictor for the intended use. The standard Gaussian model also assumes independent responses with a common conditional variance and Gaussian conditional distributions. These assumptions are particularly relevant to likelihood-based inference and prediction intervals; the equality of coefficient calculations with OLS alone does not verify them.
After fitting, examine residuals against fitted values and relevant predictors for patterns such as curvature or changing spread. Large residuals may indicate observations that merit investigation, but they do not by themselves establish that a data point is erroneous. Consider the measurement process and the modeling objective before taking action. Also check that the prediction range is meaningful: because the identity link imposes no bounds, a fitted mean can be negative even when negative values are impossible for the target.
A further distinction concerns the conditional mean and an observed outcome. The fitted value \(\hat{\mu}_i\) estimates the mean response at the predictor values; it is not a promise that a future observation will equal that value. Under the model, individual outcomes vary around their conditional means. Communicating this distinction helps prevent a point prediction from being interpreted as a guaranteed result.
Check Your Understanding
Work through these original problems, keeping the design and objective explicit.
- A hypothetical monitoring study has four continuous responses \(4,6,8,10\) and an intercept-only Gaussian identity model. Calculate the fitted mean. (Answer: \(7\).)
- Fit an intercept-and-slope Gaussian identity model to \((x,y)=(0,3),(1,5),(2,8)\). Give the slope, intercept, and prediction at \(x=4\). (Answer: slope \(2.5\), intercept \(17/6\), prediction \(77/6\), or approximately \(12.8333\).)
- A model is constrained to pass through the origin and uses observations \((1,3),(2,7)\). Calculate its Gaussian identity/OLS slope and prediction at \(x=3\). (Answer: slope \(3.4\), prediction \(10.2\).)
- An intercept-only Gaussian fit uses responses \(10\) and \(16\), with prior weights \(1\) and \(3\). What is its weighted fitted mean, and what would unweighted OLS produce? (Answer: weighted mean \(14.5\); unweighted mean \(13\).)
- For \((x,y)=(0,2),(1,4),(2,5)\), fit an intercept-and-slope Gaussian identity model. Give the coefficients, prediction at \(x=4\), and residual sum of squares. (Answer: intercept \(13/6\), slope \(1.5\), prediction \(49/6\), and residual sum of squares \(1/6\).)