Profile-Based Predictions in Multiple Regression

From a Predictor Profile to a Fitted Mean

A multiple regression uses several predictors together to describe the conditional mean of an outcome. To obtain a fitted mean for a particular case, we need a value for every predictor in the model. That complete set of values is the case’s predictor profile. Substituting the profile into the fitted regression equation gives an estimated mean outcome for cases with those predictor values.

This calculation is useful when a question concerns a specified combination of characteristics rather than a change in just one predictor. It also makes clear that a coefficient is interpreted in relation to the other predictors in the model: its contribution is assessed while their values are held fixed.

Definition: For a model with predictors \(X_1,\ldots,X_p\), a complete predictor profile is a specified set of values \(x_1,\ldots,x_p\), one for each predictor. The fitted conditional mean at that profile is the model’s estimated mean outcome for cases with those predictor values, calculated by substituting the profile into the fitted regression equation.

The Additive Multiple Regression Model

For this tutorial, consider a multiple regression with quantitative predictors and an additive conditional mean: each predictor has its own term, and the model has no interactions or curved terms. A population model for observation \(i\) is

Y_i=\beta_0+\beta_1X_{i1}+\beta_2X_{i2}+\cdots+\beta_pX_{ip}+\varepsilon_i.

Here \(Y_i\) is the outcome, \(X_{ij}\) is predictor \(j\) for observation \(i\), \(\beta_0\) is the intercept, and \(\beta_j\) is the coefficient for predictor \(j\). The model assumes that the conditional mean is linear and additive in the specified predictors, expressed by \(E(\varepsilon_i\mid X_{i1},\ldots,X_{ip})=0\). Thus, at fixed predictor values, the error has mean zero and the conditional mean is the sum of the intercept and predictor terms. This is a model assumption, not a conclusion established by plugging in numbers.

Once the coefficients have been estimated from data, denote them by \(\hat{\beta}_0,\hat{\beta}_1,\ldots,\hat{\beta}_p\). The fitted equation for a profile is

\widehat{E}(Y\mid X_1=x_1,\ldots,X_p=x_p) =\hat{\beta}_0+\hat{\beta}_1x_1+\hat{\beta}_2x_2+\cdots+\hat{\beta}_px_p.

Each term \(\hat{\beta}_jx_j\) is that predictor’s contribution at its specified value, measured in the outcome’s units. The intercept is the fitted value when every predictor equals zero. That may be a useful mathematical reference point but need not describe a realistic case. The fitted conditional mean is an estimated average for cases at the profile; it is not a promise that a particular individual will have exactly that outcome.

Worked Example: Estimating a Mean Exam Score

Suppose a hypothetical analysis provides this fitted equation for exam score, measured in points, using study hours, prior score, and attendance days:

\widehat{E}(Y\mid x_1,x_2,x_3) =42+2.1x_1+0.35x_2+0.8x_3.

Here \(x_1\) is study hours, \(x_2\) is prior score in points, and \(x_3\) is the number of attended class days. Consider a complete profile of 6 study hours, a prior score of 70 points, and 18 attended days. Assume these values lie within the ranges used to fit the model, and use the stated additive equation.

Substitute each value, keeping its matching coefficient:

\begin{aligned} \widehat{E}(Y\mid 6,70,18) &=42+(2.1)(6)+(0.35)(70)+(0.8)(18)\\ &=42+12.6+24.5+14.4\\ &=93.5\text{ points}. \end{aligned}

For this profile, the fitted conditional mean exam score is 93.5 points. The three predictor terms contribute 12.6, 24.5, and 14.4 points to the fitted value, in addition to the 42-point intercept. This does not mean every student with this profile would score 93.5; individual scores can differ from the conditional mean.

What “Holding Other Predictors Fixed” Means

In an additive model, the coefficient \(\hat{\beta}_j\) describes the change in the fitted conditional mean associated with a one-unit increase in predictor \(j\), holding the other predictors fixed. The units matter: the coefficient’s units are outcome units per unit of that predictor. This is a statement about the fitted model’s conditional mean, not automatically a causal effect.

The reasoning follows directly from the equation. Suppose two profiles have the same values for all predictors except \(X_j\), which changes from \(x_j\) to \(x_j+\Delta\). Subtracting their fitted means cancels the intercept and every unchanged predictor term. The remaining difference is \(\hat{\beta}_j\Delta\):

\begin{aligned} \widehat{E}(Y\mid \ldots,x_j+\Delta,\ldots) -\widehat{E}(Y\mid \ldots,x_j,\ldots) &=\hat{\beta}_j(x_j+\Delta)-\hat{\beta}_jx_j\\ &=\hat{\beta}_j\Delta. \end{aligned}

This justification depends on the additive model specified above. If the model included an interaction or a curved term, the change associated with a predictor could depend on the other predictor values or on the starting value. Here, each coefficient’s fitted contribution is a constant slope in its own predictor, with the remaining predictors fixed.

Worked Example: Area Contribution at a Fixed Occupancy

A hypothetical fitted model predicts monthly household electricity use \(Y\) in kilowatt-hours from floor area \(x_1\), in square feet, and number of occupants \(x_2\):

\widehat{E}(Y\mid x_1,x_2)=120+0.085x_1+95x_2.

Suppose both profiles below are within the ranges represented in the data. For a 900-square-foot home with 3 occupants, the complete profile gives

\begin{aligned} \widehat{E}(Y\mid 900,3) &=120+(0.085)(900)+(95)(3)\\ &=120+76.5+285\\ &=481.5\text{ kWh per month}. \end{aligned}

Now keep the number of occupants at 3 and use a floor area of 1,100 square feet. The fitted mean is

\begin{aligned} \widehat{E}(Y\mid 1100,3) &=120+(0.085)(1100)+(95)(3)\\ &=120+93.5+285\\ &=498.5\text{ kWh per month}. \end{aligned}

The occupancy term and intercept are unchanged. The floor area rises by 200 square feet, so its fitted contribution rises by \((0.085)(200)=17.0\) kWh per month. Accordingly, the fitted mean for the larger home is 17.0 kWh per month higher at the same occupancy. The coefficient \(0.085\) is measured in kilowatt-hours per month per square foot; this fitted relationship alone does not establish that increasing a home’s area causes electricity use to rise.

Completeness, Units, and the Range of the Data

A profile must include every predictor in the stated equation. If a predictor value is missing, the equation does not give a unique fitted mean until that value is supplied or a separate prediction procedure is specified. Do not silently drop a term: doing so changes the model calculation. Before substituting, match each value to its predictor and check the measurement units. A value in minutes cannot be entered as though the predictor were measured in hours.

Also consider whether the profile resembles values in the data used to fit the model. Substituting values outside the observed predictor ranges is extrapolation. The arithmetic still produces a number, but the model’s linear-additive pattern may not be a credible description far beyond the data. Even within the separate observed ranges, an unusual combination of predictor values may be poorly represented. A calculated fitted mean should therefore be reported with the profile and the model context, not as a context-free number.

Worked Example: A Complete Profile for Recovery Time

A hypothetical model predicts recovery time \(Y\), in days, from age \(x_1\), in years; a pre-operative risk score \(x_2\); and planned procedure duration \(x_3\), in hours. Suppose the fitted equation is

\widehat{E}(Y\mid x_1,x_2,x_3) =0.5+0.04x_1+0.8x_2+1.6x_3.

Consider a complete profile of age 65 years, risk score 4, and planned procedure duration 2.5 hours. Assume this combination is represented in the fitting data and the supplied additive model is being used. Substituting all three values gives

\begin{aligned} \widehat{E}(Y\mid 65,4,2.5) &=0.5+(0.04)(65)+(0.8)(4)+(1.6)(2.5)\\ &=0.5+2.6+3.2+4.0\\ &=10.3\text{ days}. \end{aligned}

The fitted conditional mean is 10.3 recovery days for cases with this profile under the model. The coefficient units are days per year for age, days per risk-score point for the score, and days per hour for procedure duration. The calculation does not say that changing a patient’s age, risk score, or procedure duration would cause the corresponding fitted change; other factors and the way the model was constructed matter.

Common Mistakes

Leaving out a predictor term. A student substitutes age and risk score but omits procedure duration. The result is not the fitted mean from the displayed three-predictor equation. Use a value for every predictor, or state that the profile is incomplete.
Reading a coefficient as an unadjusted relationship. A coefficient in a multiple regression describes the fitted association with its predictor when the other included predictors are fixed, not a comparison in which they may also change.
Calling the fitted mean an individual prediction with certainty. The fitted conditional mean is an estimated average at the profile. A particular outcome can be above or below it.
Treating a fitted association as causal. Holding measured predictors fixed in an equation does not, by itself, remove confounding or establish that changing one predictor would change the outcome.
Ignoring units or extrapolation. Check the units and compare the profile with the data used to fit the model. A plausible-looking numerical result can still arise from an incorrectly scaled or poorly supported profile.

Key takeaway: For an additive multiple regression, a fitted conditional mean is obtained by substituting one value for every predictor and summing the intercept and all predictor contributions. A coefficient describes the fitted mean’s change per predictor unit while the other predictors are held fixed.

Check Your Understanding

Answer each question using the fitted-equation ideas above.

A model is \(\widehat{E}(Y\mid x_1,x_2)=10+3x_1-2x_2\). Calculate the fitted conditional mean at \(x_1=4\), \(x_2=1\), and show each term.
In that model, what does the coefficient 3 describe about the fitted conditional mean, and which values must be held fixed for that interpretation?
For the same model, explain why a profile giving only \(x_1=4\) is insufficient to calculate a unique fitted mean.
If two profiles have the same values of every predictor except \(x_2\), which terms cancel when their fitted means are subtracted?
Why is a fitted conditional mean not a guarantee of the outcome for one individual at the stated profile?