Multiple Regression as a Conditional Mean Model

Why Condition on Several Predictors?

A response may vary with more than one feature of the people or objects being studied. For example, a clinic might record a patient's recovery time along with age and the number of days of treatment. Looking at age alone would not describe the mean recovery time for patients who also differ in treatment duration. Multiple regression provides a model for the mean response conditional on several predictor values at once.

The word “conditional” is important. The model does not claim that every patient with the same age and treatment duration has exactly the same recovery time. It describes the average response among cases with those predictor values, under the model. Individual outcomes can differ around that mean.

Definition: In a regression model, the response variable \(Y\) is the outcome being described or predicted. The predictor variables \(X_1,\ldots,X_k\) are the measured variables used to describe differences in the response. A multiple linear regression model specifies the conditional mean \(E(Y\mid X_1=x_1,\ldots,X_k=x_k)\) using several predictors.

The response and predictors are chosen to match the question and the data collection. Calling a variable a predictor means it is used on the right-hand side of the model; it does not, by itself, mean that changing it causes the response to change.

The Basic Multiple Linear Regression Model

For \(n\) observations and \(k\) quantitative predictors, the basic model for observation \(i\) is

Y_i=\beta_0+\beta_1x_{i1}+\beta_2x_{i2}+\cdots+\beta_kx_{ik}+\varepsilon_i.

Here, \(Y_i\) is the response for observation \(i\), and \(x_{ij}\) is that observation’s value of predictor \(j\). The \(\beta\) quantities are model parameters: \(\beta_0\) is the intercept and \(\beta_j\) is the coefficient attached to predictor \(j\). The error \(\varepsilon_i\) represents the part of the response not accounted for by the model’s conditional mean. The corresponding conditional-mean equation is

E(Y_i\mid x_{i1},\ldots,x_{ik}) =\beta_0+\beta_1x_{i1}+\cdots+\beta_kx_{ik}.

The basic model is additive: its mean is the intercept plus one term for each predictor. In this form it contains no product terms or curved functions of predictors. “Linear” refers to the model’s linear combination of coefficients. This tutorial uses the basic additive form; other model forms can be introduced when needed.

Each coefficient is attached to a particular predictor, and the conditional-mean equation includes all predictors together. In this additive model, the coefficient \(\beta_j\) represents the change in the modelled mean response for a one-unit increase in \(X_j\), with the other predictor values held fixed. That is a statement about the model’s conditional mean, not necessarily about a causal effect.

What the Conditional Mean Says

Conditioning means specifying the predictor values being considered. For example, \(E(Y\mid X_1=40,X_2=5)\) is the mean response among observations whose first predictor is 40 and whose second predictor is 5, as represented by the model. The conditional mean is a population-level quantity in the model; it is not an assertion about the exact response of any one individual.

The model separates a response into two parts: the mean associated with the specified predictors and an error around that mean. A central assumption is that the errors have conditional mean zero:

E(\varepsilon_i\mid x_{i1},\ldots,x_{ik})=0.

This assumption is what makes the expression without \(\varepsilon_i\) the conditional mean. It says that, at any given set of predictor values, positive and negative deviations balance on average. It does not say that every error equals zero.

Worked Example: Naming the Response and Predictors

Suppose a hypothetical rehabilitation service records a patient's number of days to complete a recovery milestone, age in years, and number of supervised therapy sessions during the first week. The question is how mean recovery time varies with age and therapy sessions. Identify the response and predictors, then express the conditional mean in model notation.

Response: \(Y\) is recovery time, measured in days. Predictors: \(X_1\) is age, measured in years, and \(X_2\) is the number of supervised sessions. The response is the outcome whose mean the question concerns; the two predictors are the variables used to describe that mean.

The basic model is

Y_i=\beta_0+\beta_1x_{i1}+\beta_2x_{i2}+\varepsilon_i, \qquad E(Y_i\mid x_{i1},x_{i2})=\beta_0+\beta_1x_{i1}+\beta_2x_{i2}.

For this example, suppose the model’s conditional-mean and error assumptions are appropriate for the observations being considered. The equation describes the mean number of recovery days at specified ages and session counts; it does not establish that age or sessions cause a change in recovery time.

Deriving the Conditional-Mean Equation

The conditional-mean equation follows directly from the response model and the zero-mean error assumption. At fixed predictor values, the intercept and predictor terms are constants. Taking the conditional expectation of both sides therefore gives the following:

Start with the response model.
\(Y_i=\beta_0+\sum_{j=1}^{k}\beta_jx_{ij}+\varepsilon_i\).

Condition on the predictor values.
With \(x_{i1},\ldots,x_{ik}\) fixed, \(\beta_0\) and every \(\beta_jx_{ij}\) are constants. By linearity of conditional expectation, \(E(Y_i\mid x_{i1},\ldots,x_{ik})=\beta_0+\sum_{j=1}^{k}\beta_jx_{ij}+E(\varepsilon_i\mid x_{i1},\ldots,x_{ik})\).

Apply the zero-mean error assumption.
Since \(E(\varepsilon_i\mid x_{i1},\ldots,x_{ik})=0\), the final term vanishes. Thus \(E(Y_i\mid x_{i1},\ldots,x_{ik})=\beta_0+\sum_{j=1}^{k}\beta_jx_{ij}\), as claimed.

The derivation shows why the zero conditional mean matters: without it, the stated regression expression would not equal the conditional mean. Other assumptions are important for estimation and inference, but they are not needed for this algebraic step.

Assumptions and Scope

A model is useful only when its assumptions are reasonable for the question and data. In the basic multiple regression model, the main conditions are stated in terms of the response after conditioning on the predictors:

Correct conditional-mean form: the mean response is adequately represented by the specified additive linear expression over the predictor values being studied.
Zero conditional mean: \(E(\varepsilon_i\mid x_{i1},\ldots,x_{ik})=0\). No systematic part of the response remains in the errors at fixed predictor values.
Constant conditional variance: \(Var(\varepsilon_i\mid x_{i1},\ldots,x_{ik})=\sigma^2\), the same for all predictor combinations in the model’s scope.
Independent errors: responses’ errors are independent across observations, conditional on the predictors. Dependence can arise, for example, when measurements from the same person are treated as separate independent observations.

For exact small-sample inference based on the usual normal-error model, a further assumption is that the errors are conditionally normally distributed: \(\varepsilon_i\mid x_{i1},\ldots,x_{ik}\sim N(0,\sigma^2)\). Normality is not required simply to define a conditional mean, and it is not a guarantee that the model’s mean form is correct.

These are model assumptions, not facts established just by writing down an equation. Data can help assess whether a model is plausible, but a model can still be inadequate. In particular, observational data and a good-fitting conditional-mean model do not by themselves justify a causal conclusion. Other variables, the way cases entered the data, and the study design matter to such a claim.

Worked Example: Calculating a Conditional Mean

For illustration, suppose a hypothetical model for a laboratory measurement \(Y\), in milligrams per liter, uses two predictors: \(X_1\), hours since a procedure, and \(X_2\), a baseline score. The stated model is \(E(Y\mid X_1,X_2)=12-0.40X_1+0.80X_2\). Calculate the modelled conditional mean for a case with \(X_1=5\) hours and \(X_2=10\).

Substitute the two predictor values into the conditional-mean expression:

E(Y\mid X_1=5,X_2=10) =12-0.40(5)+0.80(10) =12-2+8 =18\text{ mg/L}.

The model’s mean response for cases with these predictor values is 18 mg/L. This value is not a claim that each such case has a measured response of exactly 18 mg/L. For this calculation, suppose the additive mean form and the zero-mean error condition are appropriate at these values. The calculation alone cannot check those assumptions; nor does the hypothetical equation provide evidence that either predictor causes the response to change.

Several Predictors Describe a Profile

A set of predictor values can be viewed as a profile. The model associates a conditional mean with each profile within the range where the model is being used. Changing one component of a profile while holding the others fixed changes the modelled mean according to the coefficient for that predictor. This way of describing the model is useful because it makes clear that the mean is conditional on the whole set of included predictor values, not on one predictor in isolation.

Worked Example: Comparing Two Predictor Profiles

Suppose a hypothetical model for weekly study hours has the conditional mean \(E(Y\mid X_1,X_2)=6+0.5X_1+1.2X_2\), where \(X_1\) is scheduled class hours per week and \(X_2\) is the number of planned study sessions per week. Compare the modelled means for profiles \((X_1=12,X_2=3)\) and \((X_1=12,X_2=5)\).

For the first profile,

E(Y\mid X_1=12,X_2=3)=6+0.5(12)+1.2(3)=6+6+3.6=15.6\text{ hours}.

For the second profile,

E(Y\mid X_1=12,X_2=5)=6+0.5(12)+1.2(5)=6+6+6=18\text{ hours}.

The modelled means differ by \(18-15.6=2.4\) hours per week. The profiles have the same class hours and differ by two planned sessions; the model’s conditional means differ by \(1.2(5-3)=2.4\) hours. This comparison describes the equation, not necessarily a causal effect of scheduling sessions. Suppose the stated mean form and error assumptions hold over these profiles; nothing in the arithmetic itself verifies that supposition.

Common Mistake: Treating the Mean as an Individual Outcome

A frequent error is to say, “The model gives 18 mg/L, so every case with those predictor values will have a response of 18 mg/L.” That confuses a conditional mean with an individual response. Under the model, an individual response also includes its error:

Y_i=E(Y_i\mid x_{i1},\ldots,x_{ik})+\varepsilon_i.

Different cases with the same predictor values can therefore have different responses. The zero-mean condition says their errors average to zero at those values; it does not require each error to be zero. Also avoid interpreting “predictor” as “cause”: a conditional association is not, without suitable design and supporting assumptions, a causal conclusion.

Key takeaway: Multiple linear regression models a response’s mean at specified values of several predictors. The conditional mean is the additive regression expression; an individual response can differ from it, and the model’s conditional description alone does not establish causation.

Check Your Understanding

Answer each question using the meaning of the model, not just its notation.

A study records blood pressure as an outcome and age and daily sodium intake as predictors. Name the response and both predictors.
In \(E(Y\mid X_1=x_1,X_2=x_2)=\beta_0+\beta_1x_1+\beta_2x_2\), what does the conditional mean refer to?
Why does the assumption \(E(\varepsilon_i\mid x_{i1},\ldots,x_{ik})=0\) make the additive expression the conditional mean?
Explain why a conditional mean of 20 units does not imply that every individual with those predictor values has a response of 20 units.
Does calling a variable a predictor establish that it causes a change in the response? Explain briefly.