Conditional Means and Random Errors

One Predictor Value, Many Possible Responses

A regression line describes a systematic relationship between a predictor and a response. It does not generally say that every individual response lies on the line. For a fixed predictor value, different observations can have different outcomes because of random variation not represented by the line. The line describes the conditional mean response; the difference between an individual response and that mean is its random error.

This distinction matters whenever a model is used to describe a group or a portfolio. A predicted mean at a particular risk characteristic is not a promise about the next individual claim, policy, or customer. To interpret a regression model correctly, keep track of which quantity is being discussed: the mean response, an individual response, or the random error connecting them.

Definition: In a simple linear regression model, the response $Y$ at predictor value $x$ is written $$ Y=\beta_0+\beta_1x+\varepsilon, $$ where $\beta_0+\beta_1x$ is the systematic part of the model and $\varepsilon$ is the random error. The model’s mean-zero condition is $E[\varepsilon\mid X=x]=0$ for predictor values under consideration. Consequently, the conditional mean response is $$ E[Y\mid X=x]=\beta_0+\beta_1x. $$

Here, $\beta_0$ and $\beta_1$ are the model’s population parameters, not estimates computed from a particular sample. The response $Y$ is random even after $x$ is specified, unless the error is always zero. The error $\varepsilon$ is the part of an individual response not accounted for by the regression line. It can be positive or negative.

The conditional mean is an average over the possible responses associated with the same predictor value. It need not be an outcome that actually occurs for any one individual. For example, a model might give a conditional mean of 37.4 hours even if the response is recorded in whole hours. The mean describes the center of the conditional response distribution, not a required value for each observation.

Why the Conditional Mean Is the Line

The key mathematical condition is that the error has conditional mean zero. This means that, among responses with the same predictor value $x$, positive and negative errors balance on average. Taking a conditional expectation of the model gives the regression line as the conditional mean.

Start with the model.
At predictor value $x$, write $Y=\beta_0+\beta_1x+\varepsilon$. The coefficients and $x$ are fixed when taking the conditional expectation.

Take the conditional expectation.
By linearity of conditional expectation, $E[Y\mid X=x]=\beta_0+\beta_1x+E[\varepsilon\mid X=x]$.

Apply the mean-zero condition.
Since $E[\varepsilon\mid X=x]=0$, the final term vanishes, leaving $E[Y\mid X=x]=\beta_0+\beta_1x$.

This justification uses the conditional mean-zero assumption; it does not require the errors to have a particular probability distribution. Nor does the mean result by itself require equal error variances at different predictor values. Those properties may matter for other purposes, but they are not needed to identify the conditional mean in this calculation.

Worked Example: Separating the Mean and Individual Errors

A hypothetical insurer uses a simple regression model for claim-handling time $Y$, measured in hours, with complexity score $x$: $$ Y=18+2.5x+\varepsilon,\qquad E[\varepsilon\mid X=x]=0. $$ For a claim with score $x=8$, the observed handling time is 44 hours. For another claim with the same score, it is 31 hours. Find the conditional mean and each claim’s error.

Method. First evaluate the line at $x=8$. Then define each error as the observed response minus that conditional mean.

The line gives $$ 18+2.5(8)=18+20=38\text{ hours}. $$ Thus $E[Y\mid X=8]=38$ hours. For the first claim, $$ \varepsilon=44-38=6\text{ hours}. $$ For the second claim, $$ \varepsilon=31-38=-7\text{ hours}. $$ The first response is 6 hours above its conditional mean, while the second is 7 hours below it.

Check. Adding each error back to the mean gives $38+6=44$ and $38-7=31$, the two observed responses. The mean-zero condition describes the average error at $x=8$ across the model’s response process; it does not require either individual error to equal zero.

Common mistake avoided. Treating 38 hours as the actual time for every claim with score 8 confuses a conditional mean with an individual response.

The Random Error Is a Deviation from the Mean

Rearranging the model makes the role of the error explicit: $$ \varepsilon=Y-(\beta_0+\beta_1x). $$ For a particular response, the error is the signed distance between that response and the model’s conditional mean at the same predictor value. A positive error means the response exceeds the mean; a negative error means it falls below the mean.

The error is called random because the response is not fully determined by the predictor and the regression line. Before observing a particular response, its error is not known. Once $Y$ is observed, the difference from the model’s conditional mean is well-defined under the model. The error is part of the model’s description of the response process, not an additional predictor.

The mean-zero condition has a precise interpretation. For any fixed $x$, it says that the average error over the conditional distribution of $Y$ is zero. It does not say that every error is zero, that positive and negative errors must occur equally often in a finite sample, or that an individual response equals its conditional mean.

Worked Example: Checking a Conditional Error Distribution

A hypothetical model for the annual number of service calls from a commercial vehicle is $Y=50+3x+\varepsilon$, where $x$ is a vehicle-use score. At $x=4$, suppose the possible errors are $-8$, $0$, and $8$ calls, with probabilities $0.25$, $0.50$, and $0.25$, respectively. Find the conditional mean response and check the mean-zero condition.

Step 1: Evaluate the regression line. $$ 50+3(4)=50+12=62. $$ The line gives a conditional mean of 62 calls if the conditional error mean is zero.

Step 2: Compute the conditional mean error. $$ E[\varepsilon\mid X=4] =(-8)(0.25)+(0)(0.50)+(8)(0.25) =-2+0+2=0. $$ The specified error distribution satisfies the model’s mean-zero condition at $x=4$.

Step 3: Identify the possible responses. Adding each error to 62 gives $62-8=54$, $62+0=62$, and $62+8=70$ calls. Their conditional mean is $$ (54)(0.25)+(62)(0.50)+(70)(0.25) =13.5+31+17.5=62. $$

Check. The conditional mean of the responses is 62, matching the line, and the conditional mean error is zero. Individual responses can nevertheless be 54 or 70.

Common mistake avoided. The model’s line is not the only possible response at $x=4$; it is the probability-weighted mean of the possible responses in this example.

From Conditional Means to an Overall Mean

If the predictor itself varies across a population, the overall mean response averages over its possible values. The law of iterated expectations states that, when the relevant expectations exist, $$ E[Y]=E\big[E[Y\mid X]\big]. $$ When $E[Y\mid X=x]=\beta_0+\beta_1x$, this gives $E[Y]=\beta_0+\beta_1E[X]$. This is an average across predictor values, not a statement that every response equals the line evaluated at the average predictor.

Worked Example: Averaging Across Predictor Values

A hypothetical portfolio model for annual repair cost, in thousands of dollars, is $Y=12+5x+\varepsilon$, with $E[\varepsilon\mid X=x]=0$. In the portfolio, 40% of vehicles have use score $x=2$, and 60% have score $x=6$. Find the conditional means at both scores and the overall expected repair cost.

Step 1: Calculate the conditional mean at each score. $$ E[Y\mid X=2]=12+5(2)=22 $$ and $$ E[Y\mid X=6]=12+5(6)=42. $$ The values are in thousands of dollars.

Step 2: Weight those means by the score probabilities. $$ E[Y]=(0.40)(22)+(0.60)(42)=8.8+25.2=34. $$ Thus the overall expected repair cost is 34 thousand dollars, or $34,000.

Step 3: Check using the mean predictor. The expected score is $$ E[X]=(0.40)(2)+(0.60)(6)=0.8+3.6=4.4. $$ The line evaluated at this average score gives $12+5(4.4)=12+22=34$, the same overall mean.

Check. The weighted conditional means and the line at the average score both give $34,000. This equality follows from the linear form of the conditional mean and the zero conditional mean of the error.

Common mistake avoided. Averaging the two score values without their probabilities would give $x=4$, which is not the portfolio’s expected score. The correct weights are 0.40 and 0.60.

Common Interpretive Traps

Calling the line an individual prediction. At a specified $x$, $\beta_0+\beta_1x$ is the conditional mean under the stated model. Individual responses may lie above or below it.
Reversing the error sign. The model defines $\varepsilon=Y-(\beta_0+\beta_1x)$, so error equals response minus conditional mean. If $Y$ is below the mean, the error is negative.
Assuming a zero mean error means no variation. A distribution can have positive and negative errors that average to zero while individual errors are nonzero.
Ignoring the conditioning value. The relevant error condition is $E[\varepsilon\mid X=x]=0$. An overall mean error of zero alone does not establish that errors average to zero at every predictor value.
Mixing up conditional and overall means. $E[Y\mid X=x]$ refers to responses at one specified predictor value; $E[Y]$ averages over the population distribution of predictor values as well.

Key takeaway: In the simple linear regression model, the line gives the conditional mean response when the error has conditional mean zero. An individual response equals that mean plus its own random error; the error may be positive or negative and is not generally zero.

Check Your Understanding

For each question, distinguish the conditional mean from an individual response and use the stated error condition where relevant.

A hypothetical model for inspection time in minutes is $Y=24+1.8x+\varepsilon$, with $E[\varepsilon\mid X=x]=0$. At score $x=10$, an inspection takes 47 minutes. Find the conditional mean and the error. (Answer: conditional mean $42$ minutes; error $5$ minutes.)
At a specified predictor value, an error is $-10$ with probability 0.20, $0$ with probability 0.50, and $10$ with probability 0.30. Calculate the conditional mean error and state whether this distribution satisfies the mean-zero condition. (Answer: conditional mean error $1$; no.)
A model is $E[Y\mid X=x]=40+4x$. In a hypothetical group, $X=1$ with probability 0.25 and $X=5$ with probability 0.75, and the conditional mean error is zero at both values. Find the overall expected response. (Answer: $56$.)
A model for annual service cost, in dollars, is $Y=70-6x+\varepsilon$, with zero conditional mean error. At score $x=3$, a customer’s cost is $45. Find the conditional mean and the customer’s error. (Answer: conditional mean $52; error $-$7$.)
At $x=7$, a model has conditional mean $15+2x$. The error is $-4$ or $4$, each with probability 0.50. Find the possible responses, verify their conditional mean, and calculate $P(Y>30\mid X=7)$. (Answer: responses $25$ and $33$; conditional mean $29$; probability $0.5000$.)

What You'll Learn