Regression As a Model for Conditional Means

Regression Describes a Mean Response

A response variable can vary substantially among observations that share the same predictor value. A regression model can describe a systematic feature of that variation without claiming to predict every observation exactly. In simple linear regression, that feature can be the conditional mean response: the average value of the response among cases with a specified predictor value.

This distinction matters in actuarial work. A model might describe the average hypothetical warranty cost associated with a level of product use. It does not thereby specify the probability of every possible cost, nor does an association between use and cost prove that changing use would cause a particular change in cost.

Definition: Let $X$ be a predictor and $Y$ a response, with a finite conditional expectation for the predictor values under consideration. A simple linear regression model for the conditional mean assumes that $$ E[Y\mid X=x]=\beta_0+\beta_1x $$ for each $x$ in the model's stated range. Here, $E[Y\mid X=x]$ is the mean response among cases for which $X=x$, and $\beta_0,\beta_1$ are fixed model parameters. This equation specifies the conditional mean; on its own, it does not specify the full conditional distribution of $Y$.

The model is a statement about averages at predictor values, not a claim that every response lies on a straight line. For a particular case with $X=x$, the actual $Y$ may be above or below the conditional mean. Also, the equation is meant to apply over a relevant range of $x$. Using it far outside that range is extrapolation, and the linear relationship may not remain plausible there.

Conditional Mean Is Not the Full Distribution

A conditional distribution gives all the probabilities or relative frequencies of possible response values at a specified predictor value. The conditional mean is just one summary of that distribution. In a discrete setting, if the possible responses at $X=x$ are $y_1,\ldots,y_k$, with conditional probabilities $p_1(x),\ldots,p_k(x)$, then

E[Y\mid X=x]=\sum_{j=1}^{k} y_jp_j(x), \qquad \sum_{j=1}^{k}p_j(x)=1.

The probabilities must sum to one, and each probability is nonnegative. The mean does not determine those probabilities. For example, a response that is always 10 has mean 10, as does a response that equals 0 or 20 with probability one-half each. Their variability and probabilities of particular outcomes are different.

Consequently, a linear conditional mean does not by itself assert that responses are normally distributed, that their variability is constant across predictor values, or that a specified fraction of responses will fall close to the line. Such claims require additional assumptions. A mean model is useful when the question concerns average response; questions about probabilities, quantiles, or variability require more information.

Worked Example: A Mean Model Does Not Determine Warranty Costs

Method: Calculate conditional means from the stated discrete distributions, then compare what those means reveal with what they leave unspecified. A hypothetical warranty portfolio uses product-use score $X$ and monthly warranty cost $Y$, in dollars. Assume finite conditional means and the model $E[Y\mid X=x]=20+30x$ over scores from 2 through 5.

At score $x=2$, suppose the conditional distribution assigns probabilities $0.25, 0.50,$ and $0.25$ to costs of $\$0,\$80,$ and $\$160$, respectively. Its conditional mean is

E[Y\mid X=2] =0(0.25)+80(0.50)+160(0.25) =0+40+40 =\$80.

The model gives $20+30(2)=\$80$, so this distribution agrees with the stated mean model at score 2.

At score $x=5$, suppose the probabilities are again $0.25,0.50,$ and $0.25$, now for costs of $\$0,\$170,$ and $\$340$. Then

E[Y\mid X=5] =0(0.25)+170(0.50)+340(0.25) =0+85+85 =\$170,

which agrees with $20+30(5)=\$170$. At score $x=4$, the model predicts the conditional mean $20+30(4)=\$140$. It does not tell us which costs can occur at score 4. For instance, either a cost of $\$140$ with probability 1 or costs of $\$100$ and $\$180$ with probability $0.50$ each has mean $\$140$.

Check: The two score-4 distributions each have mean $(100+180)(0.50)=\$140$ or $140(1)=\$140$, respectively, but differ in their possible outcomes. The common mistake is to treat a predicted mean as the cost every policyholder will incur. The model predicts an average, not an individual cost or a full distribution.

Using the Model for Average Responses

Once the conditional mean model has been stated, its basic use is direct: substitute a predictor value within the stated range and evaluate the right-hand side. The result is a model-based average response for cases at that predictor value. It is not automatically an observed response, and it is not a guarantee that a future group will have exactly that average.

For example, if a hypothetical service model states that the average repair time is $12+4x$ hours for complexity scores from 2 to 4, the model's mean at score 3 is $12+4(3)=24$ hours. It says nothing on its own about the distribution of repair times around 24 hours. The coefficients describe the specified line; their separate interpretations are not needed to calculate a conditional mean.

Worked Example: Computing a Conditional Mean From Response Probabilities

Method: Use the definition of conditional expectation and verify that the resulting means agree with the given line. In a hypothetical equipment service study, $X$ is a complexity score and $Y$ is service duration in hours. Assume scores from 2 through 4 are in range, all stated conditional means are finite, and the proposed model is $E[Y\mid X=x]=12+4x$.

At score $x=2$, service durations of 10, 20, and 30 hours have conditional probabilities $0.20, 0.60,$ and $0.20$. Therefore,

E[Y\mid X=2] =10(0.20)+20(0.60)+30(0.20) =2+12+6 =20\text{ hours}.

The line gives $12+4(2)=20$ hours, matching the calculation.

At score $x=4$, durations of 18, 28, and 38 hours have the same respective probabilities. Thus,

E[Y\mid X=4] =18(0.20)+28(0.60)+38(0.20) =3.6+16.8+7.6 =28\text{ hours}.

The line gives $12+4(4)=28$ hours. At score 3, the model's conditional mean is $12+4(3)=24$ hours. This calculation concerns the average at score 3; no response distribution at score 3 has been supplied, so probabilities of particular durations cannot be calculated.

Check: The probabilities at each supplied score sum to $0.20+0.60+0.20=1.00$, and both calculated means match the line. The common mistake is to report 24 hours as the duration for each score-3 case. It is the model's conditional mean, not a statement about any one case.

Why the Mean Equation Holds

The conditional mean comes from averaging responses within a predictor value, weighting each possible response by its conditional probability. Suppose that at every $x$ in the relevant range, the conditional probabilities are $p_j(x)$ and the responses are $y_j$. If the model asserts that the weighted average is $\beta_0+\beta_1x$, then the equation follows directly from the definition of conditional expectation:

List possible responses at $x$.
Write each response value $y_j$ and its conditional probability $p_j(x)$. The probabilities are nonnegative and sum to one.

Form the weighted average.
By definition, $E[Y\mid X=x]=\sum_j y_jp_j(x)$.

Apply the model assertion.
If this weighted average equals $\beta_0+\beta_1x$ for every in-range $x$, the conditional mean is linear in $x$.

This justification also shows the limit of the conclusion. Knowing the weighted average does not reveal each response value or probability. Many different sets of conditional probabilities can produce the same average.

Association Is Not Necessarily Causation

A regression relationship describes how conditional means vary across predictor values in the population or data being modeled. It does not, by itself, establish what would happen if someone intervened to change the predictor. Other variables may be associated with both the predictor and the response, making groups with different predictor values differ for reasons besides the predictor itself.

The distinction can be made precise with conditional averages. If $Z$ is an additional characteristic, then the mean at a given $X=x$ averages over the values of $Z$ present among cases with that $x$. If that mix differs across $x$, the overall conditional means can differ even when comparisons at a fixed value of $Z$ show a different pattern. A causal conclusion needs suitable design or assumptions beyond a linear conditional mean.

Worked Example: A Mean Association Does Not Identify a Training Effect

Method: Calculate the marginal conditional means by weighting stratum-specific means by the stratum proportions, then distinguish that association from a causal conclusion. A hypothetical claims office records $X=0$ for an employee without a new training course and $X=1$ for an employee with the course. Let $Y$ be the expected number of weekly processing errors and let $Z$ indicate case complexity. Assume the following stratum means and group proportions.

Training status	Low-complexity proportion and mean	High-complexity proportion and mean
$X=0$	0.90; mean 2 errors	0.10; mean 10 errors
$X=1$	0.10; mean 1 error	0.90; mean 9 errors

For employees without training, the weighted conditional mean is

E[Y\mid X=0] =0.90(2)+0.10(10) =1.8+1 =2.8\text{ errors}.

For employees with training, it is

E[Y\mid X=1] =0.10(1)+0.90(9) =0.1+8.1 =8.2\text{ errors}.

Thus the observed marginal conditional means are 2.8 and 8.2. For a binary predictor, the line through these two conditional means is $2.8+5.4x$: at $x=0$ it gives 2.8, and at $x=1$ it gives $2.8+5.4=8.2$. Yet within each complexity group, the listed mean is one error lower for trained employees: 1 versus 2 in the low-complexity group and 9 versus 10 in the high-complexity group.

Check: The marginal means are correctly weighted, and the training groups have very different complexity mixes. The higher marginal mean among trained employees does not establish that training causes more errors; the comparison is confounded by complexity mix. The stratum means alone also do not prove a causal training effect, since other differences or the way employees entered the groups could matter.

Common Interpretive Traps

Treating the mean as an individual prediction. $E[Y\mid X=x]$ is an average for the specified predictor value, not a promise about one response.
Assuming the line gives a full probability model. A conditional mean does not specify response probabilities, variability, or quantiles.
Assuming normality or equal variability from linearity. Neither follows from a linear conditional mean alone; those are additional modeling claims.
Extending the line without checking its range. A linear mean relationship stated for one range need not hold far beyond it.
Reading a causal effect from an association. Differences between predictor groups can reflect other characteristics or selection into those groups.

Key takeaway: Simple linear regression as a conditional-mean model asserts that the average response at predictor value $x$ is $\beta_0+\beta_1x$, over a specified range. This statement concerns the mean alone. Distributional and causal conclusions require additional information and assumptions.

Check Your Understanding

For each problem, distinguish the model's mean prediction from any stronger conclusion that the information does not support.

A hypothetical equipment model states $E[Y\mid X=x]=6+3x$ dollars for usage scores from 1 to 8. Calculate the conditional mean at score 5. (Answer: $\$21$)
At a specified predictor value, $Y$ is 0, 40, or 100 with probabilities 0.20, 0.50, or 0.30. Calculate its conditional mean and state whether that determines the probability of $Y=40$ from the mean alone. (Answer: mean $=\$50$; the mean alone does not determine that probability.)
A linear conditional-mean model applies for $2\leq x\leq 6$, and gives $E[Y\mid X=x]=15+2x$. Calculate the mean at $x=4$, and say whether the model supports using its formula at $x=9$. (Answer: 23 response units; no, $x=9$ is outside the stated range.)
Among low-complexity cases, group A has mean 3 errors and group B has mean 2; among high-complexity cases, the means are 11 and 10. Group A is 80% low-complexity and group B is 20% low-complexity. Calculate each group's marginal mean and identify the conclusion warranted by these means. (Answer: A has $0.80(3)+0.20(11)=4.6$; B has $0.20(2)+0.80(10)=8.4$. The marginal association does not by itself establish a causal group effect.)
At $X=2$, a response is always 12. At $X=4$, it is 4 or 20 with equal probabilities. Compare the conditional means and explain whether the means alone imply equal response distributions. (Answer: both means are 12; no, the conditional distributions differ.)

Training status	Low-complexity proportion and mean	High-complexity proportion and mean
\(X=0\)	0.90; mean 2 errors	0.10; mean 10 errors
\(X=1\)	0.10; mean 1 error	0.90; mean 9 errors

What You'll Learn