From a Conditional Mean to an Observed Response
A regression model for conditional means describes an average response for cases with a given predictor value. It does not say that every individual response equals that average. Individual outcomes can lie above or below the conditional mean, and the difference is represented by an error term.
This distinction is important when interpreting regression results. A population mean function describes the centre of the response distribution at each predictor value; an observed response is one particular outcome from that distribution. Confusing the two can lead to claims about individuals that the model does not support.
In the simple linear regression model for conditional means, the conditional mean function has the form \(m(x)=\beta_0+\beta_1x\). The decomposition is then \(Y_i=\beta_0+\beta_1x_i+\varepsilon_i\). Here, \(\beta_0+\beta_1x_i\) is the population conditional mean at \(x_i\), while \(\varepsilon_i\) represents the individual deviation from that mean.
The error is not an additional fixed amount that must always be positive or always be negative. It varies across possible outcomes. Its distribution, spread, and dependence across observations are separate features of a model; none is determined merely by specifying a linear conditional mean. Unless a problem supplies additional assumptions, do not infer a normal distribution, a particular variance, or independence from the mean function alone.
Why the Decomposition Works
The error is defined as the response minus its conditional mean. Rearranging that definition gives the response decomposition. Taking the conditional expectation also shows why the conditional mean remains the centre of the response distribution.
The last equality follows because \(m(x_i)\) is fixed once we condition on \(X_i=x_i\), and the conditional mean of the deviation from that conditional mean is zero. If a model instead writes \(Y_i=m(x_i)+u_i\) for some other error term \(u_i\), then \(m(x_i)\) is the conditional mean only when \(E[u_i\mid X_i=x_i]=0\). Otherwise the conditional mean is \(m(x_i)+E[u_i\mid X_i=x_i]\).
Substitute the stated predictor value into \(m(x)\). This gives the population average response at that predictor value.
Subtract the conditional mean from the observed response: \(\varepsilon_i=y_i-m(x_i)\).
Check that \(y_i=m(x_i)+\varepsilon_i\). A positive error means the response is above the conditional mean; a negative error means it is below.
Worked Examples
Worked Example: A Claim-Review Time Above Its Conditional Mean
In a hypothetical claims operation, let \(X\) be a file-complexity score and let \(Y\) be review time in minutes. The population conditional mean is \(m(x)=18+2.5x\). At a complexity score of \(4\), the possible errors are \(-6,-2,2,6\) minutes, each with probability \(1/4\). A particular file takes 30 minutes to review. Find its error, the possible response values, and the probability that a file at this score takes more than 30 minutes. Use the stated model and no additional distributional assumptions.
Method. First evaluate the conditional mean at the file's score:
The observed response is 30 minutes, so its error is the observed response minus the conditional mean:
For the four possible errors, add each error to 28. The possible response times are \(28-6=22\), \(28-2=26\), \(28+2=30\), and \(28+6=34\) minutes. Only 34 is greater than 30, so the requested probability is \(1/4=0.2500\).
As a check, the equally likely responses average \((22+26+30+34)/4=112/4=28\) minutes, which is the stated conditional mean. The common mistake is to call 30 minutes the conditional mean because it is the observed time. It is instead one outcome, 2 minutes above the mean.
Worked Example: A Mean and Error Distribution for Administrative Cost
Suppose, hypothetically, that \(Y\) is an insurer's annual administrative cost in thousands of dollars and \(X\) is the number of policies administered. The population conditional mean is \(m(x)=12+0.004x\), in thousands of dollars. At \(x=2{,}500\), the error has this distribution: \(-3\) with probability \(0.2\), \(-1\) with probability \(0.3\), \(1\) with probability \(0.3\), and \(3\) with probability \(0.2\), with errors measured in thousands of dollars. Find the conditional mean, check that the errors are centred at zero, list the possible response amounts, and calculate the probability that the cost is at least 25 thousand dollars.
Method. Substitute the policy count into the conditional mean function:
Thus, the conditional mean cost is 22 thousand dollars. Check the error mean using the given probabilities:
Add each possible error to the mean of 22. The possible costs are 19, 21, 23, and 25 thousand dollars, with probabilities \(0.2,0.3,0.3,0.2\), respectively. Only the response of 25 thousand dollars meets the condition “at least 25,” so the probability is \(0.2000\).
The response expectation is also 22 thousand dollars:
The calculation confirms the decomposition. A common mistake is to use the probability of a cost strictly greater than 25; that event has probability zero. “At least 25” includes 25, so its probability is 0.2.
Worked Example: Two Outcomes at the Same Predictor Value
A hypothetical risk-scoring model has response \(Y\), measured in thousands of dollars, and population conditional mean \(m(x)=40+6x\). At \(x=3\), an individual error has probability \(0.25\) of being \(-8\), probability \(0.50\) of being \(0\), and probability \(0.25\) of being \(8\). Two cases with this score have observed responses of 50 and 66 thousand dollars. Decompose each response and find the conditional mean and the probability of an outcome at least as large as 66.
Method. The population conditional mean at score 3 is:
For the first case, the error is \(50-58=-8\) thousand dollars. For the second case, the error is \(66-58=8\) thousand dollars. Thus the two decompositions are \(50=58+(-8)\) and \(66=58+8\).
The stated error distribution gives possible responses \(58-8=50\), \(58+0=58\), and \(58+8=66\), with probabilities \(0.25,0.50,0.25\). Therefore, the probability of a response at least as large as 66 thousand dollars is \(0.25=0.2500\). The error mean is \((-8)(0.25)+(0)(0.50)+(8)(0.25)=0\), providing a check that the conditional mean is 58.
The first and second cases differ by 16 thousand dollars even though they have the same predictor value and share the same conditional mean. The common mistake is to treat the observed response of 66 as the mean for all cases with score 3. It is one possible response, 8 thousand dollars above the population conditional mean.
Population Mean, Individual Outcome, and Model Interpretation
A conditional mean is a property of a population model: it describes the average response among cases at a specified predictor value. An individual response is a realization from the conditional distribution at that value. Even if the conditional mean function is exactly linear, individual responses need not lie on the line.
For the linear mean function \(m(x)=\beta_0+\beta_1x\), a response at \(x_i\) consists of the mean \(\beta_0+\beta_1x_i\) plus its deviation \(\varepsilon_i\). If the response is above the line, the realized error is positive; if below, it is negative. Neither sign alone establishes that the model's mean function is wrong. The deviation may simply be an individual outcome permitted by the response distribution.
This decomposition also clarifies the difference between a population quantity and an estimate. In an application, the parameters of a population mean function are generally unknown, and a line calculated from data is an estimate of that function. The estimated value at a predictor setting is not automatically the observed response for a particular individual. A particular response can differ from the estimated mean because it includes an individual deviation, and the estimated line itself can differ from the population mean function because it is based on data.
Keep the distinction precise: the population conditional mean is \(E[Y\mid X=x]\); the error is the response's deviation from that conditional mean; and the observed response is their sum. A mean model alone does not specify how large individual deviations tend to be or how likely any particular response is. Those questions require additional information about the error distribution.
Common Mistakes and Exam Traps
- Calling an observed response the conditional mean. If \(m(x)=28\) and the observed response is 30, then the error is \(30-28=2\), not zero. The observation is not the population average.
- Reversing the subtraction. The error is observed response minus conditional mean, \(\varepsilon_i=y_i-m(x_i)\). Reversing the order changes the sign and reverses the interpretation.
- Assuming a mean model determines individual outcomes. A conditional mean gives an average, not a guarantee that every response equals that average. Additional distributional information is needed for probabilities or prediction ranges.
- Using an uncentred error as though it were the deviation from the conditional mean. If \(Y=m(x)+u\), then \(E[Y\mid X=x]=m(x)+E[u\mid X=x]\). The function \(m(x)\) is the conditional mean only if that error has conditional mean zero.
- Ignoring units. If the response and error are in thousands of dollars, their sum is also in thousands of dollars. Convert to dollars only after completing the calculation, if requested.
Check Your Understanding
For each question, distinguish the conditional mean from the observed response and use only the assumptions stated.
- A hypothetical service model has \(m(x)=7+1.5x\). At \(x=6\), an observed service time is 14 hours. Find the conditional mean and the observed error. (Answer: conditional mean \(16\) hours; error \(-2\) hours.)
- At \(x=10\), a response has conditional mean 30. Its error is \(-4,0,\) or \(4\), with probabilities \(0.25,0.50,\) and \(0.25\), respectively. Find the possible responses and \(P(Y>30\mid X=10)\). (Answer: responses \(26,30,34\); probability \(0.2500\).)
- A model has \(m(x)=5+0.8x\). At \(x=15\), an individual response is 20. Find the conditional mean and the error, with both in the response's units. (Answer: conditional mean \(17\); error \(3\).)
- At \(x=2\), a response has mean function \(m(x)=8+3x\). The error is \(-2,0,\) or \(4\), with probabilities \(0.50,0.25,\) and \(0.25\). Find the response expectation and \(P(Y\geq18\mid X=2)\). (Answer: expectation \(14\); probability \(0.2500\).)
- A model is written \(Y=20+2X+u\), and at \(X=4\), \(E[u\mid X=4]=1.2\). What is the actual conditional mean at \(X=4\), and is \(20+2(4)\) that conditional mean? (Answer: conditional mean \(29.2\); no, \(28\) is the stated function's value, while the nonzero conditional error mean raises the actual conditional mean by \(1.2\).)