Why the Predictor’s Status Matters
A regression model describes how a response \(Y\) varies with an explanatory variable \(X\). But the same conditional-mean relationship can arise in two sampling settings. In one, the predictor values are selected in advance or treated as fixed for the analysis. In the other, the predictor is itself a random quantity observed along with the response. Distinguishing the settings clarifies what is being conditioned on and which averages describe the study.
For example, a researcher might assign participants to specified training durations, then record a performance score. Those durations are fixed by the design. In a different study, the researcher might sample people and record each person’s naturally occurring training duration and score. The durations are random across sampled people. In either case, the conditional mean of the score can be modeled as a linear function of duration.
The Conditional-Mean Model in Both Settings
The simple linear conditional-mean model uses an intercept \(\beta_0\) and a slope \(\beta_1\). At a predictor value \(x\), the model says that the mean response among cases with predictor value \(x\) is \(\beta_0+\beta_1x\). The conditional-mean statement, rather than the predictor’s fixed or random status, is the central feature.
For fixed design values, write the responses as \(Y_1,\ldots,Y_n\) and the fixed predictor values as \(x_1,\ldots,x_n\). The corresponding model is
Because the \(x_i\) values are fixed in this formulation, conditioning on the design values does not average over a distribution of predictors. Each expected response is associated with its particular fixed \(x_i\). In repeated studies with the same design, the predictor values remain the same while the responses may vary.
For random predictors, \(X\) has a distribution, and the model is stated at predictor values in its support: \(E[Y\mid X=x]=\beta_0+\beta_1x\). The response mean for the population is generally a different quantity: it averages the conditional means over the distribution of \(X\). Thus a conditional mean at one \(x\) should not be mistaken for the overall mean response.
Worked Example: Responses at Fixed Training Durations
Worked Example: Responses at Fixed Training Durations
A hypothetical study assigns six participants to training durations of 2, 4, or 6 hours per week. Two participants receive 2 hours, three receive 4 hours, and one receives 6 hours. Suppose the conditional-mean model is \(E[Y_i\mid x_1,\ldots,x_6]=10+1.5x_i\), where \(Y_i\) is a performance score. Find the expected score at each duration and the average expected score over the six participants. Treat the assigned durations as fixed. No probability distribution for the response is specified; no interest or mortality assumptions are relevant.
Method. Substitute each fixed predictor value into the conditional-mean equation. Then average across participants, preserving the number assigned to each duration.
At 2 hours, the conditional mean is \(10+1.5(2)=10+3=13\). At 4 hours, it is \(10+1.5(4)=10+6=16\). At 6 hours, it is \(10+1.5(6)=10+9=19\).
The six expected scores are therefore 13, 13, 16, 16, 16, and 19. Their average is
As a check, the average fixed predictor value is \((2(2)+3(4)+1(6))/6=22/6=11/3\). Applying the model at that average gives \(10+1.5(11/3)=10+5.5=15.5\), the same result. Common mistake avoided: averaging the three distinct group means equally would ignore that the groups have different sizes.
Random Predictors and the Overall Mean
When \(X\) is random, the conditional mean is defined for each possible predictor value, but the overall mean \(E[Y]\) averages across those values. Provided the relevant expectations exist, the law of total expectation gives
If the conditional-mean model is linear, substitute \(E[Y\mid X]=\beta_0+\beta_1X\) into this identity. Since expectation is linear, the result is \(E[Y]=\beta_0+\beta_1E[X]\). This is a statement about the overall mean, not a replacement for the conditional-mean model.
For a discrete predictor taking values \(x_j\) with probabilities \(p_j=P(X=x_j)\), the same calculation is a probability-weighted average: \(E[Y]=\sum_j p_j E[Y\mid X=x_j]\). The probabilities must sum to 1. For a continuous predictor, the corresponding average is taken over its density, when the required expectations exist.
Use the specified model to calculate \(E[Y\mid X=x]\) at each relevant value, or retain the expression as a function of \(X\).
For a discrete predictor, record each value and its probability; for a continuous predictor, use its density or a stated property such as its mean.
Calculate \(E[E[Y\mid X]]\), weighting each conditional mean by the probability of its predictor value or integrating over the predictor distribution.
Worked Example: A Discrete Random Predictor
Worked Example: A Discrete Random Predictor
In a hypothetical survey of work teams, let \(X\) be the number of weekly coaching sessions, with \(P(X=1)=0.20\), \(P(X=3)=0.50\), and \(P(X=6)=0.30\). Let \(Y\) be a team’s score, and suppose \(E[Y\mid X=x]=2+3x\). Find the conditional mean at each possible value and the overall mean score. The predictor has the stated discrete distribution; no further distribution for \(Y\) is given. No interest or mortality assumptions are relevant.
Method. Calculate each conditional mean, then average them using the probabilities of the corresponding predictor values.
For \(X=1\), \(E[Y\mid X=1]=2+3(1)=5\). For \(X=3\), \(E[Y\mid X=3]=2+3(3)=11\). For \(X=6\), \(E[Y\mid X=6]=2+3(6)=20\).
The predictor probabilities sum to \(0.20+0.50+0.30=1.00\), as required. The weighted average of the conditional means is
Check by first finding \(E[X]=1(0.20)+3(0.50)+6(0.30)=0.20+1.50+1.80=3.50\). Then \(2+3E[X]=2+3(3.50)=12.5\), which agrees. Common mistake avoided: the conditional means must be weighted by the predictor probabilities; they are not averaged equally.
Worked Example: Averaging Over a Continuous Predictor
Worked Example: Averaging Over a Continuous Predictor
In a hypothetical population of equipment operators, let \(X\) be weekly practice hours, uniformly distributed from 0 to 8. Suppose the conditional mean of a skill score is \(E[Y\mid X=x]=4+0.75x\). Find the conditional mean for an operator with 2 practice hours, the overall mean score, and the mean score among operators practicing more than 6 hours. The predictor distribution is \(X\sim\operatorname{Uniform}(0,8)\); no distribution for \(Y\) is assumed. No interest or mortality assumptions are relevant.
Method. Substitute the specified \(x\) for the first conditional mean. For the overall and subgroup means, average the conditional mean over the appropriate distribution of \(X\).
For an operator with 2 hours, \(E[Y\mid X=2]=4+0.75(2)=4+1.5=5.5\).
A uniform distribution on the interval from 0 to 8 has mean \(E[X]=(0+8)/2=4\). Therefore,
For operators with \(X>6\), the conditional distribution of \(X\) is uniform from 6 to 8, whose mean is \((6+8)/2=7\). Thus
As a check, the specified subgroup comprises the upper quarter of the predictor range, since \((8-6)/(8-0)=2/8=0.25\). Its mean predictor value of 7 lies inside that interval, and the resulting mean score 9.25 exceeds the overall mean 7, consistent with the positive slope. Common mistake avoided: using \(E[Y\mid X=6]\) as the mean for everyone with \(X>6\) would treat a range of predictor values as though all were exactly 6.
What the Formulation Does and Does Not Say
A fixed-predictor analysis is natural when predictor levels are selected in advance, as in a designed experiment. Its conditional-mean statements compare the expected responses at those chosen levels. A random-predictor analysis is natural when the predictor values vary across sampled units. It makes the predictor distribution part of the description of the population and allows an overall response mean to be obtained by averaging conditional means.
These formulations can use the same line \(\beta_0+\beta_1x\). The slope describes the change in conditional mean associated with a one-unit increase in \(x\), when the model applies over the values being compared. Whether \(x\) was fixed by design or observed as random does not alter that conditional interpretation. However, the overall distribution of observed \(X\) differs by setting, and so can the overall average response.
The conditional-mean equation alone does not specify how scattered responses are around their means. It does not supply a variance, a probability distribution for \(Y\), or a claim that the model is appropriate in every setting. Those are separate modeling features. In particular, do not infer a response distribution merely from knowing the predictor’s distribution and the conditional mean.
Common Mistakes and Exam Traps
- Calling an observed predictor fixed by nature. A predictor may be random in the population but treated as fixed after conditioning on the observed design values. State the sampling formulation being used.
- Confusing a conditional mean with the overall mean. \(E[Y\mid X=x]\) refers to cases at a particular predictor value; \(E[Y]\) averages over the predictor distribution.
- Forgetting probabilities in a discrete average. Use \(\sum_j P(X=x_j)E[Y\mid X=x_j]\), not an unweighted average unless all predictor values are equally likely.
- Using a group’s boundary value as its mean. For an event such as \(X>c\), average over the conditional distribution within that event rather than substituting \(c\) into the line.
- Claiming more than the mean model gives. A conditional mean does not determine the full conditional distribution or the variability of responses.
Check Your Understanding
For each problem, distinguish the conditional-mean calculation from any averaging over predictor values.
- A study fixes four participants’ predictor values at 0, 2, 5, and 5. The conditional-mean model is \(E[Y_i\mid x_1,\ldots,x_4]=7+2x_i\). Find each expected response and their average. (Answer: 7, 11, 17, and 17; average \(=13\).)
- A random predictor \(X\) takes values 0, 2, and 5 with probabilities 0.25, 0.50, and 0.25. If \(E[Y\mid X=x]=7+2x\), find the three conditional means and \(E[Y]\). (Answer: 7, 11, and 17; \(E[Y]=11.5\).)
- Let \(X\sim\operatorname{Uniform}(0,10)\), and suppose \(E[Y\mid X=x]=3+0.4x\). Find \(E[Y]\) and \(E[Y\mid X>8]\). (Answer: \(E[Y]=5\); \(E[Y\mid X>8]=6.6\).)
- Four fixed design values are \(1,1,1,7\), and the model is \(E[Y_i\mid x_1,\ldots,x_4]=5+2x_i\). Find the average expected response. Explain why averaging only the distinct conditional means equally gives a different result. (Answer: average expected response \(=10\); the distinct means are 7 and 19, whose unweighted average is 13 and ignores their frequencies.)
- A random predictor takes value 0 with probability 0.80 and value 1 with probability 0.20. If \(E[Y\mid X=x]=12-3x\), find \(E[Y\mid X=1]\) and \(E[Y]\). (Answer: \(E[Y\mid X=1]=9\); \(E[Y]=11.4\).)