Conditioning on Observed Predictor Values

Why Condition on the Observed Predictor Values?

Simple regression studies how a response changes with a predictor. In applications, the predictor values may be deliberately chosen, such as specified exposure levels, or they may arise from sampling, such as observed ages or account sizes. These are different descriptions of how the data were collected. In the standard regression analysis, however, inference about the regression coefficients is usually made conditional on the predictor values that were observed.

Conditioning does not mean that the predictor must have been fixed in advance. It means that, for the coefficient calculations, we regard the observed predictor values as given and describe the randomness in the responses through the regression errors. This distinction helps clarify what the standard errors and confidence intervals do—and do not—represent.

The Standard Conditional Model

Write the simple linear regression model as \(Y_i=\beta_0+\beta_1x_i+\varepsilon_i\), where \(\beta_0\) and \(\beta_1\) are fixed but unknown parameters. For the standard conditional model, the predictor values are treated as given, and the errors satisfy \(E[\varepsilon_i\mid x_1,\ldots,x_n]=0\), \(\operatorname{Var}(\varepsilon_i\mid x_1,\ldots,x_n)=\sigma^2\), and are uncorrelated across observations. Thus, conditional on the observed predictors, the responses have means on the regression line and equal error variances.

For the usual exact small-sample \(t\)- and \(F\)-inference, the additional assumption is that the errors are conditionally independent and normally distributed with mean zero and variance \(\sigma^2\). If normality is not assumed, the least-squares estimates can still have useful properties under the mean, variance, and independence conditions, but the exact small-sample \(t\) distributions do not follow from those conditions alone.

Let \(\bar{x}\) be the mean of the observed predictor values and let \(S_{xx}=\sum_{i=1}^n(x_i-\bar{x})^2\). If \(S_{xx}>0\), the ordinary least-squares slope estimate is \(\hat{\beta}_1=\sum_{i=1}^n(x_i-\bar{x})Y_i/S_{xx}\). Under the conditional model, its mean is \(\beta_1\) and its variance is \(\sigma^2/S_{xx}\). The observed predictor spread therefore matters directly: greater \(S_{xx}\) gives a smaller conditional variance for the slope estimate, all else equal.

The notation emphasizes the conditioning. It describes the distribution of the estimate over repeated response samples with these same predictor values. It does not say that the predictor values have no randomness in the process that generated the data.

Why the Conditional Variance Formula Holds

The slope estimate can be rewritten by substituting the regression model into its formula. Since \(\sum_i(x_i-\bar{x})=0\), the intercept term contributes nothing:

Conditional on the predictors, the errors have mean zero, so the second term has conditional expectation zero. If the errors are conditionally uncorrelated and each has variance \(\sigma^2\), its conditional variance is

Conditional normality of the errors makes the weighted sum of errors normal, which gives the stated normal distribution for \(\hat{\beta}_1\). This argument also shows why the formula requires \(S_{xx}>0\): if every observed predictor value is the same, the slope cannot be estimated from those data.

When the Predictor Is Random

In many observational studies, predictor values are sampled rather than fixed by design. Suppose the observations are generated by \(Y_i=\beta_0+\beta_1X_i+\varepsilon_i\), where the \(X_i\) are random. If the conditional error assumptions hold given the observed predictors, then after observing \(X_1,\ldots,X_n\), the same conditional regression formulas apply. The random-predictor description concerns how the \(X_i\) arose; it does not by itself change the conditional response model.

For example, if \(E[\varepsilon_i\mid X_i]=0\), then the conditional mean response is \(E[Y_i\mid X_i]=\beta_0+\beta_1X_i\). If the errors are also independent of the predictors and have constant variance, conditioning on the observed predictor values leaves the familiar error variance \(\sigma^2\). If predictor and error behavior is dependent, or if the conditional error variance changes with the predictors, the standard conditional formulas may not apply as stated.

The conditional and unconditional statements should not be confused. If the random design satisfies the standard conditional assumptions and \(S_{XX}>0\), then \(E[\hat{\beta}_1\mid X_1,\ldots,X_n]=\beta_1\). Averaging this conditional expectation over the random predictors also gives \(E[\hat{\beta}_1]=\beta_1\), provided the expectation exists. But the unconditional variance includes the variation in conditional precision across possible predictor samples:

This expression is generally not equal to \(\sigma^2/E[S_{XX}]\). The reciprocal of a random quantity cannot generally be replaced by the reciprocal of its mean. Standard regression output usually reports precision conditional on the predictor values in the data, rather than averaging over all possible predictor samples.

Estimating the Error Variance

When \(\sigma^2\) is unknown, it is estimated from the residual sum of squares. Under the simple regression model with an intercept and slope, \(s^2=\mathrm{RSS}/(n-2)\), where \(\mathrm{RSS}=\sum_i(Y_i-\hat{Y}_i)^2\). The two fitted coefficients use two degrees of freedom. Under independent normal conditional errors, the slope statistic \((\hat{\beta}_1-\beta_1)/(s/\sqrt{S_{xx}})\) has a \(t\) distribution with \(n-2\) degrees of freedom, conditional on the observed predictors. The exact \(t\) result requires the normal-error assumption; it is not a consequence of merely having random or fixed predictors.

Common Interpretation Traps

• Assuming random predictors invalidate regression inference. They do not, provided the conditional error assumptions hold. The standard calculations condition on the observed predictor values.
• Treating the observed predictors as the source of response error. The standard conditional model assigns the response randomness to the errors. The predictor spread controls coefficient precision through quantities such as \(S_{xx}\); it is not added to \(\sigma^2\).
• Using the conditional formula without its assumptions. If errors have nonzero conditional means, unequal conditional variances, or dependence that is not accounted for, the displayed standard errors and exact \(t\) inference need not be valid.
• Confusing a conditional statement with an unconditional one. A confidence interval based on the observed \(S_{xx}\) describes uncertainty conditional on those predictors. It does not average over the possible predictor samples that might have been collected.