The Sampling Distribution of a Regression Slope

Why a Regression Slope Would Vary Across Samples

A least-squares slope is calculated from observed data, so a different sample will generally produce a different estimate. Imagine repeating the same study many times, using the same predictor values but obtaining new outcomes each time. The resulting slopes would form a distribution. This distribution describes the uncertainty in the slope estimate across repeated samples; it is not the distribution of individual outcomes or predictor values.

The center and spread of this distribution help explain what a reported slope can tell us. Under suitable conditions, the estimates are centered on the population slope. Their spread depends in part on the study design: a design that gives more information about how the response changes with the predictor can produce less variable slope estimates.

Definition: The sampling distribution of a regression slope is the probability distribution of the least-squares slope estimates that would result from repeatedly taking samples and calculating a slope each time, under a specified model and sampling or study design. The population slope, denoted \(\beta_1\), is the slope in the population regression model. A particular sample produces an estimate, denoted \(b_1\).

The Model and What Repeated Sampling Means

For simple linear regression, write the response for observation \(i\) as

Y_i=\beta_0+\beta_1 x_i+\varepsilon_i.

Here \(x_i\) is the predictor value, \(\beta_0\) is the population intercept, \(\beta_1\) is the population slope, and \(\varepsilon_i\) is the random error. In a repeated-sample description with a fixed design, the \(x_i\) values are kept the same and new errors, and therefore new responses, are generated for each repetition. The resulting value of \(b_1\) can change from repetition to repetition.

For the slope estimates to be centered on \(\beta_1\), the model needs a linear conditional mean and errors with mean zero at each fixed predictor value: \(E(\varepsilon_i\mid x_i)=0\). The errors must also be independent for the usual repeated-sampling model. These conditions do not require normal errors for the centering result. If, in addition, the errors are independent normal random variables with common variance \(\sigma^2\), then the slope estimate has an exact normal sampling distribution for a fixed design. Without normal errors, the sampling distribution need not be exactly normal.

These conditions are about the model and the way the data arise. A curved mean relationship, dependent errors, or a sampling process that systematically selects unusual outcomes can undermine the interpretation of repeated estimates. In an observational setting, the model describes an association; it does not by itself establish that changing the predictor would cause a change in the response.

Why the Estimates Are Centered on the Population Slope

A short derivation shows why zero-mean errors lead to slopes centered on \(\beta_1\). For fixed predictor values, define \(S_{xx}=\sum_{i=1}^{n}(x_i-\bar{x})^2\), which is positive when the predictor values are not all identical. The least-squares slope can be written as

b_1=\frac{\sum_{i=1}^{n}(x_i-\bar{x})Y_i}{S_{xx}}.

Substitute the regression model into this expression. The intercept contribution vanishes because the centered predictor values sum to zero:

\begin{aligned} b_1 &=\frac{\sum_{i=1}^{n}(x_i-\bar{x})(\beta_0+\beta_1x_i+\varepsilon_i)}{S_{xx}}\\ &=\beta_1+\frac{\sum_{i=1}^{n}(x_i-\bar{x})\varepsilon_i}{S_{xx}}. \end{aligned}

Now take the expected value over repeated errors, conditional on the fixed \(x_i\) values. Each error has conditional mean zero, so

E(b_1\mid x_1,\ldots,x_n) =\beta_1+\frac{\sum_{i=1}^{n}(x_i-\bar{x})E(\varepsilon_i\mid x_i)}{S_{xx}} =\beta_1.

Thus the slope estimate is unbiased under the stated zero-mean condition: across repetitions, its average is the population slope. This does not mean any one estimate must equal \(\beta_1\), or that a small simulation's average must equal it exactly. Random variation remains.

Worked Example: Calculating One Estimate in a Repeated-Sample Setup

Suppose a hypothetical study uses predictor values \(x=0,1,2,3\). The population model has intercept \(10\) and slope \(2\). One possible sample has responses \(10,12,14,18\). Calculate its least-squares slope and relate it to the population slope.

The predictor mean is \(\bar{x}=1.5\), and the response mean is \(\bar{y}=13.5\). The slope is

\begin{aligned} b_1 &=\frac{\sum (x_i-\bar{x})(y_i-\bar{y})}{\sum (x_i-\bar{x})^2}\\ &=\frac{(-1.5)(-3.5)+(-0.5)(-1.5)+(0.5)(0.5)+(1.5)(4.5)} {(-1.5)^2+(-0.5)^2+(0.5)^2+(1.5)^2}\\ &=\frac{5.25+0.75+0.25+6.75}{2.25+0.25+0.25+2.25} =\frac{13}{5}=2.6. \end{aligned}

This sample slope is \(2.6\) response units per predictor unit, while the population slope is \(2\) response units per predictor unit. The difference does not contradict the model: a particular sample includes random errors. Repeating the study with new errors would generally give another slope.

A Small Repeated-Sample Simulation

The table below illustrates six hypothetical repetitions under a model with population slope \(\beta_1=2\), using the same predictor values \(0,1,2,3\) each time. Suppose the errors are generated independently with mean zero. Each entry is one slope estimate from a repetition. The listed outcomes are simulation results for illustration, not real study data.

Repetition	1	2	3	4	5	6
Estimated slope \(b_1\)	2.0	2.6	1.4	2.2	1.6	1.8

Think of each estimate as a dot placed on a horizontal slope axis. Two dots fall at \(1.4\) and \(1.6\), two at \(1.8\) and \(2.0\), and two at \(2.2\) and \(2.6\). The dots vary around the population value \(2\); they do not all land exactly on it. The mean of these six estimates is

\bar{b}_1=\frac{2.0+2.6+1.4+2.2+1.6+1.8}{6} =\frac{11.6}{6}\approx1.933.

This small set of repetitions averages to about \(1.933\), not exactly \(2\). With only six repetitions, that difference is unsurprising. The centering result is a statement about the expected estimate over the model's full repeated-sampling distribution, not a promise that every finite simulation will have exactly the expected mean.

Worked Example: Reading a Simulated Sampling Distribution

In one of the repetitions above, suppose the errors are \(0,0,0,2\). With the model \(Y_i=10+2x_i+\varepsilon_i\), the responses are \(10,12,14,18\), giving the slope \(2.6\) as calculated earlier. For another repetition, suppose the errors are \(0,0,0,-2\), giving responses \(10,12,14,14\).

For the second repetition, \(\bar{y}=12.5\), while \(\bar{x}=1.5\) and \(S_{xx}=5\). Thus

\begin{aligned} b_1 &=\frac{(-1.5)(-2.5)+(-0.5)(-0.5)+(0.5)(1.5)+(1.5)(1.5)}{5}\\ &=\frac{3.75+0.25+0.75+2.25}{5} =\frac{7}{5}=1.4. \end{aligned}

The first repetition gave a slope \(0.6\) above \(2\); the second gave a slope \(0.6\) below \(2\). This pair illustrates how different errors can move the estimate to opposite sides of the population slope. It does not imply that repeated estimates will occur in balanced pairs.

How Study Design Changes the Spread

The spread of the sampling distribution is not determined by the population slope alone. It also depends on how the study is designed and how much random error the outcomes contain. In particular, predictor values that cover a wider range generally give more information about the slope: the fitted line must account for response changes across a broader set of \(x\) values. If the predictor values are tightly clustered, random changes in the responses can have more influence on the estimated tilt of the line.

The following paired illustration isolates the effect of predictor range. Use the same six error patterns as above, but compare predictor values \(0,1,2,3\) with \(0,2,4,6\). For the second design, each predictor value is doubled. The responses are generated from the same intercept and slope, with the same errors attached to each corresponding observation. These are hypothetical calculations designed to compare designs, not separate claims about real studies.

Worked Example: Comparing Narrow and Wide Predictor Ranges

For the narrow design, the six slope estimates are \(2.0, 2.6, 1.4, 2.2, 1.6, 1.8\). In the wide design, the predictor values are doubled while the same response errors are used. For any one error pattern, doubling every centered predictor value doubles the numerator of the slope calculation and multiplies its denominator by four. Therefore the error-related deviation of the slope from \(2\) is divided by two.

For example, the narrow-design estimate \(2.6\) is \(0.6\) above \(2\). With the wider predictor range, that same error pattern gives an estimate \(2+0.6/2=2.3\). Applying this calculation to all six paired repetitions gives:

Repetition	1	2	3	4	5	6
Narrow design: \(x=0,1,2,3\)	2.0	2.6	1.4	2.2	1.6	1.8
Wide design: \(x=0,2,4,6\)	2.0	2.3	1.7	2.1	1.8	1.9

The narrow-design estimates range from \(1.4\) to \(2.6\), a range of \(1.2\). The wide-design estimates range from \(1.7\) to \(2.3\), a range of \(0.6\). In this paired illustration, the wider predictor range makes the slope estimates less dispersed around \(2\). The same principle applies to repeated samples: with other conditions held comparable, a design with greater predictor spread tends to produce more precise slope estimates.

Other design features matter too. Increasing the number of observations generally reduces sampling variability when the added observations provide useful information and the model conditions remain appropriate. Larger random errors tend to increase the spread of slope estimates. The precise relationship among sample size, predictor values, and error variation is a separate calculation; the important point here is that the sampling distribution describes their combined consequence for repeated slope estimates.

Common Mistakes and Why This Distribution Matters

Confusing two kinds of variation. The spread of individual response values around a regression line is not itself the sampling distribution of the slope. The sampling distribution tracks how the calculated slope changes across repeated samples.
Expecting every estimate to equal the population slope. The estimates are centered on \(\beta_1\) under the stated model conditions, but individual estimates vary. A finite set of simulated estimates need not average exactly to \(\beta_1\).
Assuming a bell shape without conditions. Exact normality follows for a fixed design when errors are independent and normal. Zero-mean errors alone establish centering, not an exact normal shape.
Ignoring the predictor design. Two studies can use the same model and error variation but produce slope estimates with different spreads if their predictor values cover different ranges.

The sampling distribution is the foundation for measuring uncertainty in a fitted slope. Standard errors summarize its spread, and confidence intervals use that uncertainty to express a range of plausible population slopes. A clear understanding of repeated-sample variation also helps distinguish a stable estimate from one that may shift substantially if the study were repeated.

Key takeaway: Under a correct linear mean model with zero-mean errors, repeated least-squares slopes are centered on the population slope. Their spread reflects random outcome variation and the study design, including how broadly the predictor values are distributed.

Check Your Understanding

Answer these questions using the meaning of repeated sampling, not just the formulas.

In a fixed-design repeated-sampling setup, which quantities are held fixed and which are regenerated?
What does it mean for slope estimates to be centered on the population slope?
Why does a six-repetition simulation average need not equal the population slope exactly?
Under what additional condition is the sampling distribution of the slope exactly normal for a fixed design?
In the paired illustration, why did doubling every predictor value halve each estimate's deviation from the population slope?