Checking Conditions Before Slope Inference

Why Check Conditions Before Slope Inference?

A slope test or confidence interval can be computed from a regression output, but its reported uncertainty is meaningful only to the extent that the model and its conditions are reasonable. Diagnostic checks are not a formality: a curved mean relationship, dependent observations, or changing error variance can make the usual slope standard error and t procedure unreliable. A normal probability plot can also reveal departures from the normal-error model used for exact small-sample inference.

No single graph can certify that a model is correct. Study design provides essential information about how observations were collected, while residual plots help identify patterns that the fitted line has not captured. The goal is to look for important contradictions of the model, not to prove every condition from a sample.

The Conditions for a Slope t Procedure

For a simple linear regression, let \(Y_i\) be the response for observation \(i\), and let \(x_i\) be its predictor value. The conditional error model is

Y_i=\beta_0+\beta_1x_i+\varepsilon_i,\qquad i=1,\ldots,n.

Here \(\beta_0+\beta_1x_i\) is the population mean response at predictor value \(x_i\), and \(\varepsilon_i\) is the deviation of an individual response from that mean. The usual exact t procedure for the population slope \(\beta_1\) relies on the following conditions, stated conditional on the predictor values:

Linearity: the conditional mean response is a straight-line function of the predictor, \(E(Y_i\mid x_i)=\beta_0+\beta_1x_i\).
Independence: the errors for different observations are independent. In particular, knowing one error does not give information about another.
Constant variance: the errors have the same variance at every predictor value, \(\operatorname{Var}(\varepsilon_i\mid x_i)=\sigma^2\).
Normal errors: for exact small-sample t inference, the errors are normally distributed at each predictor value, with mean zero and the common variance \(\sigma^2\).

These conditions concern errors, which are not directly observed. A residual \(e_i=y_i-\hat{y}_i\) is an estimate of the error for observation \(i\), calculated by subtracting its fitted value \(\hat{y}_i\) from its observed response \(y_i\). Plots of residuals are therefore diagnostic approximations, not direct views of the errors.

Key distinction: Study design is the main source of evidence about independence. Residual plots are especially useful for checking linearity and constant variance, and a normal probability plot can reveal departures from normality. None of these checks proves a condition.

What the Regression Diagnostic Plots Show

A residual-versus-fitted plot places the residuals \(e_i\) vertically and the fitted values \(\hat{y}_i\) horizontally. A residual-versus-predictor plot places the residuals against the observed predictor values \(x_i\). In a suitable linear model, either plot should show residuals scattered around zero without a systematic curve. The points should also have roughly similar vertical spread across the horizontal axis.

A smooth curve, repeated wave, or other organized pattern can indicate that the straight-line mean model is inadequate. A fan or funnel shape, in which the vertical spread gets wider or narrower across the plot, suggests that error variance changes with the fitted value or predictor. These plots often tell a similar story in simple regression because fitted values are a linear function of the predictor. Examining both can still help make the horizontal scale more interpretable.

A normal probability plot compares the ordered residuals with the values expected from a normal distribution. If the normal-error condition is plausible, the plotted points should lie approximately along a straight line. Marked curvature, strong asymmetry, or unusually distant tail points can raise concern. Small departures from a straight line are common, particularly in modest samples; interpretation should consider how pronounced the pattern is and how much it could affect the inference.

What Least-Squares Fitting Forces the Residuals to Do

Residual plots must be interpreted with one feature of least squares in mind. In regression with an intercept, the fitted line is chosen to minimize the sum of squared residuals. The resulting residuals are constrained: they sum to zero, and their products with the predictor values sum to zero. This follows directly from minimizing the squared-error function.

SSE(b_0,b_1)=\sum_{i=1}^{n}(y_i-b_0-b_1x_i)^2.

At the least-squares estimates \(\hat{\beta}_0\) and \(\hat{\beta}_1\), the partial derivatives of \(SSE\) with respect to both coefficients are zero. Writing \(e_i=y_i-\hat{\beta}_0-\hat{\beta}_1x_i\), this gives

\frac{\partial SSE}{\partial b_0}=-2\sum_{i=1}^{n}e_i=0, \qquad \frac{\partial SSE}{\partial b_1}=-2\sum_{i=1}^{n}x_i e_i=0.

Thus \(\sum e_i=0\) and \(\sum x_i e_i=0\). These identities do not mean that the errors are independent, normally distributed, or constant in variance. Nor do they mean that the residual plot must look random: a curved pattern can still be present even though these sums equal zero.

Worked Example: A Curved Residual Pattern

Suppose a hypothetical data set records study hours \(x\) and quiz score \(y\) for six students. The observations are shown below. The fitted line is \(\hat{y}=10+2x\), with scores in points.

Study hours, \(x\)	Observed score, \(y\)	Fitted score, \(\hat{y}=10+2x\)	Residual, \(e=y-\hat{y}\)
1	13.0	12	1.0
2	13.8	14	-0.2
3	15.2	16	-0.8
4	17.2	18	-0.8
5	19.8	20	-0.2
6	23.0	22	1.0

The residuals are positive at the low and high ends and negative in the middle. Their sum is \(1.0-0.2-0.8-0.8-0.2+1.0=0\). They are also consistent with the stated least-squares fit: \(\sum x_ie_i=1.0-0.4-2.4-3.2-1.0+6.0=0\). The fitted slope is consistent as well: the mean predictor is \(3.5\), the mean score is \(17\), and

\hat{\beta}_1 =\frac{\sum (x_i-\bar{x})(y_i-\bar{y})}{\sum (x_i-\bar{x})^2} =\frac{35}{17.5}=2.

A residual-versus-predictor plot would show a U-shaped pattern: positive residuals at both ends and negative residuals in the middle. That organized shape is evidence against a straight-line mean relationship for these data. It suggests that the linear model may be missing curvature; it does not, by itself, establish which alternative model is appropriate.

Independence: Start With How the Data Were Collected

A residual plot can sometimes reveal dependence, especially when the observations have a meaningful sequence such as time or location. But a plot without an obvious pattern cannot establish independence. The design must be examined first: Were observations sampled independently? Were several measurements taken on the same person, household, school, or clinic? Is there a time sequence that could create related errors? Repeated or clustered observations can be dependent even if a conventional residual plot appears unremarkable.

Worked Example: Repeated Measurements From Clinics

A hypothetical project records the average waiting time at each of 12 clinics on each of four consecutive Mondays, then relates waiting time to the number of staff on duty. There are 48 clinic-week records, but each clinic appears four times.

The number 48 is the number of records; it does not guarantee 48 independent observations. Records from the same clinic may share features such as layout, staffing practices, or patient volume. In addition, consecutive Mondays may be connected by ongoing conditions at that clinic. Therefore, treating every clinic-week error as independent needs justification that this design description alone does not provide.

A residual plot against week might expose a visible sequence, but a lack of a sequence pattern would not remove the concern that four records from one clinic are related. The right conclusion is that independence should not be assumed solely because there are many rows in the data table. The sampling or measurement structure must be accounted for before using the usual slope t procedure.

Constant Variance: Look for Similar Vertical Spread

Constant variance means that the variability of responses around the population regression line is the same throughout the range of predictor values. In a residual plot, this condition is plausible when the vertical spread is roughly similar from left to right. The residuals need not have exactly the same range in every small region; random variation will produce some unevenness.

Worked Example: A Fan Shape in a Residual Plot

Suppose a hypothetical software residual-versus-fitted plot for a clinical measurement shows residuals mostly between about \(-1\) and \(1\) unit when fitted values are low, but ranging from about \(-5\) to \(5\) units when fitted values are high. The residuals remain centered near zero, but their vertical spread increases across the plot.

The widening spread is a fan shape and is evidence against constant error variance. The fact that residuals are centered near zero does not resolve this problem: centering concerns the location of the residuals, whereas constant variance concerns their spread. The usual standard error for the slope relies on a common error variance, so the pattern makes the usual t inference questionable. It is not enough to report the slope test without addressing the concern.

The plot is a warning, not a definitive diagnosis. A few extreme points or a small number of observations can make the spread look uneven by chance. The pattern should be considered alongside the study context and the amount of data across the fitted range.

Normality: Read the Shape, Not Just the Endpoints

A normal probability plot is used to assess whether the error distribution is reasonably compatible with normality. The check matters most for exact t inference in small samples: with independent normal errors and constant variance, the slope statistic has a t distribution with \(n-2\) degrees of freedom under the null hypothesis. A plot cannot demonstrate that the errors are normal, because it displays estimated residuals rather than the unobserved errors.

Worked Example: Curvature in a Normal Probability Plot

Imagine that a hypothetical regression uses measurements from 20 independent units. In its normal probability plot, the middle 16 ordered residuals lie close to a straight reference line, but the largest four residuals bend well above that line. The lower tail remains close to the line.

This one-sided tail departure suggests that the residual distribution may be right-skewed or may contain unusually large positive errors. It is a reason to question the normal-error condition, particularly if the sample is small and the slope inference depends heavily on the exact t model. It is not proof of nonnormality: tail points naturally vary, and one plot should be considered with the data and study context. Conversely, a nearly straight plot is reassuring but cannot prove normality.

A Practical Sequence for Checking Conditions

A disciplined order helps keep the checks tied to their relevant evidence. The following sequence separates design-based reasoning from plot-based assessment.

Describe the observational units and collection process.
Identify repeated measurements, clusters, matching, or time ordering that could make errors dependent. Use the design, not just a graph, to assess independence.

Inspect residuals against predictor and fitted values.
Look for curves or other systematic structure that would conflict with a straight-line conditional mean.

Assess the vertical spread.
Check whether residual variability appears roughly stable across the predictor or fitted range, rather than forming a fan or funnel.

Inspect a normal probability plot.
Look for substantial curvature, asymmetry, or unusual tail departures, and judge how consequential these would be for the sample size.

State the limitation clearly.
Report which checks appear reasonable and which raise concern. A diagnostic plot supports a judgment; it does not certify a model.

Common Mistake: Treating a Clean Plot as Proof

A common mistake is to say, “The residual plot looks random, so all regression conditions are satisfied.” That conclusion goes too far. Residual plots can help assess linearity and constant variance, but they cannot rule out dependence created by the study design. Nor can a visually ordinary plot prove normality, especially when the sample is small.

A better conclusion is specific and qualified: “The residual plots show no marked curvature or change in spread, and the normal probability plot is reasonably close to a line. The sampling design gives no indication of repeated or clustered observations, so the conditions appear plausible, though they cannot be verified from these data alone.” If the design or plots raise a material concern, state it before interpreting the usual slope t result.

Key takeaway: Before slope inference, use the study design to assess independence, residual-versus-predictor and residual-versus-fitted plots to assess linearity and constant variance, and a normal probability plot to assess normal-error concerns. Treat each as evidence, not proof.

Check Your Understanding

Try these questions without looking back.

State the four error conditions used for the usual exact t procedure for a population slope.
Why can a residual plot with no visible sequence not establish independence when the data contain repeated measurements on the same clinics?
In a residual-versus-fitted plot, what does a U-shaped pattern suggest about the linearity condition?
Residuals remain centered near zero, but their spread grows as fitted values increase. Which condition is in question, and why does centering not settle it?
What does a substantial bend in the upper tail of a normal probability plot suggest? Why is this evidence rather than proof?
For a least-squares line with an intercept, why must the residuals sum to zero, and why does that fact not guarantee that the linear model is appropriate?