Why Slope Inference Needs Model Conditions
A fitted line describes a relationship in observed data, but slope inference asks a broader question: what does the evidence say about the population slope? The t-based procedures for that question rely on a model for how responses vary around their population mean at each predictor value. The conditions in this model are not cosmetic requirements. They describe the pattern of response variation that makes the inferential procedure appropriate.
Suppose \(X_i\) is the predictor value and \(Y_i\) is the response for observation \(i\), for \(i=1,\ldots,n\). The population slope, \(\beta_1\), describes the change in the conditional mean response for a one-unit increase in the predictor. The model for t-based slope inference is:
with errors independent across observations. Here \(\beta_0\) and \(\beta_1\) are the population intercept and slope, and \(\varepsilon_i\) is the error for observation \(i\): the difference between its response and the population mean response at its predictor value. The notation for the error distribution states both that errors are normal at a fixed predictor value and that their variance, \(\sigma^2\), is the same for all predictor values. Independence is an additional condition across observations.
What Each Condition Says at a Fixed Predictor Value
The phrase “at a fixed predictor value” is central. The model does not require all responses in the data to have the same mean or to follow one unchanging distribution. Instead, it describes the distribution of responses among observations with the same value of \(X\), or at a particular value considered by the model.
- Linearity: the mean response changes along a straight line as \(X\) changes. In symbols, \(E(Y_i\mid X_i=x)=\beta_0+\beta_1x\).
- Independence: knowing the error for one observation does not give information about the error for another, conditional on the predictor values and the study design.
- Constant variance: at every predictor value, the spread of responses around the conditional mean is the same: \(\operatorname{Var}(Y_i\mid X_i=x)=\sigma^2\).
- Normal errors: at every predictor value, the response distribution is normal and centered on its conditional mean: \(Y_i\mid X_i=x\sim N(\beta_0+\beta_1x,\sigma^2)\).
The last statement follows from the model: adding the fixed quantity \(\beta_0+\beta_1x\) to a normal error with mean zero shifts its mean but does not change its variance. Thus the response distribution can have different means at different \(x\) values while retaining the same variance and normal shape.
Linearity: A Straight-Line Mean, Not a Straight-Line Data Pattern
Linearity concerns the conditional mean, not whether every observed point lies on a line. Individual responses vary around their mean because of the errors. A scatterplot can therefore look like a broad band and still be consistent with a linear mean relationship. Conversely, a curved pattern in the average response across predictor values suggests that a straight-line model is not adequate.
For a straight-line conditional mean, equal changes in \(X\) correspond to equal changes in the expected response. This offers one way to inspect grouped data when there are repeated observations at selected predictor values. In a residual plot, systematic curvature is another warning: residuals are observed responses minus fitted responses, \(e_i=y_i-\hat{y}_i\). Random scatter around zero is more consistent with linearity than a curved residual pattern, though a plot cannot prove the condition.
Worked Example: Checking the Shape of the Mean Response
In a hypothetical study, a researcher records a response at four predictor values, with two observations at each value. The paired responses are: at \(x=0\), 1 and 3; at \(x=1\), 3 and 5; at \(x=2\), 7 and 9; and at \(x=3\), 13 and 15. Compare the group means to assess whether a straight-line mean is plausible.
For each predictor value, calculate the sample mean response as the sum of the two responses divided by 2:
The predictor increases by 1 each time, but the successive increases in the means are \(4-2=2\), \(8-4=4\), and \(14-8=6\). These changes are not approximately constant; the means bend upward rather than follow a straight-line pattern. The two observations at each value vary around their mean, but that within-value variation does not remove the evidence of curvature. A simple linear conditional mean would be questionable for these hypothetical data.
Independence: A Condition About Observations and Design
Independence is mainly justified by understanding how the data were collected. A random sample of distinct individuals or a properly randomized experiment may support independence, provided the design does not create dependence. In contrast, measurements taken repeatedly on the same person, observations from matched pairs, or individuals grouped within the same household or clinic can be related. Such data should not be treated as independent merely because they appear in separate rows.
The condition applies to the model errors, not to the predictor values needing to be unrelated to one another. Also, residuals calculated after fitting a line are generally constrained by the fitting process, so they are not literally independent observations. Residual plots can reveal patterns, but they do not replace knowledge of the study design.
Worked Example: Repeated Measurements Create Dependence
Suppose a hypothetical study records blood pressure for several patients on two visits. A patient’s stable baseline may make both visits higher or lower than the population mean. To illustrate, write a response as a population line plus a patient-specific component \(b_j\) and a visit-specific component \(\varepsilon_{ij}\), where \(j\) indexes patients. Assume, for illustration, that the patient component has variance 4 and the visit-specific components are independent with variance 1.
For two visits from the same patient, the shared \(b_j\) contributes to both errors. If the visit-specific components are independent of the patient component and of each other, the covariance between the two errors is:
Each error has variance \(4+1=5\), so the correlation between the two errors is:
The errors from the two visits are therefore positively related in this illustration, not independent. Treating all visits as independent observations would fail to reflect the study design. The example does not show that repeated measurements can never be analyzed; it shows why the ordinary independent-error condition cannot simply be assumed for them.
Constant Variance: The Spread Should Not Systematically Change
Constant variance, also called homoscedasticity, means that the conditional spread of responses around the line is the same across predictor values. A scatterplot may suggest a violation when the vertical spread becomes wider or narrower as \(X\) changes. A residual-versus-fitted plot can reveal the same kind of pattern: a fan or funnel shape is a warning, while a roughly even vertical band is more consistent with constant variance.
A changing spread is different from a changing mean. A curved pattern raises a concern about linearity; a fan-shaped pattern raises a concern about variance. Both can appear at the same time, so it is important to inspect where observations lie around the fitted line rather than relying on a single feature of the plot.
Worked Example: Comparing Spread Across Predictor Values
Consider hypothetical responses measured at three predictor values. At each value the mean response is 10, but the observations are: at \(x=1\), 9, 10, 11; at \(x=2\), 7, 10, 13; and at \(x=3\), 5, 10, 15. Calculate the sample variance within each group and compare the spreads.
For three observations at a predictor value, the sample variance is the sum of squared deviations from that group’s mean divided by \(3-1=2\). The means are all 10. Thus:
The conditional means do not change in this example, but the observed spreads increase substantially, from a sample variance of 1 to 25. This pattern is evidence against constant variance. The group variances are estimates based on only three observations each, so they do not establish the population variances; the important feature is the pronounced widening pattern.
Normal Errors: Normality at Each Predictor Value
The normality condition is not that the predictor values are normally distributed, nor that the combined response values across all predictor values form one normal distribution. Because the mean response changes with \(X\), the combined responses may not have a single normal shape even when the model conditions hold. The condition is that errors at a fixed predictor value are normally distributed around zero. Equivalently, responses at that value are normally distributed around the line.
A normal Q-Q plot of residuals can help assess whether the error distribution is reasonably compatible with normality. Strong curvature or an extreme departure at one end can indicate skewness or unusually heavy tails. Residuals are estimates of the errors, not the errors themselves, and a diagnostic plot—especially with a small sample—cannot establish exact normality. Normality matters for the exact t-based reference distribution; departures can be more consequential in small samples.
Worked Example: A Mean of Zero Does Not Ensure Normal Errors
Imagine eight hypothetical responses measured at the same predictor value, where the population mean response is 50 units. Their deviations from that mean—the errors for this illustration—are \(-1,-0.8,-0.5,-0.2,0,0.1,0.2,2.2\) units. Check their mean, then consider what their shape suggests.
The sum of the deviations is:
A zero average is consistent with errors centered on the conditional mean, but it is not enough to establish normality. Seven deviations lie between \(-1\) and \(0.2\), while one positive deviation is 2.2 units. That pronounced one-sided tail is a warning of possible skewness. With only eight observations, the shape is difficult to judge reliably; a Q-Q plot and substantive knowledge would inform the assessment, but neither can prove that the population errors are normal.
Why the Conditions Matter Together
The conditions have distinct roles. Linearity specifies the population mean relationship whose slope is the inferential target. Independence prevents one observation’s error from carrying information about another’s. Constant variance supplies a common scale of error across predictor values. Normality describes the conditional error shape. Under the stated model, t-based slope inference has its intended reference distribution. If a condition is doubtful, the resulting inference may be unreliable, and the reason for concern should be identified rather than hidden behind a reported test or interval.
A practical assessment combines several sources of information. Use the study design to evaluate independence; examine the relationship and residual patterns for linearity and changing spread; and use a residual distribution plot, such as a normal Q-Q plot, to look for notable departures from normality. These checks diagnose plausibility, not truth. A plot with no obvious warning does not prove an assumption, and a formal test alone cannot replace understanding how the observations were obtained.
Common Mistakes
- Wrong: “The responses are not normally distributed overall, so the normal-error condition fails.” Correction: The condition concerns the response distribution at each fixed predictor value, after accounting for its mean on the line.
- Wrong: “The residuals look uncorrelated, so independence is established.” Correction: Independence is primarily a design question. Repeated or clustered observations may be dependent even if a residual plot looks unremarkable.
- Wrong: “The fitted line is linear, so the variance must be constant.” Correction: Linearity concerns the mean; constant variance concerns the spread around that mean. Assess them separately.
- Wrong: “The average error is near zero, so the errors are normal.” Correction: Centering and normal shape are different properties. A distribution can have mean zero and still be strongly skewed.
Check Your Understanding
For each question, distinguish what the condition states from what a diagnostic or study description can establish.
- In a scatterplot, the average response rises more rapidly at larger predictor values. Which condition is in doubt, and why?
- A study records three measurements on each participant. Why might the independence condition fail even if all rows contain different measurement times?
- What does constant variance require about the response distributions at two different predictor values?
- Why does a normal Q-Q plot of residuals not prove that the errors are normally distributed?
- A residual plot has a roughly straight, even band, but the data came from observations clustered within clinics. What condition still needs careful evaluation, and what information is most relevant?