Why Diagnose a Single-Variant Linear Fit?
A small p-value or a narrow confidence interval does not show that a fitted model describes the data adequately. For a quantitative trait, the single-variant linear model represents the conditional mean of the trait as a straight-line function of genotype dosage. Its usual standard errors and tests also rely on assumptions about the unexplained variation. Diagnostic plots help identify patterns that are inconsistent with these assumptions and observations that may have unusual influence on the fitted line.
Suppose \(x_i\) is the dosage of the allele of interest for individual \(i\), coded as 0, 1, or 2, and \(y_i\) is that individual’s quantitative trait value. This tutorial focuses on the model with an intercept and dosage as the predictor. Diagnostics do not establish that an association is causal, nor do they prove that a model is correct. They are tools for finding features of the data that deserve investigation.
A residual is the observed trait value minus its fitted value. It is not the unobserved error \(\epsilon_i\), although it is used to assess whether the error assumptions are plausible. In this model, the straight-line mean assumption means that the expected trait value changes linearly with dosage. The common-variance assumption means that residual variation should not systematically grow or shrink across the range of fitted values.
Residuals Versus Fitted Values
A residual-versus-fitted plot places \(\hat{y}_i\) on the horizontal axis and \(e_i\) on the vertical axis. A useful reference is a horizontal line at zero. If the linear mean and constant-variance assumptions are reasonable, residuals should generally scatter around that line without a systematic shape and with roughly similar vertical spread across the fitted range. Random scatter is not a proof of the assumptions, particularly in a small sample, but strong structure is a warning.
- Curvature: a U-shaped, inverted U-shaped, or other systematic bend suggests that a straight-line mean may not capture the pattern.
- Changing spread: a funnel or fan shape suggests that residual variance may depend on the fitted mean.
- Isolated residuals: a point far from the rest vertically may indicate an unusual trait value given its fitted value.
- Groups or stripes: these can arise because dosage has only three possible values. Look for systematic differences in the residual distributions across dosage groups, rather than expecting a continuous cloud of predictor values.
Worked Example: Recognizing Curvature in Residuals
Consider hypothetical data from five individuals with generic numerical predictor values \(x=0,1,2,3,4\) (not allele dosages) and trait values \(y=7,6,12,10,15\), respectively. Fit the intercept-and-predictor OLS model and inspect its residuals.
The predictor mean is \(\bar{x}=2\), and the trait mean is \(\bar{y}=10\). The OLS slope is the cross-product of centered predictor and trait divided by the sum of squared centered predictors:
The intercept is \(\hat{\beta}_0=\bar{y}-\hat{\beta}_1\bar{x}=10-2(2)=6\). The fitted values are therefore 6, 8, 10, 12, and 14, and the residuals are \(1,-2,2,-2,1\). They sum to zero, as OLS residuals from a model with an intercept should.
| Predictor \(x_i\) | Trait \(y_i\) | Fitted value \(\hat{y}_i\) | Residual \(e_i\) |
|---|---|---|---|
| 0 | 7 | 6 | 1 |
| 1 | 6 | 8 | -2 |
| 2 | 12 | 10 | 2 |
| 3 | 10 | 12 | -2 |
| 4 | 15 | 14 | 1 |
Plotted against fitted values, the residuals are positive at the two ends and negative at fitted values 8 and 12, with another positive residual at 10. The pattern is not convincing evidence of a particular nonlinear biological mechanism; there are only five observations. It does, however, illustrate why a residual plot is more informative than looking only at the fitted slope: residuals can show structure that a single coefficient and its standard error do not describe.
Normal Quantile Plots and Residual Tails
A normal quantile plot, also called a normal Q-Q plot, compares ordered residuals with the quantiles expected from a normal distribution. If the conditional error distribution is approximately normal, the points should lie roughly along a straight line, allowing for sampling variation. Consistent bending, pronounced tail departures, or a few isolated points can indicate skewness, heavy or light tails, or unusual observations. A Q-Q plot assesses distributional shape; it does not assess linearity or constant variance as directly as the residual-versus-fitted plot.
The vertical scale of a Q-Q plot may use raw residuals or standardized residuals, so check which is shown. Standardizing changes the scale, not the ordering. Small samples make the plot noisy: modest departures from a line are not, by themselves, proof that normality is false. Conversely, an apparently straight plot in a small sample does not verify normality.
Worked Example: Reading a Normal Quantile Plot
Suppose a hypothetical sample has six individuals, two at each dosage \(0,1,2\), with trait values \((43,57)\) at dosage 0, \((51,55)\) at dosage 1, and \((55,57)\) at dosage 2. The fitted line is \(\hat{y}=50+3x\): the mean trait values within the three dosage groups are 50, 53, and 56, which lie on that line.
The six residuals are \(-7,7,-2,2,-1,1\), which in increasing order are \(-7,-2,-1,1,2,7\). Their squared sum is \(49+4+1+1+4+49=108\). There are \(n-2=4\) residual degrees of freedom, so the residual standard deviation is
For an illustrative Q-Q plot with residuals scaled by the residual standard deviation, divide each ordered residual by \(s\). The plotted vertical coordinates are approximately \(-1.347,-0.3849,-0.1925,0.1925,0.3849,1.347\). One common set of normal reference quantiles uses plotting probabilities \((i-0.5)/6\), for \(i=1,\ldots,6\); the corresponding approximate quantiles are \(-1.383,-0.6745,-0.2104,0.2104,0.6745,1.383\).
The endpoints are fairly close, while the inner points are less extreme than their corresponding normal quantiles. With only six observations, this pattern is too sparse to support a firm conclusion about the tails. The appropriate reading is that the plot merits cautious inspection, not that it proves a particular error distribution or validates the normality assumption.
Leverage and Influence
Leverage describes how unusual an observation’s predictor value is relative to the other predictor values. In the intercept-and-dosage model, let \(\bar{x}\) be the sample mean dosage and let \(S_{xx}=\sum_i(x_i-\bar{x})^2\). The leverage of observation \(i\), denoted \(h_{ii}\), is
Thus, an individual with a dosage far from the sample mean can have high leverage. Leverage depends on the predictor values, not on the trait value. In the intercept-and-dosage model, the leverages sum to 2, the number of fitted coefficients, so their average is \(2/n\). A high-leverage observation is not automatically influential: if its residual is small, removing it may make little difference to the fitted line.
Influence describes how much the fitted results change when an observation is omitted. Cook’s distance combines residual size and leverage. For this model, let \(p=2\) be the number of fitted coefficients, and let \(s\) be the residual standard deviation from the full fit. The internally studentized residual and Cook’s distance are
Large values of \(D_i\) flag observations for closer review, but no single cutoff makes an observation automatically erroneous or removable. Heuristics such as \(4/n\) can be used as prompts for investigation, not as decision rules. A point can also have a large residual but modest leverage, so both its vertical departure and its position in the predictor space matter.
Worked Example: A High-Leverage Observation and Cook’s Distance
Consider generic numerical predictor values \(0,0,1,1,8\) (not allele dosages) and trait values \(11.75,11.75,9,9,18.5\). The predictor value 8 is unusual in this sample. The predictor mean is \(\bar{x}=2\), and
The OLS line is \(\hat{y}=10+x\). For example, the mean trait at predictor values 0, 1, and 8 is 11.75, 9, and 18.5, and the fitted values at those predictor values are 10, 11, and 18. The residuals are \(1.75,1.75,-2,-2,0.5\). Their sum is zero, and their predictor-weighted sum is also zero, consistent with OLS residuals for a model with an intercept and predictor. The residual sum of squares is
Here \(n=5\), \(p=2\), and \(df=n-p=3\), so \(s=\sqrt{14.375/3}=\sqrt{4.7917}=2.189\). The leverage for the predictor-8 observation is
Its internally studentized residual is \(r_i=0.5/[2.189\sqrt{1-0.9826}]\), approximately \(1.732\). Using the same rounded leverage in the Cook’s distance calculation gives
This very large Cook’s distance reflects the combination of a high-leverage predictor value and a nonzero residual; the residual alone is only 0.5 trait units. The point strongly affects the fitted line in this tiny hypothetical sample. It should prompt checks of the dosage, trait measurement, and model fit, not automatic deletion. If the observation is valid, its influence is part of the evidence and should be reported transparently.
A Practical Diagnostic Sequence
The plots answer different questions, so use them together rather than treating one as a general model-quality score. A practical sequence is:
Look for a systematic bend, changing spread, or isolated vertical departures. Note that discrete dosages may produce only a few vertical groupings.
Look for broad departures from a straight-line pattern, especially pronounced tail behavior or isolated points. Judge apparent departures in light of sample size.
Identify predictor values far from the sample center, then assess whether high leverage coincides with a consequential residual and a large Cook’s distance.
Check data quality and the scientific context, and consider whether the chosen model is adequate. Do not remove valid observations solely to make a plot look more regular.
A common mistake is to see one unusual point and conclude either that the model is invalid or that the point must be excluded. The correction is to separate the questions: Is the point a data error? Is it unusual in dosage, in trait conditional on dosage, or both? How much does it change the fitted association? A diagnostic flag identifies a question to resolve; it does not determine the answer.
Check Your Understanding
Use the definitions and examples above to answer these questions.
- In a residual-versus-fitted plot, what does a funnel-shaped cloud suggest, and which model assumption is relevant?
- Why can a dosage value have high leverage even when its residual is close to zero?
- What kind of departure from a straight line in a normal quantile plot would make you inspect the residual tails more closely?
- For the five-observation example, explain why the dosage-8 individual is influential even though its residual is only 0.5 trait units.
- Why is a diagnostic flag not, by itself, a sufficient reason to exclude an observation?