Deriving OLS Association Estimates and Standard Errors

Why Derive the Association Estimate?

In a quantitative-trait association analysis, a genotype dosage is used to predict a measured phenotype. The fitted slope describes the estimated change in the phenotype associated with one additional dosage copy. Its standard error describes how precisely that slope is estimated under the regression model. Both quantities can be obtained directly from sums of squares and cross-products, making clear what information in the data drives the estimate and its precision.

This tutorial considers ordinary least squares (OLS) with one genotype predictor and an intercept. It focuses on deriving and calculating the slope and its standard error, not on confidence intervals or tests.

For an additive dosage, \(g_i\) is commonly coded 0, 1, or 2 copies of an allele. The slope is then expressed in phenotype units per dosage copy. It describes an association in the specified model; by itself, it does not establish that changing genotype causes a change in phenotype.

Centered Sums of Squares

Write \(\bar{g}\) for the sample mean of the genotype dosages and \(\bar{Y}\) for the sample mean phenotype. The centered genotype sum of squares, phenotype sum of squares, and genotype–phenotype cross-product are

Centering separates the slope calculation from the intercept: the slope depends on how genotype and phenotype vary together around their means, relative to how much genotype varies around its mean. If \(S_{gg}=0\), every individual has the same dosage and there is no information to distinguish a genotype slope from the intercept.

Why the Slope Has This Form

OLS chooses the intercept and slope to minimize the residual sum of squares, the sum of squared differences between observed and fitted phenotypes. For any fixed slope, the minimizing intercept makes the mean residual zero, so it equals \(\bar{Y}-\hat{\beta}_1\bar{g}\). Substituting that intercept into the residuals gives

The resulting residual sum of squares, viewed as a function of the slope \(b\), is

Differentiating with respect to \(b\) gives \(-2S_{gY}+2bS_{gg}\). Setting this derivative to zero yields \(b=S_{gY}/S_{gg}\). When \(S_{gg}>0\), the second derivative \(2S_{gg}\) is positive, so this is the minimizing slope. Substitution into the intercept formula gives the OLS intercept. This derivation also shows why both the cross-product and genotype variation matter: the cross-product sets the direction and size of the estimated slope, while \(S_{gg}\) is its denominator.

Residual Variation and the Standard Error

The fitted residual for individual \(i\) is \(e_i=Y_i-\hat{\beta}_0-\hat{\beta}_1g_i\). The residual sum of squares is \(SSE=\sum e_i^2\). With an intercept and one slope, two parameters have been estimated, leaving \(n-2\) residual degrees of freedom. The residual mean square is \(MSE=SSE/(n-2)\), and the estimated residual standard deviation is \(\sqrt{MSE}\).

To justify the variance result, write \(x_i=g_i-\bar{g}\). The slope formula and the model imply

Conditional on the genotypes, the errors have variance \(\sigma^2\) and are independent. Therefore, the variance of their weighted sum is \(\sigma^2\sum x_i^2\). Since \(\sum x_i^2=S_{gg}\), dividing by \(S_{gg}^2\) gives

Replacing the unknown \(\sigma^2\) with \(MSE\) produces the estimated standard error. This formula relies on the common-variance model; if the residual variance changes with genotype, this model-based standard error need not be appropriate.

How Genotype Variability Affects Precision

The standard error formula makes the role of genotype variation explicit. If residual variance is held fixed, increasing \(S_{gg}\) decreases \(SE(\hat{\beta}_1)\) in inverse proportion to \(\sqrt{S_{gg}}\). A sample with more spread in genotype dosage provides more information for distinguishing the slope from the intercept. A larger sample often increases \(S_{gg}\), but sample size alone does not determine it: the distribution of dosages also matters.

Assumptions and Interpretation in Practice

The calculations are simple, but their meaning depends on the model. Before interpreting a slope or its standard error, check that the analysis has an intercept, that genotype dosage varies, and that a linear change in mean phenotype per dosage copy is a reasonable description for the chosen coding. The model-based standard error additionally relies on independent errors with a common variance. Normal errors give the standard finite-sample distributional result; with nonnormal errors, exact small-sample inference does not follow from these formulas alone.

The slope is an estimated mean difference for a one-copy dosage increase under this linear model. It is not the change in every individual's phenotype, nor does it assert a causal effect. The standard error is not the standard deviation of individual phenotypes: it measures the estimated sampling variability of the slope under the model. It is also not determined by genotype variability alone, because larger residual variation increases the standard error through \(MSE\).

Common Mistakes

• Reversing numerator and denominator. The slope is \(S_{gY}/S_{gg}\), not \(S_{gg}/S_{gY}\). The cross-product measures joint variation; the genotype sum of squares scales it by predictor variation.
• Using phenotype spread in place of residual spread. The standard error uses \(MSE=SSE/(n-2)\), not \(S_{YY}/(n-1)\). The residual variation is what remains after fitting the intercept and slope.
• Using the wrong degrees of freedom. One intercept and one slope are fitted, so the residual degrees of freedom are \(n-2\), not \(n-1\).
• Claiming that more genotype variation always guarantees a precise estimate. It lowers the standard error only with residual variation held fixed. A large residual variance can still make the slope imprecise.
• Trying to estimate a slope when all dosages are identical. If \(S_{gg}=0\), the slope formula divides by zero. The data contain no genotype contrast from which to estimate a per-copy association.