Genotype Indicators For Nonadditive Effects

Why Represent Genotypes With Indicators?

A linear model using allele dosage treats the outcome mean as changing by the same amount for each additional copy of the effect allele. That is a useful model when the equal-increment pattern is appropriate. But a variant’s three genotype classes may have different mean trait values without following that pattern. Indicator variables let a model estimate each class’s mean separately, relative to a specified reference class.

This representation is about the statistical form of the mean, not a claim about biological mechanism. The coefficient for a genotype class describes a difference in the modeled outcome mean, conditional on any included covariates. Choosing a reference class is a coding decision that makes those differences easy to read.

Definition: Let $G\in\{0,1,2\}$ denote the number of copies of the effect allele. With $G=0$ as the reference class, define $D_1=I(G=1)$ and $D_2=I(G=2)$, where an indicator $I(A)$ equals 1 when statement $A$ is true and 0 otherwise. The genotype-indicator linear model is $$ Y_i=\beta_0+\beta_1D_{1i}+\beta_2D_{2i}+\boldsymbol{\gamma}^{\mathsf T}\mathbf C_i+\varepsilon_i. $$ Here $Y_i$ is the quantitative trait for individual $i$, $\mathbf C_i$ is a vector of covariates, and $\varepsilon_i$ is the error.

The indicator pair takes values $(0,0)$ for $G=0$, $(1,0)$ for $G=1$, and $(0,1)$ for $G=2$. There is no indicator for the reference class: its mean is represented by the intercept and covariate terms. Including an intercept and all three genotype indicators would make the columns linearly dependent, so the separate genotype coefficients could not be uniquely estimated.

Model Assumptions and Coefficient Meanings

The central model assumption is that the conditional mean of $Y_i$, at a given covariate value, has the stated form. In particular, the model allows three genotype-specific levels but uses the same covariate slopes across genotype classes. For the usual least-squares estimates to have the standard interpretation as unbiased estimates of the model coefficients, the conditional error mean must be zero: $E(\varepsilon_i\mid G_i,\mathbf C_i)=0$. Observations should be independent for the usual independent-error standard errors. Constant conditional error variance supports the conventional standard-error formula; normally distributed errors additionally give exact small-sample $t$ inference under the model.

The design matrix must have full column rank. In practice, the genotype indicators must not be redundant with the intercept or covariates, and the model parameters must be identifiable. Each represented genotype class needs observations to estimate its class-specific mean from data. If a class is absent, the corresponding contrast cannot be estimated from that sample without additional assumptions.

When there are no covariates, write the three modeled genotype means as $\mu_0,\mu_1,\mu_2$. Substitution into the model gives

\mu_0=\beta_0,\qquad \mu_1=\beta_0+\beta_1,\qquad \mu_2=\beta_0+\beta_2.

Thus $\beta_0$ is the mean for $G=0$, $\beta_1$ is the difference between the $G=1$ and $G=0$ means, and $\beta_2$ is the difference between the $G=2$ and $G=0$ means. The difference between the two nonreference classes is not a single coefficient; it is $\beta_2-\beta_1$.

With covariates, these same contrasts apply at a fixed value of $\mathbf C$. For example, the model’s mean for $G=1$ minus its mean for $G=0$, at the same covariate values, is $\beta_1$. If genotype groups have different covariate distributions, this conditional contrast need not equal the difference between their unadjusted sample averages.

Worked Example: Reading Three Genotype-Class Means

Suppose a hypothetical sample has a quantitative trait measured in milligrams per deciliter. The observed values are 2, 4, and 6 for $G=0$; 5, 7, and 9 for $G=1$; and 6, 8, and 10 for $G=2$. Fit the indicator model with an intercept and no covariates.

The group means are $$ \bar y_0=(2+4+6)/3=4,\quad \bar y_1=(5+7+9)/3=7,\quad \bar y_2=(6+8+10)/3=8. $$

With an intercept and separate indicators for the two nonreference classes, the fitted means equal these three group means. Therefore, $\hat\beta_0=4$ mg/dL, $\hat\beta_1=7-4=3$ mg/dL, and $\hat\beta_2=8-4=4$ mg/dL. The fitted model is $ \hat Y=4+3D_1+4D_2$.

The estimated $G=1$ versus $G=0$ difference is 3 mg/dL, and the estimated $G=2$ versus $G=0$ difference is 4 mg/dL. To compare $G=2$ directly with $G=1$, subtract the two coefficients: $4-3=1$ mg/dL. These are descriptive fitted contrasts; the calculation alone does not establish that any difference is statistically distinguishable from zero.

Derivation: Why the Model Fits Separate Group Means

With no covariates, each observation in a genotype class receives that class’s fitted value. For any class $g$, the least-squares contribution from its observations is the sum of squared deviations from a common fitted value $m_g$. The value minimizing this sum is the class sample mean. To see this, let $y_{gj}$ be observation $j$ in class $g$, and differentiate:

\frac{\partial}{\partial m_g}\sum_{j=1}^{n_g}(y_{gj}-m_g)^2 =-2\sum_{j=1}^{n_g}(y_{gj}-m_g)=0 \quad\Longrightarrow\quad m_g=\bar y_g.

The indicator model can represent any three such fitted values: set $\beta_0=\bar y_0$, $\beta_1=\bar y_1-\bar y_0$, and $\beta_2=\bar y_2-\bar y_0$. Substitution gives fitted values $\bar y_0,\bar y_1,\bar y_2$ for the three classes. This establishes both the group-mean interpretation and the coefficient mapping. With covariates included, least squares estimates the genotype contrasts jointly with the covariate effects; the simple raw group means need not equal the adjusted fitted means.

Calculating Other Contrasts

Any pairwise genotype contrast can be formed from the coefficients. Under the reference coding above, the $G=1$ versus $G=0$ contrast is $\beta_1$, the $G=2$ versus $G=0$ contrast is $\beta_2$, and the $G=2$ versus $G=1$ contrast is $\beta_2-\beta_1$. Reversing the order reverses the sign. For example, the $G=1$ versus $G=2$ contrast is $\beta_1-\beta_2$.

Inference for a contrast uses its standard error, not necessarily the standard error of either coefficient alone. If the estimated coefficient vector is $\hat{\boldsymbol\beta}$ with estimated covariance matrix $\widehat{\operatorname{Var}}(\hat{\boldsymbol\beta})$, and a contrast is $L\hat{\boldsymbol\beta}$, then

SE(L\hat{\boldsymbol\beta}) =\sqrt{L\,\widehat{\operatorname{Var}}(\hat{\boldsymbol\beta})\,L^{\mathsf T}}.

For $G=2$ versus $G=1$, $L$ assigns coefficient 1 to $\beta_2$, coefficient $-1$ to $\beta_1$, and 0 to the intercept and other unrelated terms. The covariance between the two estimated coefficients is part of the calculation. Treating them as independent without justification can give the wrong standard error.

Worked Example: Interpreting Covariate-Adjusted Contrasts

Consider hypothetical measurements of a blood trait in mmol/L. A fitted model uses $G=0$ as reference and includes age, centered at 50 years: $$ \widehat E(Y\mid G,\text{age})=10+0.08(\text{age}-50)-1.2D_1-3.5D_2. $$ The coefficient values are stipulated as the fitted estimates for this example.

At age 60, the reference-class fitted mean is $10+0.08(60-50)=10.8$ mmol/L. The $G=1$ fitted mean is $10.8-1.2=9.6$ mmol/L, and the $G=2$ fitted mean is $10.8-3.5=7.3$ mmol/L.

At the same age, the estimated $G=1$ versus $G=0$ contrast is $-1.2$ mmol/L, and the estimated $G=2$ versus $G=0$ contrast is $-3.5$ mmol/L. The $G=2$ versus $G=1$ contrast is $-3.5-(-1.2)=-2.3$ mmol/L. The negative sign means the model estimates a lower trait mean for $G=2$ than for $G=1$, conditional on age. It does not imply that genotype causes the difference.

Changing the Reference Class

The reference class determines which contrasts appear directly as coefficients. It does not change the fitted mean for any genotype or the underlying pairwise differences. Changing the reference is a reparameterization: coefficients must be recomputed so that the same three fitted means are represented.

Worked Example: Re-expressing the Same Means With a New Reference

Suppose hypothetical trait measurements are 98, 100, and 102 units for $G=0$; 104, 106, and 108 units for $G=1$; and 109, 111, and 113 units for $G=2$. Their means are 100, 106, and 111 units, respectively. With $G=0$ as reference, the intercept is 100 and the two indicator coefficients are $106-100=6$ and $111-100=11$.

Now use $G=2$ as the reference, with indicators for $G=1$ and $G=0$. The new intercept is 111 units. The coefficient for $G=1$ is $106-111=-5$ units, and the coefficient for $G=0$ is $100-111=-11$ units.

The $G=1$ versus $G=0$ contrast in this coding is $(-5)-(-11)=6$ units, just as before. The fitted means are unchanged: $111$ for $G=2$, $111-5=106$ for $G=1$, and $111-11=100$ for $G=0$. Only the coefficient meanings relative to the chosen reference have changed.

Model Checks, Alternatives, and Limitations

Before interpreting estimates, check the genotype counts and the coding. A table of counts should confirm that the intended reference class and both indicator classes are present. Examine outcome distributions or residuals by genotype class for extreme observations, markedly different spreads, and patterns that may conflict with the common-error assumptions. For adjusted models, also check that the design is estimable: a genotype indicator perfectly explained by another design column cannot have a separate identifiable coefficient. Sparse classes can produce imprecise estimates even when the model is technically estimable.

A dosage-only model is a more constrained alternative. It represents the three genotype means as a straight-line pattern across $G=0,1,2$. In the reference-indicator parameterization, that equal-increment pattern requires $\beta_2=2\beta_1$. The indicator model does not impose that restriction. This relation describes the constraint; deciding which coding is suitable for a particular analysis requires considering the scientific question and data.

The indicator model’s flexibility has a cost: it estimates two genotype contrasts rather than one dosage slope, and small or absent genotype groups limit precision or estimability. It also assumes that genotype contrasts are the same across covariate values unless genotype-by-covariate terms are added. A fitted association is a model-based comparison, not by itself evidence of a causal effect or a biological mechanism.

Reference confusion: Saying “the coefficient is the effect of genotype” is incomplete. State the genotype class being compared with the reference and the outcome units.
Wrong baseline: The intercept is the reference-class mean only when covariates are zero. With covariates, it is the reference mean at the covariate values coded as zero.
Pairwise contrast error: The $G=2$ versus $G=1$ contrast is $\beta_2-\beta_1$, not $\beta_2$. Its standard error must account for coefficient covariance.
Overinterpreting fitted differences: A point estimate does not establish statistical evidence, causation, or a mechanism. Consider uncertainty and the study design before drawing conclusions.

Key takeaway: With $G=0$ as reference, two indicators represent $G=1$ and $G=2$. Their coefficients are contrasts against $G=0$; any other genotype comparison is obtained by subtracting the corresponding fitted means or coefficients.

Check Your Understanding

Work through each question, then compare with the brief solutions below.

If the fitted coefficients are $\hat\beta_0=12$, $\hat\beta_1=2$, and $\hat\beta_2=-1$, what are the three fitted means when there are no covariates?
Using those same coefficients, calculate the $G=2$ versus $G=1$ contrast and state its sign and units if the trait is measured in grams.
What condition on $\beta_1$ and $\beta_2$ makes the three genotype means follow equal increments across $G=0,1,2$?
Why is an indicator model with an intercept and all three genotype indicators not uniquely identifiable?

Solutions. (1) The fitted means are $\hat\mu_0=12$, $\hat\mu_1=12+2=14$, and $\hat\mu_2=12-1=11$. (2) The contrast is $\hat\beta_2-\hat\beta_1=-1-2=-3$ grams, so the fitted mean for $G=2$ is 3 grams lower than for $G=1$. (3) Equal increments require $\beta_1=\beta_2-\beta_1$, or $\beta_2=2\beta_1$. (4) For every individual, the three genotype indicators sum to 1, exactly duplicating the intercept column. This linear dependence prevents unique estimation of all four coefficients.