Categorical Covariates, Reference Levels, and Contrasts

Why Categorical Covariates Need a Coding Rule

A categorical covariate records group membership rather than a numerical quantity: examples include study site, laboratory batch, or a prespecified group label. Regression requires numerical predictors, so group membership must be encoded. The encoding determines how the model’s coefficients describe group differences. It does not, by itself, make a category biologically ordered or change the information in the data.

In genetic association regression, the genotype effect is usually interpreted conditional on the covariates included in the model. If a covariate has several categories, its encoding must represent those categories without duplicating the intercept. The reference-level approach does this with a set of indicators and gives each coefficient a direct comparison with one specified category.

Definition: For a categorical covariate with \(K\) observed levels, choose one level as the reference level. Create \(K-1\) indicator variables, each equal to 1 for one non-reference level and 0 otherwise. In a regression containing an intercept, each indicator coefficient represents that level’s difference from the reference, conditional on the other predictors. This representation is also called treatment coding or reference-cell coding.

Indicator Coding in a Genetic Association Model

Suppose \(G_i\) is an individual’s genotype dosage, coded \(0,1,\) or \(2\), and \(C_i\) is a categorical covariate with levels A, B, and C. Take A as the reference. Define \(I_{Bi}=1\) when individual \(i\) is in B and 0 otherwise, and define \(I_{Ci}\) in the same way for C. The quantitative-trait model is

Y_i=\beta_0+\beta_G G_i+\gamma_B I_{Bi}+\gamma_C I_{Ci}+\varepsilon_i.

Here \(\beta_0\) is the mean trait value for the reference category at genotype dosage zero, under the model. \(\beta_G\) is the adjusted difference in mean trait value for a one-allele increase in dosage, assumed constant across the categories in this model. The coefficients \(\gamma_B\) and \(\gamma_C\) are the adjusted mean differences for B versus A and C versus A, respectively, at the same genotype dosage. The model has no genotype-by-category interaction, so it assumes a common genotype slope across categories.

For a binary trait, the same indicators can enter a logistic regression:

\log\left(\frac{P(Y_i=1\mid G_i,C_i)}{1-P(Y_i=1\mid G_i,C_i)}\right) =\beta_0+\beta_G G_i+\gamma_B I_{Bi}+\gamma_C I_{Ci}.

In this model, \(\gamma_B\) is the conditional log odds ratio for B versus A at the same genotype dosage; exponentiating it gives the corresponding odds ratio. Likewise, \(\exp(\beta_G)\) is the conditional odds ratio for a one-allele dosage increase under the assumed common genotype slope. These are adjusted associations, not automatically causal effects.

Worked Example: Interpreting Category Coefficients for a Quantitative Trait

In a hypothetical association analysis, a quantitative trait is measured in units. Researchers fit the linear model above, coding three collection sites A, B, and C, with A as the reference. The fitted coefficients are \(\hat{\beta}_0=10.0\), \(\hat{\beta}_G=0.8\), \(\hat{\gamma}_B=1.2\), and \(\hat{\gamma}_C=-0.5\). Interpret the category coefficients and calculate predicted means at genotype dosage 1.

At a fixed genotype dosage, the fitted difference for B versus A is \(\hat{\gamma}_B=1.2\) trait units. The fitted difference for C versus A is \(\hat{\gamma}_C=-0.5\) trait units. These comparisons adjust for genotype dosage; they are not comparisons of the unadjusted site averages.

For an individual at site A with \(G=1\), both indicators are zero, so the fitted mean is

\hat{Y}_A=10.0+(0.8)(1)=10.8.

At site B, \(I_B=1\) and \(I_C=0\), giving \(\hat{Y}_B=10.0+(0.8)(1)+1.2=12.0\) units. At site C, \(I_B=0\) and \(I_C=1\), giving \(\hat{Y}_C=10.0+(0.8)(1)-0.5=10.3\) units. The B-minus-A difference is \(12.0-10.8=1.2\) units, and the C-minus-A difference is \(10.3-10.8=-0.5\) units, as required by the model.

Reference Choice and Other Comparisons

A coefficient under reference coding compares its category with the omitted level, not with every other category. To compare two non-reference categories, form a contrast of their coefficients. In the three-level model, the adjusted B-minus-C difference is \(\gamma_B-\gamma_C\). Its uncertainty depends on the estimated covariance between the two coefficient estimates, so subtracting the coefficients and then treating their standard errors as independent is generally incorrect.

More generally, write the \(K-1\) non-reference coefficients as a vector \(\boldsymbol{\gamma}\). A linear contrast is \(L\boldsymbol{\gamma}\), where \(L\) is a specified row vector. If the estimated covariance matrix of \(\hat{\boldsymbol{\gamma}}\) is \(\widehat{\mathrm{Var}}(\hat{\boldsymbol{\gamma}})\), then

\widehat{\mathrm{SE}}(L\hat{\boldsymbol{\gamma}}) =\sqrt{L\,\widehat{\mathrm{Var}}(\hat{\boldsymbol{\gamma}})\,L^{\mathsf T}}.

For the B-minus-C comparison, \(L=(1,-1)\), so its variance is \(\widehat{\mathrm{Var}}(\hat{\gamma}_B)+\widehat{\mathrm{Var}}(\hat{\gamma}_C)-2\widehat{\mathrm{Cov}}(\hat{\gamma}_B,\hat{\gamma}_C)\). This contrast is reported in trait units for a linear model, or as a log odds ratio (and, after exponentiation, an odds ratio) for a logistic model.

Changing the reference category changes which group comparisons appear as individual coefficients. For example, choosing B as reference makes A-versus-B and C-versus-B comparisons available directly. It does not change the fitted values or the model’s implied differences when the same categorical predictor is represented fully. The intercept and coefficient labels change because the comparisons have changed.

Worked Example: Comparing Two Non-Reference Categories

Continue with the hypothetical linear model, and suppose the estimated standard errors of \(\hat{\gamma}_B\) and \(\hat{\gamma}_C\) are 0.40 and 0.50, with estimated covariance 0.04. Calculate the estimated B-minus-C difference and its standard error.

The estimated contrast is \(\hat{\gamma}_B-\hat{\gamma}_C=1.2-(-0.5)=1.7\) trait units. Its estimated variance is

$$ 0.40^2+0.50^2-2(0.04)=0.16+0.25-0.08=0.33. $$

Thus its standard error is \(\sqrt{0.33}=0.5745\) trait units. The fitted difference says the trait mean at B is 1.7 units higher than at C at the same genotype dosage, under the fitted model. The standard error quantifies sampling uncertainty in that estimated contrast; it is not the standard deviation of individual trait values.

Testing Category Effects Jointly

A single coefficient test asks whether one category differs from the reference. A joint test instead asks whether the covariate contributes any category differences at all, after adjustment for genotype and other included predictors. With \(K\) levels and reference coding, the null hypothesis is that all \(K-1\) category coefficients are zero:

H_0:\gamma_1=\gamma_2=\cdots=\gamma_{K-1}=0.

For a large-sample Wald test, let \(\hat{\boldsymbol{\gamma}}\) be the coefficient vector and let \(\widehat{V}\) be its estimated covariance matrix. If the model and regularity conditions justify the large-sample approximation and \(\widehat{V}\) is nonsingular, the statistic

W=\hat{\boldsymbol{\gamma}}^{\mathsf T}\widehat{V}^{-1}\hat{\boldsymbol{\gamma}}

is approximately distributed as a chi-square random variable with \(K-1\) degrees of freedom under \(H_0\). The p-value is the probability, under that null model and approximation, of a statistic at least as large as the observed \(W\). A small p-value is evidence that at least one category effect differs from zero; it does not identify which category differs, establish a particular pairwise contrast, or show that category membership causes a trait difference.

Worked Example: Jointly Testing a Three-Level Covariate

In a hypothetical genetic association analysis, the adjusted estimates for two non-reference categories are \(\hat{\boldsymbol{\gamma}}=(1.2,-0.5)^{\mathsf T}\). Suppose their estimated covariance matrix is

\widehat{V}= \begin{pmatrix} 0.16 & 0.04\\ 0.04 & 0.25 \end{pmatrix}.

Test \(H_0:\gamma_B=\gamma_C=0\) using the large-sample Wald statistic. The determinant of \(\widehat{V}\) is \((0.16)(0.25)-(0.04)^2=0.0384\), so

\widehat{V}^{-1} =\frac{1}{0.0384} \begin{pmatrix} 0.25 & -0.04\\ -0.04 & 0.16 \end{pmatrix} \approx \begin{pmatrix} 6.5104 & -1.0417\\ -1.0417 & 4.1667 \end{pmatrix}.

Substitution gives

W= \begin{pmatrix}1.2&-0.5\end{pmatrix} \begin{pmatrix} 6.5104 & -1.0417\\ -1.0417 & 4.1667 \end{pmatrix} \begin{pmatrix}1.2\\-0.5\end{pmatrix} \approx 11.67.

There are \(K-1=2\) degrees of freedom. Under a chi-square distribution with 2 degrees of freedom, the upper-tail p-value is approximately 0.0029. This is evidence against the joint null of no adjusted category differences. It does not mean that both categories differ from A, or that the B-versus-C contrast is significant; those questions require their own estimates and tests.

Why the \(K-1\) Indicators Are Needed

With an intercept, including one indicator for every category creates exact redundancy: for each individual, the category indicators sum to one. The intercept column is therefore the sum of those indicator columns, so the model matrix cannot uniquely estimate all coefficients. Omitting one category removes this linear dependence and makes the intercept correspond to the reference level.

This explains the basic indicator count. A categorical covariate with \(K\) levels has \(K-1\) free category differences relative to a baseline. A joint test of all of those differences has \(K-1\) restrictions, provided every level is represented and the model is identifiable. If a level has no observations in the analyzed data, or the design makes a coefficient non-identifiable, the nominal coding does not create information that is absent from the data.

List the levels.
Specify the categories represented in the analysis and choose a meaningful reference level before interpreting coefficients.

Create indicators.
For each non-reference category, create a 0/1 indicator. Confirm that a person in the reference category has zero for every such indicator.

State the comparison.
Describe each coefficient relative to the reference, conditional on genotype dosage and the other model predictors.

Match the test to the question.
Use a single coefficient or contrast for a specified comparison; use a joint test to assess all category coefficients together.

Common Mistakes and Interpretation

Including all category indicators with an intercept. This creates exact collinearity. Omit a reference category or use a different full-rank parameterization.
Interpreting a category coefficient as an unadjusted difference. In an adjusted genetic association model, it is a conditional comparison at the same values of the other predictors included in that model.
Calling the reference category’s coefficient zero evidence of no effect. The reference is the comparison baseline; its zero indicator coefficient is a coding convention, not a separately estimated finding.
Assuming a joint test identifies the responsible category. A rejection says at least one tested category coefficient is nonzero under the model. It does not identify a pairwise difference.
Changing the reference and claiming the model changed. With a complete, equivalent parameterization, the numerical coefficient meanings change but the fitted comparisons do not.
Reading adjusted associations causally. Regression adjustment alone does not establish that the covariate or genotype caused the observed outcome difference.

For genetic association results, report the category definitions, reference level, model scale, and the comparison being summarized. For a logistic model, distinguish a log-odds coefficient from its exponentiated odds ratio. For a quantitative-trait model, give differences in the trait’s units. These details make clear what the adjustment means and prevent a reference-coded coefficient from being mistaken for a comparison it does not represent.

Key takeaway: A categorical covariate with \(K\) levels is represented by \(K-1\) indicators when the model includes an intercept. Each coefficient compares one category with the reference, while contrasts answer other specified comparisons and a joint test evaluates all category effects together.

Check Your Understanding

Answer each question using the model and comparison that it asks about.

A four-level covariate is included with an intercept. How many reference-coded indicators are needed, and how many degrees of freedom does a joint test of its category coefficients have?
In a logistic model, what does \(\exp(\gamma_B)\) represent when A is the reference category?
With A as reference, how would you express the adjusted B-versus-C contrast in a three-level model?
Why is it generally incorrect to calculate the standard error of \(\hat{\gamma}_B-\hat{\gamma}_C\) as if the estimates were independent?
A joint Wald test rejects the null that all category coefficients are zero. What can you conclude, and what can you not conclude from that test alone?