Why Use Effect Coding?
A nominal covariate can describe groups such as formulation, genotype, or study site. In an NLME model, its coding determines how category-specific population parameters are represented by fixed effects. Reference-cell coding expresses categories relative to one selected category. Effect coding instead expresses category differences around a grand mean on the model’s linear-predictor scale. This can be useful when no category is a natural reference, or when the scientific question concerns how each category departs from the overall category average.
The scale matters. If a parameter is modeled with an identity link, the grand mean and deviations are on the parameter scale. If it is modeled on the log scale, they are on the log-parameter scale: exponentiated contrasts then describe ratios. The mean in this coding scheme is an unweighted average across the defined categories, not an average weighted by how many individuals belong to each category.
Building the Coding and Interpreting the Coefficients
Let \(c_{kj}\) denote the entry for category \(k\) in coding column \(j\), and let \(\beta_j\) be its fixed-effect coefficient. For individual \(i\), suppose the population parameter \(\theta_i\) is modeled as
where \(k(i)\) is the individual’s category, \(g\) is the chosen parameter-scale transformation, and \(\eta_i\) is the individual random effect, conventionally modeled with mean zero. For interpreting the fixed category effects, set \(\eta_i=0\): this gives the category-specific typical value on the modeled scale. The model also requires the usual NLME assumptions for its random-effects and residual components; effect coding itself does not make those assumptions true.
Write \(\mu_k=\beta_0+\sum_j c_{kj}\beta_j\) for the category-specific linear predictor with the random effect set to zero. Averaging over the \(K\) categories gives
The last equality holds because each coding column sums to zero. Thus \(\beta_0\) is the unweighted grand mean of the category-specific linear predictors, and each category’s deviation from that mean is \(\sum_j c_{kj}\beta_j\). For the common coding scheme, the first \(K-1\) coefficients are the deviations for their corresponding categories; the final category’s deviation is the negative sum of those coefficients.
This statement applies on the linear-predictor scale. If \(g\) is nonlinear, transforming \(\beta_0\) does not generally give the arithmetic average of the category-specific parameters. For example, with a log link, \(\exp(\beta_0)\) is the geometric mean of the category-specific typical parameters, not their arithmetic mean.
Worked Example: Coding Three Nominal Categories
Suppose a hypothetical covariate has categories A, B, and C. Using the common effect-coding scheme, construct the coding matrix and verify the zero-sum condition.
| Category | \(c_1\) | \(c_2\) | Row sum |
|---|---|---|---|
| A | 1 | 0 | 1 |
| B | 0 | 1 | 1 |
| C | -1 | -1 | -2 |
| Column sums | 0 | 0 |
The relevant check is on the columns: \(1+0+(-1)=0\) for \(c_1\), and \(0+1+(-1)=0\) for \(c_2\). The row sums are 1, 1, and \(-2\), so it would be incorrect to require each row to sum to zero.
If the model is \(g(\theta_i)=\beta_0+c_1\beta_1+c_2\beta_2+\eta_i\), the category-specific linear predictors at \(\eta_i=0\) are A: \(\beta_0+\beta_1\), B: \(\beta_0+\beta_2\), and C: \(\beta_0-\beta_1-\beta_2\). Their average is \[ \frac{(\beta_0+\beta_1)+(\beta_0+\beta_2)+(\beta_0-\beta_1-\beta_2)}{3} =\beta_0. \] The category deviations, \(\beta_1\), \(\beta_2\), and \(-\beta_1-\beta_2\), sum to zero.
Deriving and Estimating Pairwise Contrasts
A pairwise contrast is a difference between two category-specific linear predictors. Subtracting their expressions cancels the intercept:
This is a linear combination of the fitted category coefficients. If the contrast is on a log-parameter scale, exponentiating it gives the ratio of the two category-specific typical parameters. On an identity scale, the contrast is a parameter difference in the parameter’s units.
For inference, let \(\widehat{\boldsymbol{\beta}}\) be the vector of fitted coefficients and \(V\) their estimated covariance matrix. If a contrast is \(L=\boldsymbol{\ell}^{T}\boldsymbol{\beta}\), its estimate is \(\widehat{L}=\boldsymbol{\ell}^{T}\widehat{\boldsymbol{\beta}}\), and its estimated variance is \(\boldsymbol{\ell}^{T}V\boldsymbol{\ell}\). A Wald confidence interval uses an appropriate normal or \(t\) critical value, subject to the model’s estimation and large-sample approximations. For an exponentiated log-scale contrast, first form the interval on the log scale, then exponentiate both endpoints.
Worked Example: Contrasts Among Three Categories
For categories A, B, and C, suppose fitted log-scale coefficients are \(\widehat{\beta}_1=0.10\) and \(\widehat{\beta}_2=0.04\), with \(\widehat{\beta}_0=\log(8)\). Assume the modeled parameter is clearance in L/h and the link is \(g(\theta)=\log(\theta)\). Find each category’s typical clearance and compare A with C.
The fitted log clearances are A: \(\log(8)+0.10\), B: \(\log(8)+0.04\), and C: \(\log(8)-0.10-0.04=\log(8)-0.14\). Exponentiating gives approximately 8.841 L/h for A, 8.327 L/h for B, and 6.955 L/h for C. Their average on the log scale is \(\log(8)\), as required; their arithmetic average on the clearance scale need not be 8 L/h.
The log-scale A-versus-C contrast is \[ \widehat{\mu}_A-\widehat{\mu}_C =2\widehat{\beta}_1+\widehat{\beta}_2 =2(0.10)+0.04=0.24. \] Thus the estimated ratio of typical clearances is \(\exp(0.24)=1.271\). In this hypothetical fit, category A’s typical clearance is estimated to be 1.271 times category C’s, or about 27.1% higher. The corresponding B-versus-C log contrast is \(\widehat{\beta}_1+2\widehat{\beta}_2=0.18\); the A-versus-B contrast is \(\widehat{\beta}_1-\widehat{\beta}_2=0.06\). These expressions follow by subtracting the category predictors, not by treating either effect-coded coefficient as a reference-category comparison.
Worked Example: Uncertainty for a Pairwise Contrast
Continue with the hypothetical fitted log-clearance model. Suppose the estimated covariance matrix for \((\widehat{\beta}_1,\widehat{\beta}_2)\) is \[ V=\begin{pmatrix}0.0025&-0.0005\\-0.0005&0.0016\end{pmatrix}. \] Calculate a Wald standard error and approximate 95% confidence interval for the A-versus-C log contrast, then express the interval as a ratio.
The contrast is \(L=2\beta_1+\beta_2\), so its coefficient vector is \(\boldsymbol{\ell}=(2,1)^T\). Its estimated variance is \[ \widehat{\operatorname{Var}}(\widehat{L}) =2^2(0.0025)+1^2(0.0016)+2(2)(1)(-0.0005) =0.0100+0.0016-0.0020=0.0096. \] Therefore \(SE(\widehat{L})=\sqrt{0.0096}=0.09798\) log units. Using the approximate normal critical value 1.96, the interval is \(0.24\pm1.96(0.09798)\), or approximately \([0.0480,\,0.432]\) log units.
Exponentiating the endpoints gives \([\exp(0.0480),\,\exp(0.432)]=[1.049,\,1.540]\). Under the fitted model and the Wald approximation, this 95% confidence interval for the typical A-to-C clearance ratio ranges from 1.049 to 1.540. It excludes 1, the value corresponding to equal typical clearances. The interval’s stated coverage is a repeated-sampling property of the procedure; it is not a 95% probability that this fixed ratio lies in this particular interval.
Assumptions, Diagnostics, and Alternatives
Effect coding is a parameterization, not a remedy for inadequate data or a misspecified model. Before interpreting its coefficients, check that the categories are defined consistently, that the model includes the intended categorical effect, and that the fitted model is identifiable. The \(K-1\) coding columns must be linearly independent; the common scheme above has rank \(K-1\). Each category must be represented well enough by the data to estimate its effect. Sparse or absent categories can make estimates unstable or non-identifiable, regardless of the coding convention.
Inspect the usual NLME diagnostics for the fitted model, including convergence, parameter uncertainty, and whether residual and random-effects assumptions are plausible. For the categorical effect, compare observed data support across categories and examine whether category-specific predictions or residual patterns suggest lack of fit. A very large standard error, strong coefficient correlation, or sensitivity to small modeling changes can signal weak information. Effect coding does not impose equal category effects, nor does it test whether the covariate matters.
The central alternatives are reference-cell coding and a cell-means parameterization. Reference-cell coding makes coefficients direct comparisons with one selected category; effect coding makes the intercept the unweighted mean linear predictor across categories. A cell-means model estimates a separate linear predictor for every category, after which contrasts can be calculated. With the same data, model, and fixed-effect structure, these are alternative parameterizations of the same category means and fitted values. They differ in coefficient interpretation, not in the comparisons the model can represent.
A common mistake is to check that each category row sums to zero. That is not the effect-coding condition: it is each coding column that must sum to zero over all category rows. Another mistake is to call \(\beta_0\) the sample-frequency-weighted average across individuals; it is the unweighted average of the category-specific linear predictors under this coding. Finally, do not interpret a log-scale coefficient as an additive change in clearance: exponentiation is needed to obtain a multiplicative comparison.
State the nominal levels and whether the parameter is modeled on its original, log, or another transformed scale.
Use \(K-1\) independent columns and confirm that each column sums to zero across the \(K\) categories.
Substitute each category’s coding row into the model, setting the random effect to zero when interpreting typical category values.
Subtract the relevant predictors, calculate the contrast’s standard error from the coefficient covariance matrix, and transform results back when needed.
Check Your Understanding
Answer each question, then compare with the brief solutions below.
- For four categories and three effect-coding columns, what must sum to zero: each row or each column?
- With three categories, the common coding gives category predictors \(\beta_0+\beta_1\), \(\beta_0+\beta_2\), and \(\beta_0-\beta_1-\beta_2\). What is the first-versus-third contrast?
- If a log-scale contrast is \(-0.20\), what is the estimated ratio of typical parameters for the first category relative to the second?
- Does effect coding make the intercept the category-frequency-weighted mean across individuals? Explain.
Solutions: (1) Each column sums to zero across the category rows; individual row sums need not be zero. (2) Subtraction gives \(2\beta_1+\beta_2\). (3) The ratio is \(\exp(-0.20)=0.819\), so the first category’s typical parameter is estimated at about 0.819 times the second’s. (4) No. The intercept is the unweighted mean of the category-specific linear predictors on the modeled scale, regardless of category frequencies.