Loglinear Parameters and Identifiability

Why Constraints Are Needed

For a two-way table, a loglinear model describes each cell through its expected count. If every cell has a positive mean, its logarithm can be expressed as a grand mean plus row, column, and interaction contributions. This decomposition is useful because it separates overall level, row differences, column differences, and departures from an additive pattern.

The decomposition alone does not identify the individual parameters. A constant can be moved from one component to another while leaving every cell log mean unchanged. Constraints remove that ambiguity. Reference-category constraints and sum-to-zero constraints are two common choices. They give different parameter values, but, when used to represent the same cell log means, they give the same fitted means.

Here \(\lambda\) is the grand-mean parameter, \(\alpha_i\) is a row effect, \(\beta_j\) is a column effect, and \(\gamma_{ij}\) is an interaction parameter. Without constraints, these names do not give the parameters unique values. There are \(1+I+J+IJ\) parameters in the expression but only \(IJ\) cell log means, so the representation necessarily has redundancy.

Reference-Category Constraints

Choose row 1 and column 1 as reference categories. Set their row and column effects to zero and set interaction terms involving either reference category to zero. The remaining parameters are then determined by differences from the reference cell and reference categories.

These formulas apply at reference levels too: they give \(\alpha_1=0\), \(\beta_1=0\), and \(\gamma_{i1}=\gamma_{1j}=0\). To verify the decomposition, substitute the formulas: \(\lambda+\alpha_i+\beta_j+\gamma_{ij}=\eta_{11}+(\eta_{i1}-\eta_{11})+(\eta_{1j}-\eta_{11})+(\eta_{ij}-\eta_{i1}-\eta_{1j}+\eta_{11})=\eta_{ij}\).

Sum-to-Zero Constraints

A second convention makes each set of effects average to zero. This is sometimes called effect coding. The grand-mean parameter is the unweighted average of the cell log means; row and column effects are deviations of their corresponding averages from that grand average. The interaction then accounts for what remains after those averages are removed.

The constraints are \(\sum_i\alpha_i=0\), \(\sum_j\beta_j=0\), \(\sum_j\gamma_{ij}=0\) for every row \(i\), and \(\sum_i\gamma_{ij}=0\) for every column \(j\). For example, \(\sum_i\alpha_i=\sum_i\eta_{i\cdot}-I\eta_{\cdot\cdot}=0\), because the average of the row averages is \(\eta_{\cdot\cdot}\). The column-effect constraint follows in the same way. For each row, \(\sum_j\gamma_{ij}=\sum_j\eta_{ij}-J\eta_{i\cdot}-\sum_j\eta_{\cdot j}+J\eta_{\cdot\cdot}=0\); the column calculation is analogous.

Proof. First, adding the four formula components gives \(\eta_{\cdot\cdot}+(\eta_{i\cdot}-\eta_{\cdot\cdot})+(\eta_{\cdot j}-\eta_{\cdot\cdot})+(\eta_{ij}-\eta_{i\cdot}-\eta_{\cdot j}+\eta_{\cdot\cdot})=\eta_{ij}\). The constraints follow from the average identities just established.

For uniqueness, suppose a constrained decomposition represents the same \(\eta_{ij}\). Averaging it over columns gives \(\eta_{i\cdot}=\lambda+\alpha_i\), since the column effects sum to zero and the interaction terms in row \(i\) sum to zero. Averaging this identity over rows gives \(\eta_{\cdot\cdot}=\lambda\), because the row effects sum to zero. Thus \(\lambda\) and every \(\alpha_i=\eta_{i\cdot}-\eta_{\cdot\cdot}\) are determined. Averaging over rows instead gives \(\eta_{\cdot j}=\lambda+\beta_j\), determining every \(\beta_j\). Finally, \(\gamma_{ij}=\eta_{ij}-\lambda-\alpha_i-\beta_j\) is determined as well. No second constrained decomposition is possible. \(\square\)

What Is Identifiable and What Is Invariant?

A parameter is identifiable when distinct allowed parameter values cannot give the same probability model. Without constraints, the separate components \(\lambda,\alpha_i,\beta_j,\gamma_{ij}\) are not identifiable: they are coordinates with redundant freedom. For example, adding a constant to the grand mean and compensating in the interaction array leaves all \(\eta_{ij}\) unchanged. The reference and sum-to-zero constraints each select one unique coordinate representation for a given complete array of cell log means.

In either constrained representation, the cell log means \(\eta_{ij}\), and therefore the fitted means \(\mu_{ij}=\exp(\eta_{ij})\), are invariant. The numerical values called row effects, column effects, or cell interactions are not generally invariant across the two conventions. Their interpretations depend on the constraint: reference effects compare to a chosen baseline, whereas sum-to-zero effects are deviations from averages.

A useful exception is the two-factor difference-in-differences. For any two rows \(i,k\) and columns \(j,\ell\), the contrast \(\eta_{ij}-\eta_{i\ell}-\eta_{kj}+\eta_{k\ell}\) is unchanged by adding any row-only or column-only contributions. Under the reference constraints, the cell interaction is a difference-in-differences relative to row 1 and column 1. Under sum-to-zero constraints, \(\gamma_{ij}\) is a double-centered component; its individual value need not equal the reference interaction. The underlying difference-in-differences, however, is determined by the cell log means.

Common Mistake

A frequent error is to interpret a parameter as though its numerical value were independent of the constraint. For example, one might see a reference interaction equal to \(1\) and conclude that the corresponding sum-to-zero interaction must also equal \(1\). That is incorrect: these parameters are defined by different formulas. What must agree is the reconstructed cell log mean, or an explicitly defined invariant contrast. When reporting a row effect or interaction, state the constraint convention so its meaning is clear.

Constraints are not additional assumptions about the observed counts. For a full two-way decomposition they choose a unique representation of the same cell log means. Confusing a coordinate convention with a restriction on fitted means can lead to incorrect comparisons across analyses or software outputs.