Tutorials › Generalized Linear Models › Recognizing Nested GLM Models

Home  /  Tutorials  /  Nested Models and Deviance Tests

Nested Models and Deviance Tests · Tutorial 306 of 981

Recognizing Nested GLM Models

Determine whether one GLM is a restriction of another by checking whether its possible linear predictors lie within the larger model’s predictor space.

Intermediate 10 min read

What You'll Learn

  • Define nesting through inclusion of the models’ possible linear-predictor vectors.
  • Check nesting using the column spaces of the design matrices.
  • Translate column-space inclusion into a coefficient mapping.
  • Distinguish nested models from models that merely use different-looking formulas.
  • Recognize how coding choices and the observed covariate design affect the comparison.

What It Means for One GLM to Be Nested in Another

Before comparing two GLMs, identify what each model permits the fitted linear predictors to be. A model with more terms is often more flexible, but formula length alone is not a reliable test: different codings can describe the same model, and two different-looking models can permit overlapping but incomparable sets of fitted values. The key is to compare their linear-predictor spaces on the observations being analyzed.

Suppose there are \(n\) observations, the same response and link are used in both models, and each model has an unrestricted coefficient vector. For a design matrix \(X\) and coefficient vector \(\beta\), write the vector of linear predictors as \(\eta=o+X\beta\), where \(o\) is a fixed offset vector. The column space \(\mathcal C(X)\) is the set of all linear combinations of the columns of \(X\).

Definition: The reduced model is nested in the full model if every linear-predictor vector allowed by the reduced model is also allowed by the full model. With the same offset \(o\), the two sets of predictors are \(o+\mathcal C(X_R)\) and \(o+\mathcal C(X_F)\); thus nesting means \(\mathcal C(X_R)\subseteq\mathcal C(X_F)\). The reduced model is then a restriction of the full model.

This definition concerns the predictor vectors at the observed covariate values. With a common link and response family, the same predictor vector gives the same fitted mean vector, so inclusion of predictor sets also gives inclusion of the corresponding fitted-mean sets. The definition does not require the coefficient vectors to have the same length or a unique representation: design matrices can have linearly dependent columns.

The Column-Space Test

The most useful practical test is to ask whether each column of the reduced design matrix can be formed as a linear combination of columns of the full design matrix. Equivalently, ask whether there is a matrix \(A\) satisfying \(X_R=X_FA\). This tests the predictor spaces directly, rather than comparing the names or order of terms in two formulas.

Theorem: For unrestricted coefficients and a common offset, \(\mathcal C(X_R)\subseteq\mathcal C(X_F)\) if and only if there exists a matrix \(A\) such that \(X_R=X_FA\). When this holds, every reduced-model coefficient vector \(\beta_R\) has a full-model representation \(\beta_F=A\beta_R\).

Proof. Suppose first that \(\mathcal C(X_R)\subseteq\mathcal C(X_F)\). Each column of \(X_R\) is in \(\mathcal C(X_F)\), so for each column there is a vector of coefficients that expresses it as a linear combination of the columns of \(X_F\). Put those coefficient vectors side by side to form \(A\). Then \(X_R=X_FA\). Conversely, if \(X_R=X_FA\), then for every \(\beta_R\),

$$ X_R\beta_R=X_FA\beta_R\in\mathcal C(X_F). $$

Thus every reduced-model predictor \(o+X_R\beta_R\) is also a full-model predictor, with \(\beta_F=A\beta_R\). This proves both directions. \(\square\)

If the offsets differ, comparing column spaces alone is not enough. The predictor sets are affine spaces, \(o_R+\mathcal C(X_R)\) and \(o_F+\mathcal C(X_F)\). Inclusion holds precisely when \(o_R-o_F\in\mathcal C(X_F)\) and \(\mathcal C(X_R)\subseteq\mathcal C(X_F)\). In routine comparisons, first check that the offsets are the same.

Worked Example: An Added Covariate

Consider three observations, with reduced-model predictors \(\eta_R=\beta_0+\beta_1x\), and full-model predictors \(\eta_F=\gamma_0+\gamma_1x+\gamma_2z\). Let the observed covariates be \(x=(0,1,2)^T\) and \(z=(1,0,1)^T\). The design matrices are

$$ X_R= \begin{pmatrix} 1&0\\ 1&1\\ 1&2 \end{pmatrix}, \qquad X_F= \begin{pmatrix} 1&0&1\\ 1&1&0\\ 1&2&1 \end{pmatrix}. $$

The first two columns of \(X_F\) are exactly the columns of \(X_R\). In particular,

$$ X_R=X_F \begin{pmatrix} 1&0\\ 0&1\\ 0&0 \end{pmatrix}. $$

Therefore the reduced model is nested in the full model. For any \((\beta_0,\beta_1)\), choosing \((\gamma_0,\gamma_1,\gamma_2)=(\beta_0,\beta_1,0)\) gives the same predictor vector. The full model also allows \(\gamma_2\ne0\), so it can represent additional predictor vectors. This conclusion is about inclusion; no comparison of test statistics is needed to recognize nesting.

Different Parameterizations Can Describe the Same Model

Model formulas are not unique descriptions of predictor spaces. For example, a categorical predictor with three levels can be represented using an intercept and two indicator columns, or using three level-specific means. These are different coordinate systems for the same set of possible predictors. The relevant comparison is whether the spans agree or one span is contained in the other.

Result: Replacing a design matrix \(X\) by \(XB\), where \(B\) is square and invertible, does not change its column space. Consequently, it does not change the GLM’s set of linear predictors when the offset is fixed.

Proof. Each column of \(XB\) is a linear combination of columns of \(X\), so \(\mathcal C(XB)\subseteq\mathcal C(X)\). Since \(B\) is invertible, \(X=(XB)B^{-1}\), giving \(\mathcal C(X)\subseteq\mathcal C(XB)\). The spaces are equal. \(\square\)

Worked Example: Two Codings of a Three-Level Factor

Suppose the observed factor levels, in order, are A, B, and C. A cell-means design matrix and a treatment-coded design matrix with A as the reference level are

$$ X_{\text{means}}= \begin{pmatrix} 1&0&0\\ 0&1&0\\ 0&0&1 \end{pmatrix}, \qquad X_{\text{treatment}}= \begin{pmatrix} 1&0&0\\ 1&1&0\\ 1&0&1 \end{pmatrix}. $$

The columns of the treatment-coded matrix are the cell-means columns combined as follows:

$$ X_{\text{treatment}}=X_{\text{means}} \begin{pmatrix} 1&0&0\\ 1&1&0\\ 1&0&1 \end{pmatrix}. $$

The coefficient matrix here is invertible: its determinant is 1. Therefore the two matrices have the same column space. More explicitly, the cell-means coefficients \((\mu_A,\mu_B,\mu_C)\) correspond to treatment-coded coefficients \((\alpha,\delta_B,\delta_C)=(\mu_A,\mu_B-\mu_A,\mu_C-\mu_A)\). Every set of three group means is available under either coding. These are equivalent models, not a reduced and a full model.

Examples of Inclusion and Non-Inclusion

Worked Example: A Common Mean as a Restriction of Group Means

Use the three factor levels A, B, and C with one observation per level. A common-mean model has design matrix \(X_R=(1,1,1)^T\); a model with a separate mean for each level has \(X_F=I_3\), the \(3\) by \(3\) identity matrix. The common-mean column is in the column space of \(I_3\), because

$$ X_R=X_F \begin{pmatrix} 1\\ 1\\ 1 \end{pmatrix}. $$

For any common mean \(\beta\), the full model gives the same predictor vector by taking its three level means to be \((\beta,\beta,\beta)\). Hence the common-mean model is nested in the separate-means model. The reverse inclusion does not hold: for example, the vector \((0,1,0)^T\) is allowed by the separate-means model but is not a constant vector. Thus the models are not equivalent.

Worked Example: Two Models That Are Not Nested

Suppose the observed \(x\)-values are \(-1,0,1\). Compare a model with predictor \(\eta=a+bx\) to one with predictor \(\eta=c+dx^2\). Their design matrices are

$$ X_A= \begin{pmatrix} 1&-1\\ 1&0\\ 1&1 \end{pmatrix}, \qquad X_B= \begin{pmatrix} 1&1\\ 1&0\\ 1&1 \end{pmatrix}. $$

To check whether model B is nested in model A, ask whether the \(x^2\) column \((1,0,1)^T\) can be written as \(a(1,1,1)^T+b(-1,0,1)^T\). The first and third entries would require \(a-b=1\) and \(a+b=1\), giving \(a=1\) and \(b=0\). The middle entry would then be \(1\), but it must be \(0\), a contradiction. So model B is not nested in model A.

For the reverse direction, ask whether \((-1,0,1)^T\) is \(c(1,1,1)^T+d(1,0,1)^T\). The first and third entries would require \(c+d=-1\) and \(c+d=1\), which cannot both hold. So model A is not nested in model B either. The two models are nonnested on this design.

Practical Checks and Common Pitfalls

A systematic check helps separate genuine inclusion from superficial similarity. With a common offset, the column-space test can be carried out by solving \(X_F A=X_R\), one reduced-model column at a time. A solution for every column establishes nesting; a column with no solution disproves it. For small examples, the required linear combinations can often be checked directly.

1
Specify what is being compared.
Confirm that the models refer to the same observations and identify their offsets and design matrices.
2
Write the predictor spaces.
For unrestricted coefficients, each space is \(o+\mathcal C(X)\).
3
Test inclusion.
With a common offset, check whether every reduced-model column is in the full-model column space, or solve \(X_R=X_FA\).
4
Interpret the result.
If inclusion holds, each reduced predictor can be represented by the full model. If neither space contains the other, the models are nonnested.

Common mistake: “The model with more named terms must contain the other.” This reasoning can fail because terms may be redundant on the observed design, because two codings may span exactly the same space, or because the terms may represent different directions that do not contain one another. The correction is to check column-space inclusion, not count terms or compare formula strings. Likewise, a term’s effect may be unidentifiable from the observed covariate pattern; in that case its design column may add no new direction to the space.

The design-specific qualification matters. The examples above establish inclusion or non-inclusion at the covariate values actually observed. If the scientific claim is that one predictor formula is contained in another for every possible covariate value, that is a stronger function-level claim and must be checked over the full covariate domain, not merely at the observed rows.

Key takeaway: For GLMs with the same link and offset and unrestricted coefficients, recognize nesting by comparing the design matrices’ column spaces. A reduced model is a restriction precisely when every predictor vector it allows is also available in the full model.

Check Your Understanding

Use the predictor-space definition to answer these questions.

  1. For a common offset, what matrix equation provides a direct test that a reduced model is nested in a full model?
  2. Two design matrices differ by an invertible change of basis. What does this imply about their model spaces?
  3. Why does a formula containing more named terms not necessarily define a model that contains another formula’s model?
  4. In a model with a common mean for three groups and a model with a separate mean for each group, which model is nested in the other?
  5. Why can nesting on the observed design differ from a claim of inclusion for every possible covariate value?

Previous

Next

Back to Tutorials
Clinical Biostats