Defining Candidate GLM Sets

Why Define the Candidate Set Before Fitting?

Model comparison is meaningful only after the candidate models have been stated clearly. A candidate set is not simply a list of formulas generated from whichever variables happen to be available. Each model encodes assumptions about the response distribution, the predictors and their functional form, the link between the mean and the linear predictor, any known offsets, and the interactions under consideration. Those choices should reflect the study design and scientific question, rather than being improvised after looking for a favorable result.

This tutorial concerns how to construct and describe that set. It does not decide which candidate is best. For now, the goal is to make each candidate precise enough that its implied probability model and scientific interpretation are unambiguous.

Specify the Components

For independent responses \(Y_1,\ldots,Y_n\), a common GLM specification gives each observation a distribution from a stated family, with mean \(\mu_i=E[Y_i]\) and a link relation \(\eta_i=g(\mu_i)\). A predictor structure with design matrix \(X\), coefficient vector \(\beta\), and known offset vector \(o\) has \(\eta=o+X\beta\). The offset enters with its coefficient fixed at one. Which distributions, links, and mean values are valid depends on the response and the family.

A clear candidate description answers the following questions.

• What is the response? State what is observed for each observational unit, its support, and the assumed family. For grouped binary data, for example, specify successes and trials rather than describing a proportion as if it were an unweighted binary outcome.
• Which predictors are included, and how are they represented? Identify transformations, factor contrasts, and any scientifically motivated grouping or functional form. A predictor name alone does not specify its coding or form.
• Which link is used? State the link and check that its domain and the resulting means are valid for the response family.
• Are there offsets? Name each known offset and explain why its coefficient is fixed at one. An offset is not an ordinary predictor whose coefficient is estimated.
• Which interactions are included? State the interacting variables and how the interaction enters the linear predictor. Do not leave “interaction model” undefined.
• What parameter restrictions apply? State any constraints, such as a restricted coefficient set, because they affect which predictor and mean vectors the model can attain.

These are model-definition decisions, not results of fitting. In particular, a candidate set should be recorded in terms of a scientific rationale and explicit specifications before its members are compared. A variable can be scientifically important even if it is not included in every candidate; conversely, adding terms solely because a search found them useful can make the set difficult to interpret.

Admissible Predictors and Attainable Means

The design matrix alone does not always tell the whole story about a candidate. Coefficients may be constrained, and some links or mean domains may rule out certain linear predictors. It is useful to define the sets involved explicitly.

This definition makes explicit why comparing column spaces without checking admissibility can be wrong. For instance, the identity link for a Bernoulli response requires every fitted mean to lie in \([0,1]\); not every vector in a design matrix’s column space will satisfy that requirement. The logit link, in contrast, maps every real predictor to a mean strictly between zero and one.

Proof. Because the link is one-to-one, its inverse is one-to-one on its image. Applying \(g^{-1}\) componentwise therefore maps distinct admissible predictor vectors to distinct mean vectors. If \(A_1\subseteq A_2\), applying this map gives \(\mathcal{M}_1\subseteq\mathcal{M}_2\). Conversely, suppose \(\mathcal{M}_1\subseteq\mathcal{M}_2\). For any \(\eta\in A_1\), its mean vector belongs to \(\mathcal{M}_1\), hence to \(\mathcal{M}_2\). There is an \(\eta'\in A_2\) with the same componentwise inverse-link image. Injectivity implies \(\eta=\eta'\), so \(\eta\in A_2\). Thus \(A_1\subseteq A_2\).

Under the additional conditions, every \(\beta\) is permitted and every real predictor vector is admissible. Thus \(A_j=o+\mathcal{C}(X_j)\) for \(j=1,2\). Since the offset is the same, \(A_1\subseteq A_2\) holds exactly when \(\mathcal{C}(X_1)\subseteq\mathcal{C}(X_2)\): subtracting \(o\) from both sets yields the column-space inclusion. The first equivalence now gives the claimed mean-vector result. \(\square\)

The additional assumptions matter. Without them, the column-space condition need not characterize attainable means. For example, take two Bernoulli observations, zero offsets, and the identity link. With \(X_1=(1,-1)^\mathsf{T}\), a predictor vector has the form \((\beta,-\beta)^\mathsf{T}\). Requiring both means to lie in \([0,1]\) forces \(\beta=0\), so this candidate attains only \((0,0)^\mathsf{T}\). With \(X_2=(0,0)^\mathsf{T}\), the only attainable mean vector is also \((0,0)^\mathsf{T}\). Yet \(\mathcal{C}(X_1)\) is not a subset of \(\mathcal{C}(X_2)\). Describing the admissible predictor sets avoids this false conclusion.

Equivalent Predictor Codings

A model’s formula can change without changing its candidate mean structure. For example, different full-rank contrast codings for a factor can describe the same predictor space. This is why candidate definitions should be understood through the implied predictors and constraints, not just by comparing the written coefficient names.

Proof. For each design matrix \(X_j\), unrestricted coefficients produce exactly the predictor vectors \(o+v\) for \(v\in\mathcal{C}(X_j)\). Equality of the column spaces therefore gives equality of the predictor sets. The common inverse link maps equal predictor sets to equal mean-vector sets. \(\square\)

For example, suppose \(X_2=X_1A\) for a nonsingular square matrix \(A\). Then \(\mathcal{C}(X_2)=\mathcal{C}(X_1A)=\mathcal{C}(X_1)\), since multiplication by \(A\) merely gives a one-to-one reparameterization of the coefficient vector. The resulting fitted means are unchanged as a set. This result requires the stated unrestricted coefficients and admissibility; restrictions introduced by a particular parameterization must also be considered.

State Interaction Structures Explicitly

An interaction allows the association for one predictor to vary across levels of another. It is a structural choice in the candidate definition. For two binary factors \(A\) and \(T\), treatment coding with indicators \(a,t\) gives the predictor \(\eta=\beta_0+\beta_Aa+\beta_Tt+\beta_{AT}at\). Without the interaction term, the predictor is additive on the link scale. Whether that is scientifically plausible is a question about the study and its mechanism; the interaction should not be an unexplained extra column in a design matrix.

Common Mistakes and a Practical Specification Check

• Treating a fitted coefficient vector as the candidate. A candidate is the full model class, including all permitted parameter values and restrictions. The estimates are one fitted instance.
• Leaving out the response denominator or exposure. A binomial sample size and a count-model exposure affect the mean specification. State them, and distinguish a known offset from an estimated coefficient.
• Assuming equal formula labels imply equal models. A term’s coding, transformation, link, and constraints can change the set of attainable means.
• Assuming column-space inclusion always proves nesting. It does so under appropriate common-link, common-offset, unrestricted-coefficient, and admissibility conditions. In general, compare the admissible predictor or attainable mean sets instead.
• Defining candidates after searching for a favorable result. A set assembled opportunistically can make the scientific interpretation and later assessment unreliable. Record the rationale and specifications before fitting and comparison.

Before proceeding, write each candidate in a compact registry: response and distribution; observational units; predictors and coding; link; offsets; interaction terms; and coefficient constraints. Check that every candidate refers to the intended observations and that its means are valid for the response. The resulting set is a transparent basis for later assessment, without prematurely deciding how those candidates should be ranked.