Set Up Null Hypotheses for Fixed Effects

Why Hypothesis Setup Requires More Than Naming a Predictor

A scientific question such as “Does treatment matter?” does not yet specify a statistical hypothesis. A GLMM can represent treatment with one coefficient, several coefficients, or coefficients that interact with time or another predictor. The null hypothesis depends on which of these features the question targets. It also depends on the scale of the model: a coefficient in a conditional linear predictor is not generally a difference in marginal response probabilities.

This tutorial focuses on translating a scientific question into a precise restriction on the fixed effects of a specified conditional GLMM. It does not select a test statistic or a reference distribution. Those choices come after the parameter and its null value have been defined.

Definition: In a GLMM with conditional mean \(\mu_{ij}=E(Y_{ij}\mid b_j)\), a fixed-effect hypothesis is a claim about one or more components of the fixed-effects vector \(\beta\), on the scale of the conditional linear predictor. A linear hypothesis can be written \(H_0:C\beta=c\), where \(C\) is a specified matrix and \(c\) is a specified vector. Its alternative states which values outside that null restriction are of interest.

State the Conditional Model and the Target

For observations \(i\) in cluster \(j\), write a conditional GLMM as

g(\mu_{ij})=\mathbf{x}_{ij}^{\mathsf T}\beta+\mathbf{z}_{ij}^{\mathsf T}b_j, \qquad b_j\sim N(\mathbf{0},D).

Here \(g\) is the link function, \(\mathbf{x}_{ij}\) and \(\mathbf{z}_{ij}\) are the fixed- and random-effects design vectors, \(\beta\) is the fixed-effects vector, and \(b_j\) is a cluster-specific random-effects vector. The usual GLMM specification also states the conditional distribution of the outcomes given the random effects and the model covariates. For setting up a fixed-effect hypothesis, the crucial point is that \(\beta\) describes the conditional linear predictor, with the random-effects structure held as specified.

Before writing \(H_0\), identify the response, link, reference levels and contrasts, and the exact scientific comparison. State whether the question concerns a single coefficient, a comparison between fitted linear predictors, a slope difference, or a whole block of coefficients. If a direction such as “higher” is important, define what higher means for the response and justify a one-sided alternative before examining the results. Otherwise, a two-sided alternative is the usual formulation.

The hypothesis is conditional-model-scale because it restricts \(\beta\), not an average over the distribution of \(b_j\). With a nonlinear link, a zero coefficient can represent no conditional effect on the linear-predictor scale while response probabilities or means involve a nonlinear transformation. Do not silently substitute a marginal probability difference for a coefficient hypothesis.

Coefficients and Contrasts

A single-coefficient hypothesis has the form \(H_0:\beta_k=0\). This is appropriate only if \(\beta_k\), under the chosen coding and the other terms in the model, corresponds to the scientific question. A contrast compares specified linear combinations of coefficients. For \(a\) a known vector, the contrast hypothesis is \(H_0:a^{\mathsf T}\beta=c\), often with \(c=0\).

The reason contrasts work is direct: for two covariate settings that differ in the fixed-effects design vector by \(\Delta\mathbf{x}\), their conditional linear predictors differ by \(\Delta\mathbf{x}^{\mathsf T}\beta\) when their random-effects contribution is the same. Thus a question about that difference becomes a linear restriction on \(\beta\). For a logit link, for example, a difference in conditional linear predictors is a log odds ratio; exponentiating it gives the corresponding conditional odds ratio.

Worked Example: Comparing Two Treatment Arms

Suppose a hypothetical binary response records whether a patient improved. Patients are clustered within clinics. The conditional logit model includes a clinic random intercept \(u_j\) and has three treatment arms, with arm A as the reference:

\operatorname{logit}\{P(Y_{ij}=1\mid u_j)\} =\beta_0+\beta_B I(B)_{ij}+\beta_C I(C)_{ij}+u_j.

The scientific question is whether arm C has a higher conditional probability of improvement than arm B. Because the logit function is increasing, the conditional probability ordering agrees with the ordering of the corresponding linear predictors. At the same clinic random intercept, the predictor for C minus the predictor for B is

(\beta_0+\beta_C+u_j)-(\beta_0+\beta_B+u_j) =\beta_C-\beta_B.

Therefore the equality null and the scientifically directed alternative are

H_0:\beta_C-\beta_B=0, \qquad H_a:\beta_C-\beta_B>0.

Suppose, for illustration, the fitted coefficients are \(\hat{\beta}_B=0.31\) and \(\hat{\beta}_C=0.72\). The estimated contrast is \(0.72-0.31=0.41\) log odds. Exponentiating gives \(\exp(0.41)\approx1.507\), an estimated conditional odds ratio for C versus B, at the same random intercept. This arithmetic identifies the contrast and its scale; it does not establish that the alternative is supported. A one-sided alternative is appropriate only if the direction was specified in advance and the scientific question truly excludes effects in the other direction.

A coefficient’s meaning can change when reference levels or contrasts change, even though a properly specified scientific comparison can still be expressed through the appropriate linear combination. In particular, “the treatment coefficient is zero” may compare one treatment level with the reference level rather than compare two non-reference levels. Write the contrast itself when the question is not exactly the coefficient’s coded comparison.

Interactions and Joint Predictor Blocks

When a model includes an interaction, a main-effect coefficient is usually conditional on the interacting predictor’s reference or zero value. Consequently, a question about whether a treatment effect differs across occasions concerns the interaction coefficient, not simply the treatment main effect. Likewise, testing a multi-level categorical predictor as a whole generally requires a joint null for its several coefficients.

Worked Example: Testing Whether a Treatment Effect Changes Over Time

Consider a hypothetical binary outcome measured at baseline and follow-up in patients clustered within clinics. Let \(T=1\) denote treatment and \(P=1\) denote follow-up. Use the conditional model

\operatorname{logit}\{P(Y_{ij}=1\mid u_j)\} =\beta_0+\beta_TT_{ij}+\beta_PP_{ij} +\beta_{TP}T_{ij}P_{ij}+u_j.

At baseline (\(P=0\)), the conditional treatment effect on the log odds scale is \(\beta_T\). At follow-up (\(P=1\)), it is \(\beta_T+\beta_{TP}\). Their difference is

(\beta_T+\beta_{TP})-\beta_T=\beta_{TP}.

Thus the question “Does the conditional treatment effect differ between baseline and follow-up?” translates to

H_0:\beta_{TP}=0, \qquad H_a:\beta_{TP}\ne 0.

For illustration, if \(\hat{\beta}_T=0.25\) and \(\hat{\beta}_{TP}=-0.40\), the estimated baseline treatment effect is \(0.25\) log odds, while the follow-up effect is \(0.25-0.40=-0.15\) log odds. Their difference is \(-0.15-0.25=-0.40\), exactly the interaction coefficient. The negative estimate describes a smaller treatment effect on the conditional log odds scale at follow-up than at baseline; by itself, it does not answer whether the difference is statistically distinguishable from zero.

Worked Example: A Joint Null for a Four-Level Predictor

Suppose a hypothetical count outcome is modeled with a log link and a random intercept for clinic. A four-level predictor has level A as its reference, and no interaction involving this predictor is included:

\log(\mu_{ij})=\beta_0+\beta_2 I(B)_{ij} +\beta_3 I(C)_{ij}+\beta_4 I(D)_{ij} +\mathbf{w}_{ij}^{\mathsf T}\gamma+u_j.

The question is whether the predictor is associated with the conditional mean at all, comparing each non-reference level with A. The null must set all three corresponding coefficients to zero. In matrix form, with \(\beta_F=(\beta_2,\beta_3,\beta_4)^{\mathsf T}\), this is

H_0: \begin{pmatrix} 1&0&0\\ 0&1&0\\ 0&0&1 \end{pmatrix} \begin{pmatrix} \beta_2\\ \beta_3\\ \beta_4 \end{pmatrix} = \begin{pmatrix} 0\\ 0\\ 0 \end{pmatrix}, \qquad H_a:\text{ at least one of }\beta_2,\beta_3,\beta_4\ne0.

For illustrative fitted values \((\hat{\beta}_2,\hat{\beta}_3,\hat{\beta}_4)=(0.20,-0.10,0.40)\), the estimated vector of comparisons with A is \((0.20,-0.10,0.40)^{\mathsf T}\). Since the model uses a log link, these are conditional log mean ratios; exponentiating gives ratios \((\exp(0.20),\exp(-0.10),\exp(0.40))\approx(1.221,0.905,1.492)\). A joint hypothesis concerns whether all three population coefficients are zero, not whether any one estimate happens to equal zero.

Scope, Assumptions, and Checks Before Testing

A hypothesis is meaningful only relative to a model and a parameterization. The conditional mean and link must reflect the intended scientific estimand, the fixed-effects design must encode the intended comparisons, and the random-effects structure must be stated. The response distribution and dependence assumptions also matter for the validity of subsequent inference, though they do not change the algebraic definition of \(H_0\). Keep the specified random-effects structure the same when the question is solely about a fixed-effect restriction.

Before proceeding from a hypothesis to an inferential procedure, check the design matrix and the data support for the proposed restriction. Exact linear dependence can make separate coefficients unidentified; sparse outcome patterns can cause unstable or non-finite estimates, particularly in binary GLMMs. Such problems do not justify changing the scientific null after seeing results. They signal that the model or data may not provide reliable information for the intended comparison, and should be investigated before interpreting an inferential result.

Also check whether an interaction changes the meaning of a main effect. If a treatment-by-time interaction is in the model, \(\beta_T=0\) describes no treatment effect at the reference time, not necessarily no treatment effect at every time. A claim of no treatment effect at all modeled times may require a joint restriction on the treatment main effect and relevant interaction coefficients. The exact restrictions depend on which times and contrasts the scientific question covers.

The main alternatives differ in the question they ask, not merely in notation. A single coefficient targets one coded comparison; a contrast targets a specified linear combination; and a joint block asks whether all included components of a predictor contribution satisfy their null restrictions. A conditional coefficient hypothesis is also distinct from a marginal hypothesis about response means averaged over the random-effects distribution. With nonlinear links, those marginal quantities generally cannot be read off as the same coefficient or contrast.

Common pitfall: “The treatment effect is zero” is not precise enough when treatment has multiple levels or interacts with time. Setting only one coefficient to zero may answer a different question from the scientific one. Name the reference comparison or write the relevant contrast; for an overall no-effect claim, specify every coefficient restriction needed to cover the intended levels or conditions.

Key takeaway: First define the conditional-model estimand, then express it as \(C\beta=c\). State whether the alternative is two-sided, directional, or a joint departure, and verify that the chosen coefficients actually represent the scientific comparison.

Check Your Understanding

For each item, identify the fixed-effect restriction and interpret its scope. Solutions follow the questions.

In a logit GLMM with a binary treatment indicator \(T\), no interactions, and treatment coded 1 versus control coded 0, write the two-sided null for no conditional treatment effect.
A model has treatment levels A, B, and C with A as reference. What restriction compares C with B, and what is the contrast if \(\beta_B=0.15\) and \(\beta_C=0.50\)?
In a treatment-by-period model, explain what \(H_0:\beta_T=0\) means when baseline is the reference period, and give the null for no change in treatment effect between periods.
A five-level factor uses one reference level and has no interactions. How many coefficient restrictions are needed to state that the factor has no conditional effect relative to the reference?

Solutions. (1) \(H_0:\beta_T=0\) versus \(H_a:\beta_T\ne0\); this concerns the conditional linear predictor, and with a logit link is a conditional log odds comparison. (2) Use \(\beta_C-\beta_B=0\); the estimated contrast is \(0.50-0.15=0.35\) log odds. (3) \(\beta_T=0\) concerns the treatment effect at baseline only. No change in treatment effect is \(H_0:\beta_{TP}=0\), usually against a two-sided alternative. (4) Four restrictions are required, one coefficient for each non-reference level; the joint null sets all four to zero.