Using an Encompassing Model for Competing Hypotheses

Why Use an Encompassing Model?

Sometimes two scientific explanations lead to different regression models, but neither model is a special case of the other. For example, one hypothesis may predict an outcome from age, while another predicts it from a biomarker. A deviance difference between those two nonnested models is not, by itself, a likelihood-ratio test. A useful alternative is to ask whether both models can be represented as restrictions of a broader, scientifically defensible model.

That broader model is an encompassing model. Comparing each candidate with it gives two nested tests: one asks whether the restrictions defining the first candidate are compatible with the data, and the other asks the corresponding question about the second. These tests can assess the candidates’ restrictions, but they do not automatically identify which candidate is true or which is better.

Building the Common Model

For GLMs with the same response, distributional family, link, offset, and other fixed assumptions, a common construction is to combine the candidate linear-predictor terms. Suppose the candidates are \(g(\mu_i)=\alpha+\beta x_i\) and \(g(\mu_i)=\alpha+\gamma z_i\). An encompassing model is \(g(\mu_i)=\alpha+\beta x_i+\gamma z_i\). The first candidate is obtained by imposing \(\gamma=0\), and the second by imposing \(\beta=0\). Neither candidate has to contain the other.

The construction is valid only if the encompassing model is scientifically and statistically coherent. Combining columns of a design matrix is not enough if the candidate models use different response observations, likelihoods, or incompatible assumptions. Nor does simply placing two different link functions side by side produce a standard GLM that encompasses both. The common model must be specified before the tests are interpreted.

To verify the result, define the deviance relative to a common saturated-model log-likelihood \(\ell_{\mathrm{sat}}\) by \(D(M)=2\{\ell_{\mathrm{sat}}-\ell_M\}\), where \(\ell_M\) is the maximized log-likelihood under model \(M\). Subtracting gives \(D(A)-D(E)=2\{\ell_{\mathrm{sat}}-\ell_A-\ell_{\mathrm{sat}}+\ell_E\}=2(\ell_E-\ell_A)\). Because every parameter value allowed by \(A\) is also allowed by \(E\), maximizing over \(E\) cannot produce a smaller likelihood than maximizing over \(A\). Hence \(\ell_E\geq\ell_A\), proving the inequality. The same reasoning applies to \(B\). \(\square\)

What Each Deviance Test Asks

The comparison \(A\) versus \(E\) tests the null hypothesis that the restrictions defining \(A\) hold, against the larger model \(E\). The comparison \(B\) versus \(E\) tests the restrictions defining \(B\). When regularity conditions for Wilks’ theorem hold—among them an identifiable, finite-dimensional model, a true null parameter in the interior of the parameter space, and nonsingular information—the likelihood-ratio statistic has an asymptotic chi-square distribution under the null. Its degrees of freedom are the difference in model dimensions, or equivalently the number of independent restrictions in the regular case.

These are separate null hypotheses. Rejecting \(A\) says the data are not adequately described by the restrictions defining \(A\), relative to the specified \(E\), at the chosen calibration and significance level. Failure to reject \(B\) says the test did not find sufficient evidence against the restrictions defining \(B\); it does not establish that \(B\) is true. In particular, it does not follow that \(B\) is better than \(A\).

When the Encompassing Model Is a Sound Comparison

An encompassing model is useful when the scientific question can be expressed as a small number of restrictions within a common model. The restriction should correspond to a meaningful hypothesis, such as excluding a predictor or setting a block of coefficients to zero. The terms added to form \(E\) should be selected for that purpose, not merely because a larger model can always be written down.

A valid comparison requires a common likelihood for the same observed response data. The response definition, sampling units, likelihood family, and fixed modeling assumptions must be aligned. For example, a Poisson model and a binomial model do not become comparable likelihood-ratio models just because both contain a predictor named “age.” When candidate models differ in distribution or link, a common encompassing construction may require additional justification and may not exist as a regular GLM. In that situation, the ordinary nested deviance test is not licensed by the terminology “encompassing.”

The choice of \(E\) also shapes the conclusion. A candidate can fit poorly relative to an overly flexible or misspecified encompassing model for reasons that do not isolate the scientific restriction of interest. The interpretation is always conditional on the encompassing model and its assumptions. A non-significant test is not a goodness-of-fit guarantee, and it does not rule out departures the encompassing model cannot represent.

What the Tests Cannot Establish

The deviance tests against \(E\) do not directly compare the candidates’ fit to one another. The test statistics are \(2(\ell_E-\ell_A)\) and \(2(\ell_E-\ell_B)\), each measuring a candidate-to-encompassing improvement. Their difference is related algebraically to the candidate log-likelihoods, but the usual chi-square calibration applies to each nested null comparison, not to that difference as a test of \(A\) against \(B\).

Nor does “not rejected” mean “confirmed.” The test may have limited power, especially when data are sparse or the restriction is weakly identified. Interpretation also depends on the calibration being suitable for the model and data; boundary parameters, overdispersion, or small samples can invalidate a routine asymptotic reference distribution. The relevant question is therefore not merely whether a software output reports a small p-value, but which restriction was tested, against which encompassing model, and under what assumptions.

Common Pitfall