Likelihood-Ratio Tests for Selection-Model Constraints

Why Test a Selection-Model Constraint?

A selection model specifies a distribution for the outcome and a model for the response or observation process, with their parameters linked through the observed-data likelihood. Researchers may want to test a restriction on that response model: for example, whether a parameter describing dependence of response on the outcome is zero, or whether several response parameters take specified values. The question is whether imposing that restriction worsens the fit enough to be inconsistent with the data under the null model.

A likelihood-ratio test (LRT) makes this comparison using the same observed-data likelihood under two nested parameter spaces. It is especially useful when a restriction concerns several parameters jointly. Its familiar chi-square calibration, however, is an asymptotic result that requires regularity conditions. A large likelihood-ratio statistic is not automatically evidence from a valid chi-square test if the model is weakly identified, the maximum is on a boundary, or the likelihood has not been maximized reliably.

Definition: Let $\ell(\theta)$ be the observed-data log likelihood for a selection model, based on the observed data. The unrestricted model allows $\theta$ to range over a parameter space $\Theta$. A null hypothesis specifies a restricted parameter space $\Theta_0$ contained in $\Theta$. Write $\hat{\theta}$ for an unrestricted maximum likelihood estimate and $\hat{\theta}_0$ for a maximizer subject to $H_0$. The likelihood-ratio statistic is $$ D=2\{\ell(\hat{\theta})-\ell(\hat{\theta}_0)\}. $$

The alternative is the unrestricted model, $H_a:\theta\in\Theta$ but $\theta\notin\Theta_0$. Because the restricted space is contained in the unrestricted one, the maximized unrestricted log likelihood cannot be smaller than the restricted maximum, so $D\geq 0$. In practice both likelihoods must be computed for the same observed data and under the same likelihood specification; only the parameter restriction changes.

Setting Up the Constraint and the Test

A common formulation divides the parameter into $\theta=(\psi,\lambda)$. Here $\psi$ contains the selection-model parameters being tested, while $\lambda$ contains nuisance parameters, including any outcome-model parameters not constrained by the null. For a specified value $\psi_0$, the test is $H_0:\psi=\psi_0$ against $H_a:\psi\ne\psi_0$. Both models estimate $\lambda$: the unrestricted fit maximizes over $(\psi,\lambda)$, and the restricted fit maximizes over $\lambda$ with $\psi$ fixed at $\psi_0$.

If $q$ independent, smooth restrictions are imposed, the usual reference distribution is chi-square with $q$ degrees of freedom. The count is the number of independent restrictions, not the total number of parameters in either model. For instance, setting two free parameters to zero gives $q=2$, provided those are genuinely two independent constraints. More generally, $q$ is the reduction in dimension between the unrestricted and restricted parameter spaces locally at the null value.

Theorem: Under $H_0$, if the model is correctly specified and identifiable, the true parameter is an interior point of a smooth parameter space, the likelihood is sufficiently differentiable, and the Fisher information is finite and nonsingular, the unrestricted and restricted maxima are regular, and the null imposes $q$ independent smooth restrictions, then $$ D \ \xrightarrow{d}\ \chi^2_q $$ as the sample size increases. The approximate p-value is $P(\chi^2_q\geq D_{\mathrm{obs}})$, where $\chi^2_q$ denotes a chi-square random variable with $q$ degrees of freedom.

These conditions deserve particular attention in selection models. The observed-data likelihood can provide little information to distinguish an outcome distribution from a response mechanism. A likelihood ridge or nearly singular information matrix may therefore make a selection parameter weakly identified. The earlier material on weak identification and likelihood ridges is relevant to diagnosing this problem: a numerical estimate and a reported standard error alone do not establish that regular likelihood theory is a good approximation.

Why the Chi-Square Reference Appears

The justification is a local quadratic approximation to the log likelihood near the true parameter under the null. In a regular model, the score is approximately normal and the curvature is described by nonsingular information. After rescaling parameter directions so that the information matrix becomes the identity, the local log likelihood has the form of a squared-distance objective.

The unrestricted fit can move in all local parameter directions. The restricted fit is confined to the null surface. If the null imposes $q$ independent restrictions, that surface has $q$ fewer local directions. The improvement from allowing those $q$ directions is asymptotically the sum of squares of $q$ independent standard normal score components. A sum of $q$ squared independent standard normal variables has a $\chi^2_q$ distribution. This gives the stated limit for twice the log-likelihood difference.

This argument also explains the limits of the result: if the local quadratic approximation fails, the null surface is not smooth, a parameter is not identified under the null, or the information is singular, the ordinary chi-square conclusion need not hold. Nonstandard calibration is a separate issue; the chi-square result should not be applied merely because a statistic has the algebraic form $2(\ell_U-\ell_R)$.

Worked Example: Testing One Outcome-Dependence Parameter

Consider a hypothetical study with a selection-model response equation $\operatorname{logit}\{P(R=1\mid Y,X)\}=\alpha_0+\alpha_1X+\gamma Y$, where $R=1$ indicates that the outcome is observed. Suppose $Y$ is scaled in standard-deviation units. The scientific restriction is $H_0:\gamma=0$, against $H_a:\gamma\ne0$. The null has one fewer free parameter, so $q=1$. Assume both fits use the same observed-data likelihood and that optimization diagnostics show converged interior maxima, a locally nonsingular information matrix, and no indication of a ridge severe enough to undermine identification.

Suppose the hypothetical unrestricted maximum log likelihood is $\ell(\hat{\theta})=-184.20$, and the restricted maximum is $\ell(\hat{\theta}_0)=-186.60$. Then $$ D=2\{(-184.20)-(-186.60)\} =2(2.40)=4.80. $$

Under the stated regularity conditions, compare $D=4.80$ with $\chi^2_1$. The upper-tail probability is approximately $0.0285$. At significance level $\alpha=0.05$, reject $H_0$: the observed-data likelihood fit worsens enough under $\gamma=0$ to provide evidence against that restriction. In this model, $\gamma$ describes a conditional response association per one-standard-deviation increase in $Y$; the test does not give that coefficient a causal interpretation.

Worked Example: A Joint Test of Two Selection Parameters

In a second hypothetical analysis, let two parameters $\gamma_1$ and $\gamma_2$ describe outcome dependence in two response-model terms. Test $H_0:\gamma_1=0,\ \gamma_2=0$ against the unrestricted alternative. Assume the restrictions are independent, both parameters are identified under the null, and the unrestricted and restricted fits satisfy the regularity and optimization checks described above. The null imposes $q=2$ restrictions.

Suppose the unrestricted and restricted maximized log likelihoods are $-92.70$ and $-95.50$, respectively. Then $$ D=2\{(-92.70)-(-95.50)\} =2(2.80)=5.60. $$

For two degrees of freedom, the chi-square upper-tail probability is $P(\chi^2_2\geq5.60)=\exp(-5.60/2)\approx0.0608$. At $\alpha=0.05$, do not reject the joint null. This result is not a demonstration that both parameters are zero; it says that this test does not find sufficient evidence against their joint restriction at the chosen level. The joint test addresses the pair together and is not interchangeable with two separate one-parameter tests.

Worked Example: Testing a Specified Nonzero Value

Suppose a hypothetical analysis asks whether a selection parameter equals $0.50$, rather than whether it equals zero. Let $\gamma$ be measured on a specified outcome scale and test $H_0:\gamma=0.50$ against $H_a:\gamma\ne0.50$. Assume $0.50$ is an interior value, the model is identifiable there, and the constrained and unconstrained maxima are regular. This is one independent constraint, so $q=1$.

Suppose the unrestricted maximum log likelihood is $-118.40$, and the maximum with $\gamma$ fixed at $0.50$ is $-121.10$. The calculation is $$ D=2\{(-118.40)-(-121.10)\} =2(2.70)=5.40. $$

The $\chi^2_1$ upper-tail probability is approximately $0.0201$. At $\alpha=0.05$, reject the specified-value null, conditional on the regularity assumptions. The evidence is against $\gamma=0.50$; the test does not by itself establish which alternative value is correct.

Diagnostics, Alternatives, and Limitations

Before interpreting the test, check that the comparison actually represents nested fits. The restricted model must be obtainable from the unrestricted one by imposing the stated constraint, and both log likelihoods must refer to the same observations, response patterns, and likelihood definition. A restricted log likelihood larger than the unrestricted one is a warning of a calculation, convergence, or model-specification error, because a genuine unrestricted maximum must be at least as large.

Also examine whether the reported values are credible maxima rather than merely converged numerical solutions. Compare starting values or optimization runs, inspect gradients and curvature, and check that the constrained estimates satisfy the restrictions. Near-flat likelihood directions, extreme estimates, or an information matrix that is nearly singular are warnings that the usual asymptotic approximation may be unreliable. A small p-value does not repair a poor fit or establish that a selection parameter is well identified.

The LRT is invariant to smooth one-to-one reparameterizations when the same model spaces are represented, but the constraint itself must be stated on the intended parameter scale. It is also sensitive to the likelihood specification: a misspecified outcome or response model can make the formal reference distribution or its interpretation inappropriate. As with other likelihood-based procedures, the test is not a substitute for checking implied response probabilities and whether the model is scientifically plausible.

A Wald test uses an estimate and its estimated standard error to measure distance from the null. Near a likelihood ridge, that local standard-error calculation can be unstable, which is why the earlier material on Wald intervals failing near ridges matters here. The LRT instead compares maximized likelihoods, but it too can be unreliable when the likelihood is weakly identified or optimization is poor. A profile-likelihood analysis fixes the parameter of interest at a sequence of values and examines the resulting likelihood loss; it gives a fuller view of shape than a single LRT at one null value. These approaches answer related questions but are not numerically interchangeable in finite samples.

The ordinary chi-square calibration is an asymptotic reference, not a guarantee of accurate finite-sample p-values. In particular, do not use it automatically when a null constraint places a parameter on a boundary, when a parameter is unidentified under the null, or when regularity conditions otherwise fail. Such cases require separate calibration arguments rather than a routine change to the degrees of freedom.

A Common Mistake

Pitfall: “The null fixes two parameters, so the LRT has two degrees of freedom, and the chi-square test is valid.” The first part is correct only if the two restrictions are independent; the second does not follow from the parameter count. The chi-square reference also requires regularity, including local identifiability, an interior null value, suitable smoothness, and reliable maximization. Determine the dimension reduction and assess those conditions separately.

A second frequent error is reversing the log-likelihood subtraction. Since the unrestricted fit contains the restricted fit, calculate $2(\ell_U-\ell_R)$, not $2(\ell_R-\ell_U)$. The resulting statistic should be nonnegative, apart from negligible numerical rounding.

Check Your Understanding

Use the assumptions in each question when deciding whether the usual chi-square calibration applies.

A null fixes one selection parameter, and the unrestricted and restricted log likelihoods are $-40.1$ and $-42.0$. What is $D$, and what is its usual reference distribution?
A null jointly sets three independent, identified parameters to zero. How many degrees of freedom does the regular LRT use?
Why is it not enough to count two restrictions to justify a $\chi^2_2$ reference distribution?
Suppose the restricted log likelihood is larger than the unrestricted log likelihood. What should be checked before reporting a test?

Solutions. (1) $D=2\{(-40.1)-(-42.0)\}=2(1.9)=3.80$; under regularity it is compared with $\chi^2_1$. (2) Three degrees of freedom, if all three restrictions are independent and the other regularity conditions hold. (3) The chi-square result additionally requires regularity, including a smooth interior null, local identifiability, suitable information, and reliable maxima. (4) Check that the models are nested and use the same data and likelihood, and inspect the constrained fit and numerical optimization; a true unrestricted maximum cannot have a smaller log likelihood than the restricted maximum.