Boundary Constraints and Nonstandard Likelihood Limits

Why a Boundary Changes the Likelihood-Ratio Limit

Likelihood-ratio tests are often calibrated using Wilks’ theorem: under regularity conditions, the likelihood-ratio statistic converges to a chi-square distribution whose degrees of freedom equal the number of restrictions. A key condition is that the null parameter lies in the interior of the parameter space. Variance parameters, probabilities constrained to an interval, and other restricted parameters can instead put the null on the boundary. Then the fitted parameter cannot move in every local direction, and the usual chi-square calibration can fail.

A basic example is a variance component $\tau$, which must satisfy $\tau\geq 0$. Testing $H_0:\tau=0$ asks whether the component is absent. Under the null, the parameter is exactly at the lower edge of its allowed range: estimates can move upward from zero, but not downward. The resulting likelihood-ratio statistic has a nonstandard limit even when the likelihood is smooth and the information is positive.

Definition: A parameter value is on the boundary of its parameter space if it lies at the edge of the allowed values, rather than in its interior. A chi-bar-square distribution is a probability mixture of chi-square distributions, possibly including $\chi^2_0$, which is a point mass at zero. The mixture weights depend on the local constraint geometry and, in general, the information structure.

The One-Boundary Limit

First consider one scalar parameter $\tau\geq 0$, with null value $\tau=0$. Suppose the likelihood is smooth near the null, the parameter is locally identified with positive information, and the profile log likelihood has a valid local quadratic approximation. Under these conditions, rescale local departures from zero as $h=\sqrt{nI}\,\tau$, where $I>0$ is the per-observation information for the profiled parameter. After profiling any regular nuisance parameters, the local log-likelihood difference has the form

\ell_n(h)-\ell_n(0)=hZ_n-\frac{1}{2}h^2+o_p(1), \qquad h\geq 0,

where $Z_n$ converges in distribution to $Z\sim N(0,1)$. The restriction $h\geq 0$ is essential. Without it, the quadratic is maximized at $h=Z_n$, allowing movement in both directions. With the boundary constraint, its maximizer is $h=\max(Z_n,0)$. Therefore the likelihood-ratio statistic $LR_n=2\{\sup_{\tau\geq 0}\ell_n(\tau)-\ell_n(0)\}$ satisfies

LR_n \ \overset{d}{\longrightarrow}\ \{\max(Z,0)\}^2.

A standard normal variable is positive with probability $1/2$ and negative with probability $1/2$. If it is negative, the constrained maximum stays at zero, so the limiting statistic is zero. If it is positive, its square has a $\chi^2_1$ distribution. Thus the limit is an equal mixture: half its probability is a point mass at zero, and half follows a one-degree-of-freedom chi-square distribution.

Theorem: For a single regular scalar parameter constrained to a half-line, with the null at its boundary and other parameters regular and interior, the likelihood-ratio statistic for testing the boundary value against the allowed one-sided alternative has the limit $$ \frac{1}{2}\chi^2_0+\frac{1}{2}\chi^2_1. $$ Here $\chi^2_0$ denotes a point mass at zero. If nuisance parameters are profiled, the local score and information must be those adjusted for the nuisance parameters; the same mixture applies when the resulting one-dimensional boundary problem satisfies the stated regularity conditions.

This is a chi-bar-square limit. The mixture is not an arbitrary adjustment: it records how often the unconstrained local estimate points outside the feasible region. With several boundary restrictions, the limit can involve more chi-square components and different weights. The equal-weight mixture above is specific to a single boundary direction under the stated conditions.

Worked Example: A Negative Local Score Meets the Boundary

Suppose a hypothetical profile log likelihood, in local units, is $\ell(h)-\ell(0)=hZ-h^2/2$ for $h\geq 0$. For one realized sample, let the standardized local score be $Z=-0.8$. Find the constrained maximum and the likelihood-ratio statistic.

The derivative with respect to $h$ is $Z-h=-0.8-h$, which is negative for every $h\geq 0$. The log likelihood therefore decreases as $h$ moves upward from zero, and its maximum over the allowed values is at $h=0$. Consequently,

\widehat{h}=0,\qquad LR=2\{0-0\}=0.

The unconstrained maximizer would be $h=-0.8$, but that value is inadmissible. This is the mechanism that creates the point mass at zero in the limiting distribution. For an observed statistic of zero, the upper-tail probability under the limiting mixture is one, because the distribution itself places probability $1/2$ at zero and the other half at positive values.

A Gaussian Variance-Component Derivation

Consider independent clusters indexed by $j=1,\ldots,n$, each with $m$ observations. In a simple random-intercept model, let $Y_j=b_j\mathbf{1}+\varepsilon_j$, where $b_j\sim N(0,\tau)$, $\varepsilon_j\sim N(0,\sigma^2 I_m)$, and the random effects and errors are mutually independent across clusters. Take $\sigma^2$ as known and positive, and test $H_0:\tau=0$ against $H_a:\tau>0$. The component variance $\tau$ is constrained to be nonnegative.

Project each cluster vector onto the normalized constant direction, defining $W_j=\mathbf{1}'Y_j/\sqrt{m}$. This projection has distribution $N(0,\sigma^2+m\tau)$. The remaining orthogonal projections have variance $\sigma^2$ and do not depend on $\tau$, so they cancel from the likelihood ratio. The test is thus equivalent to testing whether the variance of the independent $W_j$ values exceeds its known lower bound $\sigma^2$. This reduction supplies a direct likelihood calculation and exhibits the boundary explicitly.

Write $S_W=n^{-1}\sum_{j=1}^n W_j^2$. The likelihood for the $W_j$ values, apart from constants, is $\ell(v)=-\tfrac{n}{2}\log(v)-\tfrac{1}{2v}\sum_j W_j^2$, where $v=\sigma^2+m\tau\geq\sigma^2$. Its unconstrained maximum is at $v=S_W$, so the constrained maximum is $\widehat{v}=\max(S_W,\sigma^2)$. When $S_W>\sigma^2$, substituting the constrained and null estimates into the log likelihood gives

LR =2\{\ell(S_W)-\ell(\sigma^2)\} =n\left\{\frac{S_W}{\sigma^2}-1-\log\left(\frac{S_W}{\sigma^2}\right)\right\}.

When $S_W\leq\sigma^2$, the constrained maximum is already at the null value and $LR=0$. Under $H_0$, the local score for $\tau$ has mean zero and, after standardization, converges to a standard normal variable; the nonnegative constraint therefore yields the half-$\chi^2_0$, half-$\chi^2_1$ limit derived above.

Worked Example: A Hypothetical Variance-Component Sample

Suppose there are $n=100$ hypothetical clusters, each of size $m=4$, and the known residual variance is $\sigma^2=1$ in squared measurement units. Suppose the projected observations $W_j$ consist of 50 values equal to $+\sqrt{1.2}$ and 50 equal to $-\sqrt{1.2}$, in the corresponding measurement units. Then

S_W=\frac{50(1.2)+50(1.2)}{100}=1.2.

Because $S_W=1.2>1$, the constrained variance estimate is $\widehat v=1.2$, corresponding to $\widehat\tau=(1.2-1)/4=0.05$ squared measurement units. The likelihood-ratio statistic is

LR=100\{1.2-1-\log(1.2)\} =100(0.2-0.1823216) =1.7678.

This statistic is positive, so its asymptotic upper-tail probability under the boundary mixture is $\tfrac12 P(\chi^2_1\geq 1.7678)$. Equivalently, since $\chi^2_1$ is the square of a standard normal variable,

p_{\mathrm{mix}} =\frac{1}{2}\,2\{1-\Phi(\sqrt{1.7678})\} =1-\Phi(1.3296) \approx 0.0918.

Using this asymptotic calibration, the data do not reject the zero variance-component null at the 5% significance level. This conclusion concerns evidence against $\tau=0$; it does not establish that the component variance is exactly zero.

Calibration and the Wilks Pitfall

For a positive observed statistic $t$, the one-boundary mixture has upper-tail probability $\tfrac12 P(\chi^2_1\geq t)$. At $t=0$, its upper-tail probability is one. This convention matters because the limiting distribution has a discrete atom at zero as well as a continuous part. At significance level $\alpha$, a positive critical value $c_\alpha$ satisfies

\frac{1}{2}P(\chi^2_1>c_\alpha)=\alpha, \qquad c_\alpha=F^{-1}_{\chi^2_1}(1-2\alpha),

where $F^{-1}_{\chi^2_1}$ is the chi-square quantile function. For $\alpha=0.05$, $c_\alpha\approx 2.7055$. Ordinary Wilks calibration would instead use the 95th percentile of $\chi^2_1$, approximately $3.841$, which is not the appropriate boundary critical value.

Worked Example: Choosing the Boundary Critical Value

For the variance-component statistic $LR=1.7678$ above, the mixture p-value is approximately $0.0918$. The same decision can be made with the 5% critical value: reject only if $LR>2.7055$. Since $1.7678<2.7055$, the test does not reject at 5%.

If one incorrectly used the ordinary $\chi^2_1$ calibration, the p-value would be $P(\chi^2_1\geq1.7678)\approx0.1836$, twice the boundary-mixture p-value for this positive statistic. The ordinary 5% cutoff of $3.841$ is also larger than the correct boundary cutoff. Both discrepancies arise from ignoring the probability mass at zero in the limiting distribution.

Common mistake: Counting one restriction and automatically comparing the likelihood-ratio statistic with $\chi^2_1$, as if the null were an interior point. The correction is to inspect the parameter space before applying Wilks’ theorem. If the null is at a boundary, determine the appropriate constrained local geometry and its limiting mixture; for a single regular half-line boundary, use $\tfrac12\chi^2_0+\tfrac12\chi^2_1$.

The boundary result matters in applications because variance components are naturally nonnegative, and fitting them under a null of zero often places the estimate exactly at zero. A likelihood-ratio statistic can then equal zero for an appreciable fraction of samples even as the sample size grows. Applying an interior-point reference distribution overlooks this feature and can lead to incorrect p-values and rejection thresholds. The result is asymptotic: it describes the large-sample limit under the stated regularity conditions, not an exact finite-sample distribution.

Key takeaway: At a single regular boundary, the constrained local estimate is the positive part of an asymptotically normal unconstrained estimate. Squaring it produces a likelihood-ratio limit with a point mass at zero and a $\chi^2_1$ component, so ordinary Wilks calibration does not apply.

Check Your Understanding

Use the definitions and results above to answer the following questions.

Why does the constrained local maximizer equal $\max(Z_n,0)$, rather than $Z_n$?
What are the two components and their weights in the one-boundary likelihood-ratio limit?
In the Gaussian variance-component model, why do the orthogonal projections of each cluster cancel from the likelihood ratio?
For a positive observed likelihood-ratio statistic $t$, express its one-boundary asymptotic p-value using a $\chi^2_1$ upper-tail probability.
At a 5% significance level, why is the one-boundary critical value approximately $2.7055$ rather than $3.841$?

Boundary Constraints and Nonstandard Likelihood Limits

What You'll Learn