Why a Boundary Changes the Test
The generalized likelihood-ratio statistic compares the best fit under a null hypothesis with the best fit under a larger alternative. For nested models, it is commonly written \(G^2=2(\ell_1-\ell_0)\), where \(\ell_0\) and \(\ell_1\) are the maximized log-likelihoods under the null and alternative. Under the regularity conditions for Wilks’ theorem, its large-sample distribution is chi-square, with degrees of freedom equal to the difference in the number of free parameters.
A key condition is that the null parameter value is an interior point of the alternative parameter space. If the alternative restricts a parameter to be nonnegative and the null sets it to zero, zero is instead a boundary point. The usual chi-square result is then not guaranteed. This issue arises naturally for a variance component, which cannot be negative, or a mixture weight, which must lie between zero and one.
A boundary is not just a technical detail about how a parameter is written. It restricts which parameter changes are feasible near the null. The likelihood cannot improve by moving in a forbidden direction, so the fitted alternative may behave differently from an unconstrained fit. Consequently, counting the extra parameter and automatically using a chi-square distribution with one degree of freedom can give an unjustified test.
Why the Interior-Point Argument Breaks
The usual justification for the chi-square limit uses a local quadratic approximation to the log-likelihood around the true parameter. For an interior null value, small changes in both directions are allowed. In a one-parameter problem, a small positive change and a small negative change are both feasible. The maximizer can therefore respond to the random score in either direction.
At a lower boundary such as \(\tau^2=0\), only nonnegative changes are allowed. If the unconstrained likelihood would improve by moving to a negative value, that movement is prohibited; the constrained alternative can instead remain at zero. The shape of the feasible parameter space therefore affects the limiting behavior of the likelihood improvement. This is why the ordinary Wilks calibration cannot be assumed merely because the alternative adds one parameter.
The appropriate reference distribution depends on the model and its constraints. In some regular boundary problems it is a mixture involving a point mass at zero and a chi-square distribution; other complications, such as nuisance parameters that are not identified under the null, can change the analysis further. The essential exam-level decision is to identify the boundary and not invoke the ordinary chi-square result without justification.
Worked Example: A Nonnegative Variance Component
Worked Example: Testing Whether an Added Variance Is Needed
Suppose hypothetical standardized observations \(Y_1,\ldots,Y_5\) are independent and normally distributed with known mean zero. Under the null, \(Y_i\sim N(0,1)\). Under the alternative, \(Y_i\sim N(0,1+\tau^2)\), where \(\tau^2\geq0\). The observed sum of squares is \(Q=\sum_{i=1}^5Y_i^2=15\). Calculate the likelihood-ratio statistic and explain why its ordinary chi-square calibration is not automatic.
Method. Maximize the normal log-likelihood subject to the variance constraint, then compare the alternative maximum with the null likelihood. The null value \(\tau^2=0\) is the lower boundary of the permitted values.
For a candidate variance \(V=1+\tau^2\), the log-likelihood, apart from no omitted parameter-dependent terms, is
Differentiating with respect to \(V\) gives \(\ell'(V)=-5/(2V)+15/(2V^2)\). Setting this equal to zero gives \(15=5V\), so the unconstrained maximum is at \(V=3\). Since \(3\geq1\), it is permitted. Thus \(\widehat{\tau^2}=3-1=2\).
The null variance is \(V=1\). The log-likelihood difference is
Therefore,
The calculation uses the constrained alternative maximum and includes both the log-variance and quadratic terms in the likelihood. The estimate is on the alternative’s permitted side, but the null is still a boundary value. Thus \(G^2=4.5069\) is the statistic; its reference distribution must be justified for this boundary problem rather than selected automatically as \(\chi^2_1\).
Check. Substituting \(\widehat V=3\) gives the alternative quadratic term \(-15/6=-2.5\), while under the null it is \(-15/2=-7.5\). Their contribution to \(\ell_1-\ell_0\) is \(5\), matching the calculation above.
Worked Example: A Mixture Weight at Zero
Worked Example: Testing for a Second Component
Consider a hypothetical binary outcome \(X\), with \(X=1\) denoting a specified event. Under a two-component mixture, the probability of the event is \(q(\pi)=(1-\pi)(0.2)+\pi(0.8)\), where the component probabilities are fixed and \(0\leq\pi\leq1\). Test \(H_0:\pi=0\) against the mixture alternative using ten independent observations, of which five equal one. Calculate \(G^2\), and state the calibration issue.
Method. Maximize the binomial likelihood over the feasible mixture weights, then compare it with its value at \(\pi=0\). Here \(q(\pi)=0.2+0.6\pi\), so feasible weights correspond to \(0.2\leq q\leq0.8\).
The sample event proportion is \(5/10=0.5\), which lies in the permitted interval. The alternative maximum is therefore at \(\widehat q=0.5\). Solving \(0.5=0.2+0.6\widehat\pi\) gives \(\widehat\pi=0.3/0.6=0.5\). Under the null, \(\pi=0\) and \(q=0.2\).
The binomial coefficient is the same in both likelihoods and cancels in their ratio. Thus
The null value \(\pi=0\) is a boundary of the allowed interval. Therefore the usual \(\chi^2_1\) calibration does not follow simply from the fact that the alternative has one additional parameter. If the component distribution were itself indexed by an unknown parameter that has no meaning when \(\pi=0\), the null would also fail to identify that parameter, adding another reason not to apply the ordinary result automatically.
Check. The fitted event probability \(0.5\) is between the component probabilities \(0.2\) and \(0.8\), and it corresponds to the feasible weight \(0.5\). The log-likelihood improvement is positive, so the computed likelihood-ratio statistic is positive as required.
Worked Example: When the Constrained Fit Stays at the Boundary
Worked Example: A Variance Estimate Clipped at Zero
Suppose four hypothetical independent observations have known mean zero and, under the alternative, distribution \(Y_i\sim N(0,1+\tau^2)\) with \(\tau^2\geq0\). The observed values are \(-0.5,0.5,-0.5,0.5\). Find the constrained alternative estimate and likelihood-ratio statistic.
Method. Compute the sum of squares and compare its unconstrained variance estimate with the minimum allowed variance of one.
The sum of squares is \(Q=4(0.5^2)=4(0.25)=1\). For a normal model with known mean zero and candidate variance \(V\), the unconstrained maximum-likelihood estimate is \(Q/n\). Hence \(\widehat V_{\mathrm{unconstrained}}=1/4=0.25\). But the model requires \(V=1+\tau^2\geq1\), so the constrained estimate is \(\widehat V=1\), or \(\widehat{\tau^2}=0\).
At this constrained estimate, the alternative has exactly the same likelihood as the null, which fixes \(V=1\). It follows that
Check. The unconstrained estimate falls below the permitted variance, so the constrained maximum occurs at the smallest allowed value. The likelihood cannot improve over the null, and the statistic is zero.
Common Exam Traps
- Counting parameters and stopping. An extra parameter does not by itself establish the degrees of freedom for a chi-square reference distribution. First check whether the null value is interior to the alternative parameter space.
- Allowing an impossible estimate. If a variance component must be nonnegative, an unconstrained negative estimate is not the fitted alternative. Maximize over the permitted parameter space; the constrained estimate may be zero.
- Dropping likelihood terms. In the first example, using \(5\log 3\approx5.4931\) as the statistic omits the quadratic contributions from the fitted and null likelihoods. The correct calculation is \(G^2=10-5\log 3\approx4.5069\).
- Assuming every boundary problem has the same correction. A variance component and a mixture weight are both constrained, but other model features matter. In particular, a parameter that is unidentified under the null can change the limiting analysis.
- Confusing the statistic with its p-value. Computing \(G^2\) does not determine a p-value until a valid reference distribution has been chosen.
Check Your Understanding
For each problem, identify the relevant boundary issue as well as the requested calculation.
- An alternative model restricts a scale parameter to \(\lambda\geq0\), and the null sets \(\lambda=0\). Is the null an interior point or a boundary point of the alternative parameter space? (Answer: Boundary point.)
- For six independent normal observations with known mean zero, test variance \(1\) against variance \(1+\tau^2\), \(\tau^2\geq0\). The observed sum of squares is \(Q=12\). Find \(\widehat{\tau^2}\) and \(G^2\). (Answer: \(\widehat{\tau^2}=1\); \(G^2=6-6\log 2\approx1.8411\); ordinary chi-square calibration is not automatic.)
- A binary mixture has event probability \(q(\pi)=0.1+0.6\pi\), \(0\leq\pi\leq1\). In eight observations, four events occur. Find the alternative estimate \(\widehat\pi\) and the likelihood-ratio statistic against \(\pi=0\). (Answer: \(\widehat\pi=2/3\); \(G^2=8\log(25/9)\approx8.1732\); the null weight is on the boundary.)
- In a nonnegative variance-component test, the unrestricted estimate of the total variance is \(0.6\), while the null variance is \(1\) and the alternative requires total variance at least \(1\). Where is the constrained maximum, and what is the likelihood-ratio statistic? (Answer: At total variance \(1\), so the variance component estimate is zero and \(G^2=0\).)
- A nested alternative adds one nonnegative mixture weight, and the null sets it to zero. A candidate proposes a \(\chi^2_1\) reference distribution solely because one parameter was added. What is the flaw? (Answer: The null weight is on the boundary, so the ordinary Wilks chi-square calibration is not justified by parameter counting alone; the model’s boundary structure must be considered.)