Why Nonlinear Restrictions Need a Constrained Fit
A null hypothesis in a generalized linear model need not set a coefficient to zero or impose a linear contrast. It may instead assert a relationship such as \(\theta_2=\theta_1^2\), or that a parameter vector lies on a circle. The likelihood-ratio test handles these hypotheses by comparing the best likelihood available without the restriction to the best likelihood available while satisfying it. The key practical step is to maximize over the restricted set itself; evaluating the likelihood at an arbitrary point satisfying the null does not generally give the test statistic.
The likelihood-ratio principle is the same as for linear restrictions, but nonlinear restrictions require care in identifying the constrained maximum and in determining whether a standard chi-square approximation applies. For a regular smooth restriction, the number of independent constraints is determined locally by the rank of its derivative, not by how complicated the constraint looks algebraically.
Definition and Exact Test Statistic
For a regular null point \(\theta_0\), the restriction has \(q\) independent local constraints when the \(q\times p\) Jacobian matrix \(G(\theta_0)=\partial g(\theta_0)/\partial\theta^\mathsf{T}\) has rank \(q\). In particular, \(q\leq p\), and the null point is an interior point of the model’s parameter space. This full-rank condition ensures that, near \(\theta_0\), the null set is a smooth surface of dimension \(p-q\). A system of equations with more rows than its effective rank does not automatically impose that many independent restrictions.
Since \(\Theta_0\subseteq\Theta\), the unrestricted maximum cannot have a smaller log-likelihood than the restricted maximum. Thus \(\Lambda\geq0\). The comparison uses the same likelihood, including the same data and any common likelihood constants, in both fits.
Worked Example: A Gaussian Mean Restricted to a Circle
Suppose \(Y\sim N_2(\theta,I_2)\), where \(I_2\) is the two-dimensional identity matrix, and the observed vector is \(y=(3,4)^\mathsf{T}\). Test \(H_0:\theta_1^2+\theta_2^2=1\). The unrestricted maximum likelihood estimate is \(\hat{\theta}=y\), because the likelihood is proportional to \(\exp\{-\|y-\theta\|^2/2\}\).
Under the null, maximizing the likelihood is equivalent to minimizing the squared distance from \(y\) to the unit circle. For a point \(\theta\) on that circle, the Cauchy–Schwarz inequality gives \(y^\mathsf{T}\theta\leq\|y\|\|\theta\|=5\). Since \(\|y-\theta\|^2=\|y\|^2+\|\theta\|^2-2y^\mathsf{T}\theta=26-2y^\mathsf{T}\theta\), the distance is smallest when \(y^\mathsf{T}\theta=5\), at \(\hat{\theta}_0=(3/5,4/5)^\mathsf{T}\). The constrained squared distance is \((3-3/5)^2+(4-4/5)^2=(12/5)^2+(16/5)^2=144/25+256/25=16\).
The unrestricted squared distance is zero. The log-likelihood is a constant minus half the squared distance, so twice the log-likelihood difference equals the difference in squared distances: \(\Lambda=16-0=16\). The null is smooth at every point on the circle because the gradient of \(g(\theta)=\theta_1^2+\theta_2^2-1\) is \((2\theta_1,2\theta_2)\), which is nonzero on the circle.
Two More Exact Nonlinear Fits
Worked Example: A Scalar Parameter with Two Null Values
Let \(Y\sim N(\theta,1)\), with observed value \(y=2\). The unrestricted estimate is \(\hat{\theta}=2\). Test the nonlinear restriction \(H_0:\theta^2=1\), whose restricted parameter set consists of \(\theta=-1\) and \(\theta=1\).
The log-likelihood, up to a constant, is \(-\tfrac12(2-\theta)^2\). At \(\theta=1\), the squared residual is \((2-1)^2=1\); at \(\theta=-1\), it is \((2+1)^2=9\). Therefore the restricted maximum is attained at \(\hat{\theta}_0=1\), and \(\Lambda=2\{0-(-1/2)\}=1\). Equivalently, the unrestricted squared residual is zero and the restricted squared residual is one. At either null value the derivative of \(g(\theta)=\theta^2-1\) is \(2\theta\neq0\), so each null point satisfies the local rank condition.
Worked Example: Fitting a Gaussian Mean on a Parabola
Suppose \(Y\sim N_2(\theta,I_2)\), observe \(y=(0,1)^\mathsf{T}\), and test \(H_0:\theta_2=\theta_1^2\). Write a null value as \(\theta=(a,a^2)^\mathsf{T}\). The unrestricted estimate is \(y\), with squared residual zero. The constrained fit minimizes \(Q(a)=(0-a)^2+(1-a^2)^2\).
Set \(t=a^2\), so \(t\geq0\). Then \(Q(a)=t+(1-t)^2=t^2-t+1=(t-\tfrac12)^2+\tfrac34\). The minimum is \(3/4\), attained at \(a=\pm1/\sqrt{2}\); both constrained estimates have second coordinate \(1/2\). Consequently, \(\Lambda=3/4\). This is a finite-sample likelihood comparison, not a claim about a chi-square approximation. The constraint function \(g(\theta)=\theta_2-\theta_1^2\) has gradient \((-2\theta_1,1)\), which is nonzero everywhere, so the null curve is smooth.
Why the Regular Nonlinear Test Has \(q\) Degrees of Freedom
Under regularity conditions, the likelihood-ratio statistic has a chi-square limit under the null. The geometry explains the degrees of freedom: near the true null value, the nonlinear null surface is approximated by its tangent space. The unrestricted fit can improve the likelihood in all parameter directions, while the restricted fit can improve it only in tangent directions. The likelihood-ratio statistic measures the improvement in the \(q\) directions excluded by that tangent space.
Here local asymptotic normality means that, for bounded \(h\), the log-likelihood has the uniform local expansion \[ \ell(\theta_0+h/\sqrt n)-\ell(\theta_0) =h^\mathsf{T}\Delta_n-\tfrac12h^\mathsf{T}I(\theta_0)h+o_p(1), \qquad \Delta_n\xrightarrow{d}N_p(0,I(\theta_0)). \] The uniform remainder on bounded sets, together with the assumed root-\(n\) localization of both estimators, permits maximizing this quadratic approximation over the local unrestricted and restricted parameter sets.
Proof. Write \(I=I(\theta_0)\) and \(G=G(\theta_0)\). For a local parameter \(\theta=\theta_0+h/\sqrt n\), differentiability of \(g\) gives \[ g(\theta_0+h/\sqrt n)=g(\theta_0)+G h/\sqrt n+o(n^{-1/2}) \] for bounded \(h\). Since \(g(\theta_0)=0\), the limiting local null directions satisfy \(Gh=0\). Thus the local restricted maximization is over the tangent space \(\mathcal{T}=\{h: Gh=0\}\).
Change coordinates to \(x=I^{1/2}h\), \(z_n=I^{-1/2}\Delta_n\), and \(A=GI^{-1/2}\). The local log-likelihood increment becomes \(z_n^\mathsf{T}x-\tfrac12 x^\mathsf{T}x\), and the tangent constraint is \(Ax=0\). Completing the square yields \[ z_n^\mathsf{T}x-\tfrac12\|x\|^2 =\tfrac12\|z_n\|^2-\tfrac12\|x-z_n\|^2. \] The unrestricted maximum is attained at \(x=z_n\). The restricted maximum is attained at the Euclidean projection \(P_{\mathcal{T}_x}z_n\) onto \(\mathcal{T}_x=\{x:Ax=0\}\). The squared distance from \(z_n\) to this subspace is \(\|P_{\mathcal{T}_x^\perp}z_n\|^2\). Taking twice the difference between the two maximized increments therefore gives \[ \Lambda=\|P_{\mathcal{T}_x^\perp}z_n\|^2+o_p(1). \]
The rank of \(A\) is \(q\), because \(I^{-1/2}\) is nonsingular and \(G\) has rank \(q\). Hence \(\mathcal{T}_x^\perp\) has dimension \(q\). Also \(z_n\xrightarrow{d}N_p(0,I_p)\). In an orthonormal coordinate system whose first \(q\) axes span \(\mathcal{T}_x^\perp\), the projection has \(q\) independent standard normal coordinates in the limit. Its squared norm therefore converges in distribution to \(\chi^2_q\). This proves the result. \(\square\)
The theorem is an asymptotic statement: it does not say that \(\Lambda\) has an exact chi-square distribution in a finite sample. In applications, the usual likelihood-ratio p-value is calculated from the \(\chi^2_q\) approximation. The relevant \(q\) is the local rank of the constraints at the null parameter, provided the stated regularity conditions hold.
What the Constrained Calculation Does—and Does Not—Guarantee
For a nonlinear null, the constrained maximum may require numerical optimization. The restriction can define a curved set, and it may have several separate pieces, as the scalar example illustrates. The goal is still to find the largest log-likelihood over the entire null set. Checking only one feasible point can produce a restricted log-likelihood that is too small and a likelihood-ratio statistic that is too large.
The full-rank condition matters as well as the algebraic form of the null. At a point where the derivative of the restriction loses rank, the null may not look locally like a smooth surface of the expected dimension. In that case, the projection argument above does not give the usual chi-square degrees of freedom; a routine rank-based reference distribution should not be asserted without further analysis.
Common Mistakes
- Using the unrestricted estimate as the restricted fit. Unless it satisfies the null, it is not feasible under \(H_0\). Maximize the likelihood over \(\Theta_0\) instead.
- Substituting any null value for the constrained maximum. A feasible value gives a likelihood no larger than the restricted maximum. The statistic requires the best feasible value.
- Counting equations instead of independent restrictions. Use the rank of \(G(\theta_0)\) at the null point to determine the local number of restrictions.
- Assuming every nonlinear null has the standard chi-square limit. The smoothness, full-rank, interior, identifiability, information, and localization conditions are part of the result.
- Changing the likelihood scale inconsistently. The statistic is twice the difference in maximized log-likelihoods. In a known-variance Gaussian model this equals the difference in squared residual distances, not half that difference.
Check Your Understanding
Use the likelihood comparison and the local geometry of the null to answer these questions.
- For \(H_0:g(\theta)=0\), what role does the rank of \(G(\theta_0)\) play in the usual chi-square approximation?
- In a Gaussian model with known identity covariance, why is twice the log-likelihood difference equal to the difference in squared residual distances?
- For the restriction \(\theta_2=\theta_1^2\), describe the tangent constraint at a specified null point \((a,a^2)\).
- Why can evaluating the likelihood at one feasible null value yield an incorrect likelihood-ratio statistic?
- Which regularity condition fails if \(G(\theta_0)\) does not have full row rank, and why does the projection proof then fail to justify the stated degrees of freedom?