When Standard Likelihood Asymptotics Fail

Why Regular Likelihood Limits Need Regular Models

The usual likelihood calculations are powerful because, under suitable conditions, different procedures converge to familiar limits. A maximum-likelihood estimator is approximately normal, and a likelihood-ratio statistic for $q$ restrictions is approximately chi-square with $q$ degrees of freedom. Those conclusions are not automatic consequences of maximizing a likelihood. They depend on the parameter being locally identifiable and interior, on information being nonsingular, and on the likelihood behaving smoothly near the truth.

This tutorial examines three distinct violations. A true parameter can lie on the boundary of its allowed set; a nuisance parameter can be unidentified under the null hypothesis; or the likelihood can keep increasing toward an infinite regression coefficient. Each mechanism changes what can be concluded from standard maximum-likelihood theory.

Definition: For a likelihood $L(\theta)$, the likelihood-ratio statistic for testing a null parameter set $\Theta_0$ against a larger set $\Theta$ is $$ \Lambda_n=2\left\{\sup_{\theta\in\Theta}\ell_n(\theta)-\sup_{\theta\in\Theta_0}\ell_n(\theta)\right\}, $$ where $\ell_n(\theta)=\log L(\theta)$. The usual Wilks limit, $\Lambda_n\xrightarrow{d}\chi^2_q$, requires regularity conditions. In particular, the true parameter must be an interior point of a locally identifiable, smooth model with nonsingular Fisher information; the null restrictions must be smooth and have fixed codimension $q$; and the likelihood must satisfy the relevant differentiability and large-sample conditions.

These conditions explain why “there are $q$ restrictions” is not enough to determine a chi-square limit. At a boundary, the local parameter space is not a full Euclidean neighborhood. Under nonidentifiability, there may be no single regular local parameterization under the null. In separation, a finite maximizer may not exist at all. The following examples isolate these failures rather than treating them as interchangeable.

A Boundary Parameter Changes the Likelihood-Ratio Limit

Consider independent observations $X_1,\ldots,X_n\sim N(\mu,1)$, with the parameter constrained to $\mu\geq 0$. Test $H_0:\mu=0$ against $H_1:\mu>0$. The likelihood is smooth as a function of an unconstrained $\mu$, and its information is positive. But the true null value is at the edge of the allowed parameter space.

Worked Example: A Normal Mean Constrained to Be Nonnegative

Up to terms not depending on $\mu$, the log-likelihood is $$ \ell_n(\mu)=-\frac{1}{2}\sum_{i=1}^n(X_i-\mu)^2. $$ Its unconstrained maximizer is $\bar X$. Restricting to $\mu\geq0$ gives $\hat\mu=\max(0,\bar X)$. Under the null, the only allowed value is $\mu=0$.

Subtracting the log-likelihood at zero from that at the constrained maximizer gives $$ \Lambda_n=2\{\ell_n(\hat\mu)-\ell_n(0)\} =n\max(0,\bar X)^2. $$ Under $H_0$, $Z=\sqrt n\,\bar X\sim N(0,1)$ exactly. Thus $$ \Lambda_n=\max(0,Z)^2. $$

If $Z\leq0$, an event of probability $1/2$, the constrained maximum is at zero and $\Lambda_n=0$. If $Z>0$, then $\Lambda_n=Z^2$. For $x>0$, $$ P(\Lambda_n\leq x)=P(Z\leq\sqrt{x})=\frac12+\frac12P(Z^2\leq x), $$ where the last probability is the chi-square distribution function with one degree of freedom. Hence the null distribution is exactly a point mass of $1/2$ at zero plus $1/2$ times $\chi^2_1$, not $\chi^2_1$.

The derivation also shows the geometry: only positive fluctuations of the unconstrained estimate can move the constrained estimate away from the null. A chi-square variable with one degree of freedom is positive with probability one, whereas this statistic is zero with probability $1/2$. The discrepancy is not a small-sample effect; it persists for every $n$ in this example.

Result: For a scalar, regular parameter tested at a single boundary point with one locally allowed direction, the likelihood-ratio limit is commonly a mixture of a point mass at zero and a chi-square distribution. The example above derives the equal-weight mixture exactly. Boundary problems with more complicated constraints can have other mixtures; the ordinary Wilks degrees-of-freedom rule should not be applied without checking the constraint geometry.

An Unidentified Parameter Under the Null

A different failure occurs when a parameter determines the direction of an alternative but has no effect on the distribution under the null. The null then cannot identify that parameter. Maximization over the alternative effectively searches across several directions, so the statistic can reflect a maximum of likelihood gains rather than a single regular quadratic form.

Worked Example: Choosing Between Two Mean Directions

Observe $Y=(Y_1,Y_2)$ with distribution $N_2(\mu,I_2)$, where $I_2$ is the two-dimensional identity matrix. Consider the model $$ \mu=t e_k,\qquad t\in\mathbb R,\quad k\in\{1,2\}, $$ where $e_1=(1,0)^\mathsf{T}$ and $e_2=(0,1)^\mathsf{T}$. Test $t=0$. When $t=0$, both choices of $k$ give the same distribution $N_2(0,I_2)$; the direction label $k$ is not identified by the null.

Ignoring a common constant, the log-likelihood for a given $t,k$ is $$ \ell(t,k)=-\frac12\|Y-t e_k\|^2 =-\frac12\|Y\|^2+tY_k-\frac12t^2. $$ For fixed $k$, completing the square gives $$ tY_k-\frac12t^2 =-\frac12(t-Y_k)^2+\frac12Y_k^2. $$ So the maximizing value is $\hat t=Y_k$, and the log-likelihood gain over the null is $Y_k^2/2$. Maximizing over both possible directions yields $$ \Lambda=2\max_{k\in\{1,2\}}\frac{Y_k^2}{2} =\max(Y_1^2,Y_2^2). $$

Under the null, $Y_1$ and $Y_2$ are independent standard normal variables. For $x\geq0$, independence therefore gives $$ P(\Lambda\leq x)=P(Y_1^2\leq x,\;Y_2^2\leq x) =\{P(\chi^2_1\leq x)\}^2. $$ This is the distribution of the maximum of two independent $\chi^2_1$ variables, not the usual chi-square limit for a fixed, identified direction. The null leaves the direction unresolved; the alternative likelihood chooses whichever direction fits better.

This example uses just two possible directions so that the distribution can be calculated directly. If the unidentified quantity ranges over many values, the limit can involve the supremum of a stochastic process. The central diagnostic remains the same: ask whether every nuisance parameter in the alternative has a well-defined, identifiable value under the null. If not, the regular likelihood-ratio theorem does not apply just by counting restrictions.

Separation Can Remove the Finite Maximum

For a finite-dimensional maximum-likelihood limit, there must first be a finite maximum in the relevant model. Logistic regression illustrates how this can fail even with a smooth likelihood. When covariates separate the outcome classes, increasing a coefficient can make fitted probabilities approach zero or one in the observed directions, while the likelihood increases toward a supremum that is not attained at any finite coefficient.

Worked Example: Complete Separation in Logistic Regression

Take two observations with covariate values $x_1=-1$, $x_2=1$, and responses $y_1=0$, $y_2=1$. Fit the logistic model without an intercept, $$ P(Y_i=1\mid x_i)=\sigma(\beta x_i),\qquad \sigma(u)=\frac{1}{1+e^{-u}}. $$ The probability of the observed response at $x_1=-1$ is $1-\sigma(-\beta)=\sigma(\beta)$; at $x_2=1$, it is also $\sigma(\beta)$. Thus $$ L(\beta)=\sigma(\beta)^2,\qquad \ell(\beta)=2\log\sigma(\beta). $$

Differentiating, $$ \ell'(\beta)=2\{1-\sigma(\beta)\}>0 $$ for every finite $\beta$. Therefore, the likelihood is strictly increasing at every finite value and has no finite maximizer. As $\beta\to\infty$, $\sigma(\beta)\to1$, so $L(\beta)\to1$, its supremum. This is complete separation: positive covariate values occur only with response 1 and negative values only with response 0. The coefficient estimate cannot be treated as a finite, approximately normal maximum-likelihood estimate.

More generally, for design vectors $x_i$, a direction $v$ strictly separates the observed responses when $(2y_i-1)x_i^\mathsf{T}v>0$ for every observation. Moving coefficients along that direction pushes each fitted probability toward the observed outcome. In ordinary unpenalized logistic regression, this can prevent a finite maximum. When some inequalities are equalities and the rest are strict, the related phenomenon is called quasi-complete separation; it also requires care in assessing existence and inference.

Diagnose Before Applying a Standard Limit

The three mechanisms have different signatures, so they call for different diagnostics. A boundary is visible in the allowed parameter space and the location of the null. Nonidentifiability is visible by comparing distributions under the null as a nuisance parameter varies. Separation is visible in the covariate-outcome pattern and in a likelihood that increases as a coefficient grows without bound.

Failure mechanism	What to check	Consequence for standard theory
Boundary parameter	Is the true value at the edge of the parameter space?	Local fluctuations are constrained; the likelihood-ratio limit may be a mixture rather than an ordinary chi-square.
Nonidentifiability under the null	Does changing a nuisance value leave the null distribution unchanged?	The alternative may optimize over multiple directions or configurations; the usual fixed-degree chi-square argument can fail.
Separation	Can a coefficient direction increasingly classify every observation correctly?	A finite maximum-likelihood estimate may not exist, so ordinary estimator and likelihood-ratio limits are not justified.

A common mistake is to reason: “The alternative adds one parameter, therefore the likelihood-ratio statistic is asymptotically $\chi^2_1$.” That conclusion silently assumes the null is an interior, identifiable point in a regular model and that the maximum is well behaved. The boundary example instead has a point mass at zero; the unidentified-direction example takes a maximum over two random gains; and the separation example has no finite maximum. The correction is to verify the regularity conditions before assigning a reference distribution.

In practice, inspect parameter constraints and model identifiability before interpreting a Wald, score, or likelihood-ratio calculation. For logistic models, examine separation and convergence diagnostics rather than treating a large coefficient or a failed optimization as an ordinary finite estimate. Depending on the problem, inference may require a boundary-aware reference distribution, a method designed for the unidentified-null structure, or a modified estimation procedure such as penalization. Such changes must be justified for the mechanism at hand; they do not make the ordinary chi-square limit valid by default.

Key takeaway: Standard likelihood asymptotics are conditional results, not universal properties of maximum likelihood. Check interiority, identifiability under the null, and existence of a finite maximizer before using normal or chi-square approximations.

Check Your Understanding

For each question, identify the relevant likelihood feature before deciding whether a standard limit is justified.

In the constrained normal-mean example, why does the likelihood-ratio statistic have probability $1/2$ of being exactly zero?
In the two-direction model, what is unidentified under the null, and how does that affect the likelihood maximization?
What feature of the covariate-response pattern in the logistic example shows complete separation?
Why is counting the number of added parameters insufficient to establish a chi-square likelihood-ratio limit?
For each of the three mechanisms, name one diagnostic you would check before reporting ordinary maximum-likelihood inference.