Wilks’ Theorem And The Chi-Square Limit

Why a Likelihood-Ratio Statistic Has a Chi-Square Limit

A likelihood-ratio test compares the best fit available under a restricted model with the best fit under a broader model. Under the null hypothesis, the restriction is correct, so the improvement from relaxing it should usually be modest. Wilks’ theorem gives that idea a precise large-sample form: under regularity conditions, minus twice the log likelihood ratio approaches a chi-square distribution.

The result is useful because it provides a reference distribution for a likelihood comparison even when the exact finite-sample distribution is difficult to obtain. Its justification is not that every likelihood is globally quadratic. Rather, near the true parameter and with a large sample, the log likelihood behaves approximately like a quadratic function. The geometry of that quadratic gives the chi-square limit.

Definition: Let $\ell_n(\theta)$ be the log likelihood from a sample of size $n$. Let $\hat{\theta}$ maximize it over the unrestricted parameter space, and let $\tilde{\theta}$ maximize it subject to the null hypothesis. The likelihood-ratio statistic in the Wilks convention is $$ W_n=2\{\ell_n(\hat{\theta})-\ell_n(\tilde{\theta})\}. $$ It is nonnegative because the restricted parameter space is contained in the unrestricted one. For a regular null model of codimension $r$, Wilks’ theorem states that, when the null hypothesis is true, $W_n$ converges in distribution to $\chi^2_r$ as $n$ increases.

The codimension $r$ is the number of independent smooth restrictions imposed by the null within the unrestricted model. The theorem’s chi-square reference distribution is an asymptotic null distribution; it is not generally an exact distribution for a particular finite sample. In applications, the statistic is compared with a chi-square critical value or used to obtain a chi-square upper-tail p-value.

How Local Quadratic Behavior Produces the Limit

Suppose the true parameter $\theta_0$ is an interior point of a regular $p$-dimensional model and satisfies the null restrictions. The score is the derivative of the log likelihood, and the Fisher information describes the local curvature of the likelihood. For parameter values within order $1/\sqrt{n}$ of $\theta_0$, a second-order expansion has the approximate form

\ell_n\left(\theta_0+\frac{h}{\sqrt{n}}\right)-\ell_n(\theta_0) = h^{\mathsf T}\Delta_n-\frac{1}{2}h^{\mathsf T}I(\theta_0)h+o_p(1).

Here $h$ is a fixed local displacement, $I(\theta_0)$ is the Fisher information per observation, and $\Delta_n$ is the score scaled by $1/\sqrt n$. Under the null and standard regularity conditions, $\Delta_n$ converges in distribution to a mean-zero normal vector with covariance matrix $I(\theta_0)$. The term $o_p(1)$ tends to zero in probability.

To see the implication for the likelihood ratio, first change coordinates to scale the parameter directions by the information matrix. In those coordinates the local quadratic has identity curvature, and the limiting score is a vector $Z$ of independent standard normal components. Maximizing the quadratic over all local directions allows a fitted displacement in every direction. Maximizing it under the null allows displacements only in the null model’s tangent space. The difference between these two maximized values is one half the squared length of the component of $Z$ perpendicular to that tangent space. Since $W_n$ is twice the log likelihood difference, the limiting statistic is the squared length of that perpendicular component.

W_n\ \Longrightarrow\ \sum_{j=1}^{r} Z_j^2\ \sim\ \chi^2_r, \qquad Z_1,\ldots,Z_r\ \text{independent }N(0,1).

The sum of squares of $r$ independent standard normal variables is, by definition, chi-square with $r$ degrees of freedom. This explains both why the limiting distribution is chi-square and why only the directions excluded by the null contribute to the likelihood improvement.

Approximate locally.
Near the true parameter, use the score and information to represent the log likelihood by a quadratic function.

Compare the allowed directions.
The unrestricted fit can move in all local parameter directions; the null fit is confined to its tangent space.

Take the squared normal component.
The improvement is asymptotically the squared length of a standard normal vector in the directions excluded by the null.

Conditions and Scope of the Approximation

Wilks’ theorem is a regular, large-sample result. Its usual form relies on conditions such as an identifiable model, a true parameter in the interior of the parameter space, a smooth likelihood, and finite, nonsingular Fisher information. The null model must be a smooth submodel of the unrestricted model, and the number of parameters and restrictions is held fixed as the sample size grows.

If a condition fails, a chi-square limit cannot simply be assumed. For example, a null value on the boundary of the parameter space does not satisfy the interior-point condition. In such a case, the standard Wilks calibration may not apply; the null distribution needs separate justification. Likewise, in a small sample, the chi-square approximation can be inaccurate even when the model is regular. A statistic with a value near a decision threshold deserves particular care.

The theorem is stated under the null. When the null is false and the alternative is separated from it, the unrestricted model will typically fit increasingly better as information accumulates, and the statistic does not have the central chi-square limit used for a null test. Thus the chi-square reference distribution is for assessing evidence against the null, not for describing the statistic under every possible data-generating model.

Worked Applications

Worked Example: A Fixed Binomial Claim Probability

A hypothetical roadside-assistance program records whether each of 100 randomly selected service contracts generated at least one tow claim. Assume the outcomes are independent Bernoulli variables with a common probability $p$. There are 46 contracts with a tow claim. Test $H_0:p=0.40$ against the unrestricted alternative, using a 5% large-sample likelihood-ratio test. State the method and avoid confusing the restricted likelihood with the unrestricted maximum.

Method and assumptions. This is a binomial likelihood comparison. Under the unrestricted model, $p$ is estimated from the data; under the null it is fixed at 0.40. The null value is interior to $0<p<1$, so the regular one-restriction chi-square approximation is applicable for this calculation.

The unrestricted maximum-likelihood estimate is $ \hat p=46/100=0.46 $. Ignoring the binomial coefficient, which cancels in the likelihood ratio, the log likelihood is $ \ell(p)=46\log(p)+54\log(1-p) $. Therefore,

\begin{aligned} W_{100} &=2\{\ell(0.46)-\ell(0.40)\}\\ &=2\left\{46\log\left(\frac{0.46}{0.40}\right) +54\log\left(\frac{0.54}{0.60}\right)\right\}\\ &=2\{46\log(1.15)+54\log(0.90)\}\\ &=2\{46(0.1397619)+54(-0.1053605)\}\\ &=2\{6.429049-5.689468\}\\ &=2(0.739582)=1.4792. \end{aligned}

The null fixes one scalar parameter, so the reference distribution is approximately $\chi^2_1$. Its 5% upper-tail critical value is 3.841. Since $1.4792<3.841$, this test does not reject $H_0$ at the 5% level. This is a large-sample conclusion, not proof that $p$ is exactly 0.40.

Common mistake avoided: Using $2\{\ell(0.40)-\ell(0.46)\}$ reverses the maximized and restricted log likelihoods and produces a negative number. The correct order is unrestricted maximum minus restricted maximum, giving a nonnegative statistic.

Worked Example: Comparing Two Hypothetical Claim Frequencies

A hypothetical insurer observes claim counts in two portfolios. Assume independent Poisson counts with constant rates within each portfolio. Portfolio A has 90 claims over 50 exposure units, and portfolio B has 110 claims over 50 exposure units. Test whether the two rates are equal against separate unrestricted rates. Use a 5% test and report the approximate p-value to four decimal places.

Method and assumptions. For a Poisson count with exposure $E$ and rate $\lambda$, the mean count is $E\lambda$. Under separate rates, the maximum-likelihood estimates are $90/50=1.8$ and $110/50=2.2$. Under the common-rate null, the estimate pools counts and exposures: $ (90+110)/(50+50)=2.0 $. The common null is an interior, regular restriction.

The Poisson log-likelihood terms involving factorials are the same under both fits and cancel. The likelihood-ratio statistic is

\begin{aligned} W &=2\left\{90\log\left(\frac{1.8}{2.0}\right) +110\log\left(\frac{2.2}{2.0}\right)\right\}\\ &=2\{90\log(0.9)+110\log(1.1)\}\\ &=2\{90(-0.1053605)+110(0.0953102)\}\\ &=2\{-9.482445+10.484122\}\\ &=2(1.001677)=2.0034. \end{aligned}

The null equality restriction has one independent direction, so use $\chi^2_1$. The upper-tail probability is approximately $P(\chi^2_1\ge 2.0033467)=0.1570$, to four decimal places. Because $0.1570>0.05$, the test does not reject equal rates at the 5% level.

Check. The pooled estimate $2.0$ lies between the two separate estimates, and the unrestricted fit cannot have a lower maximized likelihood than the restricted fit. The computed statistic is positive, as required. Rounding the statistic to 2.0034 does not change the stated four-decimal p-value.

Common mistake avoided: Pooling the two observed rates by taking an unweighted average is only justified here because exposures are equal. In general, the common-rate estimate is total claims divided by total exposure.

Worked Example: Testing an Exponential Severity Rate

Suppose 80 positive losses from a hypothetical coverage are independent and exponentially distributed with rate parameter $\lambda$, so the density is $f(x)=\lambda e^{-\lambda x}$ for $x>0$. Their observed total is 144. Test $H_0:\lambda=0.50$ against an unrestricted rate using a 5% likelihood-ratio test. Use the asymptotic chi-square calibration.

Method and assumptions. The observations are uncensored, independent exponential losses, and $\lambda>0$. The log likelihood, apart from data-independent terms, is $ \ell(\lambda)=80\log\lambda-144\lambda $. Differentiating gives $80/\lambda-144=0$, so the unrestricted estimate is $ \hat\lambda=80/144=5/9$.

Substitute the restricted and unrestricted estimates into the log likelihood difference:

\begin{aligned} W &=2\{80\log(\hat\lambda/0.50)-144(\hat\lambda-0.50)\}\\ &=160\left\{\log\left(\frac{5/9}{1/2}\right) -\frac{144}{80}\left(\frac{5}{9}-\frac12\right)\right\}\\ &=160\left\{\log\left(\frac{10}{9}\right) -1.8\left(\frac{1}{18}\right)\right\}\\ &=160\{0.1053605-0.1000000\}\\ &=160(0.0053605)=0.8577. \end{aligned}

The tested rate is one scalar restriction, giving an approximate $\chi^2_1$ reference distribution. Since $0.8577<3.841$, the test does not reject the specified rate at the 5% level.

Check. The total loss per observation is $144/80=1.8$, so the exponential rate estimate $1/1.8=0.5556$ agrees with $5/9$. The statistic is nonnegative and is smaller than the 5% critical value.

Common mistake avoided: For this rate parameterization, the exponential mean is $1/\lambda$, not $\lambda$. Treating the sample mean $1.8$ as an estimate of the rate would invert the parameter incorrectly.

Practical Interpretation and Exam Traps

Wilks’ theorem turns a difference in optimized log likelihoods into a familiar tail-area calculation, but the theorem does not remove the need to check the model and the null. In particular, the statistic must be formed from the same data under both fits, with the same likelihood convention and all relevant likelihood terms included. Terms that are identical under both fits may cancel; terms that differ may not be discarded.

Check the sign and factor. With $W_n=2(\ell_{\text{unrestricted}}-\ell_{\text{null}})$, the statistic is nonnegative. Omitting the factor 2 does not produce the Wilks statistic.
Use the null reference distribution. The chi-square approximation describes the statistic when the null is true, subject to regularity conditions.
Do not assume exactness. A finite-sample likelihood-ratio statistic need not have a chi-square distribution, even when its large-sample limit does.
Check regularity before calibrating. Boundary parameters, nonidentifiability, or other failures of the usual assumptions can invalidate the standard chi-square limit.
Keep the comparison nested. The restricted model must be included in the unrestricted model for the usual likelihood-ratio construction and the nonnegative statistic stated here.

A useful exam workflow is to write down the two models, identify their maximized log likelihoods, calculate $W_n$, and only then select the justified chi-square reference distribution. If the approximation’s conditions are doubtful, report that limitation rather than treating a standard chi-square p-value as automatically valid.

Check Your Understanding

For each problem, use the stated regular large-sample comparison and check the statistic against the 5% critical value when requested.

A hypothetical sample of 50 independent contracts has 30 contracts with at least one glass claim. Under a common Bernoulli probability, test $p=0.50$ against an unrestricted probability. Calculate the likelihood-ratio statistic and state the 5% decision. (Answer: $W=2.0136$; do not reject at 5%.)
Two hypothetical portfolios have independent Poisson counts: 24 claims over 30 exposure units and 36 claims over 30 exposure units. Test equal rates against separate rates. Calculate $W$ and state the 5% decision. (Answer: $W=2.4163$; do not reject at 5%.)
For 40 independent, uncensored exponential losses with total 100, test the rate $\lambda=0.50$ against an unrestricted rate. Calculate the statistic and state the 5% decision. (Answer: $\hat\lambda=0.4000$, $W=2.1485$; do not reject at 5%.)
A regular nested comparison has already established two independent smooth restrictions, and the observed statistic is $W=5.20$. Use the asymptotic reference distribution to obtain the upper-tail p-value to four decimals and decide at 5%. (Answer: $\chi^2_2$ upper-tail p-value $=e^{-5.20/2}=0.0743$; do not reject at 5%.)
A hypothetical frequency model imposes a nonnegative rate, and the null specifies that rate to be zero. Is the usual interior-point form of Wilks’ theorem sufficient by itself to justify a standard chi-square calibration? (Answer: No; the null is on the parameter-space boundary, so the usual interior-point condition fails.)

What You'll Learn