Wilks' Theorem For Nested Models

Why a Likelihood-Ratio Statistic Becomes Chi-Square

A nested-model likelihood-ratio test compares the best fit allowed by a restricted model with the best fit allowed by a larger model. The unrestricted model can fit at least as well, but the improvement may reflect either genuine evidence against the restriction or ordinary sampling variation. Wilks’ theorem explains how to calibrate that improvement in large samples: under regularity conditions and when the null hypothesis is true, minus twice the log generalized likelihood ratio is approximately chi-square.

This result is useful because it turns two fitted likelihood values into an approximate test statistic and p-value. The degrees of freedom are determined by how many independent parameter dimensions the restriction removes, not by the sample size or the numerical size of the parameter estimates.

Definition: Let the unrestricted model have parameter space $\Theta$ of dimension $p$, and let the null model be a nested subset $\Theta_0\subset\Theta$ of dimension $q$. For observations $X_1,\ldots,X_n$, let $L(\theta)$ be the likelihood and $\ell(\theta)=\log L(\theta)$ the log-likelihood. The generalized likelihood ratio is $$ \Lambda=\frac{\sup_{\theta\in\Theta_0}L(\theta)}{\sup_{\theta\in\Theta}L(\theta)}, \qquad T=-2\log\Lambda =2\left\{\sup_{\theta\in\Theta}\ell(\theta)-\sup_{\theta\in\Theta_0}\ell(\theta)\right\}. $$ The numerator is the best likelihood under the null; the denominator is the best likelihood under the full model. Thus $0\leq\Lambda\leq1$ and $T\geq0$.

Wilks’ Theorem and Its Conditions

The chi-square approximation is a large-sample statement under the null. In its standard form, the observations are independent and identically distributed from a correctly specified parametric model. The true parameter lies in the null model, which is a smooth, $q$-dimensional submodel inside a $p$-dimensional model. The dimensions $p$ and $q$ are fixed as the sample size grows.

The usual regularity conditions include an identifiable model, a true parameter in the interior of the unrestricted parameter space, a likelihood sufficiently smooth near that parameter, and a finite, nonsingular Fisher information matrix. The maximum likelihood estimator must behave regularly: it is consistent and has the standard asymptotic normal distribution. The null restriction must be locally smooth, with $q$ independent parameter directions. In particular, this standard result is not automatically justified when parameters lie on a boundary, parameters are not identifiable under the null, or the support of the distribution changes with the parameter.

Theorem (Wilks’ theorem): Under the null hypothesis and the regularity conditions above, if the unrestricted model has dimension $p$ and the null model has dimension $q$, then as $n$ increases, $$ T=-2\log\Lambda\ \xrightarrow{d}\ \chi^2_{p-q}. $$ For a large but finite sample, the approximate upper-tail p-value is $P(\chi^2_{p-q}\geq T_{\mathrm{obs}})$. A small value indicates that the unrestricted model improves the fit more than would usually be expected under the null, subject to the approximation’s assumptions.

The degrees of freedom are the difference in model dimensions because the null removes $p-q$ locally available directions of variation. Equivalently, they are the number of independent restrictions imposed by the null, provided those restrictions are genuinely independent near the true parameter. Counting written equations without checking independence can give the wrong answer.

Why the Chi-Square Limit Holds

A short local argument gives the central idea. Near the true null parameter $\theta_0$, measure a parameter displacement in units of its Fisher information and the sample size. In these standardized local coordinates, write the displacement as $h/\sqrt{n}$. Under the stated regularity conditions, the log-likelihood has the quadratic approximation

\ell_n\left(\theta_0+\frac{h}{\sqrt n}\right)-\ell_n(\theta_0) = h^{\mathsf T}Z_n-\frac12 h^{\mathsf T}h+o_p(1), \qquad Z_n\xrightarrow{d}N_p(0,I_p).

Here $I_p$ is the $p$-dimensional identity matrix, and $o_p(1)$ denotes a term that converges to zero in probability. The normal limit for $Z_n$ comes from the asymptotic normality of the score; the negative quadratic term comes from the local curvature of the log-likelihood. Information standardization makes this curvature the identity matrix.

In the unrestricted model, the quadratic expression is maximized at $h=Z_n$, giving a maximum improvement of $\tfrac12 Z_n^{\mathsf T}Z_n$ over its value at $h=0$. The null restriction allows only displacements in a $q$-dimensional tangent subspace $S$. If $P_S$ is the orthogonal projection onto $S$, the restricted maximum is attained at $P_SZ_n$, and its improvement is $\tfrac12 Z_n^{\mathsf T}P_SZ_n$. Subtracting the restricted maximum from the unrestricted maximum and multiplying by two therefore gives

T \ \xrightarrow{d}\ Z^{\mathsf T}(I_p-P_S)Z = \left\|(I_p-P_S)Z\right\|^2, \qquad Z\sim N_p(0,I_p).

The matrix $I_p-P_S$ projects onto the directions excluded by the null and has rank $p-q$. In an orthonormal coordinate system for those directions, the displayed squared length is the sum of squares of $p-q$ independent standard normal variables. That is the definition of a $\chi^2_{p-q}$ random variable, which establishes the stated limit.

Applying the Approximation

In an application, first identify the two models and their dimensions. Then compare their maximized log-likelihoods, calculate $T$, and use the chi-square distribution with $p-q$ degrees of freedom to obtain an approximate p-value. The fitted log-likelihoods below are hypothetical outputs from maximum-likelihood fitting; the examples illustrate the theorem rather than report real data.

Identify the nesting and dimensions.
Confirm that every null-model parameter choice is allowed by the unrestricted model, and count the independent dimensions in each.

Calculate the likelihood-ratio statistic.
Use twice the unrestricted maximized log-likelihood minus twice the restricted maximized log-likelihood.

Use the appropriate chi-square tail.
Set the degrees of freedom to $p-q$, then find the probability of a value at least as large as the observed statistic.

Worked Example: Testing a Fixed Weibull Shape

Method: Apply Wilks’ approximation to a one-dimensional restriction. A hypothetical sample consists of independent, completely observed positive times from a Weibull model with shape $k>0$ and scale $\lambda>0$, whose density is $f(t;k,\lambda)=\frac{k}{\lambda}(t/\lambda)^{k-1}\exp[-(t/\lambda)^k]$, for $t>0$. Test $H_0:k=2$ against the unrestricted Weibull model. Suppose the maximized log-likelihood is $-143.20$ under the null and $-141.28$ under the unrestricted model.

The full parameter vector $(k,\lambda)$ has dimension $p=2$. Under $H_0$, $k$ is fixed and $\lambda$ remains free, so $q=1$. The null is an interior restriction because $k=2$ is strictly positive. The degrees of freedom are therefore $p-q=2-1=1$.

The likelihood-ratio statistic is $$ T=2\{\ell_{\mathrm{full}}-\ell_{\mathrm{null}}\} =2\{-141.28-(-143.20)\} =2(1.92)=3.84. $$

Under the null and the regularity conditions, compare this with $\chi^2_1$. Its upper-tail probability at $3.84$ is approximately $0.0500$ (four decimal places). Thus the asymptotic p-value is approximately $0.0500$, close to a 5% significance threshold. This is an approximation, not an exact finite-sample calibration.

Common mistake avoided: Using the likelihood difference $1.92$ itself as the statistic omits the required factor of two. The correct statistic is $3.84$, and its p-value must be read from a one-degree-of-freedom chi-square distribution.

Worked Example: A Smooth Constraint on Gamma Parameters

Method: Count the dimensions after imposing the constraint, then use the resulting chi-square reference distribution. Consider hypothetical independent positive measurements modeled by a Gamma distribution with shape $\alpha>0$ and rate $\beta>0$, with density $f(x;\alpha,\beta)=\beta^\alpha x^{\alpha-1}e^{-\beta x}/\Gamma(\alpha)$, for $x>0$. Test $H_0:\beta=\alpha$ against the unrestricted shape-rate model. Suppose the restricted and unrestricted maximized log-likelihoods are $-89.50$ and $-87.95$, respectively.

The unrestricted parameter space has dimension $p=2$. The equation $\beta=\alpha$ leaves one free parameter, so the null model has dimension $q=1$. This is one independent smooth restriction, with positive parameters in the interior. Thus the reference distribution has $2-1=1$ degree of freedom.

The observed statistic is $$ T=2\{-87.95-(-89.50)\} =2(1.55)=3.10. $$

For $\chi^2_1$, the upper-tail probability at $3.10$ is approximately $0.0784$ (four decimal places). The approximate p-value is therefore $0.0784$. At a 5% significance level this value would not lead to rejection, while at a 10% level it would.

Common mistake avoided: The restriction is one equation, not two restrictions merely because it involves both $\alpha$ and $\beta$. Since it leaves one freely varying parameter in the two-dimensional model, the dimension difference is one.

Worked Example: Testing a Point in a Lognormal Model

Method: Use the difference between the full and null dimensions to select the reference distribution. Suppose a hypothetical sample consists of independent positive measurements $X$ with $\log X\sim N(\mu,\sigma^2)$, where $\mu$ is real and $\sigma^2>0$. Test the point null $H_0:\mu=0,\ \sigma^2=1$ against the unrestricted lognormal model. Let the null and unrestricted maximized log-likelihoods be $-64.70$ and $-61.90$.

The unrestricted model has two free parameters, so $p=2$. The point null fixes both parameters and has dimension $q=0$. Both null values lie in the interior of the allowed parameter space. The chi-square degrees of freedom are $p-q=2$.

Calculate the statistic: $$ T=2\{-61.90-(-64.70)\} =2(2.80)=5.60. $$

For two degrees of freedom, the chi-square upper-tail probability has the form $P(\chi^2_2\geq t)=e^{-t/2}$. Hence $$ P(\chi^2_2\geq5.60)=e^{-5.60/2}=e^{-2.80}\approx0.0608. $$

The approximate p-value is $0.0608$ (four decimal places). It is above $0.05$, so this large-sample calculation does not reject the point null at the 5% level.

Common mistake avoided: A point null does not always imply one degree of freedom. Here it fixes two independent parameters, so the reference distribution is $\chi^2_2$, not $\chi^2_1$.

Common Traps and Practical Interpretation

Reversing the likelihood ratio. The null maximum cannot exceed the unrestricted maximum. Keep the statistic in the nonnegative form $2(\ell_{\mathrm{full}}-\ell_{\mathrm{null}})$.
Using the wrong degrees of freedom. Count the dimension removed by the null, not the number of data points, model terms, or parameter names appearing in a written restriction.
Treating the result as exact. Wilks’ theorem gives a limiting distribution. A modest sample can yield a poor approximation even when the model is regular; the theorem alone does not guarantee accuracy for a particular sample size.
Applying the standard theorem without regularity. A boundary value, nonidentifiability, changing support, or a nonsmooth restriction can invalidate this chi-square conclusion. In such settings the reference distribution needs separate justification.
Using a lower-tail probability. Large values of $T$ indicate a poor fit under the null, so the relevant p-value is the upper tail.

Wilks’ theorem gives a common large-sample calibration across many regular parametric models. The crucial checks are that the models are genuinely nested, the null is regular and interior, and the dimension difference is correct. When these checks hold, the fitted likelihood improvement has a standard chi-square benchmark aligned with the SOA VEE Mathematical Statistics syllabus.

Check Your Understanding

For each problem, assume the stated model is regular and that the null is true when applying Wilks’ theorem. Treat each likelihood pair as hypothetical fitted output.

Independent positive observations follow a Weibull model with free shape $k>0$ and scale $\lambda>0$. The null fixes $k=1$, leaving $\lambda$ free. The null and unrestricted maximized log-likelihoods are $-52.4$ and $-49.9$. Find the degrees of freedom, statistic, and approximate p-value. (Answer: $1$ degree of freedom; $T=5.00$; $p\approx0.0253$.)
Independent positive observations follow a Gamma model with free shape $\alpha>0$ and rate $\beta>0$. The null imposes $\beta=3\alpha$. The null and unrestricted maxima are $-101.8$ and $-100.2$. Find the degrees of freedom, statistic, and approximate p-value. (Answer: $1$ degree of freedom; $T=3.20$; $p\approx0.0736$.)
For independent positive observations with $\log X\sim N(\mu,\sigma^2)$, compare the full model with the null $\mu=0$ while $\sigma^2$ remains free. The null and unrestricted log-likelihood maxima are $-77.6$ and $-75.9$. Find the degrees of freedom, statistic, and approximate p-value. (Answer: $1$ degree of freedom; $T=3.40$; $p\approx0.0652$.)
A regular unrestricted model has dimension $7$, and its smooth nested null model has dimension $4$. The observed likelihood-ratio statistic is $7.815$. State the reference distribution and the approximate upper-tail p-value. (Answer: $\chi^2_3$; $p\approx0.0500$.)
A proposed likelihood-ratio comparison has a null restriction that fixes a parameter at the boundary of its allowed range. Is the standard Wilks chi-square conclusion justified solely by the dimension difference? (Answer: No; the usual interior regularity condition fails, so the standard $\chi^2_{p-q}$ calibration is not guaranteed.)

What You'll Learn