Diagnosing The Wilks Approximation

Why Diagnose the Approximation?

A likelihood-ratio statistic is often compared with a chi-square distribution because of Wilks’ theorem. That comparison is justified under specific large-sample and model conditions; it is not an automatic consequence of calculating a likelihood ratio. A statistic can be computed correctly and still have a poor chi-square calibration if data are sparse, parameters are not identifiable, or the model fails a regularity condition.

The practical question is not simply whether the total sample size is large. It is whether the information in the data supports the fitted model and the particular chi-square approximation being used. This diagnosis is aligned with the SOA VEE Mathematical Statistics syllabus.

Definition: For a null model nested in an alternative model, the likelihood-ratio statistic is $$ G^2=2\left\{\ell(\widehat{\theta}_{\mathrm{alt}})-\ell(\widehat{\theta}_{0})\right\}, $$ where $\ell$ is the log-likelihood, $\widehat{\theta}_{\mathrm{alt}}$ maximizes it over the alternative model, and $\widehat{\theta}_{0}$ maximizes it subject to the null hypothesis. Under the conditions of Wilks’ theorem, $G^2$ is approximately distributed as $\chi^2_q$, where $q$ is the difference in the numbers of independently identifiable parameters in the alternative and null models.

What Wilks’ Approximation Requires

The chi-square reference distribution is an asymptotic result. In its usual setting, observations are generated from a correctly specified parametric model, the null hypothesis is true, and the models are nested. The parameter dimensions are fixed as the sample size grows. The true parameter is in the interior of the parameter space, the model is identifiable, and the likelihood is sufficiently smooth near the true parameter, with nonsingular Fisher information. These conditions allow the likelihood near its maximum to behave approximately like a quadratic function.

Each condition matters. If distinct parameter values generate the same distribution, the model is not identifiable in the relevant direction. If the information matrix is singular, the usual quadratic approximation can fail. A parameter at the edge of its allowed range also requires care; a standard chi-square reference distribution should not be assumed just because a parameter-count difference can be written down. The precise behavior of boundary cases is a separate issue.

There is no universal sample-size cutoff that guarantees a good approximation. For multinomial goodness-of-fit calculations, expected cell counts are more informative than the total alone: a large total can still produce small expected counts in rare categories. Rules of thumb about expected counts are diagnostics, not theorems. They flag situations for scrutiny; they do not prove that calibration is accurate or inaccurate.

A Practical Diagnostic

Check the model and nesting.
Confirm that the null model is contained in the alternative and that both describe the data-generating process plausibly.

Check the parameter count.
Count independently identifiable parameters, not merely symbols in a formula. Redundant parameters do not create genuine degrees of freedom.

Inspect the information in the sample.
For count data, examine fitted or null expected counts by cell. Also look for a small sample, extreme estimates, or categories with few observations.

Check regularity.
Ask whether the null parameter is interior, the model is identifiable, and the likelihood is locally smooth with nonsingular information.

Choose calibration to match the diagnosis.
Use the chi-square approximation when the regularity and information conditions are credible. Prefer an exact null distribution when available, or a carefully designed simulation when it is not.

Worked Examples

Worked Example: A Regular Multinomial Goodness-of-Fit Test

A hypothetical service center classifies $100$ incoming requests into three categories. Under a prespecified null model, the category probabilities are $0.50$, $0.30$, and $0.20$. The observed counts are $55$, $27$, and $18$. Assess whether chi-square calibration is plausible and calculate the approximate upper-tail probability.

Method. Compare observed counts with their null expected counts, calculate the multinomial likelihood-ratio statistic, and check the model conditions. The probabilities are fixed under the null, so the saturated three-category model has two independent parameters more than the null model.

The expected counts are $100(0.50)=50$, $100(0.30)=30$, and $100(0.20)=20$. Each is comfortably above the small-count range, and the total sample is $100$. This supports, but does not prove, an asymptotic approximation. The null probabilities are specified and the category probabilities are positive, so the usual multinomial setup has no apparent identifiability or boundary issue.

For multinomial counts $O_j$ and expected counts $E_j$, the likelihood-ratio statistic is $2\sum_j O_j\log(O_j/E_j)$, with a zero-count contribution defined as zero. Here,

\begin{aligned} G^2 &=2\left[55\log\left(\frac{55}{50}\right) +27\log\left(\frac{27}{30}\right) +18\log\left(\frac{18}{20}\right)\right]\\ &=2\left[5.242060-2.844734-1.896489\right]\\ &=2(0.500837)=1.001673. \end{aligned}

The degrees of freedom are $3-1=2$: the saturated model has two free cell probabilities, while the null model has none. For $\chi^2_2$, the upper-tail probability is $\exp(-G^2/2)$. Thus it is $\exp(-1.001673/2)=0.6060$ to four decimals. The statistic is not unusual under this null model according to the approximation.

Check and common mistake. The weighted log terms sum to $0.500837$, so the required factor of $2$ gives $1.001673$, not $0.500837$. The expected counts sum to $50+30+20=100$, matching the observed total. The diagnostic supports chi-square calibration here; it does not establish that the null model itself is true.

Worked Example: A Small Binomial Sample and Exact Calibration

A hypothetical inspection tests $12$ units, each independently defective with null probability $p_0=0.5$. Ten units are defective. Compare the Wilks approximation for the two-sided likelihood-ratio test with exact null calibration.

Method. Under the null, $X\sim\operatorname{Binomial}(12,0.5)$. The unrestricted estimate is $\widehat p=10/12=5/6$. There is one additional identifiable parameter under the alternative, so the nominal Wilks reference is $\chi^2_1$.

The null expected counts for defective and nondefective units are both $12(0.5)=6$. Those counts are not extremely small, but the sample is modest and the observation is in a tail. Calculate the statistic directly:

\begin{aligned} G^2 &=2\left[10\log\left(\frac{5/6}{1/2}\right) +2\log\left(\frac{1/6}{1/2}\right)\right]\\ &=2\left[10\log\left(\frac{5}{3}\right) +2\log\left(\frac{1}{3}\right)\right]\\ &=2(5.108256-2.197225)=5.822063. \end{aligned}

The $\chi^2_1$ upper-tail probability is approximately $0.0158$. For exact calibration, use the null binomial distribution and include outcomes whose likelihood-ratio statistic is at least as large as the observed one. The statistic is symmetric around $6$; the qualifying outcomes are $X\leq2$ or $X\geq10$. Therefore,

P_{0.5}(X\leq2\text{ or }X\geq10) =2\frac{\binom{12}{0}+\binom{12}{1}+\binom{12}{2}}{2^{12}} =2\frac{1+12+66}{4096} =\frac{158}{4096}=0.0386.

The exact probability differs materially from the asymptotic one for this sample and observed tail. The exact calculation is preferable because the binomial null distribution is available directly. The key trap is to call expected counts of $6$ a guarantee of accurate calibration; they are a useful check, not a guarantee. Also, the exact tail here is defined by the likelihood-ratio statistic, not by automatically doubling a one-sided probability.

Worked Example: Redundant Parameters and False Degrees of Freedom

A hypothetical binary-response model specifies the success probability as $p=\{1+\exp[-(\alpha+\beta)]\}^{-1}$. The null hypothesis is $\beta=0$; under the alternative, both $\alpha$ and $\beta$ are free. In $10$ independent observations there are $7$ successes. Determine whether a one-degree-of-freedom Wilks approximation is justified.

Method. Check whether the additional parameter changes the set of distributions the model can represent. The likelihood for $7$ successes and $3$ failures depends on the parameters only through $p$:

\ell(\alpha,\beta)=7\log(p)+3\log(1-p), \qquad p=\frac{1}{1+\exp[-(\alpha+\beta)]}.

Under the null, $\beta=0$, and $\alpha$ can produce any probability strictly between $0$ and $1$. Under the alternative, $\alpha+\beta$ can also produce any such probability. Both models therefore have the same set of possible distributions. The maximum occurs at $\widehat p=7/10=0.7$ in both models:

\widehat{\ell}_{\mathrm{alt}}=\widehat{\ell}_{0} =7\log(0.7)+3\log(0.3), \qquad G^2=2(0)=0.

It would be wrong to assign one degree of freedom just because the alternative formula contains one more parameter symbol. Only the sum $\alpha+\beta$ is identifiable from these observations; the parameters cannot be separated. The nominal parameter difference does not represent an additional identifiable direction, and the regular Wilks conditions fail. The appropriate correction is to reparameterize in terms of the identifiable quantity or specify a model in which the parameters have distinct effects. No ordinary one-degree-of-freedom calibration is justified by this setup.

Choosing Between Approximation and Alternatives

When chi-square calibration is doubtful, the alternative depends on the model and the hypothesis. If a discrete null distribution can be calculated or enumerated, an exact test can calibrate the statistic without relying on a large-sample limit. Exact tests may be conservative because attainable significance levels are discrete, but their calibration is based on the stated null model.

Simulation is useful when exact enumeration is impractical. A parametric simulation should generate data under the null model, calculate the likelihood-ratio statistic for each simulated data set, and compare the observed statistic with that simulated null distribution. For a composite null, nuisance parameters must be handled explicitly. A common approach is to estimate them under the null, simulate at those estimates, and refit both null and alternative models in every replicate. This is a plug-in calibration, not automatically an exact one; results may depend on the nuisance-parameter estimate and simulation design.

A large sample does not repair every model defect. It does not make a nonidentifiable parameter identifiable, and it does not by itself establish that the assumed probability model is correct. Conversely, a modest total sample does not automatically invalidate Wilks’ approximation if the model is regular and the relevant information is adequate. The diagnosis uses the model structure and the data together.

Common Exam Traps

Using total sample size alone. Check expected counts where the model assigns probability, especially in rare categories.
Counting parameter symbols. Count identifiable dimensions in the distributions, not names appearing in the likelihood.
Treating a rule of thumb as a theorem. A count threshold is a warning signal, not a proof of calibration accuracy.
Assuming every nested likelihood ratio is chi-square. Nesting is necessary for the usual result, but regularity and identifiability also matter.
Comparing unlike calibrations without defining the tail. State whether an exact test includes outcomes at least as extreme according to the likelihood-ratio statistic.

Key takeaway: Use a chi-square reference distribution only after checking the nesting, identifiable parameter difference, sample information, and regularity conditions. If sparse data or model structure makes the approximation doubtful, use an exact null distribution when available or a null-based simulation with the fitting procedure repeated.

Check Your Understanding

For each problem, decide what the calculation says about Wilks calibration as well as computing any requested quantity.

A hypothetical three-category process has null probabilities $0.50,0.30,0.20$, sample size $40$, and observed counts $22,10,8$. Find expected counts, $G^2$, degrees of freedom, and the approximate upper-tail probability. (Answer: Expected counts $20,12,8$; $G^2=0.5472$; $2$ degrees of freedom; approximate upper-tail probability $0.7606$.)
Under a binomial null with $n=10$ and $p_0=0.5$, observe $9$ successes. Calculate $G^2$, then give the exact two-sided likelihood-ratio tail probability. (Answer: $G^2=7.3613$; exact probability $22/1024=0.0215$, compared with an approximate $\chi^2_1$ tail of about $0.0067$.)
A multinomial null assigns probabilities $0.70,0.20,0.10$, and a sample of $15$ has counts $14,1,0$. Find the expected counts and state a suitable calibration strategy. (Answer: Expected counts $10.5,3,1.5$; sparse expected counts make the chi-square approximation questionable, so use exact multinomial calibration if feasible or null-based simulation.)
A model has success probability $p=\{1+\exp[-(\gamma+\delta)]\}^{-1}$, and the null is $\delta=0$, with $\gamma$ unrestricted. Explain whether adding $\delta$ gives one valid Wilks degree of freedom. (Answer: No; both models represent every $0<p<1$, only $\gamma+\delta$ is identifiable, and the added parameter does not create a distinct identifiable direction.)

What You'll Learn