When Chi-Square Calibration Is Unreliable

Why the Reference Distribution Needs a Check

A likelihood-ratio statistic for covariance restrictions is often compared with a chi-square distribution. That comparison is justified by an asymptotic theorem, not by the fact that the statistic is a likelihood ratio. In finite samples, the approximation can be poor; in some designs, the likelihood or the unrestricted covariance estimate may not even be well-defined. A boundary in the parameter space can also change the limiting reference distribution.

The practical question is therefore not simply whether a chi-square table is available. It is whether the model, sample size, dimension, and constraints support the approximation being used. The discussion below assumes independent observations from a multivariate normal model unless stated otherwise. It concerns calibration of covariance likelihood-ratio tests, not a general comparison of covariance testing procedures.

Definition: For a null hypothesis $H_0$ nested in an alternative model $H_1$, the likelihood-ratio statistic is $$ T=2\{\ell(\hat{\theta}_1)-\ell(\hat{\theta}_0)\}, $$ where $\ell(\hat{\theta}_1)$ and $\ell(\hat{\theta}_0)$ are the maximized log-likelihoods under the alternative and null, respectively. Under regularity conditions and a fixed difference $q$ in the dimensions of the models, Wilks' theorem gives $T\xrightarrow{d}\chi^2_q$ under $H_0$.

The degrees of freedom $q$ count the independent restrictions in a regular, locally identifiable model. For example, a symmetric $p\times p$ covariance matrix has $p(p+1)/2$ free entries, but the relevant $q$ is the difference in free parameter dimensions between the two models, including any restrictions after nuisance parameters have been accounted for. Counting written equations is not always enough: restrictions may be redundant or may fail to define a smooth model at the null.

What Wilks' Theorem Requires

For the usual chi-square calibration, the null parameter must be an interior point of a smooth parameter space, the model must be identifiable near that point, and the Fisher information must be nonsingular. The likelihood must be sufficiently smooth, and the estimators must behave regularly as the sample size grows. In covariance inference, the dimension $p$ and the number $q$ of tested restrictions are held fixed as $n$ grows. A positive-definite covariance matrix is an interior point of the space of positive-definite matrices; a singular covariance matrix is not.

A useful way to see the theorem is through the local quadratic approximation to the log-likelihood. After nuisance parameters are accounted for and the information is used to rescale coordinates, the local score is asymptotically a standard normal vector. The unrestricted fit can move in all local directions, whereas the null fit is restricted to a subspace. The likelihood improvement is asymptotically the squared length of the score component in the directions excluded by the null.

Theorem (Wilks' theorem, regular nested case): Suppose the model is identifiable and differentiable in a neighborhood of the null, the true parameter is an interior point, the Fisher information is positive definite, and the maximum-likelihood estimators satisfy the usual consistency and local asymptotic normality conditions. If $H_0$ is a smooth nested restriction of codimension $q$, then under $H_0$, $T\xrightarrow{d}\chi^2_q$.

Use local coordinates.
Whiten the local parameter coordinates by the information matrix. The score then converges in distribution to $Z\sim N(0,I_d)$, where $d$ is the dimension of the full model.

Compare the local fits.
The quadratic log-likelihood approximation is a constant minus one-half the squared distance from the score coordinate to the candidate parameter displacement. The unrestricted fit projects onto the full $d$-dimensional space; the null fit projects onto its tangent subspace, of dimension $d-q$.

Take the difference.
The likelihood-ratio statistic converges to the squared norm of the component of $Z$ orthogonal to the null tangent space. This component has $q$ independent standard normal coordinates, so its squared norm has a $\chi^2_q$ distribution. The local quadratic approximation and regular estimator behavior make the remainder converge to zero in probability. $\square$

This result explains both the usefulness and the limitation of chi-square calibration. It is an asymptotic statement under regular conditions. It does not promise that the finite-sample distribution is close to the limiting distribution for a particular $n$, nor does it cover a boundary point or an ill-defined fit.

Worked Examples: When the Approximation Can Mislead

Worked Example: A Small-Sample Variance Likelihood Ratio

Let $X_1,\ldots,X_n$ be independent $N(\mu,\sigma^2)$ observations, with $\mu$ unknown. Test $H_0:\sigma^2=\sigma_0^2$ against an unrestricted positive variance. Set $U=\sum_{i=1}^n(X_i-\bar X)^2/\sigma_0^2$. Under the null, $U\sim\chi^2_{n-1}$.

The unrestricted maximum-likelihood estimates are $\hat{\mu}=\bar X$ and $\hat{\sigma}^2=\sum_i(X_i-\bar X)^2/n=\sigma_0^2 U/n$. Under the null, the estimate of $\mu$ remains $\bar X$. Substituting these estimates into the normal log-likelihood gives $$ T=n\log\!\left(\frac{\sigma_0^2}{\hat{\sigma}^2}\right)+\frac{\sum_i(X_i-\bar X)^2}{\sigma_0^2}-n =n\log\!\left(\frac{n}{U}\right)+U-n. $$

For $n=4$ and observed $U=1.5$, this is $$ T=4\log(4/1.5)+1.5-4\approx 1.4233. $$

The routine asymptotic reference is $\chi^2_1$, whose upper-tail probability at $1.4233$ is about $0.233$. But the exact null tail is obtained from $U\sim\chi^2_3$, not by treating $T$ as exactly $\chi^2_1$. The function $4\log(4/u)+u-4$ decreases up to $u=4$ and increases after $u=4$, since its derivative is $1-4/u$. Thus $T\geq1.4233$ corresponds approximately to $U\leq1.5$ or $U\geq8.38$. The exact tail probability is therefore $P(\chi^2_3\leq1.5)+P(\chi^2_3\geq8.38)$, about $0.36$. This substantial difference at $n=4$ illustrates why an asymptotic reference should not automatically be treated as exact.

Worked Example: Scatter Is Singular When Dimension Is Too High

Suppose $n=4$ independent $p=4$ dimensional observations are centered by their sample mean. The centered data vectors sum to zero, so they span a space of dimension at most $n-1=3$. The sample scatter matrix, which is the sum of their outer products, therefore has rank at most three. A $4\times4$ matrix of rank at most three has determinant zero.

In an unrestricted multivariate normal model, the log-likelihood as a function of a positive-definite covariance matrix includes a term proportional to $-\tfrac n2\log|\Sigma|$ and a term involving the inverse of $\Sigma$. With singular sample scatter, one can shrink the covariance along a direction with no observed centered variation, making the likelihood unbounded as that covariance eigenvalue approaches zero. There is no finite positive-definite unrestricted maximum-likelihood estimate in this setting.

Consequently, a likelihood-ratio statistic that requires that unrestricted fit cannot be computed by simply taking a determinant ratio and comparing it with a chi-square distribution. The obstacle is not merely that the chi-square approximation might be rough: the assumed regular likelihood calculation has broken down. A different inferential method or an explicitly justified lower-dimensional model is needed.

Worked Example: A Boundary Produces a Mixture Reference

Consider a Gaussian variance parameter constrained under the alternative to $\sigma^2\geq\sigma_0^2$, and test $H_0:\sigma^2=\sigma_0^2$. The null lies at the boundary of this one-sided alternative. In a regular one-parameter setting, the local score is asymptotically $Z\sim N(0,1)$, but the fitted local displacement cannot be negative. The likelihood-ratio statistic therefore converges to $\{\max(Z,0)\}^2$.

If $Z\leq0$, the constrained fit stays at the null and the limiting statistic is zero; this occurs with probability $1/2$. If $Z>0$, the statistic is $Z^2$. Hence the limit is a mixture with probability $1/2$ at zero and probability $1/2$ distributed as $\chi^2_1$. For a 5% upper-tail test, the critical value $c$ solves $\tfrac12P(\chi^2_1\geq c)=0.05$, so $c$ is the 90th percentile of $\chi^2_1$, approximately $2.706$. Using the ordinary $\chi^2_1$ 95th percentile, $3.841$, instead gives limiting rejection probability $0.025$, not $0.05$, for this one-sided case.

This mixture conclusion depends on the stated local one-sided geometry and regularity; it is not a universal correction for every covariance constraint. In particular, singular covariance matrices and unidentified parameters require their own analysis.

When to Seek Another Calibration

Several warning signs deserve attention before interpreting a chi-square p-value:

Small $n$ relative to the number of covariance parameters. Even with a positive-definite sample scatter matrix, the limiting approximation may be inaccurate when the information supporting each parameter is limited.
Dimension close to the rank limit. Centered sample scatter has rank at most $\min(p,n-1)$. When $p\geq n$, it must be singular, and when $p$ is near $n$, estimates and determinant-based likelihood calculations can be unstable.
A null value on a constraint boundary. Examples include a variance component constrained to be nonnegative and a correlation at an admissible endpoint. The local alternatives may form a cone rather than a full linear space, changing the limiting law.
Weak identification or nearly redundant restrictions. The nominal number of restrictions may not match the effective local dimension, undermining the claimed degrees of freedom.

A useful alternative, when a finite-sample reference is available under the model, is to use it. Otherwise, a parametric bootstrap can approximate the null distribution: fit the model under $H_0$, simulate data under that constrained fit, and in every replicate refit both $H_0$ and $H_1$ before recalculating $T$. Refitting both models matters because it reproduces the statistic actually used. If $B$ replicates give statistics $T_1^*,\ldots,T_B^*$, a Monte Carlo upper-tail estimate is $$ \hat p=\frac{1+\sum_{b=1}^B I(T_b^*\geq T_{\mathrm{obs}})}{B+1}. $$

This procedure is an approximation too: it conditions on the fitted null model, and its accuracy depends on whether that model is adequate and whether the fits are computable in the simulated samples. A bootstrap cannot repair a likelihood that is unbounded or a parameter that is not identifiable. For a boundary problem, the simulation must preserve the constraint and the observed boundary situation; using an unconstrained simulation would calibrate a different test.

Common Mistake

Wrong reasoning: “The test is a likelihood-ratio test with three restrictions, so its p-value must come from $\chi^2_3$.”

Correction: Three restrictions imply three chi-square degrees of freedom only when they define a regular, locally identifiable, interior null and the usual asymptotic conditions hold. Check that the likelihood fits exist, that the sample scatter supports the dimension, and that no constraint places the null on a boundary. Then assess whether the sample size makes the asymptotic approximation credible. If these checks fail, use a justified finite-sample law, a boundary-specific reference, or a carefully designed simulation calibration.

Key takeaway: Chi-square calibration is a consequence of regular asymptotic likelihood theory, not an automatic property of covariance likelihood ratios. Small samples can make the approximation inaccurate; high dimension can make the unrestricted fit singular; and boundary constraints can change the limiting distribution.

Check Your Understanding

Use the assumptions and calculations in this tutorial to answer the following questions.

Why does a regular likelihood-ratio test with $q$ restrictions have an asymptotic $\chi^2_q$ reference, and name two conditions needed for that result.
For centered observations in dimension $p$, why is the rank of the sample scatter at most $n-1$? What follows when $p\geq n$?
In the univariate variance example, what exact random variable determines the null distribution of the likelihood-ratio statistic?
Why does a one-sided boundary constraint lead to a point mass at zero in the example, and what is the resulting limiting mixture?
In a parametric-bootstrap calibration, why must both the null and alternative models be refitted for every simulated data set?