Wilks Calibration For Covariance Tests

Why a Likelihood-Ratio Statistic Has a Chi-Square Calibration

A covariance model may impose restrictions on a positive-definite covariance matrix: for example, several entries may be linked by a lower-dimensional parameterization. A likelihood-ratio test compares the best fit under those restrictions with the best fit in a larger model. The central calibration question is how much improvement is expected just from allowing the extra parameters to vary.

Under suitable regularity conditions, that improvement has an asymptotic chi-square distribution. Its degrees of freedom are the difference between the dimensions of the unrestricted and restricted models. This result is Wilks’ theorem. The dimension rule is useful, but it is not automatic: the local likelihood must be well behaved, and its normalized score must converge to a normal distribution.

Definition: Let $\ell(\theta)$ be the log-likelihood for a parameter $\theta$ in a model of dimension $d$, and let $\Theta_0$ be a nested null model of dimension $d_0$. The likelihood-ratio statistic is $$ W_n=2\left\{\sup_{\theta\in\Theta}\ell(\theta)-\sup_{\theta\in\Theta_0}\ell(\theta)\right\}. $$ The dimension difference is $q=d-d_0$. When the regularity conditions for Wilks’ theorem hold and the null model is true, $W_n\xrightarrow{d}\chi^2_q$.

Here, dimension means the number of locally free, identifiable parameters, not the number of entries written in a covariance matrix. For a $p$-dimensional random vector, a general symmetric covariance matrix has $$ d_{\Sigma}=\frac{p(p+1)}{2} $$ free entries: $p$ variances and $p(p-1)/2$ distinct covariances. If the model also contains a mean vector, its parameters must be included in both model dimensions when relevant; parameters common to the null and alternative cancel in the difference.

Conditions Behind the Calibration

A standard setting is an independent and identically distributed sample $X_1,\ldots,X_n$ from a regular parametric model. The parameter must be identifiable and lie in the interior of an open parameter space, and the Fisher information per observation must be finite and positive definite. The null model must be a smooth, locally identifiable submodel of dimension $d_0$, with no boundary or singularity at the true parameter. The unrestricted and restricted likelihood maximizers must be consistent and have root-$n$ tight local coordinates.

The likelihood also needs a local quadratic approximation. Write $\theta_0$ for the true parameter under the null, $I$ for its positive-definite Fisher information per observation, and $S_n=n^{-1/2}\,\partial\ell(\theta_0)/\partial\theta$ for the normalized score. A suitable local asymptotic normality condition is that, for every fixed $M<\infty$, uniformly over $\|h\|\le M$,

\ell(\theta_0+h/\sqrt n)-\ell(\theta_0) =h^\mathsf{T}S_n-\frac12h^\mathsf{T}Ih+o_p(1).

Crucially, this condition for the likelihood expansion does not by itself assert that $S_n$ is asymptotically normal. Wilks’ calibration also requires the score central limit theorem

S_n\xrightarrow{d}N_d(0,I), \qquad \Delta_n=I^{-1/2}S_n\xrightarrow{d}N_d(0,I_d).

In a regular independent and identically distributed likelihood, appropriate differentiability, integrability, and identifiability assumptions can yield this score limit. Alternatively, it can be stated directly as an assumption. The essential point is that finite, positive-definite information and a local quadratic expansion alone are not a substitute for score convergence.

Wilks’ Theorem and Its Geometric Derivation

Theorem (Wilks): Suppose the null hypothesis is true at an interior point $\theta_0$ of a regular $d$-dimensional model. Assume the local likelihood expansion and score convergence above, and assume the unrestricted and restricted maximizing local coordinates are root-$n$ tight. Suppose also that the null is a smooth submodel of dimension $d_0$, so its local alternatives, after scaling by $\sqrt n$, converge to a $d_0$-dimensional tangent space. Then $$ W_n\xrightarrow{d}\chi^2_{d-d_0}. $$

For a smooth submodel, curvature does not alter its first-order local shape: after scaling by $\sqrt n$, its allowable local directions approach its tangent space at $\theta_0$. Let $T$ denote that tangent space in the original parameter coordinates. The Fisher information supplies the local quadratic metric, so it is helpful to transform coordinates to make that metric Euclidean.

Set $u=I^{1/2}h$ and $Z_n=I^{-1/2}S_n$. The local log-likelihood increment becomes

h^\mathsf{T}S_n-\frac12h^\mathsf{T}Ih =Z_n^\mathsf{T}u-\frac12\|u\|^2, \qquad Z_n\xrightarrow{d}N_d(0,I_d).

Under this change of coordinates, the null tangent space becomes a linear subspace $L=I^{1/2}T$ of dimension $d_0$. Complete the square:

Z_n^\mathsf{T}u-\frac12\|u\|^2 =\frac12\|Z_n\|^2-\frac12\|u-Z_n\|^2.

Over all $u\in\mathbb R^d$, this expression is maximized at $u=Z_n$, with maximum $\frac12\|Z_n\|^2$. Over $u\in L$, the closest point to $Z_n$ is its orthogonal projection $P_LZ_n$. Thus the restricted maximum is $\frac12\|P_LZ_n\|^2$. The root-$n$ tightness and uniform local expansion allow these local maxima to determine the likelihood suprema up to $o_p(1)$; the smooth null’s scaled local sets converge to $L$. Therefore,

W_n=\|Z_n\|^2-\|P_LZ_n\|^2+o_p(1) =\|P_{L^\perp}Z_n\|^2+o_p(1).

Because $Z_n$ converges to a standard $d$-dimensional normal vector, its projection onto the $(d-d_0)$-dimensional space $L^\perp$ has $d-d_0$ independent standard normal coordinates in an orthonormal basis. The sum of their squares has a $\chi^2_{d-d_0}$ distribution. Convergence in distribution and the continuous mapping theorem give the claimed limit. $\square$

Worked Covariance Calibrations

Worked Example: One Smooth Restriction in a Three-Variable Covariance

Suppose $X_1,\ldots,X_n$ are independent $N_3(\mu,\Sigma)$ observations, with $\mu$ unknown and $\Sigma$ unrestricted and positive definite under the alternative. Test the null restriction $\sigma_{12}=\sigma_{13}$. The full covariance has $3(4)/2=6$ free parameters. The null imposes one independent equality between two covariance coordinates, so its covariance dimension is $6-1=5$. The common mean contributes three parameters to each model and cancels. Hence $q=1$.

For instance, the covariance matrix with diagonal entries $4,9,16$ and off-diagonal entries $\sigma_{12}=0.3,\sigma_{13}=0.3,\sigma_{23}=0.2$ satisfies the null and is positive definite: each diagonal entry exceeds the sum of the absolute off-diagonal entries in its row. Thus the null can hold at an interior covariance parameter.

If the observed likelihood-ratio statistic is $W_n=4.10$, the asymptotic reference distribution is $\chi^2_1$. Its 5% upper critical value is approximately $3.841$, and $4.10>3.841$, so the asymptotic test rejects at level 0.05. The corresponding upper-tail probability is about $0.043$. These are asymptotic, not exact finite-sample, calculations.

Worked Example: Two Independent Restrictions in a Four-Variable Covariance

In a four-variable Gaussian model, the unrestricted covariance dimension is $4(5)/2=10$. Consider the null restrictions $\sigma_{12}=\sigma_{34}$ and $\sigma_{13}=0$. These constrain distinct covariance coordinates and are locally independent, so the null dimension is $10-2=8$, and $q=2$. The identity matrix satisfies both restrictions and is positive definite, verifying that the null includes an interior parameter value.

Suppose the observed statistic is $W_n=6.40$. The reference distribution is $\chi^2_2$, whose 5% upper critical value is approximately $5.991$. Since $6.40>5.991$, reject the null at the asymptotic 5% level. For two degrees of freedom, the upper-tail probability at $6.40$ is $\exp(-6.40/2)=\exp(-3.20)\approx0.0408$.

Worked Example: A Common-Correlation Covariance Model

For $p=3$, consider the covariance model with three freely varying positive variances and one common correlation $\rho$: $\sigma_{ii}=\sigma_i^2$ and $\sigma_{ij}=\rho\sigma_i\sigma_j$ for $i\ne j$. Restrict $-1/2<\rho<1$, which makes the three-dimensional equicorrelation matrix positive definite. This null model has dimension $3+1=4$, while the unrestricted covariance has dimension $6$. Therefore $q=6-4=2$.

For example, $\sigma_1^2=1,\sigma_2^2=4,\sigma_3^2=9,\rho=0.2$ is a valid interior null parameter: the correlation matrix has eigenvalues $1-\rho=0.8$ twice and $1+2\rho=1.4$, all positive. If $W_n=3.00$, its asymptotic upper-tail probability under $\chi^2_2$ is $\exp(-3.00/2)\approx0.2231$. Since this exceeds 0.05 (equivalently, $3.00<5.991$), the test does not reject at the 5% level.

When the Dimension Rule Is Not Enough

The dimension difference is the degrees of freedom only when the null is a regular, locally smooth submodel and the other Wilks conditions hold. Boundary parameters, non-identifiability, singular information, or restrictions that meet at a singular point can produce a limit other than a single chi-square distribution. The statistic’s numerical value also does not determine the degrees of freedom; those come from the model dimensions under the null and alternative.

A particularly important pitfall is to argue that a quadratic likelihood expansion automatically gives a chi-square limit. The expansion describes the likelihood locally, but the random linear term in it must also have a limiting distribution. Without a score central limit theorem, the projection calculation has no basis for treating that term as standard normal, and the claimed chi-square calibration need not follow. State the score convergence assumption explicitly, or verify conditions that imply it.

For covariance tests, Wilks’ result supplies a general first-order reference distribution for regular restrictions. It does not claim that the chi-square approximation is exact at a finite sample size. The sample size, dimension, and model-specific likelihood behavior affect approximation quality; finite-sample refinements are a separate question.

Check Your Understanding

Use the parameter dimensions and conditions in the tutorial to answer the following.

How many free parameters does an unrestricted covariance matrix have when $p=5$?
A null covariance model has dimension 7 and its unrestricted alternative has dimension 11. What is the Wilks degrees of freedom, assuming regularity?
Why must a score central limit theorem be stated or established in addition to a local quadratic likelihood expansion?
In the common-correlation example, why are there two degrees of freedom rather than three?
Name two forms of irregularity that can invalidate the ordinary chi-square calibration.