Likelihood-Ratio Goodness-of-Fit Test for a Factor Model

Why Test a Factor Model?

A q-factor model is useful when a covariance matrix can be represented through a smaller number of shared latent sources plus variable-specific variation. But a fitted model is not automatically an adequate description of the data. A likelihood-ratio goodness-of-fit test asks whether the covariance restrictions imposed by the model are consistent with the observed covariance, compared with a model that allows every covariance entry to vary freely.

The comparison is between nested covariance models: the q-factor model is restricted, while the saturated model contains every positive-definite covariance matrix. The likelihood ratio measures how much fit is lost by imposing the factor structure. Its usual chi-square calibration depends on the factor model being locally regular and identifiable after accounting for rotational redundancy.

Models and Likelihood-Ratio Statistic

Suppose \(X_1,\ldots,X_n\) are independent observations from a multivariate normal distribution, with a common but unknown mean. Define the maximum-likelihood sample covariance \(S_n=n^{-1}\sum_{i=1}^n(X_i-\bar X)(X_i-\bar X)^{\mathsf T}\). Assume \(n>p\), so that \(S_n\) is positive definite with probability one under a nonsingular normal distribution. Let \(\widehat\Sigma_q\) be a covariance matrix that maximizes the likelihood within the q-factor model; assume this constrained maximum exists. The saturated maximum-likelihood covariance is \(S_n\).

The mean estimate is \(\bar X\) in both models, so it cancels from the likelihood ratio. For any positive-definite candidate covariance \(\Sigma\), the part of the Gaussian log-likelihood depending on \(\Sigma\), after profiling out the mean, is

where \(C\) does not depend on \(\Sigma\). Substituting the constrained and saturated fits gives the test statistic.

This expression uses the \(n\)-denominator covariance \(S_n\). If a covariance computed with denominator \(n-1\) is used instead, the corresponding convention should be stated; the two common versions are asymptotically equivalent, but their finite-sample multipliers differ.

Why the Statistic Measures Lack of Fit

The determinant terms compare covariance volume, while the trace term measures how the fitted covariance scales and aligns with the observed covariance. The expression is always nonnegative, and it is zero when the factor fit reproduces the saturated covariance.

Proof. Let \(A=\widehat\Sigma_q^{-1/2}S_n\widehat\Sigma_q^{-1/2}\). This matrix is positive definite. Denote its positive eigenvalues by \(\lambda_1,\ldots,\lambda_p\). By invariance of determinant and trace under this representation, \(\operatorname{tr}(S_n\widehat\Sigma_q^{-1})=\sum_{j=1}^p\lambda_j\) and \(\log|S_n|-\log|\widehat\Sigma_q|=\sum_{j=1}^p\log\lambda_j\). Consequently, \[ D(S_n,\widehat\Sigma_q)=\sum_{j=1}^p(\lambda_j-\log\lambda_j-1). \] For \(x>0\), \(x-1-\log x\geq 0\): its derivative is \(1-1/x\), so it decreases up to \(x=1\), increases after \(x=1\), and equals zero at \(1\). Thus every term in the sum is nonnegative. Equality requires every \(\lambda_j=1\), which implies \(A=I_p\), and hence \(S_n=\widehat\Sigma_q\). The converse follows by substitution. \(\square\)

The statistic is \(n\) times this discrepancy. A larger discrepancy means a greater loss of Gaussian likelihood from restricting the covariance to the factor model. Nonnegativity alone does not establish that the chi-square approximation applies; that requires a separate regularity argument.

Degrees of Freedom and the Chi-Square Result

A saturated covariance matrix has \(p(p+1)/2\) distinct free entries. The q-factor parameterization appears to have \(pq+p\) parameters: \(pq\) loadings and \(p\) diagonal uniquenesses. However, the loading matrix is not uniquely determined by the covariance it produces. For any orthogonal \(q\times q\) matrix \(Q\), \((\Lambda Q)(\Lambda Q)^{\mathsf T}+\Psi=\Lambda\Lambda^{\mathsf T}+\Psi\). The continuous rotational indeterminacy has \(q(q-1)/2\) dimensions.

At a generic regular point, with no additional local nonidentification, this redundancy is the only loss of dimension in the parameterization. Thus the factor covariance model has dimension \(pq+p-q(q-1)/2\). The difference between the saturated and factor-model dimensions gives the degrees of freedom.

Justification. The saturated covariance model has dimension \(p(p+1)/2\). To obtain the local dimension of the factor model, consider a perturbation \(d\Lambda\) of the loadings and a diagonal perturbation \(d\Psi\). The corresponding first-order change in covariance is \(d\Lambda\Lambda^{\mathsf T}+\Lambda d\Lambda^{\mathsf T}+d\Psi\). For each skew-symmetric matrix \(A\), setting \(d\Lambda=\Lambda A\) and \(d\Psi=0\) makes this change zero, because \(A+A^{\mathsf T}=0\). These are the \(q(q-1)/2\) infinitesimal rotational directions. Under the stated assumption that there are no further directions in the kernel, the model’s local dimension is \(pq+p-q(q-1)/2\). Subtracting this from the saturated dimension gives \(\nu\). Wilks’ likelihood-ratio theorem applies to nested, smooth, locally identifiable models at an interior null point: the statistic converges to a chi-square distribution with degrees of freedom equal to the difference in local dimensions. This yields the stated limit. \(\square\)

The dimension calculation explains the number of restrictions being assessed; Wilks’ theorem explains the asymptotic reference distribution. The conditions matter. A boundary estimate, a singular point, or additional local nonidentification can invalidate the regular chi-square calibration. Those cases should not be treated as ordinary applications of this result.

Worked Calculations

Interpretation and Common Pitfalls

A small statistic indicates that the factor restrictions cost relatively little likelihood compared with the saturated model. A large statistic indicates substantial loss of fit and, under the null and regularity conditions, counts as evidence against the proposed covariance structure. The test assesses covariance fit under the Gaussian model; it does not by itself establish that the factors have a particular substantive interpretation.

• Forgetting rotational redundancy. Counting all \(pq+p\) factor parameters as distinct covariance parameters overstates the model dimension and understates the degrees of freedom. Subtract \(q(q-1)/2\) under the generic regular conditions stated above.
• Using the formula without checking regularity. A boundary estimate or additional identification failure can make the local model nonregular, so the ordinary Wilks chi-square result need not apply.
• Using a candidate fit as though it were the MLE. The likelihood-ratio statistic requires the maximum likelihood over the factor model. Evaluating the discrepancy at an arbitrary feasible covariance can overstate the statistic.
• Mixing covariance conventions. The \(n\)-denominator likelihood formula and an \(n-1\)-based reporting convention have different finite-sample multipliers. State which one is used.