Nested Factor-Number Tests and Boundary Nonregularity

Why Adding One Factor Is Not Automatically a Regular Test

A likelihood-ratio test comparing \(q\) factors with \(q+1\) factors appears to be a standard nested-model comparison: the smaller model is obtained by setting the new factor's loadings to zero. But this nesting has a special feature. Under the null, the added factor contributes no covariance, so its orientation is not determined by the observed variables. The likelihood can therefore be locally flat in directions that a routine chi-square argument expects to be informative.

There is a second source of difficulty. Factor-analysis covariance models often constrain uniqueness variances to be nonnegative. If a fitted uniqueness reaches zero, the parameter is on the boundary of its allowed region. A regular likelihood-ratio calibration assumes an interior null point in a locally smooth model; a boundary point may instead lead to a mixture distribution or another nonstandard limit.

Definition: In the Gaussian common-factor model with \(p\) observed variables and \(q\) factors, the covariance matrix is \(\Sigma=\Lambda\Lambda^\mathsf{T}+\Psi\), where \(\Lambda\) is a \(p\times q\) loading matrix and \(\Psi\) is diagonal. The \(q\)-factor model is nested in the \((q+1)\)-factor model by writing the latter's loading matrix as \((\Lambda,a)\) and setting the added loading vector \(a\in\mathbb{R}^p\) to zero. The likelihood-ratio statistic is \(T=2\{\ell_{q+1}-\ell_q\}\), where each log-likelihood is maximized over its stated model.

For ordinary maximum-likelihood factor analysis, the covariance model is commonly specified with positive uniquenesses, \(\psi_i>0\), and positive-definite \(\Sigma\). Some formulations allow \(\psi_i=0\); that changes the parameter space by admitting boundary points. In either formulation, the parameterization also has rotational redundancy: replacing \(\Lambda\) by \(\Lambda Q\), for an orthogonal matrix \(Q\), leaves \(\Lambda\Lambda^\mathsf{T}\) unchanged. Identification restrictions or an equivalent treatment of this redundancy are needed before applying dimension arguments.

When the Usual Chi-Square Argument Applies

Wilks' theorem gives the familiar reference distribution only under regularity conditions. In particular, the null must be an interior point of a smooth, locally identifiable model; the Fisher information must be nonsingular in identifiable coordinates; and the nested models must have smooth local parameter spaces. The observations must also satisfy the usual likelihood regularity conditions, such as independent sampling from a correctly specified model with differentiable likelihood and finite nonsingular information.

Theorem: Suppose the null and alternative are regular, identifiable, nested models, with local dimensions \(d_0\) and \(d_1\), where \(d_1>d_0\), and the null is an interior point of the alternative. Under the standard maximum-likelihood regularity conditions, the likelihood-ratio statistic converges under the null to \(\chi^2_{d_1-d_0}\).

The reason is local. After centering at the true parameter and scaling by \(\sqrt n\), the log-likelihood is approximated by a quadratic whose random linear term is asymptotically normal. In coordinates whitened by the Fisher information, maximizing this quadratic over the alternative amounts to projecting a standard normal vector onto the alternative's local parameter space. Maximizing over the null makes the corresponding projection onto the smaller tangent space. For nested linear tangent spaces, the difference of the two squared projection lengths is the squared length of a standard normal projection onto a space of dimension \(d_1-d_0\), hence has a chi-square distribution with that many degrees of freedom. This projection argument depends on smooth tangent spaces and nonsingular information; it does not establish the result at a singular or boundary point. \(\square\)

A parameter count is useful only after those geometric conditions have been checked. In the range where the factor parameterization is generically locally identified and the covariance model is not saturated, its generic dimension is often counted as \(pq+p-q(q-1)/2\): there are \(pq\) loadings, \(p\) uniquenesses, and \(q(q-1)/2\) dimensions of orthogonal rotational redundancy. This expression is not a universal dimension formula. The covariance space itself has dimension \(p(p+1)/2\), so a model's dimension cannot exceed that ceiling; outside the stated range, saturation or identification failures require separate analysis.

Worked Example: A Nominal Dimension Difference

Suppose \(p=6\), and assume the one- and two-factor covariance models are generically locally identified and nonsaturated in the range considered. The dimension count gives

m_1=6(1)+6-\frac{1(1-1)}{2}=12,\qquad m_2=6(2)+6-\frac{2(2-1)}{2}=17.

The nominal dimension difference is \(17-12=5\). If the null were a regular interior submodel of the alternative with the required smoothness and information conditions, Wilks' theorem would give a \(\chi^2_5\) limit. But in the factor-number comparison, the null sets the added loading vector to zero. That point is precisely where the added factor's direction is unidentified, so the count alone does not justify using \(\chi^2_5\). It is a nominal dimension difference, not a calibration result.

Zero Loadings Create a Singular Parameterization

Under the \((q+1)\)-factor model, the covariance matrix can be written \(\Sigma=\Lambda\Lambda^\mathsf{T}+aa^\mathsf{T}+\Psi\). At the null, \(a=0\). The first-order change in the added covariance component is zero there: for any direction \(v\), \[ \left.\frac{d}{dt}\{(tv)(tv)^\mathsf{T}\}\right|_{t=0}=0. \] Thus the added factor does not provide \(p\) ordinary first-order covariance directions at the null. In addition, every possible orientation \(v\) describes the same null covariance when its magnitude is zero. This is the unidentified-direction feature of the null.

Worked Example: The Scale of a Local Loading Alternative

Take a model with null covariance \(\Sigma_0=I_3\), and suppose the added loading vector is \(a=(t,t,0)^\mathsf{T}\). Its covariance contribution is

aa^\mathsf{T}= \begin{pmatrix} t^2&t^2&0\\ t^2&t^2&0\\ 0&0&0 \end{pmatrix}.

The diagonal entries can, in principle, be offset by changes in uniquenesses, but the off-diagonal covariance \(t^2\) is a signal the diagonal uniqueness matrix cannot absorb. That signal is of order \(t^2\), not \(t\). For a Gaussian covariance model, the Kullback–Leibler divergence from a nearby covariance \(\Sigma_0+\Delta\) to \(\Sigma_0\) is of order \(\|\Delta\|^2\), provided the covariance remains positive definite and the perturbation is small. Here \(\|\Delta\|\) is of order \(t^2\), so the divergence is of order \(t^4\). To keep the total divergence over \(n\) observations of constant order, one therefore needs \(nt^4\) of constant order, or \(t\) of order \(n^{-1/4}\), rather than the usual \(n^{-1/2}\) scale for a regular parameter.

This calculation explains why ordinary quadratic likelihood theory is not automatic. It does not by itself specify the limiting distribution of the factor-number statistic: the nuisance parameters, the directions available to \(a\), and the model's local geometry all matter.

Key consequence: A zero added loading vector is not enough to claim that the likelihood-ratio statistic has a particular nonstandard distribution. It does show that the regular Wilks argument cannot be invoked without further analysis. In general, calibration must account for the actual local geometry of the covariance model, including nuisance directions and any identification constraints.

Boundary Constraints and a Derived Mixture Example

A boundary is a different failure of regularity. If uniqueness variances are constrained by \(\psi_i\geq 0\), a null fit with some \(\psi_i=0\) lies at the edge of the parameter space. The local alternatives are then restricted to a cone rather than a full linear space. In a simple one-dimensional problem this produces a familiar mixture, but that result should not be transferred mechanically to a factor-number test with unidentified loading directions.

Worked Example: A One-Sided Variance Component

Let \(Z_1,\ldots,Z_n\) be independent \(N(0,1+\tau)\) observations, where the baseline variance is known and \(\tau\geq 0\). Test \(H_0:\tau=0\) against \(\tau>0\). The log-likelihood, up to a constant, is \[ \ell(\tau)=-\frac n2\log(1+\tau)-\frac{1}{2(1+\tau)}\sum_{i=1}^n Z_i^2. \] The score at zero is \(U_n=\frac12\sum_i(Z_i^2-1)\), with variance \(n/2\). Hence \(U_n/\sqrt{n/2}\xrightarrow{d}N(0,1)\).

For local values \(\tau=h/\sqrt n\), with \(h\geq0\), the local log-likelihood difference is \(hU_n/\sqrt n-h^2/4+o_p(1)\). Write \(V_n=U_n/\sqrt n\), so \(V_n\xrightarrow{d}N(0,1/2)\). Maximizing the quadratic over \(h\geq0\) gives \(h=2\max(V_n,0)\), and twice the maximized difference is \(2\{\max(V_n,0)\}^2\). Since \(\sqrt2 V_n\) converges to a standard normal \(Z\), the limiting statistic is \(\{\max(Z,0)\}^2\). It is zero with probability \(1/2\), and otherwise has the positive half of a \(\chi^2_1\) distribution: its law is \(\frac12\chi^2_0+\frac12\chi^2_1\), where \(\chi^2_0\) denotes a point mass at zero.

This derivation relies on a single one-sided parameter with regular nuisance structure. A general added factor has a vector of loadings whose direction is unidentified under the null, and potentially several boundary constraints. Its limiting law need not be this half-and-half mixture.

How to Choose a Calibration

For a particular comparison, begin by identifying the null point in the alternative parameterization. Ask whether all added parameters are identified there, whether the covariance map has a full-rank derivative in the relevant directions, and whether any constrained variances are at their boundaries. Then assess whether the null and alternative have smooth local models with nonsingular information. A failure at any of these steps means that the routine dimension-difference chi-square calibration needs justification beyond Wilks' theorem.

Depending on the model and hypotheses, appropriate calibration may require a problem-specific asymptotic derivation or a parametric bootstrap that simulates under the fitted null and repeats the full estimation procedure. Such a bootstrap is not automatically valid: the null fit, constraints, and optimization must be represented faithfully, and its performance should be assessed for the setting at hand. A Bartlett correction, which adjusts a regular likelihood-ratio statistic's finite-sample behavior, does not repair a wrong limiting distribution caused by singularity or a boundary.

Common Mistakes

Using the parameter-count difference as a proof. A difference in nominal dimensions yields a chi-square degrees of freedom only after regularity and local identification have been established.
Calling the zero loading vector an ordinary interior null. Although zero is interior in unconstrained Euclidean loading coordinates, the covariance contribution is quadratic in the loadings and the added factor's orientation is unidentified there.
Applying the one-sided mixture universally. The \(\frac12\chi^2_0+\frac12\chi^2_1\) result is derived for a simple single boundary parameter; it is not a general factor-number reference distribution.
Confusing a boundary uniqueness with a zero added loading. They are distinct sources of nonregularity and may occur together. Each must be considered when determining the null distribution.
Assuming saturation cannot occur. A generic dimension expression is subject to the ambient covariance-space dimension and identification conditions; it cannot be applied mechanically for every \(p\) and \(q\).

Key takeaway: The \(q\)-versus-\((q+1)\)-factor comparison is nested, but nesting alone does not imply a chi-square likelihood-ratio limit. Zero added loadings create unidentified directions and a singular covariance map; boundary uniqueness estimates can add cone constraints. Use a chi-square calibration only when the regularity conditions for it have been established.

Check Your Understanding

Use the definitions and arguments above to answer the following.

Which regularity conditions are most directly threatened when the added loading vector is zero?
Why does a nominal difference of five covariance-model parameters not, by itself, justify a \(\chi^2_5\) reference distribution?
In the local-loading example, why is the covariance signal of order \(t^2\), and what scale of \(t\) keeps the total divergence over \(n\) observations of constant order?
What assumptions make the one-sided variance-component example lead to a half-and-half chi-square mixture?
How can a boundary uniqueness estimate affect the local geometry used to calibrate a likelihood-ratio test?