Counting Parameters in Exploratory Factor Analysis

Why Count Parameters?

An exploratory factor model describes the covariance among $p$ observed variables using $q$ latent factors and variable-specific residual variation. Counting parameters gives an initial way to assess whether the model could be identified from the observed covariance matrix: if the model has too many free directions relative to the available covariance entries, regular local identification cannot hold. But this count is only a diagnostic. It does not by itself show that distinct parameter values produce distinct covariance matrices.

The key is to keep three quantities separate: the number of covariance entries available, the number of model parameters before accounting for a known redundancy, and the number of distinct covariance structures actually generated by the model. The first two can be counted directly. The third may be smaller than the adjusted parameter count, so a parameter-count difference must not automatically be called the model’s actual covariance degrees of freedom.

The Covariance Model and Its Raw Count

Definition: In the $q$-factor model for $p$ observed variables, the covariance matrix is $$ \Sigma=\Lambda\Lambda^{\mathsf T}+\Psi, $$ where $\Lambda$ is a $p\times q$ matrix of factor loadings and $\Psi=\operatorname{diag}(\psi_1,\ldots,\psi_p)$ is diagonal, with $\psi_i>0$. The factor covariance is fixed to the $q\times q$ identity matrix, and the specific errors are uncorrelated with one another and with the factors. The covariance model is the set of matrices $\Sigma$ generated as the free entries of $\Lambda$ and the diagonal entries of $\Psi$ vary over their admissible values.

A symmetric $p\times p$ covariance matrix has $p$ diagonal entries and $p(p-1)/2$ distinct off-diagonal entries. Thus the ambient space of symmetric covariance matrices has dimension

m=p+\frac{p(p-1)}{2}=\frac{p(p+1)}{2}.

The loading matrix contributes $pq$ entries, and $\Psi$ contributes $p$ more. Before accounting for any redundancy, the parameterization therefore has $pq+p$ scalar parameters. This is a count of coordinates used to describe the model; it need not be a count of distinct covariance structures.

Removing the Known Orthogonal Redundancy

Even with the factor covariance fixed to the identity, the factor coordinates can be changed by an orthogonal transformation. If $Q$ is a $q\times q$ orthogonal matrix, so that $QQ^{\mathsf T}=I_q$, then

(\Lambda Q)(\Lambda Q)^{\mathsf T}+\Psi =\Lambda QQ^{\mathsf T}\Lambda^{\mathsf T}+\Psi =\Lambda\Lambda^{\mathsf T}+\Psi.

Consequently, a continuum of loading matrices can represent the same covariance matrix. At a loading matrix of full column rank $q$, the orthogonal transformations provide $q(q-1)/2$ locally redundant directions. One way to see this count is that an infinitesimal orthogonal change has the form $Q(t)=I_q+tA+o(t)$, where $A^{\mathsf T}=-A$. A skew-symmetric $q\times q$ matrix has zero diagonal and one freely chosen entry for each pair of distinct indices, hence $q(q-1)/2$ free entries.

Definition: The nominal parameter count after removing rotational redundancies is $$ r_{\mathrm{nom}}=pq+p-\frac{q(q-1)}{2}. $$ It counts the raw loading and specific-variance parameters after subtracting the known number of locally redundant orthogonal factor-coordinate directions at full-column-rank loadings. It is a parameter-count diagnostic, not a claim that the covariance model has dimension $r_{\mathrm{nom}}$.

This qualification matters. Other nonidentification or dependencies may remain. In particular, the dimension of the set of covariance matrices generated by the model cannot exceed $m$, even if $r_{\mathrm{nom}}>m$. The nominal count is not then the dimension of the covariance image.

A Necessary Count for Regular Local Identification

Theorem: At a full-column-rank loading matrix, a necessary condition for regular local identification of the factor parameters modulo orthogonal factor-coordinate changes is $$ r_{\mathrm{nom}}\leq m, $$ or, equivalently, $$ pq+p-\frac{q(q-1)}{2}\leq \frac{p(p+1)}{2}. $$ When this inequality holds, the difference $m-r_{\mathrm{nom}}$ is a formal parameter-count diagnostic. Its nonnegativity is necessary, but not sufficient, for regular local identification.

Here, regular local identification means that, after representing the parameters locally without the known orthogonal redundancy, the covariance map has a derivative with full column rank. Full column rank of that derivative is a standard local regularity requirement: a parameter change in any represented direction must change the covariance to first order.

Proof. At a full-column-rank $\Lambda$, the orthogonal transformations act locally with $q(q-1)/2$ independent directions. Removing these directions leaves $r_{\mathrm{nom}}$ local parameter coordinates. The covariance map sends those coordinates into the $m$-dimensional space of symmetric $p\times p$ matrices. Its derivative is a linear map from a space of dimension $r_{\mathrm{nom}}$ to one of dimension $m$. A linear map into an $m$-dimensional space cannot have rank greater than $m$. Thus, if $r_{\mathrm{nom}}>m$, this derivative cannot have full column rank $r_{\mathrm{nom}}$, and regular local identification is impossible. Rearranging gives the stated necessary condition. This argument does not show that the derivative has full column rank when $r_{\mathrm{nom}}\leq m$, so the condition is not sufficient. $\square$

If $r_{\mathrm{nom}}\leq m$, it is common to report $m-r_{\mathrm{nom}}$ as the formal degrees-of-freedom count from parameter counting. That label is appropriate as a diagnostic, but it must not be mistaken for a proof about the actual dimension of the covariance image. If additional dependencies reduce that image’s dimension, the actual covariance constraints may be more numerous than the formal count suggests.

Worked Example: A Two-Factor Model With a Nonnegative Diagnostic

Suppose $p=6$ observed variables are modeled with $q=2$ factors. Find the ambient covariance dimension, the raw count, the rotational adjustment, and the formal diagnostic.

The symmetric covariance matrix has $$ m=\frac{6(6+1)}{2}=\frac{42}{2}=21 $$ distinct entries. The loading matrix has $pq=6(2)=12$ entries, and the diagonal specific covariance has $p=6$ entries. Thus the raw parameter count is $12+6=18$.

For $q=2$, the number of orthogonal factor-coordinate directions is $$ \frac{q(q-1)}{2}=\frac{2(1)}{2}=1. $$ The nominal count after subtracting that known redundancy is $r_{\mathrm{nom}}=18-1=17$. The formal difference is $$ m-r_{\mathrm{nom}}=21-17=4. $$ This count passes the necessary inequality because $17\leq21$. It does not establish identification: the derivative of the covariance map could still have rank below 17 for reasons not accounted for by this count.

Worked Example: When the Nominal Count Exceeds the Covariance Dimension

Consider $p=4$ observed variables and $q=2$ factors. The ambient dimension is $$ m=\frac{4(5)}{2}=10. $$ There are $pq+p=4(2)+4=12$ raw parameters. Subtracting the $2(1)/2=1$ known rotational direction gives $$ r_{\mathrm{nom}}=12-1=11. $$

Since $11>10$, the necessary condition fails. The count therefore rules out regular local identification of the factor parameters modulo rotations at a full-column-rank loading matrix. It would be incorrect to say that the covariance model has dimension 11, because it lies inside a 10-dimensional space. It would also be incorrect to call $10-11=-1$ a number of covariance degrees of freedom. The accurate conclusion is that the nominal count exceeds the ambient covariance dimension, so regular local identification is impossible by this parameter-count argument.

Worked Example: A Three-Factor Count at the Boundary

Take $p=6$ and $q=3$. The covariance dimension is still $$ m=\frac{6(7)}{2}=21. $$ The raw count is $pq+p=6(3)+6=24$, and the rotational adjustment is $$ \frac{q(q-1)}{2}=\frac{3(2)}{2}=3. $$ Thus $r_{\mathrm{nom}}=24-3=21$, and $m-r_{\mathrm{nom}}=21-21=0$.

The count is exactly at the necessary boundary: there are no formal excess covariance entries according to this diagnostic. This does not imply that the model is identified. One must still check whether distinct locally represented parameter changes can leave the covariance unchanged, including dependencies beyond the orthogonal directions already subtracted.

What the Count Can and Cannot Tell You

The parameter-count calculation is useful as an early screen. A negative formal difference $m-r_{\mathrm{nom}}$ means the nominal count is larger than the number of available covariance coordinates. In that case, regular local identification after accounting for rotations is impossible. A nonnegative difference means only that this particular dimension obstruction has not ruled identification out.

The actual covariance image is the set of matrices $\Lambda\Lambda^{\mathsf T}+\Psi$ produced by admissible parameter values. Its dimension is at most $m$, and parameter redundancies can make it smaller than the nominal count. Therefore, actual covariance degrees of freedom, if defined as $m$ minus the dimension of that image, cannot be negative. When the nominal count exceeds $m$, report the failed necessary condition—not negative covariance degrees of freedom. When the nominal count is at most $m$, the formal difference is still only a diagnostic unless additional analysis establishes the covariance image’s dimension.

A later local-identification analysis can examine the rank of the derivative of the covariance map after suitable local parameter constraints. Parameter counting does not replace that analysis: it counts possible directions, but does not determine which directions the covariance map actually distinguishes.

Common Pitfall

Wrong reasoning: “There are $m$ covariance entries and $r_{\mathrm{nom}}$ parameters after rotation, so the covariance model has dimension $r_{\mathrm{nom}}$, and its degrees of freedom are $m-r_{\mathrm{nom}}$, even when that number is negative.”

Correction: The nominal count is not automatically the dimension of the covariance image. The image must fit inside the $m$-dimensional ambient space. If $r_{\mathrm{nom}}>m$, the count shows that regular local identification is impossible; it does not give a negative dimension or negative actual covariance degrees of freedom. If $r_{\mathrm{nom}}\leq m$, the nonnegative difference remains a necessary-count diagnostic, not proof of identification.

Key takeaway: For the $q$-factor model with diagonal $\Psi$, compare $r_{\mathrm{nom}}=pq+p-q(q-1)/2$ with $m=p(p+1)/2$. The inequality $r_{\mathrm{nom}}\leq m$ is necessary for regular local identification modulo rotations at full-rank loadings, but it is not sufficient. Never treat $r_{\mathrm{nom}}$ as the actual covariance-image dimension without further justification.

Check Your Understanding

Use the definitions and counts above to answer these questions.

For $p=7$ and $q=2$, calculate $m$, $r_{\mathrm{nom}}$, and the formal parameter-count difference.
Why does an orthogonal transformation of the loading coordinates leave $\Lambda\Lambda^{\mathsf T}+\Psi$ unchanged?
For $p=4$ and $q=2$, why is $-1$ not a valid number of actual covariance degrees of freedom?
If $r_{\mathrm{nom}}\leq m$, what does the parameter-count condition establish, and what does it not establish?
Why can the covariance image have dimension smaller than the nominal parameter count?