Iterated Communality Estimates in Principal-Axis Factoring

Why Iterate the Communalities?

In principal-axis factoring, the diagonal of the observed correlation matrix is not treated as entirely common variance. Each diagonal entry is replaced by an estimate of the corresponding variable’s communality, while the off-diagonal correlations are retained. The leading eigenvectors and eigenvalues of this reduced matrix give factor loadings; those loadings, in turn, imply new communalities. Repeating these two steps is the defining feature of iterated communality estimates.

The iteration is useful because the communalities are not generally known in advance. It is also important to distinguish an algorithmic stopping point from a guarantee that a unique, admissible factor solution has been found. The update can depend on its starting values, and some iterations can produce communalities outside the usual range for standardized variables.

Definition: For \(p\) standardized variables, let \(R\) be their \(p\times p\) correlation matrix and let \(h_i^2\) denote the communality estimate for variable \(i\). The reduced correlation matrix has off-diagonal entries \(R_{ij}\) and diagonal entries \(h_i^2\). In principal-axis factoring with \(m\) retained factors, the retained eigenpairs of this matrix determine loadings, and the next communality estimate for variable \(i\) is the sum of the squares of its \(m\) retained loadings.

The Iterative Update

Choose initial communalities \(h_{i,0}^2\), and form \(R^{(0)}\) by replacing the diagonal of \(R\) with these values. A common starting choice is the squared multiple correlation of each variable with the others. When the relevant submatrix is invertible, this starting value for variable \(i\) is \(R_{i,-i}R_{-i,-i}^{-1}R_{-i,i}\). Other starting choices are possible; they should be recorded because the sequence can depend on them.

At iteration \(t\), write the eigenvalues of \(R^{(t)}\) in descending order as \(\lambda_1^{(t)}\geq\cdots\geq\lambda_p^{(t)}\), with corresponding orthonormal eigenvectors \(v_1^{(t)},\ldots,v_p^{(t)}\). If the \(m\) retained eigenvalues are positive, the principal-axis loading for variable \(i\) on retained factor \(k\) is \(a_{ik}^{(t)}=\sqrt{\lambda_k^{(t)}}v_{ik}^{(t)}\). The next reduced matrix keeps the off-diagonal correlations and replaces its diagonal with the updated communalities.

h_{i,t+1}^2=\sum_{k=1}^{m}\left(a_{ik}^{(t)}\right)^2 =\sum_{k=1}^{m}\lambda_k^{(t)}\left(v_{ik}^{(t)}\right)^2, \qquad R_{ii}^{(t+1)}=h_{i,t+1}^2,\quad R_{ij}^{(t+1)}=R_{ij}\ (i\ne j).

This expression follows directly from the retained loading matrix \(A^{(t)}=V_m^{(t)}(\Lambda_m^{(t)})^{1/2}\): its \(i\)th row has squared length \(\sum_{k=1}^m(a_{ik}^{(t)})^2\). Thus the update is the diagonal of \(A^{(t)}A^{(t)\mathsf T}\). It is not the diagonal of the full reduced matrix unless all eigencomponents are retained.

Initialize.
Choose and record the starting communalities, then insert them on the diagonal of the reduced correlation matrix.

Extract.
Compute the leading \(m\) eigenpairs and form their loading columns as eigenvectors multiplied by the square roots of their eigenvalues.

Update and check.
Replace each diagonal entry by the sum of its retained loading squares. Compare successive communalities and check that the retained eigenvalues and communalities remain interpretable.

Worked Iterations

Worked Example: One Update for Three Variables

Suppose the correlation matrix has correlations \(R_{12}=0.6\), \(R_{13}=R_{23}=0.3\). Start all three communalities at \(0.5\) and retain one factor. The reduced matrix is

R^{(0)}= \begin{pmatrix} 0.5&0.6&0.3\\ 0.6&0.5&0.3\\ 0.3&0.3&0.5 \end{pmatrix}.

Because the first two variables are interchangeable in this matrix, \((1,-1,0)^{\mathsf T}/\sqrt{2}\) is an eigenvector with eigenvalue \(0.5-0.6=-0.1\). On the orthonormal basis \((1,1,0)^{\mathsf T}/\sqrt{2},(0,0,1)^{\mathsf T}\), the remaining block is

\begin{pmatrix} 1.1&0.3\sqrt{2}\\ 0.3\sqrt{2}&0.5 \end{pmatrix}.

Its eigenvalues solve \((1.1-\lambda)(0.5-\lambda)-0.18=0\), or \(\lambda^2-1.6\lambda+0.37=0\). The larger root is \(\lambda_1=(1.6+\sqrt{1.08})/2\approx1.3196\), so it is the leading eigenvalue. An associated unit eigenvector in this two-dimensional basis has coordinates proportional to \((1,q)\), where \(q=(\lambda_1-1.1)/(0.3\sqrt{2})\approx0.5176\). Normalizing gives first coordinate approximately \(0.8881\) and second approximately \(0.4597\). In the original variable coordinates the eigenvector is therefore approximately \((0.6280,0.6280,0.4597)^{\mathsf T}\).

The retained loading vector is this eigenvector multiplied by \(\sqrt{1.3196}\approx1.1487\), giving loadings approximately \((0.7214,0.7214,0.5281)^{\mathsf T}\). Squaring each loading gives the updated communalities \((0.5204,0.5204,0.2789)\). The update is asymmetric between variable 3 and variables 1 and 2 because their correlation pattern is different, even though the starting communalities were equal.

Worked Example: The Equicorrelation Update

Let \(p>1\) variables have common pairwise correlation \(r=0.20\), and suppose all start with the same communality \(c_t\geq0\). Retain one factor. The reduced matrix has diagonal \(c_t\) and every off-diagonal entry \(r\). The unit vector of equal coordinates is an eigenvector with eigenvalue \(c_t+(p-1)r\). Every vector orthogonal to it has eigenvalue \(c_t-r\), because the off-diagonal contribution is \(-r\) times such a vector. Since \(r>0\) and \(p>1\), the equal-coordinate eigenvalue is larger by \(pr>0\), and it is positive.

The leading unit eigenvector has every entry \(1/\sqrt{p}\). Each retained loading is therefore \(\sqrt{c_t+(p-1)r}/\sqrt{p}\), so its square gives the same updated communality for every variable:

c_{t+1}=\frac{c_t+(p-1)r}{p}.

For \(p=3\) and \(c_0=0\), the successive values are \(c_1=0.4/3\approx0.1333\), \(c_2=(0.1333+0.4)/3\approx0.1778\), and \(c_3=(0.1778+0.4)/3\approx0.1926\). They move toward \(0.20\), as shown in the convergence argument below.

Convergence in a Special Case

The equicorrelation example permits a direct convergence result. It is a useful check on the mechanics of the update, not a general convergence theorem for principal-axis factoring.

Theorem: In the equicorrelation, one-factor setting above, if \(p>1\), \(r>0\), and the common starting communality is \(c_0\geq0\), then the iteration converges to \(r\). The condition \(p>1\) is essential to this statement.

Proof. The recurrence has fixed point \(r\), since \(\{r+(p-1)r\}/p=r\). Subtracting \(r\) from both sides of the recurrence gives

c_{t+1}-r=\frac{c_t-r}{p}.

Repeated application yields \(c_t-r=(c_0-r)/p^t\). Because \(p>1\), \(p^{-t}\to0\), so \(c_t\to r\). If \(p=1\), the recurrence instead reads \(c_{t+1}=c_t\), and it need not converge to \(r\); for example, \(c_0=0\) remains zero even when \(r=0.20\). \(\square\)

Stopping Rules and Starting-Value Sensitivity

In a numerical implementation, a common convergence check is the maximum change in any communality:

\Delta_t=\max_{1\leq i\leq p}|h_{i,t+1}^2-h_{i,t}^2|.

One can stop when \(\Delta_t\) falls below a stated tolerance, such as \(0.05\) for a rough demonstration or a smaller tolerance for a more precise computation. The tolerance, the number of iterations, and the starting values should be reported. A small change is evidence that the communality sequence has stabilized at the chosen precision; by itself, it does not prove that the solution is unique, globally optimal, or substantively appropriate.

Worked Example: Comparing Starts and Applying a Tolerance

Continue with three equicorrelated variables, \(r=0.20\), and one retained factor. With \(c_0=0\), the first three iterates are \(0.1333,0.1778,0.1926\). Their successive changes are \(0.1333,0.0444,0.0148\), respectively. A rule requiring a change below \(0.05\) first passes at the second update.

Now start instead at \(c_0=0.80\). The same update gives \(c_1=(0.80+0.40)/3=0.4000\), \(c_2=(0.4000+0.40)/3=0.2667\), and \(c_3=(0.2667+0.40)/3=0.2222\). The changes are \(0.4000,0.1333,0.0444\), so this start also first passes the \(0.05\) rule at the third update. The iterates at that point differ, \(0.1926\) versus \(0.2222\); they have not become identical just because both met the stopping rule. The proved limit is \(0.20\) for both starts, but in more general problems convergence to the same limit is not guaranteed by this example.

Interpreting the Output Carefully

For standardized variables, communalities are usually expected to lie between zero and one: the common part should not exceed the total variance of a variable. An update above one is a warning of a Heywood-type boundary or inadmissible estimate, not a value that should be silently capped. Likewise, a reduced matrix can have negative eigenvalues. The method should retain the requested leading positive eigenvalues; if there are not \(m\) positive eigenvalues, the usual loading construction for \(m\) factors is not available as stated.

Eigenvectors can also be unstable when retained eigenvalues are tied or nearly tied. Their signs are arbitrary, and when a retained subspace has multiplicity greater than one, individual eigenvectors can change substantially while spanning nearly the same subspace. Communality comparisons are less affected by sign changes, but a complete assessment should also inspect the loading pattern and the eigenvalue separation. Iterated principal-axis factoring is consequently a computational procedure whose behavior must be checked, not an automatic guarantee of a well-behaved factor solution.

Common Mistake

Assuming every sequence converges just because the equicorrelation example does. That proof uses equal correlations, equal starts, one retained factor, and \(p>1\). In general, eigenvalues and eigenvectors change as the diagonal changes; the sequence need not follow a scalar contraction. Check successive communalities, retained eigenvalues, and admissibility, and report starting values. A small update is a practical stopping criterion, not a universal convergence proof.
Replacing a communality by an eigenvalue. The update for variable \(i\) is the sum of its retained loading squares, not the sum of retained eigenvalues. The eigenvector entries allocate each eigenvalue’s contribution across variables.

Key takeaway: Principal-axis iteration alternates between eigendecomposing a reduced correlation matrix and replacing its diagonal with retained loading-based communalities. Convergence checks, starting values, and admissibility checks are all part of interpreting the resulting solution.

Check Your Understanding

Answer each question by using the update definition and the stated assumptions.

Write the formula for updating the communality of variable \(i\) from \(m\) retained eigenpairs.
Why does the equicorrelation recurrence have the same update for every variable when the starting communalities are equal?
What happens to \(c_{t+1}=(c_t+(p-1)r)/p\) when \(p=1\), and why does this matter for the convergence claim?
If the maximum communality change is below a chosen tolerance, what has been established, and what has not?
Why should a communality estimate above one be reported rather than automatically truncated to one?