Communalities and Uniquenesses

Introduction

A factor model separates the variation in each observed variable into a part associated with common factors and a part specific to that variable. Two quantities summarize this separation: the communality, which measures the common-factor contribution to a variable’s variance, and the uniqueness, which measures its specific variance. These quantities are useful for understanding what a fitted factor model says about each variable, but neither should be interpreted without its assumptions and scale.

This tutorial uses the common-factor covariance decomposition. Let \(X\) be a vector of \(p\) observed variables and \(F\) a vector of \(q\) common factors. Write \(\Lambda\) for the \(p \times q\) loading matrix and \(\varepsilon\) for the vector of specific components. Assume the variables have finite second moments, and that the factors and specific components are uncorrelated. If \(\operatorname{Cov}(F)=\Phi\) and \(\operatorname{Cov}(\varepsilon)=\Psi\), then

For much of the discussion, \(\Phi=I_q\): the factors have unit variances and are mutually uncorrelated. In that case, \(\Lambda\Phi\Lambda^\top=\Lambda\Lambda^\top\). When \(\Phi\) is not the identity, the same definitions account for factor correlations. For calculations of correlations between different observed variables, we will state when the specific components have zero covariance with each other.

Definitions and the Variance Decomposition

Let \(\lambda_i^\top\) be row \(i\) of \(\Lambda\), so \(\lambda_i\) is a \(q\)-vector of loadings for observed variable \(X_i\). Let \(\psi_{ii}\) be the \(i\)th diagonal element of \(\Psi\). The common-factor contribution to \(X_i\)'s variance is the \(i\)th diagonal element of \(\Lambda\Phi\Lambda^\top\).

The notation \(h_i^2\) is conventional; it denotes a variance contribution, not the square of a loading. Because a covariance matrix is positive semidefinite, \(h_i^2\geq 0\). A specific variance is also nonnegative, since it is a variance. In the orthogonal, unit-variance factor case, the communality has the especially simple form

Thus each squared loading contributes to the communality when the factors are uncorrelated and standardized. If the factors are correlated, cross-products also matter: \(h_i^2=\lambda_i^\top\Phi\lambda_i\), not generally the sum of squared loadings.

To verify the decomposition, the \(i\)th variable can be written \(X_i=\mu_i+\lambda_i^\top F+\varepsilon_i\). Constants do not affect variance. Using the covariance of a sum and the assumed zero covariance between \(F\) and \(\varepsilon\),

Dividing this equality by the positive total variance gives \(h_i^2/\operatorname{Var}(X_i)+\psi_{ii}/\operatorname{Var}(X_i)=1\), as claimed. This is a decomposition of variance under the model, not a causal claim that changing a factor would change the observed variable by that amount. □

When observed variables are standardized to variance one, the decomposition becomes \(1=h_i^2+\psi_{ii}\). Then communality is the proportion of that variable’s variance attributed to common factors, and uniqueness is the remaining proportion. For variables on their original scales, \(h_i^2\) and \(\psi_{ii}\) are variances in squared measurement units; proportions require dividing by the total variance.

From Communalities to Covariances and Correlations

The diagonal entries of \(\Lambda\Phi\Lambda^\top\) give communalities. Its off-diagonal entries give the covariances between observed variables attributable to the common factors. For \(i\ne j\), the full covariance is

Here \(\psi_{ij}\) is the covariance between the specific components for variables \(i\) and \(j\). In the common case where \(\Psi\) is diagonal, \(\psi_{ij}=0\) for \(i\ne j\), so the shared-factor covariance is the entire covariance. A correlation then follows by dividing by the two standard deviations:

Consequently, knowing the two communalities does not in general determine the correlation. A communality describes the length, in the factor covariance metric, of one variable’s loading vector. The shared covariance also depends on how the two loading vectors align with one another. When factors are uncorrelated and unit variance, that covariance is the ordinary inner product of the two loading vectors.

To prove the bound, a positive semidefinite matrix has a positive semidefinite square root \(\Phi^{1/2}\), with \(\Phi^{1/2}\Phi^{1/2}=\Phi\). Define \(a=\Phi^{1/2}\lambda_i\) and \(b=\Phi^{1/2}\lambda_j\). Then \(\lambda_i^\top\Phi\lambda_j=a^\top b\), \(\|a\|^2=\lambda_i^\top\Phi\lambda_i=h_i^2\), and \(\|b\|^2=h_j^2\). By the Cauchy–Schwarz inequality, \[ |\lambda_i^\top\Phi\lambda_j| =|a^\top b| \leq \|a\|\|b\| =\sqrt{h_i^2h_j^2}. \] This bounds only the common-factor covariance. If \(\psi_{ij}\ne 0\), total covariance also includes that specific covariance, so the same bound need not hold for the total covariance. □

Common Mistakes and Interpretation

Communalities and uniquenesses are most useful when interpreted at the level of individual variables. They indicate how the model allocates each variable’s variance; they do not, by themselves, describe the full pattern of relationships among variables.

• Squaring loadings when factors are correlated. The sum of squared loadings gives the communality only when \(\Phi=I_q\). Otherwise use \(\lambda_i^\top\Phi\lambda_i\), including the cross-product terms.
• Calling the communality a proportion in every model. It is a variance contribution. It equals a proportion only for a standardized variable, or after dividing by the total variance.
• Assuming two high communalities imply a high correlation. Two variables can have large communalities but load in different or opposing directions. Their shared covariance depends on \(\lambda_i^\top\Phi\lambda_j\), not only on \(h_i^2\) and \(h_j^2\).
• Ignoring specific cross-covariance. The expression for total covariance includes \(\psi_{ij}\). It reduces to shared-factor covariance alone only when the specific components for those two variables have zero covariance.
• Treating a variance decomposition as a causal explanation. The model assigns covariance to common and specific components; that mathematical allocation alone does not establish causal effects.