Pattern, Structure, and Factor Correlation Matrices

Why Two Loading Matrices Appear

With orthogonal factors, the coefficient describing an observed variable’s direct contribution from a factor is also its correlation with that factor, provided both are standardized. With oblique factors, the factors are correlated, so an observed variable can be associated with a factor partly because it loads on another correlated factor. The direct coefficients and the marginal correlations then need not agree.

This distinction is central to reading an oblique factor solution. The pattern matrix records the coefficients in the factor representation; the structure matrix records correlations between observed variables and factors. The factor correlation matrix connects the two.

Definition: In an oblique factor model, write the centered vector of \(p\) observed variables as \(X=\Lambda F+\varepsilon\), where \(F\) is a vector of \(q\) factors, \(\Lambda\) is the \(p\times q\) pattern matrix, and \(\varepsilon\) is a vector of residuals. Assume \(E[F]=0\), \(E[\varepsilon]=0\), \(\operatorname{Cov}(F)=\Phi\), and \(\operatorname{Cov}(F,\varepsilon)=0\). The matrix \(\Phi\) is the factor covariance matrix; when the factors are standardized, it is their correlation matrix and has unit diagonal. The structure matrix contains the correlations between observed variables and factors when both are standardized.

The usual standardized setting has \(\operatorname{Var}(X_i)=1\) for each observed variable and \(\operatorname{Var}(F_j)=1\) for each factor. Then the entries of \(\Lambda\) are coefficients on the factor variables in the model, while the entries of the structure matrix \(S\) are ordinary correlations. If observed variables or factors are not standardized, the covariance relationship remains valid, but correlations require scale adjustments.

Deriving the Pattern–Structure Relationship

Theorem: Under the factor-model assumptions above, \(\operatorname{Cov}(X,F)=\Lambda\Phi\). If the observed variables and factors are standardized, the structure matrix is therefore \(S=\Lambda\Phi\). If \(\Phi\) is invertible, then \(\Lambda=S\Phi^{-1}\).

To derive the covariance relationship, take the covariance of \(X=\Lambda F+\varepsilon\) with \(F\). By linearity of covariance and the assumption that residuals are uncorrelated with factors,

\operatorname{Cov}(X,F) =\operatorname{Cov}(\Lambda F+\varepsilon,F) =\Lambda\operatorname{Cov}(F)+\operatorname{Cov}(\varepsilon,F) =\Lambda\Phi.

When each variable and factor has variance one, covariance equals correlation, so \(S=\Lambda\Phi\). If \(\Phi\) is invertible, multiply on the right by \(\Phi^{-1}\) to obtain \(\Lambda=S\Phi^{-1}\). Invertibility is essential for this recovery formula; the covariance relationship \(S=\Lambda\Phi\) itself does not require an inverse. \(\square\)

For variables that are not standardized, let \(D_X\) be the diagonal matrix of observed-variable variances and \(D_F\) the diagonal matrix of factor variances. If those variances are positive, the matrix of pairwise correlations is

S_{\mathrm{corr}} =D_X^{-1/2}\Lambda\Phi D_F^{-1/2}.

This scaling matters: \(\Lambda\Phi\) is a cross-covariance matrix, not automatically a correlation matrix. In the standardized case, \(D_X=D_F=I\), giving the simpler formula \(S=\Lambda\Phi\).

Worked Example: Calculating Structure Loadings and Common Variances

Consider three standardized observed variables and two standardized factors. Let the factors be jointly normal with covariance matrix \(\Phi\), and suppose the model is \(X=\Lambda F+\varepsilon\), with jointly normal residuals independent of the factors:

\Phi= \begin{pmatrix} 1&0.6\\ 0.6&1 \end{pmatrix}, \qquad \Lambda= \begin{pmatrix} 0.8&0.1\\ 0.7&0.2\\ 0.1&0.6 \end{pmatrix}.

The eigenvalues of \(\Phi\) are \(1.6\) and \(0.4\), so it is positive definite and is a valid factor correlation matrix. Calculate the structure matrix by multiplying each row of \(\Lambda\) by \(\Phi\):

S=\Lambda\Phi = \begin{pmatrix} 0.8+0.1(0.6)&0.8(0.6)+0.1\\ 0.7+0.2(0.6)&0.7(0.6)+0.2\\ 0.1+0.6(0.6)&0.1(0.6)+0.6 \end{pmatrix} = \begin{pmatrix} 0.86&0.58\\ 0.82&0.62\\ 0.46&0.66 \end{pmatrix}.

For example, the first observed variable has pattern coefficients \(0.8\) and \(0.1\), but its correlations with the two factors are \(0.86\) and \(0.58\). The first structure coefficient includes the contribution \(0.1(0.6)\): the variable’s loading on factor 2 contributes to its association with factor 1 because the factors correlate.

The common variance of an observed variable is the variance of its factor component, \(\operatorname{Var}(\Lambda_i F)\), where \(\Lambda_i\) is its row of pattern coefficients. For the first variable this is

(0.8,\ 0.1) \begin{pmatrix}1&0.6\\0.6&1\end{pmatrix} \begin{pmatrix}0.8\\0.1\end{pmatrix} =0.8(0.86)+0.1(0.58) =0.746.

The corresponding common variances for variables 2 and 3 are \(0.7(0.82)+0.2(0.62)=0.698\) and \(0.1(0.46)+0.6(0.66)=0.442\). To make all three observed variables standardized, choose residual variances \(0.254\), \(0.302\), and \(0.558\), respectively. Each is positive, and each adds to its common variance to give total variance one.

Common Variance and the Pattern–Structure Product

For standardized observed variables, the common variance is not generally the sum of squared pattern coefficients. Correlation between factors contributes cross-products. There is a useful equivalent calculation using one row of each loading matrix.

Result: Under the factor-model assumptions, the common variance of observed variable \(i\) is \(h_i^2=\Lambda_i\Phi\Lambda_i^{\mathsf T}\). If factors are standardized, and \(S_i=\Lambda_i\Phi\) is the corresponding row of the structure matrix, then \(h_i^2=\Lambda_i S_i^{\mathsf T}\).

Indeed, the factor component of \(X_i\) is \(\Lambda_iF\). Its variance is

\operatorname{Var}(\Lambda_iF) =\Lambda_i\operatorname{Cov}(F)\Lambda_i^{\mathsf T} =\Lambda_i\Phi\Lambda_i^{\mathsf T}.

Since \(S_i=\Lambda_i\Phi\) and \(\Phi\) is symmetric, \(S_i^{\mathsf T}=\Phi\Lambda_i^{\mathsf T}\). Substitution gives \(h_i^2=\Lambda_iS_i^{\mathsf T}\), as claimed. This is a complete row-wise calculation: multiply corresponding pattern and structure coefficients, then sum. \(\square\)

When \(\Phi=I_q\), this expression reduces to the familiar sum of squared pattern coefficients. For oblique factors, it includes cross-products. In the first variable of the preceding example, the sum of squared pattern coefficients is \(0.8^2+0.1^2=0.65\), not the common variance \(0.746\). The missing contribution is \(2(0.6)(0.8)(0.1)=0.096\), and \(0.65+0.096=0.746\).

Worked Example: Recovering Pattern Coefficients

Suppose there are two standardized factors with correlation \(0.5\), and a standardized observed variable has structure coefficients \(0.7\) and \(0.5\). If its factor correlation matrix and structure row are

\Phi= \begin{pmatrix}1&0.5\\0.5&1\end{pmatrix}, \qquad S_i=(0.7,\ 0.5),

then the determinant of \(\Phi\) is \(1-0.5^2=0.75\), so \(\Phi\) is invertible. Its inverse is

\Phi^{-1} =\frac{1}{0.75} \begin{pmatrix}1&-0.5\\-0.5&1\end{pmatrix}.

The pattern row is \(S_i\Phi^{-1}\):

\Lambda_i =\frac{1}{0.75}(0.7-0.25,\ -0.35+0.5) =(0.6,\ 0.2).

Check by multiplying back: \((0.6,0.2)\Phi=(0.6+0.1,\ 0.3+0.2)=(0.7,0.5)\), which is the given structure row. The common variance is \(0.6(0.7)+0.2(0.5)=0.52\). For standardized \(X_i\), a residual variance of \(0.48\) completes the unit variance.

When Do the Matrices Coincide?

The equation \(S=\Lambda\Phi\) gives an exact condition for equality with the pattern matrix:

S=\Lambda \quad\Longleftrightarrow\quad \Lambda(\Phi-I_q)=0.

If the factors are orthogonal and standardized, then \(\Phi=I_q\), so the equality holds for every pattern matrix. But orthogonality is not necessary for a particular pattern matrix: its rows might be unaffected by the nonidentity part of \(\Phi\).

Worked Example: Equal Matrices Despite Correlated Factors

Take three standardized factors with correlation matrix and a one-variable pattern row

\Phi= \begin{pmatrix} 1&0.5&0\\ 0.5&1&0\\ 0&0&1 \end{pmatrix}, \qquad \Lambda_i=(0,\ 0,\ 0.7).

The first two factors are correlated, so the factors are not mutually orthogonal. Nevertheless,

S_i=\Lambda_i\Phi=(0,\ 0,\ 0.7)=\Lambda_i.

In this case the row has zero coefficients on the correlated factors and a nonzero coefficient only on the third factor, which is uncorrelated with the first two. Equivalently, direct multiplication gives \(\Lambda_i(\Phi-I_3)=(0,0,0)\). The common variance is \(0.7^2=0.49\), so residual variance \(0.51\) makes the observed variable standardized.

Interpretation and a Common Pitfall

Pattern coefficients answer a conditional, model-based question: what coefficient multiplies each factor in the linear representation of the observed variable? Structure coefficients answer a marginal association question: how strongly is that observed variable correlated with each factor? When factors are correlated, these questions have different answers. A structure coefficient can be substantial even when the corresponding pattern coefficient is small, because the variable may load on another factor correlated with the factor in question.

Pitfall: treating every structure coefficient as a direct loading. A structure coefficient is a correlation, so it can reflect both a direct pattern coefficient and associations transmitted through correlated factors. Use the pattern matrix for the model coefficients and the structure matrix for variable–factor correlations.
Pitfall: assuming that equal matrices prove orthogonality. Orthogonality guarantees equality for every \(\Lambda\), but a particular \(\Lambda\) can satisfy \(\Lambda(\Phi-I_q)=0\) even when \(\Phi\ne I_q\). Check the matrix product rather than inferring the factor correlations from one solution’s equal rows.
Pitfall: summing squared pattern coefficients for an oblique solution. The common variance is \(\Lambda_i\Phi\Lambda_i^{\mathsf T}\), which includes factor-correlation cross-products. The sum of squares is generally correct only when \(\Phi=I_q\).
Pitfall: using an inverse without checking it exists. Recovering \(\Lambda\) from \(S\) requires an invertible \(\Phi\). If it is singular, \(S=\Lambda\Phi\) still holds, but the inverse formula cannot be used.

Key takeaway: For standardized variables and factors, \(S=\Lambda\Phi\). The pattern and structure matrices coincide exactly when \(\Lambda(\Phi-I_q)=0\). Orthogonality guarantees coincidence for every loading matrix, but it is not necessary for a particular loading matrix.

Check Your Understanding

Use the formulas and distinctions in this tutorial to answer the following questions.

For standardized variables and factors, state the matrix equation relating the pattern matrix, factor correlation matrix, and structure matrix.
Suppose \(\Lambda_i=(0.5,0.3)\) and the factor correlation is \(0.4\). Calculate both structure coefficients for this variable.
Why does the sum of squared pattern coefficients generally fail to equal common variance when factors are correlated?
If \(\Phi\) is invertible and \(S\) is known, what formula recovers the pattern matrix? What condition is needed to use it?
Explain why orthogonal factors guarantee \(S=\Lambda\), but the equality \(S=\Lambda\) for one particular solution does not establish that its factors are orthogonal.