Oblique Rotation and the Factor-Covariance Matrix

Why an Oblique Transformation Changes Two Matrices

A rotation of a factor solution changes how the common variation is represented by the factors, not the fitted covariance of the observed variables. For an orthogonal rotation, the factor covariance remains the identity matrix. Under a general oblique change of factor coordinates, that is no longer true: the loadings and the factor-covariance matrix must change together. Keeping track of both changes is essential for preserving the model.

Use a centered factor model with $p$ observed variables and $q$ factors. Let $x$ be the observed-variable vector, $f$ the factor vector, and $\varepsilon$ the unique-factor or error vector. Assume $E[f]=0$, $E[\varepsilon]=0$, and $\operatorname{Cov}(f,\varepsilon)=0$. The factor covariance matrix need not be the identity.

Definition: In the factor model $x=\Lambda f+\varepsilon$, $\Lambda$ is the $p\times q$ loading matrix, $\Phi=\operatorname{Cov}(f)$ is the $q\times q$ factor-covariance matrix, and $\Psi=\operatorname{Cov}(\varepsilon)$ is the unique covariance matrix. The model-implied covariance is $$ \Sigma=\Lambda\Phi\Lambda'+\Psi. $$ The common covariance is $\Lambda\Phi\Lambda'$. In the usual factor model, $\Psi$ is diagonal. Assume $\Phi$ is positive definite and $\Psi$ is positive semidefinite.

The factor covariance matrix records the variances and covariances of the factors. If the original factors are orthogonal and standardized, then $\Phi=I_q$. An oblique transformation can produce factors with nonzero covariances and, in general, factor variances that are not one.

The Transformation and Its Covariance

Let $T$ be any nonsingular $q\times q$ matrix. Define new factor coordinates $g=T^{-1}f$. Substituting $f=Tg$ into the model gives $x=\Lambda T g+\varepsilon$. Thus the loading matrix in the new coordinates is $\Lambda_T=\Lambda T$. The covariance of $g$ follows from the covariance rule for a linear transformation.

\Lambda_T=\Lambda T,\qquad \Phi_T=\operatorname{Cov}(g)=T^{-1}\Phi T^{-T}, \qquad T^{-T}=(T^{-1})'.

The notation fixes the transformation convention used here: the new factor vector is $T^{-1}f$, so the loadings are multiplied on the right by $T$. Reversing that convention without also changing the covariance formula is a common source of errors.

Theorem: If $T$ is nonsingular, then transforming the factors and parameters as above preserves the common covariance and the full model-implied covariance: $$ \Lambda_T\Phi_T\Lambda_T'=\Lambda\Phi\Lambda', \qquad \Lambda_T\Phi_T\Lambda_T'+\Psi=\Sigma. $$ Moreover, $\Phi_T$ is positive definite whenever $\Phi$ is positive definite.

Proof. Substitute the transformed quantities into the common covariance:

\begin{aligned} \Lambda_T\Phi_T\Lambda_T' &=(\Lambda T)(T^{-1}\Phi T^{-T})(\Lambda T)'\\ &=\Lambda T T^{-1}\Phi T^{-T}T'\Lambda'\\ &=\Lambda\Phi\Lambda'. \end{aligned}

Here $TT^{-1}=I_q$, and $T^{-T}T'=I_q$, since $T^{-T}=(T')^{-1}$. Adding the unchanged $\Psi$ to both sides proves preservation of the full covariance. To verify positive definiteness, take any nonzero vector $a\in\mathbb{R}^q$. Since $T$ is nonsingular, $T^{-T}a\ne 0$, and therefore

a'\Phi_Ta =a'T^{-1}\Phi T^{-T}a =(T^{-T}a)'\Phi(T^{-T}a)>0.

Thus $\Phi_T$ is positive definite. $\square$

The transformation is reversible. From $\Lambda_T=\Lambda T$ and $\Phi_T=T^{-1}\Phi T^{-T}$, the original parameters are recovered as $\Lambda=\Lambda_TT^{-1}$ and $\Phi=T\Phi_TT'$. These inverse formulas show that this is a change of factor coordinates, not a change in the covariance model being represented.

Worked Matrix Calculations

Worked Example: A Shear Produces Correlated Factors

Start with two orthogonal factors, so $\Phi=I_2$, and take $$ \Lambda= \begin{pmatrix} 0.8&0.2\\ 0.6&0.5\\ 0.1&0.7 \end{pmatrix}, \qquad T= \begin{pmatrix} 1&0.5\\ 0&1 \end{pmatrix}. $$ The transformation is nonsingular, with $T^{-1}=\begin{pmatrix}1&-0.5\\0&1\end{pmatrix}$. Calculate the new loadings: $$ \Lambda_T=\Lambda T = \begin{pmatrix} 0.8&0.6\\ 0.6&0.8\\ 0.1&0.75 \end{pmatrix}. $$

Because the original factor covariance is $I_2$, the transformed factor covariance is $T^{-1}T^{-T}$: $$ \Phi_T= \begin{pmatrix}1&-0.5\\0&1\end{pmatrix} \begin{pmatrix}1&0\\-0.5&1\end{pmatrix} = \begin{pmatrix}1.25&-0.5\\-0.5&1\end{pmatrix}. $$ Its determinant is $1.25(1)-(-0.5)^2=1$, and its leading diagonal entries are positive, so it is positive definite. The two transformed factors have covariance $-0.5$. Their correlation is $-0.5/\sqrt{1.25\cdot1}$, approximately $-0.447$.

For a direct check of covariance preservation, the original common covariance is $$ \Lambda\Lambda'= \begin{pmatrix} 0.68&0.58&0.22\\ 0.58&0.61&0.41\\ 0.22&0.41&0.50 \end{pmatrix}. $$ Using the transformed matrices gives the same entries. For example, the transformed covariance between observed variables 1 and 2 is $$ (0.8,0.6) \begin{pmatrix}1.25&-0.5\\-0.5&1\end{pmatrix} \begin{pmatrix}0.6\\0.8\end{pmatrix} =(0.8,0.6)\begin{pmatrix}0.35\\0.50\end{pmatrix} =0.58. $$ The three diagonal entries are $0.68$, $0.61$, and $0.50$; the other off-diagonal entries are $0.22$ and $0.41$. Thus $\Lambda_T\Phi_T\Lambda_T'=\Lambda\Lambda'$, as claimed.

Worked Example: Transforming an Already Correlated Factor Model

Suppose $$ \Lambda=\begin{pmatrix}0.7&0.2\\0.4&0.8\end{pmatrix}, \qquad \Phi=\begin{pmatrix}1&0.3\\0.3&1\end{pmatrix}, \qquad T=\begin{pmatrix}1&1\\0&1\end{pmatrix}. $$ The original factor covariance is positive definite because its leading diagonal entry is positive and its determinant is $1-0.3^2=0.91>0$. Here $T^{-1}=\begin{pmatrix}1&-1\\0&1\end{pmatrix}$. The transformed loadings are $$ \Lambda_T=\Lambda T=\begin{pmatrix}0.7&0.9\\0.4&1.2\end{pmatrix}. $$

For the transformed factor covariance, first multiply $T^{-1}\Phi$, then multiply by $T^{-T}$: $$ T^{-1}\Phi= \begin{pmatrix}0.7&-0.7\\0.3&1\end{pmatrix}, \qquad \Phi_T=T^{-1}\Phi T^{-T} = \begin{pmatrix}1.4&-0.7\\-0.7&1\end{pmatrix}. $$ Its determinant is $1.4-0.49=0.91>0$, so it is positive definite. This example does not start with orthogonal factors: the transformation formula applies to a general positive-definite $\Phi$, not just to $I_q$.

Check the common covariance entry by entry. The original first diagonal entry is $$ (0.7,0.2)\begin{pmatrix}0.76\\0.41\end{pmatrix} =0.7(0.76)+0.2(0.41)=0.614, $$ where $\Phi(0.7,0.2)'=(0.76,0.41)'$. The second diagonal entry is $$ (0.4,0.8)\begin{pmatrix}0.64\\0.92\end{pmatrix} =0.4(0.64)+0.8(0.92)=0.992. $$ The off-diagonal entry is $(0.7,0.2)(0.64,0.92)'=0.632$. Now use the transformed representation. Its first diagonal entry is $$ (0.7,0.9)\begin{pmatrix}0.35\\0.41\end{pmatrix} =0.7(0.35)+0.9(0.41)=0.614, $$ and its second diagonal entry is $$ (0.4,1.2)\begin{pmatrix}-0.28\\0.92\end{pmatrix} =0.4(-0.28)+1.2(0.92)=0.992. $$ The off-diagonal entry is $(0.7,0.9)(-0.28,0.92)'=0.632$. All three entries agree.

Worked Example: A Nonorthogonal Scaling Need Not Create Correlation

A nonorthogonal change of coordinates need not produce correlated factors. Let $\Phi=I_2$, choose $T=\operatorname{diag}(2,0.5)$, and use $$ \Lambda=\begin{pmatrix}0.6&0.4\\0.3&0.8\end{pmatrix}. $$ The transformed loadings are $\Lambda_T=\begin{pmatrix}1.2&0.2\\0.6&0.4\end{pmatrix}$, while $$ \Phi_T=T^{-1}T^{-T} =\begin{pmatrix}0.25&0\\0&4\end{pmatrix}. $$ The factors remain uncorrelated, although their variances are no longer one. The original common covariance has entries $$ (1,1):\ 0.6^2+0.4^2=0.52,\qquad (2,2):\ 0.3^2+0.8^2=0.73,\qquad (1,2):\ 0.6(0.3)+0.4(0.8)=0.50. $$ In the transformed representation, these entries are $0.25(1.2^2)+4(0.2^2)=0.52$, $0.25(0.6^2)+4(0.4^2)=0.73$, and $0.25(1.2)(0.6)+4(0.2)(0.4)=0.50$. The covariance is preserved. This scaling illustrates why “nonorthogonal” should not be taken to mean that every transformed factor covariance must have nonzero off-diagonal entries.

What the Invariance Does—and Does Not—Say

For a fixed $\Psi$, covariance preservation means that every invertible $T$ in this construction gives the same $\Sigma$. In a Gaussian factor model, the conclusion extends to the distribution of the observed vector: if $f$ and $\varepsilon$ are jointly Gaussian and independent, then $g=T^{-1}f$ is Gaussian with covariance $\Phi_T$, and $\Lambda_Tg+\varepsilon=\Lambda f+\varepsilon$. The observed distribution is therefore unchanged. The transformation changes the coordinates used to describe the latent factors, not the model-implied distribution of the data.

The invariance is a statement about a coordinated transformation. It does not say that the loading matrix alone determines the common covariance once the factors may be correlated. The covariance is $\Lambda\Phi\Lambda'$, so both matrices are needed. Nor does the algebra by itself choose a useful rotation; it only shows which parameter changes preserve the fitted covariance.

Common pitfall: One might multiply the loadings by $T$ and then continue to use $I_q$ as the factor covariance, reasoning that a rotation only changes the loading matrix. That is correct for an orthogonal transformation of an initially standardized orthogonal solution, but not for a general oblique transformation. In Worked Example 1, this mistake gives the off-diagonal common-covariance entry $(0.8,0.6)(0.6,0.8)'=0.96$, whereas the correct entry is $0.58$. The correction is to transform the factor covariance at the same time, using $\Phi_T=T^{-1}\Phi T^{-T}$.

Key takeaway: Under the convention $g=T^{-1}f$, an invertible transformation changes the loadings to $\Lambda T$ and the factor covariance to $T^{-1}\Phi T^{-T}$. Together they preserve the common covariance exactly; changing only one of these matrices generally does not.

Check Your Understanding

Use the transformation convention and covariance identity developed above.

If $g=T^{-1}f$, what are the transformed loading matrix and factor covariance matrix?
Why must $T$ be nonsingular for this to be a reversible change of factor coordinates?
Starting from $\Phi=I_q$, explain why an oblique transformation can give factors whose variances are not one.
What condition on $\Phi$ ensures that $T^{-1}\Phi T^{-T}$ is positive definite?
Suppose someone changes $\Lambda$ to $\Lambda T$ but leaves $\Phi$ unchanged. What part of the covariance calculation has been omitted?

Oblique Rotation and the Factor-Covariance Matrix

What You'll Learn