Worked Orthogonal Rotation of a Loading Matrix

Why Work Through a Rotation Numerically?

An orthogonal rotation changes the coordinate axes used to describe common variation in a factor model. The loadings can become easier to interpret even though the model’s common covariance representation is unchanged. To assess a rotation, it is useful to do more than name the criterion: calculate the rotated loadings, check the matrix arithmetic, and verify what the rotation preserves.

This tutorial works with a particular loading matrix and a specified orthogonal rotation. The numerical example uses Varimax as the criterion for comparing loading patterns. It illustrates the effect of this rotation; it does not claim that the selected rotation is the global Varimax optimum.

The multiplication convention matters: loadings are in the rows, so this tutorial rotates them by right multiplication, \(\Lambda Q\). For two factors, a convenient rotation matrix is

Its columns have unit length and are perpendicular, so \(Q(\theta)'Q(\theta)=I_2\). The angle’s sign determines the rotation direction under this convention. A computation using \(Q'\) instead of \(Q\) rotates in the opposite direction.

Worked Example: Rotate a Four-Variable Loading Matrix

Consider the loading matrix

First check orthogonality. Each column of \(Q\) has squared length \(0.8^2+(-0.6)^2=0.64+0.36=1\), and their inner product is \(0.8(0.6)+(-0.6)(0.8)=0.48-0.48=0\). Thus \(Q'Q=I_2\). This is \(Q(\theta)\) with \(\cos\theta=0.8\) and \(\sin\theta=-0.6\).

For a row \((a,b)\), right multiplication gives \((a,b)Q=(0.8a-0.6b,\ 0.6a+0.8b)\). Applying that operation to every row gives

For example, the first row becomes \((0.56+0.24,\ 0.42-0.32)=(0.80,0.10)\). In the second row the result is \((0.544+0.156,\ 0.408-0.208)=(0.70,0.20)\). The other two rows are computed by the same row-by-matrix multiplication.

The loading pattern has changed substantially. In \(A\), several rows have moderate loadings on both factors, and the first two rows include negative second-factor loadings. In \(B\), each row has one loading of \(0.7\) or \(0.8\) and one of \(0.1\) or \(0.2\). This is a more concentrated pattern of loadings in this example.

What Orthogonal Rotation Preserves

A useful check is to calculate the row lengths on both sides. In \(A\), the four communalities are

For \(B\), they are \(0.80^2+0.10^2=0.65\), \(0.70^2+0.20^2=0.53\), \(0.10^2+0.80^2=0.65\), and \(0.20^2+0.70^2=0.53\). Each matches its corresponding value for \(A\).

Proof. The rotated row \(i\) is \(\lambda_iQ\), where \(\lambda_i\) is a row vector. Its squared length is

The first equality is the definition of squared length, and the second uses \(QQ'=I_m\), which follows from orthogonality. The last expression is the original row’s squared length. This proves the result for every row. \(\square\)

For the factor model with orthogonal factors, the common covariance component is \(\Lambda\Lambda'\). It is also preserved, since \((\Lambda Q)(\Lambda Q)'=\Lambda QQ'\Lambda'=\Lambda\Lambda'\). Thus rotation changes the coordinates used to express the loadings; it does not change the common covariance represented by those loadings.

Compare the Varimax Criterion

For a \(p\times m\) loading matrix \(L=(l_{ij})\), the Varimax criterion is the sum across factors of the variances of the squared loadings within each column:

A larger value indicates greater dispersion among the squared loadings within columns under this criterion. The squares mean that changing the sign of a loading does not affect this calculation. We now calculate \(V\) for both matrices in the example, taking care to square each squared loading when finding the fourth-power sum.

This numerical comparison supports the description of \(B\) as having a more concentrated loading pattern under Varimax. It does not show that \(B\) maximizes Varimax over all possible rotations: that claim would require evaluating or optimizing the criterion over the full set of orthogonal matrices. A specified rotation can be assessed by calculating its result and criterion, but a single comparison is not an optimization proof.

Worked Example: A Forty-Five-Degree Rotation

Consider a second loading matrix and a \(45\)-degree rotation:

Since \(1/\sqrt{2}\) is approximately \(0.707107\), the first row \((0.8,0)\) becomes approximately \((0.565685,-0.565685)\). The second row becomes \((0.424264,0.424264)\), and the third becomes \((0.989949,0.141421)\). Hence

The original communalities are \(0.8^2+0^2=0.64\), \(0^2+0.6^2=0.36\), and \(0.6^2+0.8^2=1\). After rotation, the first row has squared length approximately \(0.32+0.32=0.64\); the second, \(0.18+0.18=0.36\); and the third, \(0.98+0.02=1\). Rounding accounts for any tiny discrepancy in the displayed decimal values. The individual loadings change, while each row’s squared length remains the same.

Common Mistakes and a Practical Check

• Using the wrong multiplication convention. With loadings stored as rows, the rotation is \(\Lambda Q\), not \(Q\Lambda\). Check the dimensions: a \(p\times m\) loading matrix can be multiplied on the right by an \(m\times m\) rotation matrix.
• Using the wrong rotation direction. Replacing \(Q\) by \(Q'\) gives the inverse rotation. Verify the chosen matrix and multiply a row explicitly before proceeding.
• Confusing communalities with individual loadings. The theorem preserves each row’s squared length, not the values or signs of its individual entries.
• Miscomputing the fourth-power sum. In the Varimax calculation, square each squared loading. For example, a loading of \(0.8\) contributes \(0.8^4=0.4096\), not \(2(0.8^4)\) for a single entry.
• Calling a candidate rotation optimal. A larger criterion than the starting matrix establishes an improvement for that comparison, not a global maximum over all orthogonal rotations.