Matching Factors Across Solutions With Congruence

Why Factor Matching Requires More Than Comparing Labels

Two rotated factor solutions can describe similar loading patterns while listing their factors in different orders or with opposite signs. A factor called “Factor 1” in one solution therefore need not correspond to “Factor 1” in another. To compare solutions, we need a measure of similarity between loading columns that does not depend on their names, and a systematic way to pair the columns.

The Tucker congruence coefficient measures the similarity of two loading columns across the same observed variables. It is a normalized inner product: its sign records whether the columns point in the same or opposite directions, while its absolute value measures the strength of their alignment. Congruence helps with matching, but it does not establish that two fitted models are equivalent or that a match is substantively meaningful.

Definition: Let $a=(a_1,\ldots,a_p)^\top$ and $b=(b_1,\ldots,b_p)^\top$ be two nonzero loading columns, each referring to the same $p$ observed variables and using the same type of loading (for example, pattern loadings compared with pattern loadings). Their Tucker congruence coefficient is $$ \phi(a,b)=\frac{\sum_{i=1}^p a_i b_i}{\sqrt{\sum_{i=1}^p a_i^2}\sqrt{\sum_{i=1}^p b_i^2}}. $$ It is undefined if either column has zero norm. A value near $1$ indicates similar directions, a value near $-1$ indicates similar directions after reversing one sign, and a value near $0$ indicates little linear alignment.

The coefficient is unchanged if either column is multiplied by a positive constant: it compares directions, not magnitudes. It is also a cosine between two vectors, so it lies between $-1$ and $1$. Factor columns can be compared this way only when their entries refer to the same observed variables and comparable loading conventions. For oblique solutions, use the same kind of loading in both solutions: compare pattern columns with pattern columns, or structure columns with structure columns. Pattern and structure matrices have different meanings and should not be mixed in one comparison.

Worked Example: Congruence and a Reversed Factor Sign

Suppose a factor in one solution has loading column $a=(0.8,0.6,0)^\top$, while a factor in a second solution has column $b=(-0.7,-0.7,0)^\top$. Calculate their congruence and interpret its sign.

The squared norm of $a$ is $0.8^2+0.6^2+0^2=0.64+0.36=1$. The squared norm of $b$ is $0.7^2+0.7^2+0^2=0.49+0.49=0.98$. Their inner product is $0.8(-0.7)+0.6(-0.7)+0(-0)=-0.56-0.42=-0.98$.

\phi(a,b)=\frac{-0.98}{\sqrt{1}\sqrt{0.98}} =-\sqrt{0.98}\approx -0.9899.

The negative sign indicates opposing directions. Reversing the sign of $b$ gives congruence approximately $0.9899$, so these columns are very similar after a sign reversal. This is not a conflict in the substantive loading pattern: the sign of a factor is arbitrary, provided the corresponding factor representation is changed consistently.

Properties That Make Congruence Useful

The bound on congruence follows directly from the Cauchy–Schwarz inequality. The sign behavior follows from the formula itself. Stating these properties explicitly clarifies what the coefficient can and cannot tell us.

Result: For nonzero loading columns $a$ and $b$, $-1\leq\phi(a,b)\leq 1$. Moreover, $\phi(a,-b)=-\phi(a,b)$, while $|\phi(a,-b)|=|\phi(a,b)|$.

Proof. By Cauchy–Schwarz, $|a^\top b|\leq\|a\|\|b\|$. Both norms are positive, so dividing by their product gives $|\phi(a,b)|\leq1$, which is equivalent to the stated bounds. Replacing $b$ by $-b$ changes the numerator from $a^\top b$ to $-a^\top b$, but leaves the norm of the second column unchanged. Hence $\phi(a,-b)=-\phi(a,b)$, and taking absolute values gives the second identity. $\square$

Equality in the bound occurs exactly when the two nonzero columns are proportional. Thus congruence $1$ means one column is a positive multiple of the other, and congruence $-1$ means it is a negative multiple. In applied comparisons, coefficients somewhat below these extremes may still indicate substantial similarity, but no universal cutoff should substitute for examining the loading patterns and the context.

Matching Several Factors

For two solutions with $q$ factors, calculate the congruence between every column in the first solution and every column in the second. The resulting $q\times q$ matrix is a congruence matrix. A factor match should be a one-to-one pairing: each factor in one solution is paired with at most one factor in the other. Choosing the largest coefficient independently in each row can assign the same factor twice and produce a poor overall pairing.

A common matching rule chooses the permutation $\pi$ that maximizes the sum of absolute congruences, $\sum_{j=1}^q|\phi(a_j,b_{\pi(j)})|$. Absolute values allow sign reversals to count as close matches; after pairing, each matched factor’s sign can be oriented to make its coefficient positive. This criterion is a useful convention, not a unique law of factor interpretation. If two assignments have nearly equal totals, the matching itself is uncertain.

Result: For a fixed pairing of columns, the largest possible sum of signed congruences over choices of sign for the second solution’s columns is the sum of the absolute congruences for that pairing. Consequently, maximizing the absolute-congruence sum over permutations gives the best one-to-one pairing under this signed-sum criterion.

Proof. Fix a permutation $\pi$, and let $s_j\in\{-1,1\}$ be the sign applied to column $b_{\pi(j)}$. The signed contribution for pair $j$ is $s_j\phi(a_j,b_{\pi(j)})$. If the coefficient is positive, choose $s_j=1$; if it is negative, choose $s_j=-1$. Either choice gives contribution $|\phi(a_j,b_{\pi(j)})|$; if it is zero, either sign gives zero. Since each sign can be chosen independently, the maximum sum for this fixed permutation is $\sum_j|\phi(a_j,b_{\pi(j)})|$. Maximizing that quantity over all permutations gives the best pairing under the stated criterion. $\square$

Worked Example: Why the Best Pairing Is Global

Suppose three loading columns in the first solution are the coordinate vectors $a_1=(1,0,0,0)^\top$, $a_2=(0,1,0,0)^\top$, and $a_3=(0,0,1,0)^\top$. Consider columns in a second solution $b_1=(0.75,0.65,0.05,\sqrt{0.0125})^\top$, $b_2=(0.70,0.10,0.70,0.10)^\top$, and $b_3=(0,0,0.95,\sqrt{0.0975})^\top$. Each $b$ column has norm one: their squared norms are respectively $0.5625+0.4225+0.0025+0.0125=1$, $0.49+0.01+0.49+0.01=1$, and $0.9025+0.0975=1$. The $a$ columns also have norm one, so each congruence is the corresponding coordinate of $b$.

The congruence matrix, with rows for $a_1,a_2,a_3$ and columns for $b_1,b_2,b_3$, is

\begin{pmatrix} 0.75 & 0.70 & 0\\ 0.65 & 0.10 & 0\\ 0.05 & 0.70 & 0.95 \end{pmatrix}.

Matching each row to its largest entry first assigns $a_1$ to $b_1$ (0.75), then $a_2$ to $b_2$ (0.10), and finally $a_3$ to $b_3$ (0.95). Its total is $0.75+0.10+0.95=1.80$. But the one-to-one assignment $a_1\leftrightarrow b_2$, $a_2\leftrightarrow b_1$, $a_3\leftrightarrow b_3$ has total $0.70+0.65+0.95=2.30$. It is better overall, even though it does not give $a_1$ its individually largest entry.

For completeness, the other assignment using $b_1$ and $b_2$ with $a_3\leftrightarrow b_2$ is impossible because it would reuse $b_2$. The remaining alternatives have totals no greater than $0.75+0.10+0.95=1.80$, $0.70+0.65+0.95=2.30$, or $0.75+0+0.70=1.45$; checking the six permutations confirms that 2.30 is the maximum. This example shows why the matching should be treated as a joint assignment rather than a sequence of local choices.

Sign Changes, Permutations, and Near-Duplicate Factors

A sign reversal changes the signed congruence but not its absolute value. A permutation simply changes the order in which factors appear, and therefore permutes the rows or columns of the congruence matrix. Neither operation, by itself, indicates a substantively different loading pattern. Once a match is selected, report whether its signed congruence was positive or negative before orienting the columns: that sign records whether a reversal was needed.

Near-duplicate factors create a different difficulty. If two columns have very similar loading patterns, each may have high congruence with the same column in another solution. A one-to-one assignment still returns a pairing, but that pairing may be weakly determined: small perturbations can change which factor is matched to which. High coefficients alone do not resolve this ambiguity; examine the competing assignments and their total scores.

Worked Example: A Near-Duplicate Pair

Let the first solution have unit columns $a_1=(1,0,0)^\top$ and $a_2=(0.99,\sqrt{0.0199},0)^\top$. The second is also unit length because $0.99^2+0.0199=0.9801+0.0199=1$. Suppose the second solution has columns $b_1=a_1$ and $b_2=a_2$.

The congruence of $a_1$ with $b_1$ is $1$. The congruence of $a_1$ with $b_2$ is $0.99$, since both norms are one. By symmetry, $\phi(a_2,b_1)=0.99$, and $\phi(a_2,b_2)=1$. Thus the congruence matrix is

\begin{pmatrix} 1 & 0.99\\ 0.99 & 1 \end{pmatrix}.

The diagonal assignment totals $1+1=2$; the crossed assignment totals $0.99+0.99=1.98$. The diagonal assignment wins, but by only 0.02. Both candidate matches have high congruence, and the factors are themselves almost indistinguishable in direction. A small change to the loading estimates could alter the preferred pairing, so the result should be described as an ambiguous match rather than as strong evidence that each factor has a uniquely identified counterpart.

Practical Interpretation and Common Pitfalls

Tucker congruence is useful for comparing rotated loading patterns when the same variables and loading conventions are used. A transparent comparison reports the congruence matrix, the chosen one-to-one assignment, and any sign reversals. When factors are near-duplicates, it is also helpful to report close alternative assignments rather than presenting a single algorithmic choice as definitive.

Ignoring the sign. A coefficient near $-1$ is not a weak match. Its absolute value is near one, and reversing one factor’s sign gives a positive match.
Matching by factor number. Column labels and ordering are not reliable identifiers. Compare loading patterns and consider all one-to-one assignments.
Making row-by-row choices. Independent row maxima can reuse the same factor. Use a global assignment criterion when several factors are being matched.
Treating a high coefficient as proof of equivalence. Congruence describes the alignment of two columns; it does not, by itself, prove that the solutions are equivalent or settle their substantive interpretation.
Overstating a near-tie. If several assignments have similar totals, report the ambiguity. A formal pairing can exist even when the evidence does not clearly distinguish the alternatives.
Comparing unlike columns. Use the same variables and compare the same kind of loading matrix in both solutions. A coefficient calculated from mismatched entries is not a meaningful factor comparison.

Key takeaway: Calculate congruence across corresponding loading columns, use absolute coefficients to account for arbitrary sign reversals, and choose matches jointly through a one-to-one assignment. Interpret close competing assignments as uncertainty about correspondence, not as a decisive factor identity.

Check Your Understanding

Use the definitions and matching principles above to answer these questions.

Why does the Tucker congruence coefficient remain unchanged in absolute value when one loading column is multiplied by $-1$?
What does a congruence of $-0.96$ indicate, and what sign adjustment would make the matched coefficient positive?
Why can selecting the largest entry separately in each row of a congruence matrix fail to give the best one-to-one match?
Two possible assignments have absolute-congruence totals of 2.91 and 2.90. What should be reported about the match, and why?
Why should pattern loadings from one oblique solution not be compared directly with structure loadings from another?