Confirmatory Loading Constraints and Rotational Identification

Why Constrain a Loading Matrix?

A factor model can represent the same observed covariance matrix with different loading matrices. This rotational indeterminacy is useful when exploring interpretations, but it creates a problem for confirmatory analysis: a scientific interpretation needs factors whose meanings are specified rather than chosen after inspecting a convenient rotation. Confirmatory loading constraints express that specification. The key question is not only whether a proposed restriction can be written down, but whether it actually selects an orientation.

Let \(p\) denote the number of observed variables and \(q\) the number of factors. In a common factor model, the covariance matrix is represented by a loading matrix, a factor covariance matrix, and a diagonal uniqueness matrix. A change of factor coordinates can change the loadings while leaving the represented covariance unchanged. Constraints on the loadings distinguish among these equivalent representations.

Definition: In the common factor covariance model, \(\Sigma=\Lambda\Phi\Lambda^\top+\Psi\), where \(\Sigma\) is the \(p\times p\) covariance matrix of the observed variables, \(\Lambda\) is the \(p\times q\) loading matrix, \(\Phi\) is the positive-definite \(q\times q\) factor covariance matrix, and \(\Psi\) is a diagonal uniqueness matrix. A confirmatory loading constraint is a specified restriction on entries or functions of entries of \(\Lambda\), used to express the intended factor structure and, potentially, to identify its orientation.

A zero constraint fixes a loading at zero, typically asserting that an observed variable does not load on a particular factor. An equality constraint sets two or more loadings equal, perhaps to express a common effect. A target constraint specifies preferred loading values but allows departures from them; it is often implemented by minimizing a discrepancy from the target rather than by imposing exact equalities.

Definition: A zero constraint has the form \(\lambda_{ij}=0\). An equality constraint has the form \(\lambda_{ij}=\lambda_{k\ell}\), or more generally equates a specified collection of loadings. A target constraint assigns desired values \(\tau_{ij}\) to selected loadings and assesses or penalizes departures such as \(\lambda_{ij}-\tau_{ij}\). Exact constraints define a restricted set; targets ordinarily define a criterion for choosing among orientations.

Equivalent Representations and the Need for Identification

For any nonsingular \(q\times q\) matrix \(T\), define \(\Lambda^*=\Lambda T\) and \(\Phi^*=T^{-1}\Phi T^{-\top}\). The observed covariance representation is unchanged. This is the general change of factor coordinates when \(\Phi\) is allowed to vary. In an orthogonal factor representation with \(\Phi=I\), transformations that preserve that condition are orthogonal matrices \(Q\), satisfying \(Q^\top Q=I\); then \(\Lambda^*=\Lambda Q\).

Result: Under the nonsingular transformation above, \(\Lambda^*\Phi^*\Lambda^{*\top}+\Psi=\Lambda\Phi\Lambda^\top+\Psi\). Thus loading restrictions can select a representation of the same covariance structure without changing that structure.

To verify the result, substitute the transformed quantities and use \(TT^{-1}=I\):

\begin{aligned} \Lambda^*\Phi^*\Lambda^{*\top}+\Psi &=(\Lambda T)(T^{-1}\Phi T^{-\top})(\Lambda T)^\top+\Psi\\ &=\Lambda T T^{-1}\Phi T^{-\top}T^\top\Lambda^\top+\Psi\\ &=\Lambda\Phi\Lambda^\top+\Psi. \end{aligned}

This equality explains both the value and the limits of loading constraints. A restriction may choose one representative from a set of covariance-equivalent representations, but it does not automatically resolve all ambiguities. Factor signs and labels can remain interchangeable, and factor scales require their own conventions, such as fixed factor variances or marker-loading restrictions. A locally unique orientation is also not necessarily a globally unique parameterization.

Worked Example: Zero Restrictions With Marker Loadings

Start with the two-factor loading matrix

\Lambda= \begin{pmatrix} 2&1\\ 1&2 \end{pmatrix}, \qquad T= \begin{pmatrix} a&b\\ c&d \end{pmatrix}.

Suppose the confirmatory pattern requires variable 1 to load only on factor 1 and variable 2 to load only on factor 2. To set the scale, also require both designated marker loadings to equal 1. Since \(\Lambda^*=\Lambda T\), the four restrictions are

2b+d=0,\qquad a+2c=0,\qquad 2a+c=1,\qquad b+2d=1.

From \(a+2c=0\), \(c=-a/2\). Substituting into \(2a+c=1\) gives \(3a/2=1\), so \(a=2/3\) and \(c=-1/3\). From \(2b+d=0\), \(d=-2b\). Substituting into \(b+2d=1\) gives \(-3b=1\), so \(b=-1/3\) and \(d=2/3\). Therefore

T= \begin{pmatrix} 2/3&-1/3\\ -1/3&2/3 \end{pmatrix}, \qquad \det(T)=\frac{4}{9}-\frac{1}{9}=\frac{1}{3}\ne 0.

The transformation is admissible because it is nonsingular. Direct multiplication gives \(\Lambda T=I\), so both zero restrictions and both marker restrictions hold. The example shows how zeros specify a pattern while marker values fix scale. These calculations establish a solution for this starting matrix; they do not imply that every proposed pattern or data set has a unique admissible solution.

How to Assess Whether Constraints Identify an Orientation

For a local identification check, describe allowable transformations with coordinates. For example, a two-factor orthogonal rotation can be parameterized near a given orientation by an angle \(\theta\). More generally, choose a smooth local coordinate vector \(u\) for the admissible transformations, after applying the chosen scale and normalization conventions. Let \(g(u)\) collect the left-hand sides of the exact restrictions, with each restriction written as zero. The restrictions identify the orientation locally at \(u_0\) if no distinct sufficiently nearby coordinate value satisfies them.

Theorem: Suppose \(g\) is continuously differentiable in a neighborhood of \(u_0\in\mathbb{R}^d\), \(g(u_0)=0\), and its Jacobian \(Dg(u_0)\) has column rank \(d\). Then the restrictions \(g(u)=0\) identify \(u_0\) locally: there is a neighborhood of \(u_0\) in which \(u_0\) is the only solution.

Proof. Since \(Dg(u_0)\) has column rank \(d\), some \(d\) rows form a nonsingular \(d\times d\) matrix. Let \(h(u)\) be the vector consisting of those \(d\) components of \(g(u)\). Then \(h(u_0)=0\) and \(Dh(u_0)\) is nonsingular. By the inverse function theorem, \(h\) is one-to-one in a neighborhood of \(u_0\). Any solution of \(g(u)=0\) in that neighborhood must also satisfy \(h(u)=0=h(u_0)\), and hence must equal \(u_0\). This proves local identification. \(\square\)

The rank condition is sufficient, not necessary in every irregular case, and it is local rather than global. If sign changes or factor permutations leave the constraints unchanged, they may produce distinct solutions outside the neighborhood. Nor does merely having at least \(d\) restrictions guarantee rank \(d\): restrictions can be redundant or have zero derivatives in some directions.

Worked Example: A Zero Loading Locally Identifies a Rotation

Consider the two-factor loading rows \((2,1)\) and \((1,0)\), and use the orthogonal rotation

Q(\theta)= \begin{pmatrix} \cos\theta&-\sin\theta\\ \sin\theta&\cos\theta \end{pmatrix}.

Require the first variable's loading on factor 2 after rotation to be zero. Its transformed row is

(2,1)Q(\theta) =(2\cos\theta+\sin\theta,\,-2\sin\theta+\cos\theta).

The restriction is \(g(\theta)=-2\sin\theta+\cos\theta=0\). One solution satisfies \(\tan\theta=1/2\), with \(\cos\theta=2/\sqrt{5}\) and \(\sin\theta=1/\sqrt{5}\). At this solution,

g'(\theta)=-2\cos\theta-\sin\theta =-\frac{4}{\sqrt{5}}-\frac{1}{\sqrt{5}} =-\sqrt{5}\ne 0.

There is one rotation coordinate and the derivative has rank one, so the zero restriction locally identifies this orientation. The equation also has solutions differing by \(\pi\); the zero alone therefore does not distinguish the global sign of both factors.

Worked Example: An Equality Restriction Can Be Uninformative

Use the same rotation convention with the loading matrix \(\Lambda=\begin{pmatrix}2&0\\0&1\end{pmatrix}\). Suppose the only restriction is that the two diagonal loadings after rotation are equal. The transformed first row is \((2\cos\theta,-2\sin\theta)\), and the transformed second row is \((\sin\theta,\cos\theta)\). Thus the equality restriction is

g(\theta)=2\cos\theta-\cos\theta=\cos\theta=0.

At \(\theta=\pi/2\), the restriction holds, but \(g'(\theta)=-\sin\theta=-1\), so it identifies that solution locally. In contrast, if the equality were between the two first-column loadings, the equation would be \(2\cos\theta=\sin\theta\), with derivative \(-2\sin\theta-\cos\theta\). At a solution with \(\tan\theta=2\), taking \(\cos\theta=1/\sqrt{5}\) and \(\sin\theta=2/\sqrt{5}\) gives derivative \(-5/\sqrt{5}=-\sqrt{5}\ne0\), again a locally identifying restriction. These calculations illustrate that equality restrictions can identify an orientation, but the result depends on the actual loading pattern and derivative, not on the word “equality.”

Target Restrictions and Orientation Choice

A target matrix \(\Tau\) specifies desired values for selected loading entries. A common approach chooses an allowable transformation to minimize a discrepancy, for example

J(Q)=\sum_{(i,j)\in\mathcal{T}}w_{ij}\bigl((\Lambda Q)_{ij}-\tau_{ij}\bigr)^2, \qquad w_{ij}>0,

where \(\mathcal{T}\) indexes targeted entries. Unless the target values are imposed as exact equations, this is an optimization criterion, not a set of exact loading constraints. A minimizer selects an orientation according to the chosen targets and weights. Multiple minimizers, a flat objective in some direction, or a remaining sign ambiguity can prevent unique orientation selection.

Worked Example: A Target Criterion Selects an Orientation

Take \(\Lambda=\begin{pmatrix}2&0\\0&1\end{pmatrix}\) and target the first variable's second-factor loading and the second variable's first-factor loading at zero. Under the rotation \(Q(\theta)\) above, those two loadings are \(-2\sin\theta\) and \(\sin\theta\). With unit weights, the target discrepancy is

J(\theta)=(-2\sin\theta)^2+(\sin\theta)^2=5\sin^2\theta.

Since \(0\leq\sin^2\theta\leq1\), \(J(\theta)\geq0\), with equality exactly when \(\sin\theta=0\), or \(\theta=k\pi\) for integer \(k\). The target criterion therefore selects an axis orientation, but it does not distinguish \(\theta=0\) from \(\theta=\pi\): the latter reverses both factor signs. The target is effective for choosing a sparse orientation, yet additional conventions are needed to fix signs.

What Restriction Counts Can and Cannot Tell You

The number of free rotational coordinates provides a useful first check. With unrestricted nonsingular transformations, there are \(q^2\) transformation coordinates before normalizations. For orthogonal rotations, the dimension is \(q(q-1)/2\). These counts describe the dimension of the transformation family, not a sufficient number of useful restrictions. Exact restrictions must vary independently in the relevant directions; the Jacobian rank check formalizes that requirement locally.

Restrictions also encode substantive assumptions. A zero loading asserts more than a convenient orientation when interpreted as a claim about the measurement structure. An equality restriction asserts a particular relationship among effects. A target allows approximate rather than exact adherence, but its weights and targeted entries influence which solution is preferred. The restrictions should therefore be justified by the measurement design or scientific question, not added mechanically to force a desired output.

Common Mistake

Wrong reasoning: “There are as many restrictions as rotational degrees of freedom, so the orientation must be identified.”
Correction: A count is only a preliminary check. Restrictions may be redundant, fail to change under a particular rotation, or leave discrete sign and permutation alternatives. Check the rank of the restriction Jacobian at the proposed solution, and separately consider scale normalization and global ambiguities.
Wrong reasoning: “A target loading of zero is the same as an exact zero-loading constraint.”
Correction: A target generally contributes to an objective being minimized; it need not equal zero at the optimum. Treat it as exact only when it is explicitly imposed as an equation.

Key takeaway: Confirmatory loading restrictions specify a desired factor structure, but they identify an orientation only when they constrain the allowable transformation in the relevant directions. Distinguish exact restrictions from target criteria, check local rank rather than only counting restrictions, and state the scale and sign conventions that remain.

Check Your Understanding

For each question, distinguish what the restriction says from what it establishes about identification.

What is the difference between imposing \(\lambda_{ij}=0\) exactly and targeting that loading at zero in a rotation criterion?
Why can \(\Lambda^*=\Lambda T\) and \(\Phi^*=T^{-1}\Phi T^{-\top}\) represent the same observed covariance matrix?
In a one-dimensional rotation parameterization, what does a nonzero derivative of an exact restriction at a solution imply locally?
Why does having as many restrictions as rotational degrees of freedom not guarantee identification?
What ambiguity remains in the target-criterion example when \(\theta=0\) and \(\theta=\pi\) both attain the minimum?