Indicator Patterns and Generic Identification

Why Indicator Patterns Matter

A factor model describes covariances among observed variables through a smaller set of latent factors. But having fewer parameters than observed covariances does not guarantee that the parameters can actually be recovered. The locations of the nonzero loadings matter: some patterns create informative covariance links, while others leave continuous trade-offs between loadings that no amount of covariance information can resolve.

This tutorial treats identification as a question about the model’s covariance structure. It develops a useful sufficient condition based on groups of pure indicators, and shows explicitly why a favorable parameter count can still be misleading.

Definition: In the orthogonal common-factor model, the observed vector $X$ has covariance matrix $$ \Sigma=\Lambda\Lambda^\mathsf{T}+\Psi, $$ where $\Lambda$ is the $p\times m$ loading matrix and $\Psi$ is diagonal. The diagonal entries of $\Psi$ are the specific variances. A loading pattern specifies which entries of $\Lambda$ are free and which are fixed at zero. A parameter value is generically identified if, except on exceptional lower-dimensional sets of parameter values, its covariance matrix determines it uniquely up to unavoidable factor sign changes and permutations.

The orthogonal-factor convention fixes factor variances at one and factor covariances at zero. Reversing a factor’s sign reverses the signs of all its loadings without changing $\Sigma$; permuting factors also leaves the model unchanged. These are equivalences, not failures of identification. In this tutorial, “identified” allows these sign and permutation changes.

Generic identification is not the same as identification at every possible parameter value. Zero loadings, special equalities among loadings, and other exceptional configurations can make recovery fail even when nearby, typical parameter values are recoverable. Conversely, a model may have a continuous family of parameter values giving the same covariance matrix, which is a structural failure rather than an isolated exception.

Three Indicators Can Identify One Factor

Consider one factor, with three observed variables that all load on it and have nonzero loadings. Write their loadings as $\lambda_1,\lambda_2,\lambda_3$. Because $\Psi$ is diagonal, each off-diagonal covariance is just a product of two loadings:

\sigma_{12}=\lambda_1\lambda_2,\qquad \sigma_{13}=\lambda_1\lambda_3,\qquad \sigma_{23}=\lambda_2\lambda_3.

These three links allow the individual loading magnitudes to be recovered. The sign of all three can be reversed together, but that is precisely the factor-sign equivalence.

Theorem: For a one-factor model with three nonzero loadings, if the three off-diagonal covariances are nonzero and the covariance matrix is generated by the model, then the loading magnitudes are determined by $$ \lambda_1^2=\frac{\sigma_{12}\sigma_{13}}{\sigma_{23}},\qquad \lambda_2^2=\frac{\sigma_{12}\sigma_{23}}{\sigma_{13}},\qquad \lambda_3^2=\frac{\sigma_{13}\sigma_{23}}{\sigma_{12}}. $$ The loadings are determined up to a simultaneous sign reversal, and each specific variance is then determined by the corresponding diagonal variance minus the squared loading.

Proof. Multiplying the first two covariance equations gives $\sigma_{12}\sigma_{13}=\lambda_1^2\lambda_2\lambda_3$. Since $\sigma_{23}=\lambda_2\lambda_3\ne0$, division gives the formula for $\lambda_1^2$. The other two formulas follow by the same multiplication-and-division argument. Once the magnitudes are known, the signs of the pairwise products fix the relative signs of the loadings; only reversing all three signs together leaves every product unchanged. Finally, $\sigma_{ii}=\lambda_i^2+\psi_{ii}$, so $\psi_{ii}=\sigma_{ii}-\lambda_i^2$. $\square$

Worked Example: Recovering a One-Factor Triad

Suppose three standardized variables have off-diagonal covariances $\sigma_{12}=0.30$, $\sigma_{13}=0.40$, and $\sigma_{23}=0.60$. Each diagonal variance is 1. Recover the loadings and specific variances.

First, $\lambda_1^2=(0.30)(0.40)/0.60=0.20$, so choose $\lambda_1=\sqrt{0.20}\approx0.4472$ as the sign convention. Next, $\lambda_2^2=(0.30)(0.60)/0.40=0.45$, giving $\lambda_2\approx0.6708$ with the sign consistent with $\sigma_{12}>0$. Finally, $\lambda_3^2=(0.40)(0.60)/0.30=0.80$, giving $\lambda_3\approx0.8944$.

Checking the reconstructed covariances gives $(0.4472)(0.6708)\approx0.30$, $(0.4472)(0.8944)\approx0.40$, and $(0.6708)(0.8944)\approx0.60$. The specific variances are $1-0.20=0.80$, $1-0.45=0.55$, and $1-0.80=0.20$; each is positive. Thus the proposed parameters reproduce the given covariance matrix.

The proof also shows why two indicators are not enough for this recovery: their single off-diagonal covariance gives only the product $\lambda_1\lambda_2$. A third indicator supplies another link, making the squared loadings separately recoverable. That observation is a pattern-based identification argument, not merely a comparison of the number of parameters and covariances.

Pure Indicators Give a Useful Sufficient Pattern

An indicator is pure for factor $j$ when its row of $\Lambda$ has a nonzero loading in column $j$ and zeros in every other factor column. A simple sufficient pattern is to have at least three pure indicators for each factor. The factors’ pure-indicator groups may be disjoint, and the pattern fixes which group belongs to which factor.

Theorem: In an orthogonal factor model with diagonal $\Psi$, suppose each factor has at least three pure indicators with nonzero loadings. Then, for generic parameter values, the loading magnitudes and specific variances of those indicators are determined by $\Sigma$, up to factor signs. Any additional pure indicator for an already represented factor is also identified up to that same sign.

Proof. Take any three pure indicators $a,b,c$ for factor $j$, with loadings $\lambda_{aj},\lambda_{bj},\lambda_{cj}$. Their off-diagonal covariances are $\lambda_{aj}\lambda_{bj}$, $\lambda_{aj}\lambda_{cj}$, and $\lambda_{bj}\lambda_{cj}$: no other factor contributes, and the diagonal specific covariance contributes nothing off the diagonal. The preceding theorem therefore recovers their squared loadings and relative signs. For any further pure indicator $r$ for factor $j$, its covariance with $a$ is $\sigma_{ra}=\lambda_{rj}\lambda_{aj}$. Since $\lambda_{aj}\ne0$, this determines $\lambda_{rj}$ once the sign convention for factor $j$ is chosen. Each specific variance follows from the diagonal equation $\sigma_{rr}=\sum_k\lambda_{rk}^2+\psi_{rr}$. Thus the stated parameters are recovered. $\square$

Worked Example: Two Factors With Three Pure Indicators Each

Consider six standardized variables. Variables 1–3 load only on factor 1, with loadings $0.5,0.6,0.8$; variables 4–6 load only on factor 2, with loadings $0.4,0.5,0.7$. The factors are orthogonal. The off-diagonal covariances among variables 1–3 are $0.30,0.40,0.48$, since these are the pairwise loading products. Among variables 4–6 they are $0.20,0.28,0.35$. Covariances across the two groups are zero.

For the first group, the first squared loading is $(0.30)(0.40)/0.48=0.25$, so its magnitude is $0.5$. The second squared loading is $(0.30)(0.48)/0.40=0.36$, and the third is $(0.40)(0.48)/0.30=0.64$. For the second group, the corresponding calculations are $(0.20)(0.28)/0.35=0.16$, $(0.20)(0.35)/0.28=0.25$, and $(0.28)(0.35)/0.20=0.49$. The magnitudes are therefore $0.4,0.5,0.7$.

Because each observed variance is 1, the specific variances are $0.75,0.64,0.36,0.84,0.75,0.51$, obtained by subtracting the respective squared loadings from 1. These are all positive. The two factor signs can be chosen independently, and within each group the pairwise covariances fix the relative signs. The pattern thus supplies enough links to recover the loadings and specific variances.

Why Parameter Counting Can Mislead

Parameter counting compares the number of free parameters with the number of distinct observed covariance entries, $p(p+1)/2$. It is a useful first screen, but it says nothing by itself about whether different parameters produce different covariance matrices. In an unrestricted $p\times m$ loading matrix, continuous orthogonal rotations also represent the same covariance matrix, so a count must account for that redundancy. With a restricted loading pattern, one must separately check whether rotations preserve the restrictions; it is not generally correct to subtract the unrestricted rotation count mechanically.

Even a count that properly reflects the stated restrictions is not a proof of identification. If some free parameters affect the covariance only through a product or other shared quantity, the model can have fewer distinct covariance directions than its parameter count suggests.

Worked Example: A Favorable Count but Nonidentified Loadings

Take four variables and two orthogonal factors with this loading pattern: variables 1 and 2 load only on factor 1, while variables 3 and 4 load only on factor 2. There are four free loadings and four specific variances, for eight parameters. A $4\times4$ symmetric covariance matrix has $4(5)/2=10$ distinct entries. The count $8<10$ may appear favorable.

However, the covariance between variables 1 and 2 is only $\lambda_{11}\lambda_{21}$. Set the observed variances to 1, the covariance between variables 1 and 2 to $0.30$, and the covariance between variables 3 and 4 to $0.40$, with all cross-group covariances zero. One valid parameter choice is $(\lambda_{11},\lambda_{21})=(0.50,0.60)$ and $(\lambda_{32},\lambda_{42})=(0.80,0.50)$. The corresponding specific variances are $0.75,0.64,0.36,0.75$.

A different choice is $(\lambda_{11},\lambda_{21})=(0.75,0.40)$ and $(\lambda_{32},\lambda_{42})=(0.50,0.80)$. It gives the same products, $0.30$ and $0.40$, and hence the same off-diagonal covariances. Its specific variances are $1-0.75^2=0.4375$, $1-0.40^2=0.84$, $1-0.50^2=0.75$, and $1-0.80^2=0.36$, all positive. The diagonal variances remain 1, and the cross-group covariances remain zero. Thus two distinct parameter choices give exactly the same $\Sigma$.

In fact, each pair of loadings can vary continuously while keeping its product fixed, with the specific variances adjusted to preserve the diagonal. The parameter count did not reveal that only one product per pair was observed. Each factor has only two indicators, so there is no third covariance link to separate the two loadings.

Key takeaway: A parameter count is a dimension check, not a recovery argument. To assess a loading pattern, identify which off-diagonal covariances involve each loading, and ask whether those links can separate individual loadings rather than only their products or combinations.

Common Pitfall: Treating a Count as an Identification Proof

A common mistake is to argue, “There are eight parameters and ten covariance entries, so the model is identified.” The correction is that the count only compares totals. Some covariance entries may be fixed at zero by the pattern, while others may depend on several parameters through the same product. A count also cannot determine whether distinct parameter values produce the same covariance matrix.

The reverse mistake is also possible: counting all entries of an unrestricted loading matrix even when many are fixed at zero can make a structured model appear to have too many parameters. Count the parameters actually free under the specified pattern, handle any continuous equivalences that truly remain, and then inspect the covariance links. The pure-indicator argument above supplies a sufficient recovery proof in one useful setting; other patterns require their own analysis.

This pattern-based reasoning is valuable before fitting a model. If its loadings are not identified by the covariance structure, an optimization routine cannot make them scientifically recoverable merely by returning numerical estimates. A later local-identification analysis can formalize more complicated cases, but the indicator pattern already gives a practical first diagnosis.

Check Your Understanding

Use covariance links, not parameter counts alone, to answer these questions.

Why does a third nonzero indicator allow one-factor loading magnitudes to be recovered when two indicators alone do not?
In the two-factor pure-indicator theorem, why must each factor have at least three indicators for the stated recovery argument?
For the four-variable disjoint-pair model, which covariance quantity is identified within each pair, and why does it not identify both loadings separately?
What does “generic” allow us to exclude, and what does it not allow us to ignore?
Why is it unsafe to subtract the unrestricted factor-rotation redundancy automatically when counting parameters under a restricted loading pattern?