Heywood Cases and Boundary Estimates

Why Heywood Cases Matter

In a common-factor model, each observed variable's variance is divided into common variance and unique variance. A fitted solution that assigns an observed variable more common variance than its total variance cannot be interpreted as an admissible model with nonnegative unique variance. This is the central issue in a Heywood case.

The term is used in two related ways. A uniqueness estimate equal to zero is a boundary estimate: it satisfies nonnegativity but says that the fitted model leaves no unique variance for that variable. A negative uniqueness estimate is improper under the usual common-factor model constraints. Both deserve attention, but they are not the same outcome. The distinction matters when comparing an unconstrained calculation with a fit that explicitly enforces nonnegative uniquenesses.

Definition: In the common-factor covariance model $\Sigma=\Lambda\Phi\Lambda^{\mathsf T}+\Psi$, the communality of variable $i$ is $h_i^2=(\Lambda\Phi\Lambda^{\mathsf T})_{ii}$, and its uniqueness is $\psi_i=\Psi_{ii}$. A Heywood case is commonly used for an estimated uniqueness at or below zero, $\hat\psi_i\leq 0$. If $\hat\psi_i=0$, the estimate is on the admissible boundary; if $\hat\psi_i<0$, it violates the usual constraint $\psi_i\geq0$.

For standardized variables with uncorrelated, unit-variance factors, $\Phi=I$, and the diagonal decomposition is $1=h_i^2+\psi_i$. Thus a negative uniqueness is equivalent to an estimated communality greater than one. For variables measured on other scales, the corresponding comparison is with their variance: $ \operatorname{Var}(X_i)=h_i^2+\psi_i$. The threshold is not automatically one unless the variables are standardized.

What the Nonnegative-Uniqueness Constraint Implies

The model constraint has a direct covariance interpretation. Suppose $\Phi$ is positive semidefinite and $\Psi$ is diagonal with nonnegative entries. Then the model covariance is positive semidefinite, and each variable's common variance cannot exceed its total variance.

Result: If $\Sigma=\Lambda\Phi\Lambda^{\mathsf T}+\Psi$, $\Phi$ is positive semidefinite, and $\psi_i\geq0$ for every $i$, then $\Sigma$ is positive semidefinite and $(\Lambda\Phi\Lambda^{\mathsf T})_{ii}\leq\Sigma_{ii}$ for every $i$.

Proof. For any vector $a$, the quadratic form of the common part is $a^{\mathsf T}\Lambda\Phi\Lambda^{\mathsf T}a=(\Lambda^{\mathsf T}a)^{\mathsf T}\Phi(\Lambda^{\mathsf T}a)\geq0$, because $\Phi$ is positive semidefinite. Also $a^{\mathsf T}\Psi a=\sum_i\psi_i a_i^2\geq0$. Adding these expressions gives $a^{\mathsf T}\Sigma a\geq0$, so $\Sigma$ is positive semidefinite. On the diagonal, $\Sigma_{ii}=(\Lambda\Phi\Lambda^{\mathsf T})_{ii}+\psi_i$, and $\psi_i\geq0$ gives the claimed inequality. $\square$

The converse needs care. A covariance matrix can be positive semidefinite even when a particular decomposition of it uses a negative diagonal entry in $\Psi$. Therefore, the fact that observed or sample correlations form a valid correlation matrix does not guarantee that a chosen factor extraction has nonnegative uniqueness estimates. The issue is the fitted decomposition, not necessarily the validity of the whole matrix.

A Three-Variable One-Factor Diagnostic

A simple algebraic result makes the problem visible without fitting a large model. Consider three standardized variables in a one-factor model with factor variance one, loadings $\lambda_1,\lambda_2,\lambda_3$, and diagonal uniquenesses. The off-diagonal correlations satisfy $r_{ij}=\lambda_i\lambda_j$. If all three correlations are nonzero, these equations determine the squared loadings.

Result: If $r_{12}=\lambda_1\lambda_2$, $r_{13}=\lambda_1\lambda_3$, and $r_{23}=\lambda_2\lambda_3$, with all three correlations nonzero, then $$ \lambda_1^2=\frac{r_{12}r_{13}}{r_{23}},\qquad \lambda_2^2=\frac{r_{12}r_{23}}{r_{13}},\qquad \lambda_3^2=\frac{r_{13}r_{23}}{r_{12}}. $$ For standardized variables, the corresponding uniqueness is $\psi_i=1-\lambda_i^2$. A Heywood case occurs whenever one of these squared loadings exceeds one; equality gives a zero-uniqueness boundary estimate.

Derivation. Multiplying the first two off-diagonal equations gives $r_{12}r_{13}=\lambda_1^2\lambda_2\lambda_3$. Since $r_{23}=\lambda_2\lambda_3\ne0$, division yields $\lambda_1^2=r_{12}r_{13}/r_{23}$. The other two formulas follow by multiplying the pairs of equations that contain the corresponding loading and dividing by the remaining nonzero correlation. Finally, the standardized diagonal equation is $1=\lambda_i^2+\psi_i$, so $\psi_i=1-\lambda_i^2$. The sign of a loading is not determined by its square alone; a consistent set of loading signs must also reproduce the correlations. $\square$

Worked Example: An Unconstrained One-Factor Solution

Suppose a standardized three-variable correlation matrix has $r_{12}=0.8$, $r_{13}=0.8$, and $r_{23}=0.3$. First check that these entries can belong to a valid correlation matrix. Its determinant is $1+2(0.8)(0.8)(0.3)-0.8^2-0.8^2-0.3^2=0.014$, which is positive; the principal $2\times2$ minors are also positive. Thus this example does not rely on an invalid correlation matrix.

The one-factor formulas give $\lambda_1^2=(0.8)(0.8)/0.3=2.1333\ldots$, while $\lambda_2^2=(0.8)(0.3)/0.8=0.3$ and $\lambda_3^2=(0.8)(0.3)/0.8=0.3$. Taking all loadings positive reproduces the positive correlations: approximately $(1.4606,0.5477,0.5477)$.

The resulting uniqueness estimates are $\hat\psi_1=1-2.1333\ldots=-1.1333\ldots$, and $\hat\psi_2=\hat\psi_3=1-0.3=0.7$. The off-diagonal correlations are reproduced exactly, but the first uniqueness is negative. This is an unconstrained algebraic solution, not an admissible common-factor covariance decomposition with nonnegative uniquenesses.

The example also shows why a valid input correlation matrix is not a sufficient safeguard: its determinant is positive, yet the exact one-factor off-diagonal solution is improper. Here the one-factor restrictions on the off-diagonal entries, together with the standardized diagonal, demand more than the available total variance for variable 1.

What a Constrained Boundary Fit Changes

A constrained procedure imposes $\psi_i\geq0$, or equivalently $\lambda_i^2\leq1$ in a standardized one-factor model. If the unconstrained solution violates this restriction, a constrained fit cannot retain every feature of that solution. It must alter loadings, accept residual discrepancies, or both. The exact outcome depends on the fitting criterion and any additional restrictions; merely replacing a negative estimate by zero does not, by itself, establish that the remaining parameters have been refitted optimally.

Worked Example: A Symmetric Constrained Boundary Fit

Use the correlations from the previous calculation, but now consider a deliberately restricted constrained fit: set $\lambda_1=1$ at its admissible boundary and require $\lambda_2=\lambda_3=x$, with $0\leq x\leq1$. Fit the three off-diagonal correlations by minimizing the sum of squared residuals, counting each pair once: $Q(x)=2(0.8-x)^2+(0.3-x^2)^2$.

Differentiating gives $Q'(x)=-4(0.8-x)-4x(0.3-x^2)=4(x^3+0.7x-0.8)$. The derivative of $x^3+0.7x-0.8$ is $3x^2+0.7>0$, so there is at most one root. The function is negative at $x=0$ and positive at $x=1$, so it has exactly one root in the allowed interval. Numerically, that root is $x\approx0.6846$, giving the constrained boundary loadings $(1,0.6846,0.6846)$.

The corresponding uniquenesses are $\psi_1=1-1^2=0$ and $\psi_2=\psi_3=1-(0.6846)^2\approx0.5313$, all nonnegative. The fitted correlations are approximately $0.6846$, $0.6846$, and $0.6846^2\approx0.4687$. The residuals relative to $(0.8,0.8,0.3)$ are approximately $(-0.1154,-0.1154,0.1687)$, and $Q\approx2(0.1154)^2+(0.1687)^2\approx0.0551$.

This calculation is a constrained fit within the stated boundary-and-symmetry submodel, not a claim that it is the global constrained optimum over every possible loading vector. It demonstrates the trade-off: nonnegative uniquenesses are respected, but the off-diagonal correlations are no longer reproduced exactly.

Heywood Cases with More Than One Factor

With $q$ uncorrelated unit-variance factors, variable $i$'s communality is the sum of its squared loadings, $h_i^2=\sum_{j=1}^{q}\lambda_{ij}^2$. A row of loadings whose squared length exceeds one therefore implies a negative uniqueness for a standardized variable. With correlated factors, the communality is instead $\lambda_i^{\mathsf T}\Phi\lambda_i$; it is the common-variance quadratic form, not simply the sum of squared loadings, that must be compared with the total variance.

Worked Example: A Two-Factor Row Beyond the Boundary

For a standardized variable with two uncorrelated unit-variance factors, suppose an unconstrained procedure estimates its loading row as $(0.9,0.6)$. Its estimated communality is $0.9^2+0.6^2=0.81+0.36=1.17$, so the implied uniqueness is $1-1.17=-0.17$. This is an improper estimate.

For a transparent constrained adjustment, find the closest loading row to $(0.9,0.6)$ in ordinary Euclidean distance subject to squared length at most one. The original row has length $\sqrt{1.17}$. The closest point in the unit disk lies in the same direction, scaled to length one, so the adjusted row is $(0.9,0.6)/\sqrt{1.17}\approx(0.8321,0.5547)$. Its squared length is exactly one, and its implied uniqueness is zero. This is a boundary adjustment under the stated distance criterion; it illustrates the constraint rather than prescribing a universal factor-fitting procedure.

Why Estimates Become Improper

A Heywood estimate is a diagnostic, not a single diagnosis of what went wrong. Common sources include sampling variation, a model that does not adequately represent the population covariance, an unsuitable number of factors, and weak identification or nearly redundant loading patterns. In each case, the extraction can try to reproduce the observed covariance pattern by assigning an unusually large share of variance to the common part.

The distinction between constrained and unconstrained extraction is essential. An unconstrained algebraic method or optimizer may return a negative uniqueness because it is permitted to do so, or because intermediate equations are solved without imposing the variance constraint. A constrained method rules out negative values, but a boundary estimate at zero can still signal that the fit is pressed against the edge of the admissible parameter space. It is not automatically evidence that the population uniqueness is exactly zero.

Check the scale.
For standardized variables compare the communality with one; on another scale compare it with the variable's total variance.

Check the model and the estimate.
Identify whether the reported result is unconstrained, a constrained boundary solution, or a post hoc truncation.

Inspect the fit and specification.
Consider whether the factor structure, factor count, data quality, or identification may be contributing to the result.

Common Mistake

A tempting but incorrect response to $\hat\psi_i<0$ is: “The uniqueness is a variance, so replace it by zero; the model is now valid and the fit is unchanged.” Setting the entry to zero while leaving all other parameters fixed changes the implied diagonal of the covariance matrix. Refitting the other parameters may restore a valid model, but generally changes fitted covariances and residuals. The constrained example above showed this explicitly: nonnegative uniquenesses came with nonzero off-diagonal residuals.

The correction is to report what was actually done. A zero estimate obtained under a nonnegativity constraint is a boundary estimate; a negative estimate from an unconstrained calculation is an improper solution. If a negative value is truncated or the model is refitted under constraints, that operation must be distinguished from the original estimate, and its effect on fit must be assessed.

Key takeaway: A Heywood case identifies a conflict between an estimated common variance and the available total variance. Zero uniqueness lies on the admissible boundary; negative uniqueness violates the usual model constraint. Constraining uniquenesses can remove improper values, but it may change the fitted covariance structure.

Check Your Understanding

Use the definitions and calculations above to answer these questions.

For a standardized variable with estimated communality $1.08$, what is its implied uniqueness, and is it admissible under the usual constraint?
Why does a positive determinant for an observed correlation matrix not rule out a Heywood case in a fitted factor decomposition?
In the three-variable one-factor model, what condition on $r_{12}r_{13}/r_{23}$ gives a zero uniqueness for variable 1?
Why is replacing a negative uniqueness by zero while holding all loadings fixed not generally an equivalent refit?
For correlated factors, which quantity should be compared with the total variance to diagnose an excessive communality?