Why First-Eigenvalue Scaling Limits Block Dominance

Why Bound a Block’s Leading Influence?

In a joint PCA, variables from several matched blocks are placed side by side, so the blocks compete to shape the global axes. A block with a large first principal inertia can dominate those axes even when its size is not the main scientific reason to give it influence. First-eigenvalue scaling addresses this specific imbalance: it limits how much any one block can contribute along a unit direction in the space of individuals.

The limit is useful, but it is easy to overstate what it accomplishes. It does not make blocks equally important on every axis, nor does it make their total inertia equal. The result is a bound on each block’s largest possible directional contribution; other properties of the block, including its remaining eigenvalues, still matter.

Setup and the Per-Direction Bound

Suppose $K$ blocks measure the same $n$ individuals. For block $k$, let $X_k$ be its $n$-by-$p_k$ data matrix after the chosen centering and any required variable preprocessing. Assume the blocks are matched row by row and that the same individuals and row weighting are used in every block. Define the first principal inertia $\lambda_{1,k}$ as the largest eigenvalue of $X_k^{T}X_k$. Equivalently, it is the largest eigenvalue of $X_kX_k^{T}$, apart from any additional zero eigenvalues.

For now assume $\lambda_{1,k}>0$ for every block. First-eigenvalue scaling divides every entry of $X_k$ by $\sqrt{\lambda_{1,k}}$. Write the resulting block as $Z_k$, and place the scaled blocks side by side to form $Z=[Z_1,\ldots,Z_K]$. The square root is essential: PCA is based on squared lengths, so dividing the data by $\sqrt{\lambda_{1,k}}$ divides its eigenvalues by $\lambda_{1,k}$.

Definition: The scaled block is $Z_k=X_k/\sqrt{\lambda_{1,k}}$. Its first principal inertia is the largest eigenvalue of $Z_k^{T}Z_k$, which equals 1. For a unit vector $u$ in the space of individuals, the block’s directional contribution is $u^{T}Z_kZ_k^{T}u=\|Z_k^{T}u\|^2$.

The phrase “directional contribution” here refers to the squared projection of the unit individual-space direction $u$ onto the block’s variables. The key fact follows directly from the Rayleigh-quotient characterization of the largest eigenvalue.

Theorem: For every block $k$ and every unit vector $u\in\mathbb{R}^n$, first-eigenvalue scaling gives $$ 0\leq u^{T}Z_kZ_k^{T}u=\|Z_k^{T}u\|^2\leq 1. $$ Thus no scaled block can contribute more than 1 along any unit direction. The bound is attained in a direction associated with the block’s first principal inertia.

To justify the inequality, $Z_kZ_k^{T}$ is positive semidefinite and has largest eigenvalue 1. For any unit $u$, its Rayleigh quotient is no greater than that largest eigenvalue. More generally, if $u$ is not a unit vector, the corresponding bound is $u^{T}Z_kZ_k^{T}u\leq\|u\|^2$.

u^{T}Z_kZ_k^{T}u \leq \lambda_{\max}(Z_kZ_k^{T})\|u\|^2 = \|u\|^2.

This also implies a bound on the largest global eigenvalue. The individual-space matrix used by joint PCA is $ZZ^{T}=\sum_{k=1}^{K}Z_kZ_k^{T}$. For any unit $u$, each summand’s quadratic form is at most 1, so the sum is at most $K$. Taking the maximum over unit vectors gives $\lambda_{\max}(ZZ^{T})\leq K$. This is an overall ceiling, not a claim that the global first eigenvalue must equal $K$.

Worked Examples

Worked Example: Two Blocks With Different Scales and Dimensions

Consider hypothetical centered blocks represented using two orthonormal centered score vectors $e_1$ and $e_2$, so $e_i^{T}e_j=1$ when $i=j$ and 0 otherwise. Block A has two columns, $3e_1$ and $e_2$, while block B has four columns, $4e_1$, $2e_2$, and two zero columns. The zero columns simply make B wider; they do not add inertia.

The blocks’ cross-product eigenvalues are the squared column lengths in these orthogonal constructions. Therefore, block A has eigenvalues 9 and 1, giving $\lambda_{1,A}=9$; block B has eigenvalues 16 and 4, giving $\lambda_{1,B}=16$. The scaling divisors are $\sqrt{9}=3$ and $\sqrt{16}=4$.

After scaling, A’s columns are $e_1$ and $e_2/3$, and B’s nonzero columns are $e_1$ and $e_2/2$. Their individual-space matrices are $$ Z_AZ_A^{T}=e_1e_1^{T}+\frac{1}{9}e_2e_2^{T}, \qquad Z_BZ_B^{T}=e_1e_1^{T}+\frac{1}{4}e_2e_2^{T}. $$ For the unit direction $u=e_1$, the contributions are $e_1^{T}Z_AZ_A^{T}e_1=1$ and $e_1^{T}Z_BZ_B^{T}e_1=1$. For $u=e_2$, they are $1/9$ and $1/4$, both below 1.

The combined matrix has eigenvalue 2 in direction $e_1$, since both blocks contribute 1 there. In direction $e_2$, its eigenvalue is $1/9+1/4=13/36$, approximately 0.3611. The global leading direction is therefore $e_1$, with each block at the bound. Despite the different raw scales and different numbers of columns, neither block can exceed 1 along any unit direction after scaling.

Worked Example: The Bound Does Not Equalize Total Inertia

Consider two hypothetical scaled blocks with these eigenvalues: block C has 2 nonzero eigenvalues, 1 and 0.2; block D has 10 nonzero eigenvalues, 1 and nine values of 0.7. Both have first eigenvalue 1, as required by first-eigenvalue scaling.

The total inertia of a block is the sum of its eigenvalues. For C it is $1+0.2=1.2$. For D it is $1+9(0.7)=7.3$. Each block satisfies the directional bound, but their total inertias differ substantially. In fact, the trace of the scaled block matrix equals this sum of eigenvalues, so the larger collection of substantial secondary eigenvalues gives D much greater total inertia.

There is no contradiction: the theorem limits the largest eigenvalue, not the sum of all eigenvalues. It controls how strongly a block can act in one direction at a time. It does not erase the cumulative effect of many other directions.

Worked Example: Removing a Raw Scale Imbalance

Suppose two hypothetical blocks share orthogonal score directions $e_1,e_2$. Their unscaled individual-space matrices are $X_AX_A^{T}=100e_1e_1^{T}+e_2e_2^{T}$ and $X_BX_B^{T}=4e_1e_1^{T}+4e_2e_2^{T}$. Their first eigenvalues are 100 and 4, so block A has a much larger raw leading inertia.

Without scaling, the combined matrix is $104e_1e_1^{T}+5e_2e_2^{T}$. Its leading eigenvalue is 104 along $e_1$, and block A supplies $100/104\approx0.9615$ of that eigenvalue. This shows how a raw leading scale can strongly shape the joint axis.

After first-eigenvalue scaling, divide A by $\sqrt{100}=10$ and B by $\sqrt{4}=2$. The scaled individual-space matrices become $e_1e_1^{T}+0.01e_2e_2^{T}$ and $e_1e_1^{T}+e_2e_2^{T}$. Their sum has eigenvalues 2 along $e_1$ and 1.01 along $e_2$. Along the leading direction, each block contributes 1. The raw scale imbalance is removed in that direction, but the blocks are not identical: along $e_2$, A contributes only 0.01 while B contributes 1.

Assumptions, Diagnostics, and Failure Modes

The bound is a linear-algebra result; it does not require normally distributed data, independent variables within a block, or a specific biological interpretation. It does rely on using the stated scaling consistently and defining each $\lambda_{1,k}$ from the same processed block that is then scaled. The common row space is needed to combine blocks in a joint analysis, while centering and variable preprocessing determine which structure the eigenvalues describe.

Several checks help establish that the bound is being applied as intended:

Compute the largest eigenvalue of each scaled block’s cross-product matrix. It should be 1, up to numerical rounding.
Check that all blocks contain the same individuals in the same row order. Otherwise, their axes do not refer to the same individual space.
Verify that $\lambda_{1,k}>0$. A block with all-zero variation has no usable first eigenvalue and cannot be scaled by this rule.
Inspect preprocessing and extreme observations. The algebraic bound still holds after scaling, but an outlier or unsuitable preprocessing can make the resulting axes scientifically misleading.

A frequent error is to divide each data block by $\lambda_{1,k}$ rather than by $\sqrt{\lambda_{1,k}}$. That changes the scaled first eigenvalue to $1/\lambda_{1,k}$, not 1, because eigenvalues scale with the square of a data-matrix multiplier. Another error is to conclude that all scaled blocks have equal total influence. The examples show why that conclusion does not follow: the theorem bounds each block’s contribution in any one unit direction, while total inertia sums contributions across directions.

How This Choice Compares With Alternatives

Leaving blocks unscaled preserves their original processed magnitudes. This may be appropriate when those magnitudes are directly comparable and scientifically meaningful, but a block with a much larger first eigenvalue can then dominate a leading joint direction. First-eigenvalue scaling specifically caps that leading directional influence.

An alternative is to scale blocks to have equal total inertia, using their traces rather than their first eigenvalues. That targets the sum of eigenvalues. It does not, in general, guarantee that the largest eigenvalue of every scaled block is 1: a block whose inertia is concentrated in one direction can still have a comparatively large leading eigenvalue. Conversely, first-eigenvalue scaling gives the per-direction bound but leaves total inertia dependent on the remaining eigenvalues.

Scaling each variable to unit variance is another distinct choice. It adjusts the variables individually before a joint analysis, but it does not by itself impose the block-level bound unless the resulting block is also scaled using its first eigenvalue. These choices answer different questions; the appropriate one depends on what block-level comparability is intended to mean.

Key takeaway: First-eigenvalue scaling makes the largest eigenvalue of every block equal to 1. Consequently, every scaled block contributes at most 1 along any unit individual-space direction. This controls leading directional dominance, not equality of total inertia or of all block contributions.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions below.

A block has first eigenvalue 25. What divisor is used to scale its data matrix, and what is its scaled first eigenvalue?
For a scaled block with eigenvalues 1, 0.6, and 0.2, what is its total inertia? Does the directional bound fail?
There are four scaled blocks. What upper bound follows for the largest eigenvalue of their combined individual-space matrix?
Why does dividing a data matrix by its first eigenvalue generally fail to make its scaled first eigenvalue equal to 1?

Solutions. (1) Divide by $\sqrt{25}=5$; eigenvalues are divided by 25, so the scaled first eigenvalue is 1. (2) Total inertia is $1+0.6+0.2=1.8$. The bound does not fail: it concerns the largest eigenvalue, which is 1, not the sum. (3) The upper bound is 4, since each of the four block quadratic forms is at most 1 for any unit direction. (4) Dividing the data matrix by $\lambda_{1}$ divides its cross-product eigenvalues by $\lambda_{1}^{2}$, so the first becomes $1/\lambda_{1}$, rather than 1, unless $\lambda_{1}=1$.