From Selected Ranks to a Three-Block Decomposition
JIVE separates variation in several data blocks into a component with common sample scores, components specific to individual blocks, and residual variation not represented by those components. A small numerical construction is useful because it makes the distinction between these parts visible—and because it exposes an important caution: changing a selected rank changes the set of allowed fits, so the model generally needs to be refitted.
Suppose three omics blocks have been measured on the same \(n\) samples. Write \(X_k\) for the centered data matrix of block \(k\). A JIVE representation is \(X_k=J_k+A_k+E_k\), where \(J_k\) is the joint component, \(A_k\) is the individual component for block \(k\), and \(E_k\) is the residual. The concatenated joint matrix \(J=[J_1\ J_2\ J_3]\) has rank at most \(r\); each \(A_k\) has rank at most \(r_k\). In the convention used here, the joint score space and each block’s individual score space are orthogonal. A component’s rank bounds its number of independent score directions; it does not specify which directions a refitted model will choose.
JIVE fits are commonly judged by squared Frobenius error subject to rank and orthogonality constraints. Thus the error for a proposed decomposition is \(\sum_k\|X_k-J_k-A_k\|_F^2\). The rank constraints restrict which decompositions are allowed, but they do not require a refitted component to point in the same direction as before. The rank-selection material establishes how ranks may be chosen; the calculation below focuses on interpreting the resulting matrices and what rank changes do to them.
Worked Example: Checking a Selected Three-Block Fit
Worked Example: Checking a Selected Three-Block Fit
Consider a hypothetical construction with three blocks and three features per block. Let \(u,z,a_1,a_2,a_3\) be mutually orthogonal unit-length sample vectors. Let \(e_1,e_2,e_3\) be the unit vectors for the three features within a block. Define the data as \(X_k=\sqrt{8}\,u e_1^T+2a_k e_2^T+\sqrt{2}\,z e_3^T\), for \(k=1,2,3\). This defines concrete matrices through their sample-score and feature-loading directions; the construction is hypothetical, not a real study.
Take the selected joint rank to be \(r=1\) and the individual rank in each block to be \(r_k=1\). One valid decomposition is \(J_k=\sqrt{8}\,u e_1^T\), \(A_k=2a_k e_2^T\), and \(E_k=\sqrt{2}\,z e_3^T\). Each \(J_k\) and \(A_k\) has rank one. The concatenated joint matrix has rank one because every block shares score vector \(u\). In each block, \(u\) and \(a_k\) are orthogonal, as required for the joint and individual score directions.
A rank-one outer product \(c\,v w^T\) has squared Frobenius norm \(c^2\) when \(v\) and \(w\) are unit vectors. Therefore, each joint block has squared norm \(8\), each individual block has squared norm \(2^2=4\), and each residual block has squared norm \(2\). Because the score vectors are orthogonal, the three terms are pairwise orthogonal under the Frobenius inner product within each block. The total residual sum of squares for this decomposition is \(\sum_{k=1}^3\|E_k\|_F^2=3(2)=6\). The joint and individual sums of squared norms are \(3(8)=24\) and \(3(4)=12\), respectively; the total data sum of squares is consequently \(24+12+6=42\).
This example establishes the bookkeeping for a valid decomposition. It does not make the sum of squares of an omitted component a general prediction of the residual after changing ranks. That question depends on whether components are held fixed or refitted.
Why Rank Changes Require Refitting
For a fixed decomposition, pairwise Frobenius orthogonality makes the error change easy to calculate. If a component is omitted and all remaining matrices are held unchanged, its squared norm is added to the residual sum of squares, provided it is orthogonal to the previous residual. But a refitted model is allowed to change its remaining joint and individual components. The fixed-component sum is therefore not, in general, the residual sum of squares for the new rank-constrained fit.
Worked Example: Refitting with Joint Rank Zero
Return to the three-block construction and set the joint rank to zero while keeping each individual rank at one. First hold all original individual components fixed. The former joint terms are now omitted, with total squared norm \(3(8)=24\). The original residual has sum of squares \(6\), so this fixed-component reconstruction has residual sum of squares \(6+24=30\).
Now refit under joint rank zero. With no joint component, each block can use its individual rank-one term to approximate its own \(X_k\). The three feature columns of \(X_k\) are orthogonal: their score vectors are \(u\), \(a_k\), and \(z\), and their squared norms are \(8\), \(4\), and \(2\). Hence the singular values of \(X_k\) are \(\sqrt{8}\), \(2\), and \(\sqrt{2}\). The best rank-one approximation retains the largest singular direction, \(u e_1^T\), with squared norm \(8\). Its remaining squared error in that block is \(4+2=6\). Across three blocks, the refitted residual sum of squares is \(3(6)=18\), not \(30\).
For this calculation, the best rank-one approximation follows from the standard singular-value characterization: for a matrix with orthogonal singular directions, retaining the largest singular value minimizes the squared Frobenius error among rank-one approximations. Here the three squared singular values are \(8,4,2\), so retaining the first leaves \(4+2=6\). With joint rank zero, the blocks can be optimized separately; summing their three minimum errors gives the minimum total error, \(18\).
The calculation shows why merely adding \(24\) answers a different question. In the refit, each block’s individual component can take the former joint score direction \(u\). The rank constraint permits a new component allocation; it does not freeze the original individual terms.
A Second Joint Direction Can Absorb Shared Residual Structure
A residual is the part left over by a particular fitted decomposition. It need not be unrelated across blocks. If residuals share a score direction, increasing the allowed joint rank can make that direction eligible for the joint component. Whether it is assigned there depends on the refitted model and its constraints.
Worked Example: Refitting with Joint Rank Two
In the construction, each residual is \(E_k=\sqrt{2}\,z e_3^T\). The score vector \(z\) is shared across all three blocks. If the joint rank is increased from one to two, define \(J_k^*=\sqrt{8}\,u e_1^T+\sqrt{2}\,z e_3^T\), retain \(A_k^*=2a_k e_2^T\), and set \(E_k^*=0\).
The concatenated joint matrix now has score directions \(u\) and \(z\), which are orthogonal, so its rank is at most two. Each individual score \(a_k\) remains orthogonal to both joint directions. Moreover, \(J_k^*+A_k^*=X_k\) for every block, so \(\sum_k\|E_k^*\|_F^2=0\). This is an exact rank-two representation of the constructed data.
Thus the original residual sum of squares, \(6\), is not a lower bound for a refitted rank-two model. The extra joint direction can absorb the shared-score residual structure, giving zero error here. This is a possibility demonstrated by the construction, not a general claim that increasing joint rank always eliminates residual error.
Rank changes can also affect individual components. For instance, in the original decomposition, setting block 2’s individual rank to zero while holding every other matrix fixed omits a component of squared norm \(4\). The fixed-component residual sum of squares becomes \(6+4=10\). That number does not determine the residual of a refitted model with \(r_2=0\): the joint component and the other blocks’ individual components may change. The new rank-constrained fit must be computed rather than inferred by adding \(4\).
Derivation: The Fixed-Component Error Calculation
The fixed-component rule follows directly from the Frobenius inner product \(\langle B,C\rangle_F=\sum_{i,j}B_{ij}C_{ij}\), which gives \(\|B+C\|_F^2=\|B\|_F^2+\|C\|_F^2+2\langle B,C\rangle_F\). If an omitted component \(D_k\) is orthogonal to the existing residual \(E_k\), then
Summing this equality over blocks gives the fixed-component increase \(\sum_k\|D_k\|_F^2\). The equality depends on both conditions: the other components are fixed, and the omitted term is orthogonal to the existing residual. Even when this calculation is valid, it describes only that reconstruction. Refitting searches a different set of component matrices, so the displayed equality alone cannot establish the new fit’s minimum error.
Checking a Decomposition and Comparing Alternatives
A useful audit begins with the matrices themselves rather than a verbal interpretation. Confirm that \(J_k+A_k+E_k=X_k\) for every block; check that \(\operatorname{rank}(J)\le r\) and \(\operatorname{rank}(A_k)\le r_k\); and verify the required score-space orthogonality. Then calculate \(\|E_k\|_F^2\) block by block and add the values. If a claimed shared direction appears in residuals across several blocks, note that it could potentially be represented by an added joint direction, subject to the model’s other constraints.
Keep the question being answered explicit. A fixed-component audit asks what error results if a term is removed and everything else is left alone. A refitted rank comparison asks for the minimum error after optimizing under new ranks. These are different calculations. MFA and separate or concatenated PCA provide different ways to summarize block structure; they should not be treated as substitutes for the JIVE decomposition into joint, individual, and residual matrices.
The component allocation is also a modeling result, not a causal claim. A score direction shared across blocks describes common variation in the observed matrices; it does not by itself establish a shared biological mechanism. Likewise, a residual may contain structure omitted by the chosen ranks rather than only measurement noise. The decomposition is limited by the selected ranks, the preprocessing and scaling of the blocks, and the orthogonality constraints.
- Check reconstruction: component matrices should sum to the observed block, up to rounding.
- Check rank and orthogonality: inspect the joint rank, each individual rank, and the required score-space relationships.
- Check residual accounting: compute each block’s squared residual norm before summing.
- Recompute after rank changes: do not use an omitted component’s energy as the refitted error without an explicit fixed-component condition.
A common mistake is to say, “Removing a component of squared norm \(24\) makes the residual sum of squares increase by \(24\),” and then present the result as the refitted least-squares error. That is correct only for a fixed-component calculation under the required orthogonality condition. In the rank-zero example, freezing the other terms gave \(30\), whereas refitting gave \(18\). The correction is to label the fixed calculation and optimize again for the new ranks.
Check Your Understanding
Answer each question using the distinction between a fixed-component calculation and a refitted model.
- In the constructed example, what is the total squared norm of the three individual components?
- If the joint component is removed while all other components are held fixed, what is the resulting residual sum of squares?
- With joint rank zero and individual rank one in each block, why does the refitted residual sum of squares equal \(18\)?
- What representation gives zero residual when the joint rank is two?
Solutions. (1) Each individual component has squared norm \(4\), so the total is \(3(4)=12\). (2) The original residual sum of squares is \(6\), and the fixed joint component has squared norm \(24\), giving \(6+24=30\). (3) Each block’s squared singular values are \(8,4,2\); its best rank-one approximation retains \(8\) and leaves \(4+2=6\). Three blocks therefore leave \(3(6)=18\). (4) Use joint score directions \(u\) and \(z\), with the corresponding loadings \(\sqrt{8}e_1^T\) and \(\sqrt{2}e_3^T\) in each block, retain \(2a_k e_2^T\) as individual structure, and set all residuals to zero.