Joint and Individual Structure With JIVE

Why Separate Shared and Block-Specific Variation?

A set of matched omics blocks measures the same samples in different feature spaces. For example, each sample may have a gene-expression profile and a metabolite profile. Some sample-to-sample variation may appear in both blocks, while other variation may be evident in only one. JIVE, short for Joint and Individual Variation Explained, represents these patterns separately rather than requiring one low-rank summary to account for all of them.

To make the meaning of “row space” explicit, use the orientation in which each block has features in rows and matched samples in columns. Thus block \(k\) is a matrix \(X_k\) with \(p_k\) features and \(n\) samples. The row space of \(X_k\) is a subspace of \(\mathbb{R}^n\): it is spanned by the feature profiles across the samples. This orientation makes the row spaces directly describe patterns of sample variation.

Definition: For \(K\) matched data blocks, the JIVE decomposition writes each block as \(X_k=J_k+A_k+E_k\). Here \(J_k\) is the joint component, \(A_k\) is the individual (block-specific) component, and \(E_k\) is the residual. The joint components have row spaces contained in a shared, low-dimensional subspace of sample space. The row space of \(A_k\) captures sample patterns specific to block \(k\). The ranks of these matrices describe their dimensions.

The shared subspace is a statement about patterns across samples, not about having the same features in different blocks. The blocks may have entirely different numbers and types of features. A joint sample pattern can have different feature loadings, strengths, or signs in different blocks. In contrast, an individual component represents a sample pattern included for one block and not attributed to the shared component.

In a standard JIVE analysis, blocks must refer to the same samples in the same order. The matrices are usually centered feature by feature, and may be scaled or otherwise preprocessed so that units and feature variances do not give an unintended advantage to particular blocks. The decomposition is a low-rank approximation: it assumes that the main structured variation can be represented by a small number of joint and individual sample patterns. It does not require every residual entry to be independent noise.

How the Components Encode Sample Patterns

Suppose a shared sample space has dimension \(r_J\), with basis matrix \(Q_J\) of size \(n\) by \(r_J\). A joint component for block \(k\) can be written \(J_k=B_kQ_J^T\), where \(B_k\) gives the feature loadings on those shared patterns. For sample \(i\), its joint coordinates are the entries in row \(i\) of \(Q_J\). These coordinates are common across blocks, even though \(B_k\) differs from block to block.

Likewise, an individual component can be represented as \(A_k=C_kQ_k^T\), where the columns of \(Q_k\) describe sample patterns represented specifically for block \(k\). The matrix \(C_k\) translates those patterns into feature values in that block. Thus the component ranks have a direct interpretation: \(\operatorname{rank}(J)\), the rank of the vertically stacked joint matrices, is the dimension of the joint sample space, while \(\operatorname{rank}(A_k)\) is the dimension of the individual sample space for block \(k\).

$$ X_k = B_k Q_J^T + C_k Q_k^T + E_k $$

This factorization also clarifies what the decomposition does not say. A joint sample coordinate is not itself a causal explanation, and a block-specific component does not prove that the underlying biology is exclusive to that assay. The terms describe a structured statistical representation of the observed matrices under the chosen preprocessing and rank constraints.

Worked Example: Two Blocks With One Shared Pattern

Consider four matched samples and two blocks. Let the sample patterns be \(g=(1,-1,0,0)\), \(h_1=(1,1,-1,-1)\), and \(h_2=(0,0,1,-1)\). Define block 1 to have joint rows \(2g\) and \(-g\), and an individual row \(3h_1\) in its first feature. Define block 2 to have a joint row \(4g\) and an individual row \(2h_2\). There is no residual in this example.

The component matrices and their sums are

J_1=\begin{bmatrix}2&-2&0&0\\-1&1&0&0\end{bmatrix}, \quad A_1=\begin{bmatrix}3&3&-3&-3\\0&0&0&0\end{bmatrix}, \quad X_1=\begin{bmatrix}5&1&-3&-3\\-1&1&0&0\end{bmatrix}.

For block 2, \(J_2=[4,-4,0,0]\) and \(A_2=[0,0,2,-2]\), so \(X_2=[4,-4,2,-2]\). Adding the components entry by entry gives the displayed observed matrices. Every row sums to zero, as expected for these centered sample patterns.

The stacked joint matrix has rows that are multiples of \(g\), so its rank is 1 and the joint sample space is \(\operatorname{span}(g)\). The individual component in block 1 has rank 1 and row space \(\operatorname{span}(h_1)\); that in block 2 has rank 1 and row space \(\operatorname{span}(h_2)\). The same joint pattern occurs in both blocks, while the two additional patterns are assigned to their respective blocks. The row spaces here make the sample-pattern interpretation concrete.

Ranks, Approximation, and Identification

In practice, the matrices will not usually split into perfectly clean components. JIVE seeks low-rank joint and individual matrices whose sum approximates the observed blocks. A useful general statement of the target is to minimize the total squared residual size subject to selected rank restrictions and conditions that distinguish the joint and individual spaces:

\min_{\{J_k,A_k\}_{k=1}^{K}}\ \sum_{k=1}^{K}\left\|X_k-J_k-A_k\right\|_F^2, \qquad \operatorname{rank}(J)\leq r_J,\quad \operatorname{rank}(A_k)\leq r_k .

Here \(J\) denotes the vertical stack of the \(J_k\), \(\|\cdot\|_F^2\) is the sum of squared matrix entries, and \(r_J\) and \(r_k\) are the allowed joint and individual ranks. The displayed objective states the reconstruction target; a complete JIVE specification also needs restrictions that identify which low-rank variation is joint and which is individual. Without such restrictions, the same observed variation could be reassigned between components in multiple ways. For example, if a pattern is allowed to appear freely in both \(J_k\) and \(A_k\), their sum can remain unchanged even while the component labels change. The standard JIVE formulation imposes geometric constraints on these spaces; the decomposition should be interpreted together with those constraints, not as a unique split produced by low rank alone.

The rank bounds control the dimensions of the represented spaces, not the amount of variance explained by each feature. A joint rank of one means that all joint sample variation is represented by one sample pattern, although its feature loadings can differ substantially between blocks. An individual rank of two means that the block-specific part uses at most two sample patterns. Rank selection is a separate modeling decision: ranks that are too small leave structured signal in the residual, while ranks that are too large can absorb noise or unstable patterns.

Worked Example: A Joint Space With Two Sample Dimensions

Suppose three matched samples have two independent centered patterns, \(g_1=(1,-1,0)\) and \(g_2=(0,1,-1)\). Let

J_1=\begin{bmatrix}1&-1&0\\0&1&-1\end{bmatrix}, \qquad J_2=\begin{bmatrix}2&-2&0\\-1&2&-1\end{bmatrix}.

The first row of \(J_1\) is \(g_1\), and its second row is \(g_2\). The second row of \(J_2\) is \(-g_1+g_2=(-1,2,-1)\), while its first row is \(2g_1\). Both blocks therefore use the same two-dimensional joint sample space, \(\operatorname{span}(g_1,g_2)\). The stacked joint matrix has rank 2, not 1, because \(g_1\) and \(g_2\) are linearly independent. No individual component is needed for these matrices. This example shows why “joint” does not mean “one-dimensional” or “identical block values”: it means variation represented in a shared sample subspace.

Worked Example: Residual Size After a Proposed Split

For one centered feature in one block and three samples, suppose a proposed joint component is \(J=[2,-2,0]\), an individual component is \(A=[1,1,-2]\), and the observed row is \(X=[4,-1,-3]\). The reconstructed row is

J+A=[3,-1,-2], \qquad E=X-J-A=[1,0,-1].

The squared Frobenius norm of this one-row residual is the sum of its squared entries:

\|E\|_F^2=1^2+0^2+(-1)^2=2.

Thus the reconstruction error for this block is 2 in squared measurement units. Its residual norm is \(\sqrt{2}\), in the original measurement units. The residual entries sum to zero, consistent with the centered row, but that fact alone does not establish that the chosen joint and individual ranks are appropriate. Residual patterns should be assessed across the full data set and in relation to the modeling assumptions.

Reading the Decomposition and Checking Its Fit

A useful JIVE interpretation asks three distinct questions. First, what sample patterns recur across blocks, as represented by the joint space? Second, what additional patterns are represented within each block? Third, what remains in the residual after both structured parts are removed? A large joint component is not automatically a biologically important mechanism; it is a pattern shared under the selected preprocessing and model. Likewise, a large individual component may reflect meaningful block-specific variation, technical variation, or both.

Diagnostics should test whether the fitted decomposition is a plausible description of the data rather than simply report component scores. Inspect reconstruction residuals for remaining low-rank or patterned structure, and compare their scale across blocks. Check whether fitted sample patterns are driven by a small number of samples or features. Examine sensitivity to centering, scaling, and reasonable choices of ranks. If the decomposition changes markedly under small preprocessing changes, the joint/individual distinction is not robust. If residuals retain clear sample structure, the ranks or model may be inadequate; if the fitted components mostly describe isolated noise, the ranks may be excessive.

The interpretation also depends on sample matching. If columns are misaligned across blocks, the model can label unrelated sample patterns as joint. Missing observations, strong batch effects, and differences in measurement scale may also undermine the intended comparison unless they are handled appropriately. A JIVE decomposition does not, by itself, correct confounding, establish causation, or determine why a pattern is shared.

How JIVE Differs From Other Low-Rank Summaries

Separate PCA analyzes each block on its own. It can reveal within-block structure but does not explicitly identify a common sample space. Concatenated PCA stacks features from different blocks and finds a single low-rank summary of the combined matrix; it does not, by itself, split that summary into joint and block-specific pieces. MFA provides a framework for comparing blocks through a compromise representation and block contributions. JIVE instead models each block as the sum of shared, individual, and residual matrices. These approaches answer related but distinct questions, so the choice depends on whether the analysis requires an explicit joint/individual decomposition.

A common mistake is to call any pattern visible in two blocks “joint” and any pattern visible in only one plot “individual.” Visual resemblance alone does not define the JIVE components: the decomposition is constrained by the fitted row spaces and the chosen model. Another mistake is to treat the residual as proof that the data contain only random noise. The residual is whatever remains after the fitted components; it may include noise, unmodeled signal, preprocessing artifacts, or a poor rank choice.

Key takeaway: JIVE separates variation by its representation across matched blocks. The joint row space contains shared patterns of sample variation; each individual row space captures additional variation assigned to one block; the residual records what the low-rank structured components do not explain.

Check Your Understanding

Answer each question, then compare with the solutions below.

With features in rows and samples in columns, what space contains the row space of a block, and what does it represent?
If the stacked joint matrix has rank 2, how many independent sample patterns are represented jointly?
Why can low-rank constraints alone fail to define a unique joint/individual split?
For residual \(E=[2,-1]\), calculate the squared Frobenius norm and state its units relative to the measurements.
Name one distinction between JIVE and concatenated PCA.

Solutions. (1) The row space is a subspace of \(\mathbb{R}^n\), where \(n\) is the number of matched samples; it spans feature profiles across those samples and therefore describes sample patterns. (2) Two independent sample patterns. (3) A pattern can be reassigned between joint and individual matrices while keeping their sum unchanged unless restrictions distinguish the spaces. (4) \(\|E\|_F^2=2^2+(-1)^2=5\), in squared measurement units. (5) Concatenated PCA gives a low-rank summary of the stacked matrix, while JIVE explicitly separates joint and block-specific components as well as residuals.