Selecting Joint and Individual Ranks in JIVE

Why Rank Selection Matters in JIVE

JIVE represents several data blocks measured on the same \(n\) subjects by separating structure shared across blocks from structure specific to each block. For block \(k\), write \(X_k=J_k+A_k+E_k\), where \(J_k\) is its joint component, \(A_k\) is its individual component, and \(E_k\) is residual variation. The joint rank \(r\) is the number of shared score directions; the individual rank \(r_k\) is the number of block-specific score directions in block \(k\).

Rank selection is not just a question of how much variation to retain. If \(r\) is too small, a shared pattern may be left in the residuals or represented as apparently block-specific. If \(r\) is too large, variation that is not convincingly shared can be absorbed into \(J_k\). Likewise, a large \(r_k\) can make noise look like block-specific signal. Permutation procedures and information criteria offer different ways to balance fit against complexity.

The procedures below assume blocks have the same subjects in the same order, have been put on suitable scales, and have been centered across subjects. Candidate fits use the same preprocessing and JIVE constraints. Centering and scaling choices matter: changing them can change which directions appear important and therefore can change selected ranks.

Permutation-Based Selection

A permutation method compares a component statistic from the observed data with values generated under a specified null arrangement. A common statistic is a singular value of a fitted residual matrix or a measure of the reduction in residual sum of squares (RSS) when a candidate component is added. The null must match the question: for example, whether there is additional shared structure after accounting for the current fit, or whether a block has additional coordinated structure after the joint part has been accounted for.

One possible sequential procedure is to fit candidate ranks, calculate the statistic for the next component, and generate a null distribution by permuting residuals in a way that destroys the structure under test while preserving features the null is meant to retain. To test shared structure, a construction may independently permute subject labels between blocks, breaking cross-block alignment while retaining each block's internal observations. To test additional within-block structure, a construction may independently permute subject labels feature by feature in the residual block, breaking coordinated subject patterns. These are not interchangeable operations: the null hypothesis and the dependence structure determine which, if either, is defensible.

Permutation validity requires exchangeability under the null. For example, feature-wise permutations are generally unsuitable when the null errors have important dependence across features that the permutations destroy. Shared testing also requires care if subjects have clusters, repeated measures, or other restrictions on which labels can be exchanged. Permute only within exchangeable units or use a null-generating procedure that respects the design. A permutation p-value measures how unusual the observed statistic is under that specified null; it does not establish that a component is biologically meaningful.

Sequential testing should be planned rather than treated as a collection of unrelated tests. Testing many candidate components can increase the chance of retaining at least one component by chance. A stopping rule, a multiple-testing adjustment, or a permutation procedure that calibrates the maximum statistic can address this, depending on the analysis goal. Permutation choices can also affect the result: a null that destroys too much dependence may yield an unrealistically narrow reference distribution, while one that preserves the signal under test may fail to test the intended hypothesis.

Information-Based Selection and a Centered-Data BIC

An information criterion compares model fit with model complexity. For Gaussian errors with a common variance, a useful BIC form is an RSS term plus a penalty for the number of fitted parameters. In centered JIVE data, however, the entries are not independent if one simply assumes independent errors after subtracting each feature's sample mean. The likelihood must respect the centering operation.

To see this, let \(Z_k\) denote the uncentered observations, and suppose that before centering their feature errors are independent \(N(0,\sigma^2)\), with a common variance. Write \(Z_k=\mathbf{1}\mu_k^T+S_k+\varepsilon_k\), where \(\mu_k\) contains feature means and the structured signal \(S_k\) is centered across subjects. The entries of \(\varepsilon_k\) are independent before centering. Choose an \(n\) by \(m\) matrix \(Q\), with \(m=n-1\), whose columns are an orthonormal basis for the subspace perpendicular to \(\mathbf{1}\). Then \(Q^T\mathbf{1}=0\) and \(Q^TQ=I_m\). In contrast coordinates, \(Y_k=Q^TZ_k=Q^TS_k+Q^T\varepsilon_k\). Gaussianity and orthonormality imply that the \(m\) transformed errors for each feature are independent \(N(0,\sigma^2)\). Thus there are \(M=mP\) independent contrast entries across all blocks, where \(P=\sum_k p_k\), under the stated independent-error model. This gives a likelihood for the centered information without treating the \(n\) centered entries in a feature as independent.

For a candidate rank tuple, let \(\mathrm{RSS}\) be the sum of squared residuals in these contrast coordinates. Maximizing the Gaussian likelihood over the common variance gives \(\hat{\sigma}^2=\mathrm{RSS}/M\). Omitting constants shared by candidate models, the BIC to minimize is

where \(d\) is the number of fitted structural parameters; counting the estimated variance adds one more parameter, giving \((d+1)\log M\). In the comparisons below that extra term is included. This derivation assumes the uncentered Gaussian error model just stated, and that candidate fits use the same likelihood and preprocessing. If errors have block-specific variances, missingness, or dependence, the likelihood and criterion need corresponding changes.

For the standard orthogonal-subspace parameterization, one way to count \(d\) is as follows. Let \(m=n-1\). A rank-\(r\) joint subspace in \(m\)-dimensional contrast space has \(r(m-r)\) parameters for its orientation, and its block loadings contribute \(Pr\). For ordered, mutually orthogonal individual subspaces, block \(k\)'s rank-\(r_k\) subspace is chosen from a remaining space of dimension \(m-r-\sum_{\ell<k}r_\ell\). Its orientation contributes \(r_k(m-r-\sum_{\ell<k}r_\ell-r_k)\); its loadings contribute \(p_kr_k\). Therefore,

This count applies to the stated model, in which the individual score subspaces are mutually orthogonal as well as orthogonal to the joint subspace. It counts subspaces rather than arbitrary basis rotations, avoiding redundant parameters from alternative bases for the same subspace. If a fitted JIVE variant imposes different constraints, its effective parameter count must be adjusted. BIC is an asymptotic approximation; low-rank models can be singular at rank boundaries, so it should be treated as a principled comparison rather than an exact finite-sample guarantee.

Information criteria and permutations answer different questions. BIC ranks candidate models by an approximate penalized likelihood; permutation testing asks whether a statistic is unusual under a chosen null. They can disagree without either calculation being wrong. BIC is sensitive to the assumed error distribution and parameter count. Permutation conclusions are sensitive to exchangeability and the chosen null-generating scheme. Report the method, candidate ranks, preprocessing, assumptions, and diagnostics rather than presenting a selected rank as uniquely determined by the data.

How Rank Choices Reallocate Variation

The rank tuple changes both the fitted subspaces and the residuals. Raising \(r\) permits more variation to be represented by subject patterns shared across blocks. Raising \(r_k\) permits more variation specific to block \(k\). Because the components are fitted together under orthogonality constraints, these changes need not amount to simply moving a fixed quantity from one column of a variance table to another: the fitted directions themselves can change.

Diagnostics help reveal unstable or implausible choices. Check whether selected ranks change substantially under reasonable scaling choices, modest changes to the candidate grid, or resampling of subjects. Inspect residual structure: prominent cross-block association after fitting may indicate insufficient joint rank, while coherent within-block patterns may indicate insufficient individual rank or model misspecification. Conversely, a component supported by only a few observations, or one whose direction changes sharply across resamples, may be fragile. Residual checks and stability analyses do not replace the rank-selection criterion, but they can expose its limitations.

A common mistake is to select the rank tuple that explains the largest fraction of variation and call it the best model. Larger ranks cannot increase RSS, so this rule systematically favors adding components and may absorb noise. The correction is to compare added fit with a penalty or a calibrated null, and then assess whether the selected structure is stable and interpretable. Another mistake is to permute centered entries as if they were independent. Centering imposes a sum-to-zero constraint, so those entries are dependent; use a valid contrast-space likelihood for the Gaussian BIC derivation, or describe an independent-entry criterion as a working approximation rather than an exact centered-data likelihood.