Cross-Block Alignment And Leakage

Why Cross-Block Alignment Must Respect the Split

Multi-omics studies may measure different molecular blocks on the same subjects. A learned alignment maps one block into coordinates related to another block or to a shared representation. The mapping can make paired measurements comparable, support a downstream prediction, or quantify how well one block corresponds to another. Its usefulness must be evaluated on subjects whose measurements did not help determine the mapping.

This includes alignment that does not use the outcome. For example, centering, scaling, estimating a rotation, or learning a shared latent space can all use information about the distribution of the held-out omics data. If held-out subjects contribute to those calculations, the evaluation no longer measures application of a mapping learned from training subjects alone. The contamination may be subtle: no outcome labels need to be used for leakage to occur.

Definition: Let \(X_i\) and \(Z_i\) denote two measured omics blocks for subject \(i\), and let \(\mathcal I_{\mathrm{tr}}\) and \(\mathcal I_{\mathrm{te}}\) denote the training and held-out subject sets. A cross-block mapping is a function \(g_{\hat\theta}\) that transforms one block, such as \(X_i\), into coordinates intended to correspond to \(Z_i\) or to a common representation. The alignment parameters \(\hat\theta\) must be estimated using training subjects only. For each held-out subject, apply \(g_{\hat\theta}\) with those parameters fixed.

The learned parameters can include means, scales, selected features, loading matrices, regression coefficients, rotation matrices, and any data-dependent choices used to construct the map. If a pipeline first normalizes each block and then estimates a cross-block transformation, every estimated quantity belongs to the fitted mapping. Each must be learned from the training subjects, not recomputed using the held-out subjects.

Fit First, Apply Second

A clean evaluation treats alignment as a training-fitted transformation. In each split, first identify the independent units and assign them to training or held-out sets. Estimate all alignment parameters from training measurements. Then pass held-out measurements through the resulting fixed map. If held-out paired blocks are available, their correspondence can be used to calculate a prespecified evaluation quantity, but not to refit the map.

Define the mapping target.
Specify which block is mapped, which coordinates it is mapped to, and what held-out quantity will assess alignment. Confirm that those measurements would be available at the intended use time.

Estimate parameters on training subjects.
Use training data to estimate every centering, scaling, feature, and alignment parameter. If a training-only pipeline includes several transformations, estimate each in the order it will be applied.

Freeze and apply the transformation.
For held-out subjects, use the training-fitted parameters unchanged. Do not update them using held-out distributions, paired measurements, or outcomes.

Assess without adapting.
Calculate the prespecified held-out alignment measure. If this result is used to alter the mapping, the held-out set has become part of the development process and cannot serve as an untouched evaluation of that revised map.

The split should keep together every observation that would not be independent at deployment. For instance, if one person contributes repeated samples, splitting those samples across training and held-out sets can allow person-specific structure to inform the map on both sides. The relevant independent unit is the unit that will receive a prediction or other application of the mapping; the earlier material on choosing independent units for data splits addresses that design decision.

A mapping also has an input contract. If deployment supplies only \(X_i\), a method that needs the held-out \(Z_i\) to construct the mapped value is not an applicable predictor. If both blocks are available for a prespecified paired-alignment assessment, \(Z_i\) may be used to score correspondence after applying the fixed map, but not to estimate or adjust the map. The intended availability of each block must be clear before evaluation.

Worked Examples

Worked Example: Training-Only Offset Alignment

Suppose a hypothetical assay records one feature in block \(X\) and a corresponding feature in block \(Z\), both in arbitrary intensity units. For three training subjects, the paired measurements are \((X,Z)=(1,3),(3,5),(5,7)\). Use an additive offset to align \(X\) to \(Z\), and apply it to a held-out subject with \(X=4\), whose observed \(Z\) is 7.

Estimate the offset from training pairs as the mean training difference:

\hat\delta=\frac{1}{3}\sum_{i\in\mathcal I_{\mathrm{tr}}}(Z_i-X_i) =\frac{(3-1)+(5-3)+(7-5)}{3} =\frac{6}{3}=2.

The fitted mapping is \(g_{\hat\delta}(X)=X+\hat\delta\). Applying it without alteration gives \(g_{\hat\delta}(4)=4+2=6\) intensity units. The held-out discrepancy is \(7-6=1\) intensity unit. The held-out pair did not contribute to the estimated offset. Re-estimating the offset after including that pair would use held-out information, even though the adjustment is only an offset and does not use an outcome.

Worked Example: A Linear Cross-Block Map

Consider hypothetical training pairs \((X,Z)=(0,1),(1,2),(2,2)\), where each value is a feature measured in arbitrary units. Fit the ordinary least-squares line \(Z_i=a+bX_i+\varepsilon_i\) on the three training subjects, then map a held-out \(X=3\). Assume the relationship is adequately represented by a linear map over this range, the paired training measurements are correctly matched, and the training subjects are the independent units.

The training means are \(\bar X=1\) and \(\bar Z=5/3\). The slope is the sum of centered cross-products divided by the sum of centered squared \(X\) values:

\hat b= \frac{(-1)(-2/3)+(0)(1/3)+(1)(1/3)} {(-1)^2+0^2+1^2} =\frac{1}{2}=0.5.

Therefore \(\hat a=\bar Z-\hat b\bar X=5/3-(0.5)(1)=7/6\), and the fitted map is \(g_{\hat\theta}(X)=7/6+0.5X\). At the held-out input \(X=3\), the mapped value is \(7/6+1.5=8/3\), or approximately 2.667 units. If the held-out paired measurement is \(Z=3\), the observed discrepancy \(Z-g_{\hat\theta}(X)\) is \(1/3\), or approximately 0.333 units. The held-out \(Z\) is used to assess the fit, not to recalculate \(\hat a\) or \(\hat b\).

This example uses an intercept and a slope estimated from training pairs. If the training relationship is nonlinear, the linear map may be inadequate; that is a limitation of the chosen mapping, not a reason to estimate its coefficients from held-out pairs.

Worked Example: Training-Fitted Orthogonal Alignment

Suppose two hypothetical two-feature blocks have three paired training rows. Let the rows of \(A\) be \((1,0),(0,1),(-1,-1)\), and let the corresponding rows of \(B\) be \((0,1),(-1,0),(1,-1)\). Assume the intended relationship is a rotation, so the map is an orthogonal matrix \(Q\), and assume the training measurements are centered at zero. Estimate \(Q\) from these training pairs and apply it to a held-out \(A=(2,1)\).

The orthogonal Procrustes fit minimizes the total squared distance between the transformed training rows and their partners:

\hat Q=\arg\min_{Q^\mathsf{T}Q=I} \sum_{i\in\mathcal I_{\mathrm{tr}}}\lVert A_iQ-B_i\rVert^2.

Here the candidate rotation \(Q=\begin{pmatrix}0&1\\-1&0\end{pmatrix}\) sends each training row to its paired row: \((1,0)Q=(0,1)\), \((0,1)Q=(-1,0)\), and \((-1,-1)Q=(1,-1)\). Each squared discrepancy is zero, so the objective is zero, its smallest possible value. The training rows span two dimensions, allowing this rotation to be identified from their correspondence. Thus the fitted map is \(A\mapsto A\hat Q\).

For the held-out row, \((2,1)\hat Q=(-1,2)\). If its observed paired \(B\) row is \((-1,2.2)\), the held-out residual vector is \((0,0.2)\), and its squared Euclidean discrepancy is \(0^2+0.2^2=0.04\) squared feature units. The held-out pair measures how well the training-fitted rotation transfers; it does not alter \(\hat Q\). The rotation assumption is substantive: if the blocks differ by feature-specific scaling or nonlinear structure, orthogonal alignment may leave systematic discrepancies.

Why the Split Changes the Meaning of the Result

A training-fitted map estimates how a relationship learned from one set of subjects transfers to new subjects, under the assumption that the new subjects are drawn from the deployment population of interest. Recomputing alignment parameters on all subjects changes the question: it measures correspondence after the held-out data have helped define the coordinates. Such a result can look better because the map has adapted to the evaluation sample, not because it would transfer as specified.

This concern applies whether parameters use outcomes or not. An unsupervised alignment can exploit held-out feature means, variances, covariance structure, or paired correspondences. “No labels were used” is therefore not sufficient evidence that evaluation was independent. The relevant question is whether any held-out information affected the learned transformation.

A short justification makes the issue explicit. Write the training-fitted parameters as \(\hat\theta_{\mathrm{tr}}=A(D_{\mathrm{tr}})\), where \(D_{\mathrm{tr}}\) contains the training measurements and \(A\) is the fitting algorithm. The held-out mapped value is \(g_{\hat\theta_{\mathrm{tr}}}(X_i)\). If instead the fitting algorithm receives held-out data, the computed parameter becomes \(A(D_{\mathrm{tr}},D_{\mathrm{te}})\). Unless those data have no effect on the result, this is a different mapping. In particular, the held-out discrepancy is then evaluated against a transformation that has already adapted to the held-out sample.

Key takeaway: Estimate all data-dependent alignment parameters from training subjects, freeze them, and apply them to held-out subjects. Use held-out paired data only for a prespecified assessment, not to refine the transformation being assessed.

Diagnostics, Alternatives, and Limits

Several checks help reveal when alignment is being fit or applied incorrectly:

Audit parameter provenance. Record which subjects contributed to each mean, scale, loading, feature choice, or mapping coefficient. Any held-out contribution to a fitted quantity is a warning sign.
Check the application contract. Confirm that every input required to transform a new subject will actually be available in the intended setting. A map that needs a missing block cannot be applied as specified.
Inspect training residuals and held-out discrepancies. Large or structured discrepancies can indicate a poor mapping, mismatched pairs, a misspecified relationship, or a difference between training and deployment populations. A low training discrepancy alone does not establish transfer.
Check dimensions and feature identity. Feature ordering, units, and preprocessing must match between fitting and application. A correct matrix operation on misordered features still yields an invalid alignment.
Check for split dependence. If technical replicates, repeated samples, or other dependent observations cross the split, apparent transfer may partly reflect information shared across sets rather than transfer to independent units.

The main alternative is to estimate the alignment using all available paired subjects before evaluating it. That can be appropriate for descriptive analysis of those same subjects, but it does not estimate performance of a training-fitted map on new subjects. Another possibility is explicitly transductive analysis, in which unlabeled held-out measurements are available and used at deployment to adapt the transformation. That is a different procedure: it must be declared in advance, and the evaluation must reproduce that access pattern. It should not be described as applying a fixed training-only map.

No split can guarantee that the training and held-out populations are comparable. A map may transfer poorly because of cohort composition, assay changes, sparse or noisy features, weak cross-block relationships, or insufficient training data. Held-out discrepancies are diagnostic evidence about this transfer, not a proof that the map is biologically meaningful or causal. If the held-out results are used to choose a different alignment, the reported assessment is no longer an untouched evaluation of the revised choice.

Common Mistake

Wrong reasoning: “Alignment is unsupervised, so I can estimate it from all subjects before splitting; only the final prediction model needs training-only fitting.” This overlooks that alignment parameters depend on observed data. Held-out feature distributions or paired relationships can influence the representation passed to the prediction model, giving the evaluation information from the subjects it is meant to hold out.

Correction: Put alignment inside the training portion of the evaluation procedure. Estimate all its parameters from training subjects, then transform held-out subjects with those same parameters. If the intended deployment procedure truly adapts to a batch of unlabeled incoming subjects, specify that procedure and evaluate it as such rather than treating it as a fixed mapping.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions.

A training-fitted map centers a feature using a training mean of 10. A held-out value is 13. What centered value should be passed forward, and should the held-out set mean replace 10?
Why can estimating a cross-block rotation from all paired subjects contaminate an evaluation even when no outcomes are used?
A held-out paired block is available. May it be used to calculate a prespecified alignment discrepancy? May it be used to update the alignment matrix being assessed?
What is one reason to keep repeated samples from the same person in the same split?
A fixed map performs poorly on held-out subjects, and an analyst changes the map after inspecting their paired measurements. Can the same held-out discrepancy still be presented as an independent assessment of the revised map?

Solutions. (1) The centered value is \(13-10=3\); retain the training mean of 10. (2) The held-out paired measurements can influence the estimated rotation, so the evaluated transformation is no longer training-fitted. (3) Yes, a prespecified discrepancy can assess the fixed map; no, the held-out block must not update the matrix being assessed. (4) Repeated samples from one person may share subject-specific information, so splitting them can make the held-out sample non-independent of training. (5) No. Using the held-out result to revise the map makes that set part of development; an independent assessment requires new held-out data or a separately defined evaluation design.