Co-Clustering Matrices As Stability Summaries

Why Summarize Repeated Runs Pair by Pair?

Repeated clustering runs can yield different partitions because the data, features, or training sets are perturbed. Cluster names such as “cluster 1” and “cluster 2” are arbitrary: changing those names does not change the partition. The earlier material on aligning cluster labels across resamples addresses label matching when labels themselves must be compared. A co-clustering matrix takes a different route: it asks whether each pair of samples was placed together, a question that is unchanged by relabeling.

The result is a sample-by-sample summary. A large entry means that the pair was frequently assigned to the same cluster among runs in which both samples could be evaluated. A small entry means they were rarely assigned together under those same conditions. The “among runs” qualification matters: a pair may be absent from some resamples, or a run may not produce a usable assignment for both samples.

The denominator \(E_{ij}=\sum_{b=1}^{B}O_{ij}^{(b)}\) is the number of runs eligible for the pair. It should be recorded and interpreted alongside \(C_{ij}\). In particular, the numerator sums only the defined co-assignment indicators. An ineligible run contributes neither a co-assignment nor a separation; it is not treated as a zero.

What the Entries Mean

For each pair with \(E_{ij}>0\), \(C_{ij}\) is between 0 and 1. A value of 1 means the pair was together in every eligible run; a value of 0 means they were apart in every eligible run. Intermediate values are proportions. For example, \(C_{ij}=0.75\) means that the pair was co-assigned in three quarters of its eligible runs, not that either sample has a 75% probability of belonging to a particular named cluster.

If the runs are generated as repeated random draws from a specified perturbation procedure, the target quantity can be written as the conditional probability \(P(Z_{ij}=1\mid O_{ij}=1)\): the probability of co-assignment in a run, conditional on both samples being eligible. The observed frequency estimates that probability. This interpretation depends on the run-generation procedure; changing the resampling unit, feature perturbation, preprocessing, or clustering settings can change the probability being summarized. If runs are a fixed collection rather than random draws, the frequency is a descriptive proportion for that collection.

The matrix is symmetric because the same two samples form the pair regardless of order: \(C_{ij}=C_{ji}\) whenever the entries are defined. Its diagonal is 1 when a sample is eligible in at least one run, since a sample is in the same cluster as itself. If a sample is never eligible, its diagonal entry is undefined. A zero off-diagonal entry is meaningful only when \(E_{ij}>0\); an undefined entry must not be displayed or interpreted as zero.

From Runs to a Matrix

A practical construction records two quantities for every pair: its eligible-run count \(E_{ij}\) and its co-clustering frequency \(C_{ij}\). The same eligibility rule must be applied consistently. For instance, if the analysis defines a sample as eligible only when it appears in a resample and receives a cluster assignment, both conditions must hold for each member of the pair. If assignments are available for samples outside a resample, use those assignments only if that was part of the prespecified run procedure; do not silently mix in-sample and out-of-sample rules.

The following justification shows why the frequency estimates a conditional co-assignment probability under a random-run interpretation. For an eligible run, \(Z_{ij}\) is a binary indicator. Conditional on eligibility, its expectation is the probability of co-assignment:

If eligible runs are independent draws from the same run-generating procedure, the eligible indicators \(Z_{ij}\) are identically distributed Bernoulli observations conditional on eligibility. Their sample mean is the displayed \(C_{ij}\), so as the number of eligible runs grows it converges to that conditional probability. This justification does not make the runs independent if the procedure creates dependent perturbations, nor does it make the result generalize beyond the specified procedure.

Eligibility Changes the Interpretation

Pair-specific denominators are essential when samples are absent from different resamples. The numerator and denominator must refer to the same eligible runs. A pair with a frequency of 1 based on one eligible run has weaker empirical support than a pair with a frequency of 1 based on many eligible runs. The frequencies alone hide this distinction.

Patterns, Diagnostics, and Limits

A heat map of \(C_{ij}\), with samples ordered in a meaningful way, can make stable-looking groups and uncertain interfaces visible. Blocks of high values suggest pairs that are repeatedly co-assigned; low values between blocks suggest repeated separation. Values that change gradually across a boundary, or pairs with intermediate frequencies, flag relationships that depend more strongly on the perturbation. These visual patterns guide inspection, but do not by themselves establish the number of groups or identify a unique partition.

Inspect the denominator matrix as well as the frequency matrix. Look for pairs with few eligible runs, entire samples with little eligibility, and discrepancies in eligibility caused by a coding or assignment rule. Check that the matrix is symmetric, that defined diagonal entries are 1, and that every reported frequency lies between 0 and 1. An undefined pair should be marked as missing, not colored as though its frequency were zero. If a heat map gives the same visual weight to a pair based on very few runs as to one based on many, show the eligible counts or otherwise make that limitation clear.

There are important limits to the summary. Co-clustering frequencies discard the identities of cluster labels and reduce repeated partitions to pairwise relationships. Consequently, the matrix does not retain all the information in the runs. In particular, its entries are averages across runs: they need not describe any one partition that actually occurred. A pairwise pattern should therefore not be presented as if it were itself a fitted clustering. The matrix also reflects the full analysis choices used in the runs; it is not a measure of biological truth or evidence that cluster membership causes an outcome.

Co-clustering is useful when the central question is which sample pairs repeatedly travel together. Other summaries answer different questions. Label alignment supports comparisons of named clusters when the matching of labels is meaningful. A single run-to-run agreement statistic compresses comparison into a scalar and therefore does not show which pairs account for disagreement. Cluster-specific summaries focus on individual groups. These approaches are complementary; the pairwise matrix is particularly helpful for seeing local patterns and boundaries, while its many entries should not be mistaken for independent observations or a complete account of clustering stability.

Common Mistakes

• Counting an ineligible run as a separation. “The pair was not together because one sample was absent” is incorrect. The run supplies no pairwise comparison; exclude it from both numerator and denominator.
• Using a common denominator for every pair. This is wrong when pairs have different eligibility. Calculate \(E_{ij}\) separately for each pair and divide only by that pair’s count.
• Calling an undefined entry zero. Zero means the pair was eligible and separated every time. No eligible runs means there is no observed frequency to interpret.
• Reading a high value as a membership probability. The entry is a conditional pairwise co-assignment frequency under a particular run procedure, not the probability that a sample belongs to a named subtype.
• Treating the matrix as a final partition. Pairwise averages summarize repeated partitions; they do not necessarily correspond to a partition observed in any run.