Preserving Modules Across Cohorts

Why Evaluate Modules in a Second Cohort?

A module found in one cohort is a data-dependent grouping of genes whose expression patterns support a particular network structure in that cohort. A second cohort offers a test of whether the grouping carries beyond the data that produced it. This is not a single yes-or-no question: the same genes might remain connected but change their relative connectivity, or their connectivity ordering might remain similar even as overall network density declines.

A useful comparison therefore separates membership preservation—whether the same genes belong together—from topological preservation—whether those genes retain similar within-module connectivity patterns. Neither alone establishes that a module has the same biological function or that cohort differences caused any observed network change.

Set Up a Comparable Cohort Assessment

Before calculating preservation, verify that genes are aligned by identifier and that the expression measurements are sufficiently comparable for the intended network comparison. Construct the reference and test networks using a consistent definition of association, network type, and adjacency transformation. For example, if the reference uses a signed weighted adjacency, using an unsigned adjacency in the test cohort changes what counts as connectivity. The test cohort must also have enough observations and gene-level variation to estimate associations meaningfully.

The reference module labels should remain fixed for the primary preservation assessment. Calculate test-cohort network properties for the genes in each reference module, regardless of how the test cohort's own clustering would label those genes. Detecting test-cohort modules independently can provide a separate membership-overlap comparison, but it answers a different question.

For a weighted adjacency matrix, the within-module connectivity of gene \(i\) in module \(M\) can be written as the sum of its adjacency weights to other genes in \(M\). A normalized version may divide by the number of possible connections. The precise definition should be kept consistent across cohorts. Since absolute connectivity depends on network construction and scale, also compare relative patterns, such as connectivity ranks or module-membership scores. Module membership (often called \(kME\)) is the correlation between a gene's expression profile and its module eigengene; compare these scores across cohorts with attention to the arbitrary sign of a principal-component eigengene.

Membership Overlap and Connectivity Patterns

When modules are detected separately in both cohorts, membership overlap can be summarized using set measures. For reference module \(A\) and test module \(B\), the Jaccard index is the number of shared genes divided by the number of genes in their union. It ranges from 0 (no shared genes) to 1 (identical sets). The reference coverage, \(|A\cap B|/|A|\), asks what fraction of the reference genes appear in the test module; the test coverage, \(|A\cap B|/|B|\), asks what fraction of the test module comes from the reference module. These distinguish loss of reference genes from addition of genes not in the reference set.

Membership measures do not assess network topology. For topology, compare quantities calculated for the fixed reference genes: for example, the correlation of their within-module connectivity values, the correlation of their connectivity ranks, or the correlation of their module-membership scores. Also summarize shifts in the distribution or mean of connectivity. A strong correlation can coexist with a change in mean or scale, as the preceding example demonstrates.

Permutation-Based Preservation Summaries

A common approach compares the observed preservation statistics for a module with those for random gene sets of the same size in the test network. The random sets provide a benchmark for how much preservation-like structure might occur without selecting the particular reference module. For a statistic oriented so that larger values indicate stronger preservation, a standardized score can be calculated as

where \(T_{\mathrm{obs}}\) is the observed statistic for the reference module, \(\overline{T}_{\mathrm{perm}}\) is the mean statistic over permuted gene sets, and \(s_{\mathrm{perm}}\) is their standard deviation. In WGCNA-style analyses, separate standardized components can summarize density preservation and connectivity preservation; a commonly used \(Z_{\mathrm{summary}}\) combines those components by addition. The exact component statistics depend on the implementation, so their definitions and direction should be reported.

The standardization has a direct justification: subtracting the permutation mean centers the observed statistic relative to the benchmark, and dividing by the permutation standard deviation expresses the difference in units of permutation variability. This makes statistics on different raw scales easier to summarize. It does not by itself guarantee that the standardized statistic follows a normal distribution or that its value has a particular false-positive rate.

A complementary summary is the medianRank of preservation statistics across modules. For each statistic, modules are ranked relative to one another; the ranks for a module are then summarized by their median. A lower median rank indicates stronger relative preservation among the modules included in that comparison. Unlike a standardized score, it is not a measure of distance from a random-set mean. It is useful when preservation scores are on different scales or have different variability, but it is relative to the particular set of modules being ranked: adding or removing modules can change the ranks.

Comparing Approaches and Checking Assumptions

Permutation validity depends on the benchmark matching the question. Random gene sets should be drawn from the genes eligible for module detection and should have the same size as the reference module. Yet genes differ in expression variability and network connectivity, so unrestricted random sets may not be exchangeable with the reference module. Stratifying or matching on relevant properties can make the benchmark more informative, although it changes the null comparison and should be described. Small modules, few permutations, or very low permutation variability can make standardized scores unstable.

Cohort differences can also affect the comparison. Different sample sizes change the precision of estimated correlations; a smaller cohort may show weaker or noisier connectivity even if the underlying structure is similar. Differences in measurement platform, preprocessing, range of expression, or cell composition can alter estimated associations. The earlier material on batch, cell mixture, and spurious co-expression is directly relevant: preservation statistics cannot distinguish these sources from biological differences on their own. Check sample sizes, gene coverage, expression distributions, and network settings, and report them alongside the preservation results.

Common Mistakes and Limits

• Calling a high connectivity correlation “unchanged connectivity.” A correlation describes pattern, not level. Report an appropriate density or mean-connectivity summary as well.
• Treating overlap as topology. Shared gene membership does not guarantee shared edges or connectivity ordering; assess both when both questions matter.
• Reading \(Z_{\mathrm{summary}}\) as a p-value. It is a standardized, heuristic summary based on a permutation benchmark, not a probability that the null is true or a universal significance test.
• Ignoring unequal information across cohorts. A lower preservation score in a smaller or noisier cohort may reflect estimation limitations, not necessarily a changed biological network.
• Comparing incompatible networks. Changing the adjacency definition, network type, or relevant tuning choices can make apparent differences a consequence of construction rather than cohort structure.

Preservation analysis is designed to assess whether a reference module's measured structure is evident in another cohort. It does not test whether networks differ significantly between conditions, identify a causal mechanism, or establish functional equivalence. Those are distinct questions requiring appropriate analyses and interpretation.