Why the grouping objective matters
A co-expression network represents genes as nodes and their pairwise co-expression as edges or edge weights. Once the network has been constructed, a community-detection method assigns genes to groups. That assignment is not determined by the network alone: it depends on what the method calls a good grouping. Modularity evaluates a partition against a degree-based null model; hierarchical methods build a sequence of groupings from a chosen distance and linkage rule. These objectives answer related, but not identical, questions.
The network must be specified before interpreting either result. In the calculations below, assume a fixed, undirected network with nonnegative edge weights and no self-loops. These assumptions fit an undirected weighted adjacency matrix \(A=(a_{ij})\) when its entries are nonnegative and symmetric. The edge weights and node strengths are treated as given; uncertainty in estimating co-expression or choices made while constructing the network are not incorporated into the scores.
\(Q_\gamma=\sum_C\left[\frac{l_C}{m}-\gamma\left(\frac{K_C}{2m}\right)^2\right]\).
Equivalently, \(Q_\gamma=\frac{1}{2m}\sum_{i,j}\left[a_{ij}-\gamma\frac{k_i k_j}{2m}\right]\mathbf{1}(c_i=c_j)\), where \(c_i\) is the community label of node \(i\). The term \(k_i k_j/(2m)\) is the configuration-model expected weight for the pair, and \(\mathbf{1}(c_i=c_j)\) is 1 when the nodes share a community and 0 otherwise.
The null model preserves node strengths in expectation, rather than treating every possible edge as equally likely. The resolution parameter \(\gamma\) scales the null expectation: increasing \(\gamma\) generally makes the penalty for grouping nodes together larger and can favor smaller communities; decreasing it can favor larger ones. The parameter is a feature of the objective, not a statistical confidence level or a correction for uncertainty.
A larger \(Q_\gamma\) means a greater null-adjusted within-community excess than a smaller score. It does not by itself mean that the partition has more internal edge weight than the null expectation: only \(Q_\gamma>0\) establishes that the partition’s total within-community edge weight exceeds the resolution-adjusted null expectation. A negative score is still a valid score, and one negative score can be larger than another.
Calculating and interpreting modularity
Worked Example: Comparing Two Partitions of a Complete Graph
Consider a hypothetical unweighted complete graph on four genes, with \(\gamma=1\). Every pair is connected, so \(m=6\) and every node has strength 3. Compare the partition into four singletons with the partition consisting of one singleton and a community of three genes. The graph is undirected, has nonnegative weights, and has no self-loops, as required by the stated modularity formula.
For the all-singleton partition, each community has \(l_C=0\) and \(K_C=3\). Thus \(Q_1=4\left[0/6-(3/12)^2\right]=4(-0.0625)=-0.25\).
For the partition with one singleton and one triple, the singleton contributes \(-(3/12)^2=-0.0625\). The triple has three internal edges, so \(l_C=3\), and its total strength is \(K_C=9\). Its contribution is \(3/6-(9/12)^2=0.5-0.5625=-0.0625\). Therefore \(Q_1=-0.0625-0.0625=-0.125\).
The second score is larger: \(-0.125>-0.25\). Yet it is not positive. The triple’s observed internal edge weight, 3, is below its resolution-adjusted null expectation, \(K_C^2/(4m)=81/24=3.375\). This example shows why “higher modularity” must not be paraphrased as “more within-community weight than expected” unless the score is positive.
Worked Example: Two Disconnected Gene Groups
Suppose a hypothetical network consists of two disconnected triangles, with three genes in each triangle and unit weight on each of its three edges. There are no other edges. This is a nonnegative, undirected, loop-free network, with \(m=6\); each gene has strength 2. Take \(\gamma=1\), and partition the genes by triangle.
For each triangle community, \(l_C=3\) and \(K_C=6\). Its modularity contribution is \(3/6-(6/12)^2=0.5-0.25=0.25\). Adding both contributions gives \(Q_1=0.50\), which is positive. Here the total internal edge weight is 6, while the sum of the null expectations is \(2(6^2/(4\cdot6))=3\). Thus the partition has more within-community edge weight than the null expectation.
For comparison, the one-community partition has \(l_C=m=6\) and \(K_C=12\), giving \(Q_1=1-(12/12)^2=0\). The all-singleton partition has six contributions of \(-(2/12)^2\), giving \(Q_1=-6(1/36)=-0.1667\). For these partitions, grouping each triangle separately achieves the largest of the three scores. This comparison does not establish that the triangles are biologically meaningful; it establishes only how the specified objective ranks these partitions.
Why modularity can miss small communities
The resolution limitation follows directly from the change in modularity when two communities are merged. Let \(A\) and \(B\) be distinct communities, with total strengths \(K_A\) and \(K_B\), and let \(e_{AB}\) be the total edge weight between them, counting each crossing edge once. When they are combined, their internal weight increases by \(e_{AB}\), and their total strength becomes \(K_A+K_B\).
The first line subtracts the two separate contributions from the contribution after merging. The internal weights \(l_A\) and \(l_B\) cancel; expanding \((K_A+K_B)^2\) leaves the cross-term \(2K_AK_B\), producing the last expression. Therefore merging increases the score exactly when \(e_{AB}>\gamma K_AK_B/(2m)\). This is a comparison of observed between-community weight with a threshold determined by the null model, total network weight, and \(\gamma\).
In a large network, \(m\) can be large enough that this threshold is small. As a result, even groups with dense internal connections may be merged if their between-group connection exceeds the modularity threshold. This is one form of the resolution limit: maximizing a global objective can fail to separate a small, internally coherent group from a larger network. Changing \(\gamma\) changes the threshold, but there is no universal value that guarantees recovery of biologically meaningful groups at every scale.
Worked Example: A Ring of Small Cliques
Consider a hypothetical ring of 30 five-node cliques. Within each clique, all 10 possible edges have weight 1; between each consecutive pair of cliques, including the pair closing the ring, there is one edge of weight 1. The network is undirected, nonnegative, and loop-free. It has \(m=30(10)+30=330\). Each clique has total strength \(K_C=20+2=22\): its ten internal edges contribute 20 to the sum of strengths, and its two external edges contribute 2. Set \(\gamma=1\).
For two neighboring cliques, \(e_{AB}=1\). The merge threshold is \(K_AK_B/(2m)=22(22)/(660)=484/660=0.7333\). Since \(1>0.7333\), the change in modularity is \(\Delta Q_1=1/330-484/(2\cdot330^2)=0.0008081\), which is positive. Thus merging that pair improves modularity, even though each five-node clique is internally complete. The calculation illustrates a possible scale-related failure of the objective; it does not claim that every modularity optimizer will merge every clique or that every real gene module has this structure.
Hierarchical approaches use a different objective
A hierarchical clustering starts with a measure of dissimilarity between genes and a rule for joining clusters. In agglomerative clustering, each gene begins as its own cluster, and the algorithm repeatedly merges the pair judged closest under the chosen linkage rule. The resulting dendrogram records nested groupings; a cut through that tree selects a particular grouping. Different dissimilarities, linkage rules, and cut heights can give different results.
For example, a distance could be formed from a co-expression similarity, such as \(d_{ij}=1-r_{ij}\) when correlations are in the range from 0 to 1. This choice must be appropriate to the network’s edge definition: signed correlations, unsigned similarities, and transformed adjacency weights do not automatically yield interchangeable distances. Hierarchical clustering does not, by itself, compare within-community edge weight with a degree-based null expectation. Its goal is to organize pairwise dissimilarities into a nested structure.
Worked Example: Average-Linkage Gene Clustering
Suppose three hypothetical genes have pairwise correlations \(r_{AB}=0.9\), \(r_{AC}=0.2\), and \(r_{BC}=0.3\). The correlations are positive and form a valid correlation matrix; define dissimilarity as \(d_{ij}=1-r_{ij}\). Then \(d_{AB}=0.1\), \(d_{AC}=0.8\), and \(d_{BC}=0.7\). The measurements are treated as the fixed inputs to this clustering illustration.
Under average linkage, the first merge is \(A\) with \(B\), because their distance, 0.1, is the smallest. The distance from cluster \(\{A,B\}\) to \(C\) is the average of the two gene-level distances: \(d_{\{A,B\},C}=(d_{AC}+d_{BC})/2=(0.8+0.7)/2=0.75\). A cut below 0.75, but at or above 0.1, gives clusters \(\{A,B\}\) and \(\{C\}\); a cut at 0.75 joins all three. This decision follows from the defined distance, linkage, and cut, not from a modularity null model.
| Feature | Modularity-based detection | Hierarchical clustering |
|---|---|---|
| What is optimized or constructed? | A partition score measuring within-community excess relative to a degree-based null model. | A nested sequence of merges based on a dissimilarity and linkage rule. |
| How is a final grouping selected? | Choose the partition with a high score, typically using an optimization algorithm. | Choose a cut of the dendrogram under a stated rule. |
| Key sensitivity | Network construction, \(\gamma\), and the optimization procedure. | Distance definition, linkage, and cut choice. |
| Useful interpretation | Groups have relatively high internal weight compared with the specified null model when the score is positive. | Groups reflect nested similarity under the chosen distance and linkage. |
Assumptions, diagnostics, and practical limits
Neither approach supplies a biological truth label. Modularity assumes the chosen network and its degree-based reference model are suitable for the question. The standard formula above is for a nonnegative undirected network; signed edges require a specifically defined signed-network objective rather than silently inserting negative weights into this formula. Hierarchical clustering assumes its input distances and linkage have a useful interpretation for the intended grouping. Neither method automatically accounts for uncertainty in estimated correlations, sample composition, or network construction.
For modularity, inspect the edge-weight distribution, strength distribution, community sizes, and the value of \(Q_\gamma\), but do not treat a high score as evidence that the modules are stable or biologically validated. Repeat optimization from different initializations and compare the resulting partitions: modularity landscapes can contain many different partitions with similarly high scores. Check whether conclusions change across plausible \(\gamma\) values and whether very small groups are being absorbed. Such sensitivity checks describe dependence on modeling choices; they do not identify a uniquely correct resolution.
For hierarchical methods, check whether the distance definition matches the intended meaning of co-expression, whether the linkage rule is appropriate, and whether modest changes in input correlations or cut height substantially alter groups. A dendrogram always gives a hierarchy for the supplied inputs, but that fact alone does not demonstrate that every branch is a meaningful gene module. Both approaches may also reflect technical effects or shared cell composition rather than coordinated biological regulation.
The practical choice is therefore objective-dependent. Modularity is natural when the question concerns within-group edge weight relative to a degree-aware baseline, while hierarchical clustering is useful when nested similarity structure and a transparent cut are central. They can be used as complementary views, but agreement is not proof of validity and disagreement is not necessarily a methodological failure: their criteria differ.
Common pitfalls
- Calling every higher modularity score positive evidence of excess. A higher score only ranks above a lower one; only \(Q_\gamma>0\) indicates aggregate within-community weight above the resolution-adjusted null expectation.
- Assuming modularity has a single inherent scale. The resolution parameter changes the merge threshold, and a global objective can overlook small coherent groups.
- Comparing scores across different network constructions as if only the partition changed. Edge weights, thresholding, node set, and total weight also affect the score.
- Calling a dendrogram cut a null-model test. A hierarchical cut partitions genes according to distances and linkage; it does not establish excess connectivity relative to the configuration model.
- Treating a computational grouping as a biological mechanism. Co-expression is association, and grouping alone does not show direct regulation or causal interaction.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions below.
- For modularity \(Q_\gamma\), what does a larger score establish, and what additional condition indicates aggregate within-community weight above the null expectation?
- Two communities have \(K_A=8\), \(K_B=10\), \(m=100\), \(e_{AB}=0.5\), and \(\gamma=1\). Does merging increase modularity?
- In one sentence, explain why a hierarchical clustering cut and a modularity partition can disagree without either calculation being internally inconsistent.
- Name two choices that can change the groups from hierarchical clustering and two choices that can change a modularity result.
Solutions. (1) A larger score means greater null-adjusted within-community excess than a smaller score; aggregate excess is indicated by \(Q_\gamma>0\). (2) The threshold is \(K_AK_B/(2m)=8(10)/200=0.4\). Since \(e_{AB}=0.5>0.4\), \(\Delta Q_1=0.5/100-8(10)/(2\cdot100^2)=0.001>0\), so merging increases modularity. (3) The methods use different criteria: modularity scores a partition against a degree-based null model, while hierarchical clustering builds groups from a distance, linkage, and cut. (4) Hierarchical groups can change with the distance definition, linkage, or cut height; modularity results can change with network construction, \(\gamma\), or the optimization procedure.