Tutorials › Statistical Methods for Single-Cell RNA Sequencing › Uneven Density and Community Boundaries

Graph-Based Clustering · Tutorial 260 of 700

Uneven Density and Community Boundaries

Learn to compare candidate community merges by calculating how internal density, degree, and resolution affect modularity and Constant Potts Model objectives.

Intermediate 12 min read

What You'll Learn

  • Calculate the edge densities inside and between candidate communities.
  • Derive and use the modularity merge-gain formula for an undirected graph.
  • Use the Constant Potts Model merge criterion to interpret its resolution parameter.
  • Compare how the two objectives treat the same weak boundary edge.
  • Diagnose when a community assignment is sensitive to resolution or graph changes.

Why Density and Boundaries Do Not Tell the Whole Story

A graph can contain a tightly connected group next to a region with fewer internal edges, with only a few edges linking the two. It is tempting to conclude that the dense group must be one community and the sparse region another. But a community assignment depends not just on whether a boundary looks weak. It also depends on the objective being optimized, the sizes and degrees of the candidate groups, and the chosen resolution.

This tutorial compares two objectives for undirected graphs: modularity and the Constant Potts Model (CPM). Both evaluate partitions by comparing within-community connections against a reference, but they use different references. Consequently, the same graph and boundary can lead them to favor different assignments. The calculations below compare specific candidate partitions; they do not imply that every partition found by an algorithm is globally optimal.

Definition: In a simple undirected graph, an edge joins two distinct vertices, and there is at most one edge between any pair. For a vertex set \(C\), its size is \(n_C\), its internal edge count is \(m_C\), and its internal edge density is \(m_C/\binom{n_C}{2}\). For disjoint sets \(A\) and \(B\), let \(e_{AB}\) be the number of edges between them; their boundary edge density is \(e_{AB}/(n_A n_B)\). A partition divides all vertices into non-overlapping communities.

A low boundary density means that relatively few possible edges across the boundary are present. It does not by itself determine whether merging the communities improves an objective. In particular, “few” may mean something different for two small groups than for two large groups, and modularity also accounts for vertex degrees across the whole graph.

Two Objectives, Two Reference Points

For a simple undirected graph with \(m\) edges, let \(A_{ij}\) be 1 when vertices \(i\) and \(j\) share an edge and 0 otherwise, and let \(k_i\) be the degree of vertex \(i\). The modularity of a partition is

$$ Q=\frac{1}{2m}\sum_{i,j}\left(A_{ij}-\gamma\frac{k_i k_j}{2m}\right)\mathbf{1}(c_i=c_j). $$

Here \(c_i\) is the community assigned to vertex \(i\), \(\mathbf{1}(c_i=c_j)\) is 1 when the vertices share a community and 0 otherwise, and \(\gamma>0\) is the resolution parameter. The term \(k_i k_j/(2m)\) is the expected adjacency contribution under the configuration-model reference used by modularity. Thus modularity rewards within-community edges relative to a degree-based expectation. It is not simply a score for high internal density.

For candidate communities \(C\), the same modularity can be written using \(m_C\), the number of edges internal to \(C\), and \(K_C=\sum_{i\in C}k_i\), the sum of degrees in \(C\):

$$ Q=\sum_C\left(\frac{m_C}{m}-\gamma\left(\frac{K_C}{2m}\right)^2\right). $$

The CPM objective for this unweighted graph is

$$ Q_{\mathrm{CPM}}=\sum_C\left(m_C-\gamma\binom{n_C}{2}\right). $$

CPM rewards each internal edge and subtracts a penalty for each possible pair within a community. For a fixed partition, a community’s contribution is positive when its internal edge density exceeds \(\gamma\). Unlike modularity’s degree-based reference, CPM uses a pair-count penalty; its resolution parameter is on an edge-per-pair scale. The numerical values of \(\gamma\) in these two objectives are therefore not directly comparable.

Deriving the Merge Comparisons

Suppose communities \(A\) and \(B\) are separate in one candidate partition and merged in another. For modularity, merging changes the internal edge count from \(m_A+m_B\) to \(m_A+m_B+e_{AB}\), and changes the combined degree sum from separate terms to \(K_A+K_B\). Subtracting the separate score from the merged score gives

$$ \begin{aligned} \Delta Q &=\frac{m_A+m_B+e_{AB}}{m} -\gamma\left(\frac{K_A+K_B}{2m}\right)^2\\ &\quad-\left[\frac{m_A}{m}-\gamma\left(\frac{K_A}{2m}\right)^2 +\frac{m_B}{m}-\gamma\left(\frac{K_B}{2m}\right)^2\right]\\ &=\frac{e_{AB}}{m}-\gamma\frac{K_AK_B}{2m^2}. \end{aligned} $$

The merge improves modularity precisely when \(\Delta Q>0\). The first term is the observed boundary-edge contribution; the second is the degree-based penalty for combining the groups. This comparison depends on graph-wide \(m\) and on both groups’ degree sums, not only on their boundary density.

For CPM, the merge adds \(e_{AB}\) observed edges and introduces a penalty for all \(n_A n_B\) new possible pairs between the groups. Hence

$$ \Delta Q_{\mathrm{CPM}}=e_{AB}-\gamma n_A n_B. $$

The CPM merge improves the objective when \(e_{AB}/(n_A n_B)>\gamma\): the boundary density exceeds the chosen resolution. This clean threshold helps explain CPM’s response to group size, but it is only a comparison of these two partitions. Other possible splits or merges can still matter to the full optimization.

Worked Example: A Dense Clique Beside a Sparse Path

Consider eight vertices. Set \(A\) is a four-vertex clique, so it has 6 internal edges. Set \(B\) is a four-vertex path, so it has 3 internal edges. There is one edge between the sets. The graph has \(m=6+3+1=10\) edges. The internal densities are \(6/\binom{4}{2}=1\) for \(A\) and \(3/\binom{4}{2}=0.5\) for \(B\); the boundary density is \(1/(4\cdot4)=0.0625\).

For modularity at \(\gamma=1\), the degree sum in \(A\) is \(K_A=2(6)+1=13\): its internal edges contribute 12 to the degree sum, and the boundary edge contributes 1. Similarly, \(K_B=2(3)+1=7\). The merge comparison is

$$ \Delta Q=\frac{1}{10}-\frac{13\cdot7}{2(10)^2} =0.1000-0.4550=-0.3550. $$

For these two candidate partitions, keeping the clique and path separate has the larger modularity. For CPM at \(\gamma=0.1\), the comparison is

$$ \Delta Q_{\mathrm{CPM}}=1-(0.1)(4)(4)=-0.6000. $$

CPM also favors keeping them separate at this resolution. The example illustrates that a sparse region can sit next to a very dense one without the density contrast alone deciding the assignment: the objective’s merge criterion must also be considered.

When the Same Boundary Produces Different Choices

Worked Example: Two Dense Groups With One Bridge

Now consider two disjoint triangles joined by a single edge between one vertex in each triangle. Each triangle has 3 internal edges and density 1; there are \(e_{AB}=1\) boundary edges among \(3\cdot3=9\) possible cross-pairs. The whole graph has \(m=7\) edges, and each triangle has degree sum \(K_A=K_B=7\): six degree contributions from its three internal edges and one from the bridge.

At modularity resolution \(\gamma=1\), the merge comparison is

$$ \Delta Q=\frac{1}{7}-\frac{7\cdot7}{2(7)^2} =0.1429-0.5000=-0.3571. $$

Modularity favors the partition into two triangles rather than their merger. For CPM at \(\gamma=0.1\), however,

$$ \Delta Q_{\mathrm{CPM}}=1-(0.1)(3)(3)=0.1000. $$

CPM favors merging these two candidate groups at that resolution. This does not mean CPM’s numerical resolution is “lower” than modularity’s: the parameters operate in different objective scales. The point is that CPM compares the bridge density, \(1/9\approx0.1111\), directly with \(0.1\), whereas modularity compares the bridge contribution with a degree-based expectation involving the whole graph.

Worked Example: A Sparse Boundary Changes CPM’s Decision With Resolution

Let \(A\) be a three-vertex clique and \(B\) a five-vertex path, with one edge between them. Their internal densities are \(3/\binom{3}{2}=1\) and \(4/\binom{5}{2}=0.4\), respectively. The boundary density is \(1/(3\cdot5)=1/15\approx0.06667\).

At CPM resolution \(\gamma=0.05\), merging gives

$$ \Delta Q_{\mathrm{CPM}}=1-(0.05)(3)(5)=0.2500. $$

The merge increases the CPM objective. At \(\gamma=0.08\), the same graph and same edge give

$$ \Delta Q_{\mathrm{CPM}}=1-(0.08)(3)(5)=-0.2000. $$

Now the separate candidate partition has the higher CPM objective. The change occurs because the boundary density is between the two resolutions: \(0.05<1/15<0.08\). Internal density describes the two groups, but in this pairwise CPM comparison it is the density across their boundary that determines whether the merge improves the objective.

Reading Local Density and Boundary Evidence

These comparisons suggest a disciplined way to interpret a graph partition. First measure the internal and boundary edge densities for the groups of interest. Then calculate the objective-specific merge change at the resolution actually used. A weak boundary may support keeping groups separate, but it is not a universal rule: in modularity, the relevant comparison includes degree sums and total graph size; in CPM, the boundary density is compared with \(\gamma\).

Local density can also point in different directions within one graph. A dense group may have many internal edges but also high degree, which raises modularity’s degree-based merge penalty with another group. A sparse group may have few internal edges, yet a sufficiently low CPM resolution can still make joining it to a neighbor favorable if the boundary density exceeds that resolution. Neither objective defines a community solely by being “dense” in an absolute sense.

For a practical check, calculate summaries for candidate boundaries rather than relying only on a graph drawing:

  • Record \(n_A\), \(n_B\), \(m_A\), \(m_B\), and \(e_{AB}\), and calculate each within-group and boundary density.
  • For modularity, also record \(K_A\), \(K_B\), and the graph-wide \(m\); evaluate \(\Delta Q\) at the chosen \(\gamma\).
  • For CPM, compare \(e_{AB}/(n_A n_B)\) with the chosen \(\gamma\).
  • Check whether a small change in resolution or a single boundary edge reverses the comparison. Such sensitivity is evidence that the assignment is conditional on modeling choices, not proof that one partition is biologically correct.

An edge-count comparison does not establish that a boundary corresponds to a biological separation. A graph can reflect choices made during cell representation and neighborhood construction, and a community objective only evaluates the graph it receives. A high-scoring partition is therefore a property of the graph and objective, not a causal or biological validation by itself.

Common Pitfalls and Alternatives

Pitfall: “There is only one edge between these two groups, so every community method must keep them separate.” This reasoning ignores group sizes, degree structure, and resolution. In the two-triangle example, modularity at \(\gamma=1\) favored separation while CPM at \(\gamma=0.1\) favored merging. The correction is to calculate the merge change for the particular objective and parameter, then describe the conclusion as conditional on those choices.

A related mistake is to treat a high internal density as sufficient evidence for a community. The triangle example has two internally complete groups, yet CPM can favor merging them. Conversely, a sparse group is not automatically absorbed: its boundary may be too weak under the selected objective. Internal and boundary densities are useful descriptions, not standalone decision rules.

Modularity and CPM are not the only ways to partition graphs. Flow-based methods such as Infomap evaluate how a random walk can be compressed, so their criterion is not the same as modularity’s degree-based null comparison or CPM’s pair-density threshold. The choice between objectives should follow the structure one wants to represent and be checked against plausible resolution choices. Repeated optimization from different initial conditions can reveal algorithmic variability, but agreement across runs does not remove an objective’s modeling assumptions.

Finally, a positive merge change for one pair does not prove that the resulting full partition is optimal. Other vertices may move, or other groups may split or merge. The calculations are exact comparisons between the stated alternatives; a clustering algorithm searches a much larger space and may use a heuristic. Interpret local calculations accordingly.

Key takeaway: A weak boundary is evidence to examine, not a universal community rule. Modularity compares observed edges with a degree-based graph-wide expectation; CPM compares boundary density with its resolution. State the objective, resolution, and candidate assignment whenever interpreting a density-based community decision.

Check Your Understanding

Use the definitions and merge comparisons above. Solutions are provided after the questions.

  1. Two groups have \(n_A=2\), \(n_B=5\), and \(e_{AB}=1\). What is their boundary density, and at what CPM resolution does the merge comparison change sign?
  2. For modularity with \(m=20\), \(K_A=8\), \(K_B=10\), \(e_{AB}=2\), and \(\gamma=1\), calculate \(\Delta Q\). Does this comparison favor merging?
  3. Explain why a boundary consisting of one edge is not enough information to decide whether CPM favors a merge.
  4. In the two-triangle example, why can the modularity and CPM comparisons favor different candidate partitions without either calculation being inconsistent?

Solutions. (1) The boundary density is \(1/(2\cdot5)=0.1\). CPM’s merge change is zero at \(\gamma=e_{AB}/(n_A n_B)=0.1\); it is positive below 0.1 and negative above 0.1. (2) \(\Delta Q=2/20-(8)(10)/(2(20)^2)=0.1000-0.1000=0\). The two candidate partitions tie under this comparison. (3) CPM uses \(\Delta Q_{\mathrm{CPM}}=e_{AB}-\gamma n_A n_B\), so the group sizes and resolution are needed as well as the edge count. (4) The objectives use different reference points: modularity includes a degree-based expected-edge penalty scaled by the whole graph, while CPM penalizes possible cross-pairs by its own resolution. Different comparisons are therefore expected and do not indicate an arithmetic contradiction.

Clinical Biostats