Worked GSEA Analysis of a Ranked Expression Study

From a Ranked Gene List to a Pathway Result

A ranked enrichment analysis asks whether the genes in a specified pathway tend to accumulate near one end of a gene list ordered by a gene-level statistic. Unlike over-representation analysis, it does not first divide genes into “significant” and “not significant” groups. The ranking retains information about the relative direction and strength of each gene-level result, so a pathway can show a coordinated trend even when many of its genes do not pass a gene-level cutoff.

The result depends on the ranked list and its orientation. For example, if positive statistics mean greater expression in condition A than condition B, a positive pathway enrichment score indicates that pathway genes tend to occur toward the A-associated end of the list. It does not establish that the pathway causes a change in expression or that every member gene changes in the same way.

Definition: Let \(N\) genes be ordered from rank 1 to rank \(N\), with gene-level statistics \(r_1,\ldots,r_N\). For a gene set \(S\) of size \(N_H\), define a running sum \(RS(i)\), beginning at \(RS(0)=0\). At a hit, meaning gene \(i\) is in \(S\), the sum increases by a normalized weight; at a miss, meaning gene \(i\) is not in \(S\), it decreases by \(1/(N-N_H)\). With weighting exponent \(p\geq 0\), the hit weight is \(|r_i|^p\), normalized by the sum of these weights over genes in \(S\). The enrichment score (ES) is the running-sum deviation with greatest absolute magnitude: the maximum running sum if the greatest deviation is positive, or the minimum running sum if the greatest deviation is negative.

For \(p>0\), the hit increment at rank \(i\) is \(|r_i|^p/\sum_{j\in S}|r_j|^p\). For \(p=0\), each hit receives \(1/N_H\), the unweighted version. A miss has the same decrement at every rank. If \(p>0\) and \(r_i=0\) for a hit, its increment is zero: the running sum stays where it is at that rank. It is incorrect to say that every hit necessarily increases the sum. The weighted procedure requires a positive total hit weight, \(\sum_{j\in S}|r_j|^p>0\); if all hit weights are zero, that weighted score is undefined. The unweighted version does not have this problem for a nonempty set.

Calculate the Running Sum and Enrichment Score

The running sum compares the observed placement of the pathway genes with a pattern in which hits and misses are interspersed. Since the miss steps sum to \(-1\) and the normalized hit steps sum to \(+1\), the running sum returns to zero at the end of the list. Its largest upward or downward excursion determines the ES. The sign records which end of the ranked list has the stronger concentration.

Worked Example: A Zero-Weight Hit in a Weighted Running Sum

Consider six hypothetical genes with ranked statistics \(2, 0, -1, -2, -3, -4\), ordered from highest to lowest. Let the pathway contain the genes at ranks 1, 2, and 4. Use \(p=1\). The pathway hit weights are \(2,0,2\), with total weight \(4\), and there are three misses, each with decrement \(1/3\).

At rank 1, the hit adds \(2/4=0.5\), so \(RS(1)=0.5\). Rank 2 is a hit with statistic zero, so it adds \(0/4=0\) and \(RS(2)=0.5\). Rank 3 is a miss, giving \(RS(3)=0.5-1/3=0.1667\). Rank 4 is a hit and adds \(2/4=0.5\), giving \(RS(4)=0.6667\). The remaining two misses give \(RS(5)=0.3333\) and \(RS(6)=0\).

The largest deviation is the positive maximum, \(ES=0.6667\), reached at rank 4. This example has a positive-weight hit and a zero-weight hit; the latter leaves the running sum unchanged. The total hit weight is positive, so the weighted calculation is defined.

The pathway's ES is a descriptive statistic. To assess whether it is unusually large under a specified null, compare it with scores from permuted gene sets of the same size. This is the gene-set permutation approach: the ranked statistics remain fixed while pathway membership is reassigned among the genes eligible for the analysis. It therefore tests whether the set is unusually concentrated relative to the rest of that ranked universe, under the gene-set exchangeability assumptions discussed in the earlier material on gene-set permutations and their limitations.

Permutation p-value: For a positive observed ES, the one-sided empirical p-value is the fraction of null gene sets whose ES is at least as large as the observed ES. For a negative observed ES, it is the fraction whose ES is at least as negative. If \(B\) random permutations are used and \(b\) are at least as extreme in the observed direction, a common finite-simulation estimate is \((b+1)/(B+1)\). If all possible sets of the required size are enumerated, use the exact fraction of those sets at least as extreme, including the observed set. The p-value measures extremeness under this null; it is not the probability that the null hypothesis is true.

Worked Ranked-Study Analysis and Permutation Test

The following small hypothetical ranked study makes the complete calculation visible. In a real analysis, the list would usually contain many more genes, and permutation resolution would depend on the number of permutations. Here, a short list allows exact enumeration of the gene-set null.

Worked Example: Weighted ES and an Exact Gene-Set Permutation p-Value

Suppose six hypothetical genes are ranked by a signed statistic, with positive values indicating higher expression in condition A than condition B. The ordered values are \(3,2,1,-1,-2,-3\), and the pathway consists of the genes at ranks 1 and 2. Use \(p=1\). Its hit weights sum to \(3+2=5\), so the two hit increments are \(3/5=0.6\) and \(2/5=0.4\). The four misses each decrease the running sum by \(1/4=0.25\).

The running sums are \(0.6\) after rank 1, \(1.0\) after rank 2, \(0.75\) after rank 3, \(0.50\) after rank 4, \(0.25\) after rank 5, and \(0\) after rank 6. Therefore \(ES=1.0\), a positive maximum at rank 2. The two pathway genes are both at or before that maximum, so both are in the positive leading edge.

For the exact gene-set permutation test, hold the ranked statistics fixed and consider every possible set of two genes from the six ranks. There are \(\binom{6}{2}=15\) such sets. Each candidate set gets its own hit weights from the statistics at its selected ranks, while each of its four misses has step \(-0.25\). The resulting ES values are:

Selected ranks	ES	Selected ranks	ES	Selected ranks	ES
1, 2	1.00	1, 3	0.75	1, 4	0.75
1, 5	0.60	1, 6	0.50	2, 3	0.75
2, 4	0.50	2, 5	0.25	2, 6	-0.60
3, 4	0.50	3, 5	-0.50	3, 6	-0.75
4, 5	-0.75	4, 6	-0.75	5, 6	-1.00

Only one of the 15 possible sets has an ES at least as large as the observed \(1.00\). Thus the exact positive-direction permutation p-value is \(1/15=0.0667\). At \(\alpha=0.05\), this small example does not provide sufficient evidence against the gene-set permutation null. The observed ES is the most extreme positive value in this finite null, but the attainable p-values are coarse because only 15 sets were enumerated.

The example tests concentration toward the positive end of this particular ranking. Its direction is interpretable only because the sign convention for the gene-level statistic was specified. If the statistic had instead been defined as condition B minus condition A, the same ranking would have the opposite biological interpretation.

Direction and Leading-Edge Genes

The leading edge is the subset of pathway genes that contributes to the ES excursion. For a positive ES, it consists of the pathway hits encountered up to the rank where the running sum reaches its maximum. For a negative ES, it consists of pathway hits from the rank of the minimum through the bottom of the ranked list. These definitions connect the pathway-level result to the genes that drive its running-sum extreme. A leading-edge gene is not thereby established as a causal regulator or as individually significant.

Worked Example: Negative Enrichment and Its Leading Edge

Consider five hypothetical genes ranked by a signed statistic in descending order, where positive still means higher expression in condition A. Let a pathway contain the genes at ranks 4 and 5, and use the unweighted version \(p=0\). Each hit adds \(1/2=0.5\), and each of the three misses subtracts \(1/3\).

The running sum after the first three misses is \(-1/3,-2/3,-1\). The hit at rank 4 raises it to \(-0.5\), and the hit at rank 5 raises it to \(0\). The minimum is \(-1\), so \(ES=-1\). Both pathway genes occur after the minimum and are therefore in the negative leading edge.

This negative ES says that the pathway genes in this example lie toward the low-statistic end of the ranking. Given the stated orientation, that end is associated with lower expression in condition A relative to condition B. It does not mean the pathway is “inhibited” in a mechanistic sense; the ranking supports a directional expression association, not a causal conclusion.

Assumptions, Diagnostics, and Alternatives

A GSEA result is conditional on several choices and assumptions. The ranked list should include the genes eligible for the analysis, each gene should appear once, and the pathway membership should be mapped to the same gene identifiers and universe. The ranking statistic must be defined consistently and oriented before interpreting the sign. For the gene-set permutation null, candidate memberships are treated as exchangeable among eligible genes, conditional on set size. This can be questionable when genes differ systematically in measurement quality, expression level, or other properties related to their rank.

Check the ranked list, pathway size, and running-sum plot rather than relying on a p-value alone. Confirm that the running sum returns to zero, the reported ES equals the extreme running-sum value, and the leading edge matches the ranks on the relevant side of the extreme. Inspect whether ties are handled consistently: tied gene statistics can make their within-tie ordering arbitrary, and a tie-breaking rule may affect the apparent rank of a peak. Also verify that the selected weighting exponent and ranking statistic match the analysis specification.

Permutation resolution and the null definition matter. With a finite random sample of \(B\) permutations, the plus-one estimate avoids reporting a zero p-value; its smallest possible value is \(1/(B+1)\). Exact enumeration instead gives a resolution of one over the number of eligible sets. Gene-set permutations test a competitive question—whether this set is more concentrated than comparable sets from the ranked universe. They do not preserve the sample-level dependence structure in the way a phenotype- or sample-based procedure may, and neither null should be substituted for the other without considering the scientific question and design.

The ES is not directly comparable across pathways of different sizes or across different null distributions simply because its numerical scale is bounded by the running-sum construction. Some GSEA implementations report a normalized enrichment score based on a null distribution; that is a separate scaling step, not the raw ES calculated here. Finally, pathway tests are usually conducted for many gene sets. An individual permutation p-value does not account for that multiplicity; interpretation across a family of tests requires the multiple-testing approach appropriate to the analysis.

Common pitfall: “Every hit increases the running sum, so a zero-statistic hit raises the score.” This is wrong for weighted GSEA with \(p>0\): its increment is \(|r_i|^p/\sum_{j\in S}|r_j|^p=0\) when \(r_i=0\), so it leaves the sum unchanged. Misses decrease the sum; only positive-weight hits increase it. A zero-weight hit is allowed when the total hit weight is positive.

Key takeaway: A ranked GSEA analysis combines the oriented gene-level ranking, a specified hit-weighting rule, the running-sum extreme, and a clearly defined permutation null. Report the direction and leading edge alongside significance, and interpret them as pathway-level patterns in the ranked data.

Check Your Understanding

Try each question, then check the brief solutions below.

With \(p>0\), what happens to the running sum at a hit with \(r_i=0\), and when is this allowed?
A gene set has observed positive \(ES=0.42\). Of 200 random gene-set permutations, 11 have ES at least \(0.42\). Using the plus-one estimate, calculate the one-sided p-value.
For a negative ES, which pathway hits form the leading edge?
If positive ranking statistics indicate greater expression in condition B, what does positive enrichment indicate about the pathway's ranked genes?
Why can an exact permutation test based on 15 possible sets not give a p-value smaller than \(1/15\)?

Solutions. (1) It adds zero, so the running sum is unchanged; this is allowed if the total hit weight is positive. (2) \((11+1)/(200+1)=12/201=0.0597\). (3) Hits from the running-sum minimum through the bottom of the ranked list. (4) The pathway genes tend to occur toward the B-associated, high-statistic end; this is a directional expression association, not a causal claim. (5) The smallest nonzero exact tail fraction is one extreme set divided by all 15 sets, \(1/15=0.0667\).