Why the GSEA Enrichment Score Is the Maximum Deviation

From a Running Path to an Enrichment Score

A ranked gene list gives an order to all genes in the tested universe, while a pathway or gene set identifies a subset of those genes. The running-sum statistic tracks whether set members are accumulating faster or slower than genes outside the set as we move down the ranking. The enrichment score (ES) reduces that whole path to its most pronounced departure from zero. Its sign indicates whether the greatest excursion is upward or downward; its magnitude records how far the path moved from zero.

The score is not the final value of the running sum. Under the standard unweighted construction, that final value is zero by design. Nor is ES the sum of all running-sum values. It is selected from the running sum's extremes across the ranked list.

Definition: Suppose the ranked list contains $N$ genes, of which $N_H$ belong to the gene set and $N-N_H$ do not. Let $H_k$ be the number of set members among the first $k$ genes. The unweighted running sum after rank $k$ is $$ R(k)=\frac{H_k}{N_H}-\frac{k-H_k}{N-N_H}, \qquad R(0)=0. $$ A set-member hit increases $R(k)$ by $1/N_H$; a gene outside the set decreases it by $1/(N-N_H)$. Define $R_{\max}=\max_{0\leq k\leq N}R(k)$ and $R_{\min}=\min_{0\leq k\leq N}R(k)$. The signed enrichment score is $R_{\max}$ if $R_{\max}\geq |R_{\min}|$, and $R_{\min}$ otherwise.

This definition assumes that the list contains a well-defined set of $N$ tested genes, that the gene set is represented in that list, and that it is neither empty nor the entire list: $1\leq N_H<N$. These conditions make both step sizes defined. The ranking and the mapping of genes to set membership must also be specified. If ranking statistics are tied, a tie-breaking rule can affect the order of hits and therefore the path; that choice should be consistent and documented.

The score is signed: a positive ES means the largest-magnitude excursion is above zero, and a negative ES means the largest-magnitude excursion is below zero. The score alone does not give a p-value or establish statistical significance. Significance requires a reference distribution and an appropriate null procedure, which are separate from choosing the running sum's extreme.

Why the Extreme Is a Departure

The definition can be understood by comparing the observed accumulation of set members with proportional accumulation across ranks. After the first $k$ genes, the fraction of all set members encountered is $H_k/N_H$. The fraction of the ranked list traversed is $k/N$. If set members were accumulating in exact proportion to rank position, these two fractions would match. Algebra relates their difference directly to the running sum:

\begin{aligned} R(k) &=\frac{H_k}{N_H}-\frac{k-H_k}{N-N_H}\\ &=\frac{N H_k-kN_H}{N_H(N-N_H)}\\ &=\frac{N}{N-N_H}\left(\frac{H_k}{N_H}-\frac{k}{N}\right). \end{aligned}

Thus, at each rank $k$, the running sum is a positive rescaling of the difference between the observed fraction of set members encountered and the fraction of the list traversed. A positive value means set members have accumulated ahead of that proportional benchmark; a negative value means they have lagged behind it. Selecting the greatest signed excursion asks where that difference is most pronounced, considering both directions.

At $k=0$, both fractions are zero. At $k=N$, both are one, so $R(0)=R(N)=0$. Between those endpoints, hits and misses move the path in opposite directions. The path's extreme, rather than its endpoint, contains the information about where the set members occur.

Worked Example: Set Members Cluster Near the Top

Consider a hypothetical ranked list of $N=8$ genes. Three belong to the set, at ranks 1, 2, and 4. Each hit adds $1/3$, and each of the five misses subtracts $1/5$. The running sum is:

Rank $k$	Gene status	Running sum $R(k)$
0	Start	0
1	Hit	$1/3=0.3333$
2	Hit	$2/3=0.6667$
3	Miss	$2/3-1/5=0.4667$
4	Hit	$2/3-1/5+1/3=0.8000$
5	Miss	$0.8000-0.2000=0.6000$
6	Miss	$0.6000-0.2000=0.4000$
7	Miss	$0.4000-0.2000=0.2000$
8	Miss	$0.2000-0.2000=0$

The largest positive value is $0.8000$ at rank 4; the smallest value is zero. Therefore $R_{\max}=0.8000$, $R_{\min}=0$, and $ES=0.8000$. The set members' early positions create a strong upward excursion before the remaining misses return the path to zero.

Worked Example: Set Members Cluster Near the Bottom

Now suppose a hypothetical list again has $N=8$ genes and $N_H=3$, but the hits are at ranks 6, 7, and 8. A miss contributes $-1/5$, while a hit contributes $1/3$. The first five ranks are misses, so the running sum falls by $1/5$ at each step:

R(1)=-0.2000,\quad R(2)=-0.4000,\quad R(3)=-0.6000,\quad R(4)=-0.8000,\quad R(5)=-1.000.

The three hits then give $R(6)=-1+1/3=-0.6667$, $R(7)=-1+2/3=-0.3333$, and $R(8)=-1+3/3=0$. Here $R_{\min}=-1.000$, while $R_{\max}=0$. Since the negative extreme has the larger absolute value, $ES=-1.000$. The sign records that the set members accumulate late in the ranking; the magnitude is the greatest downward departure.

Sets Distributed Through the List

A set whose members are spread through the ranked list will often have smaller excursions than a set concentrated at one end. But “spread throughout” does not mean every running sum is exactly zero. The order of the hits and misses still determines the path, and even a roughly distributed set can have a local positive or negative extreme.

Worked Example: Set Members Occur Throughout the List

Consider a hypothetical list with $N=9$ genes, $N_H=3$, and hits at ranks 2, 4, and 7. Each hit adds $1/3$, and each of the six misses subtracts $1/6$. Calculating step by step:

\begin{aligned} R(1)&=-1/6=-0.1667,\\ R(2)&=-1/6+1/3=0.1667,\\ R(3)&=0.1667-1/6=0,\\ R(4)&=0+1/3=0.3333,\\ R(5)&=0.3333-1/6=0.1667,\\ R(6)&=0.1667-1/6=0,\\ R(7)&=0+1/3=0.3333,\\ R(8)&=0.3333-1/6=0.1667,\\ R(9)&=0.1667-1/6=0. \end{aligned}

The maximum is $R_{\max}=0.3333$, and the minimum is $R_{\min}=-0.1667$. Since $0.3333>|-0.1667|$, $ES=0.3333$. The result is positive, but its excursion is modest compared with the top-clustered example. This calculation does not by itself establish whether the observed score is unusual under a null model.

How to Calculate and Check the Score

A reliable calculation separates the path from the final selection of its extreme. The following procedure is sufficient for the unweighted running sum defined here.

Check the list and set.
Confirm the list length $N$, the set size $N_H$, and that $1\leq N_H<N$. Identify hits and misses in rank order.

Apply the correct step size.
Add $1/N_H$ for each hit and subtract $1/(N-N_H)$ for each miss, retaining enough precision during calculation.

Record every partial sum.
Include the starting value $R(0)=0$ and the value after every rank. Do not inspect only the value at a selected rank or the end.

Select the signed extreme.
Compare $R_{\max}$ with $|R_{\min}|$. Keep the positive maximum if it is at least as large; otherwise keep the negative minimum.

Verify the endpoint.
After all genes are processed, the hits contribute $N_H(1/N_H)=1$ and misses contribute $(N-N_H)(-1/(N-N_H))=-1$. Hence $R(N)=0$, apart from display rounding.

Diagnostics, Alternatives, and Limits

The endpoint check is a useful arithmetic diagnostic, but it does not prove every intermediate step is correct. If $R(N)$ is not approximately zero, check that the number of hit and miss steps matches $N_H$ and $N-N_H$, and that the appropriate increment was used for each. A common error is to subtract the same amount at every miss as is added at a hit. The increments generally differ because their denominators are $N-N_H$ and $N_H$, respectively.

Another mistake is to report only the largest positive value. If the path falls farther below zero than it rises above it, the ES is negative. For example, if $R_{\max}=0.25$ and $R_{\min}=-0.60$, the score is $-0.60$, not $0.25$ and not $0.60$. Taking the absolute value alone loses the direction encoded by the sign.

The maximum-deviation summary uses the most extreme point of the whole path. That differs from the terminal sum, which is always zero, and from a sum or average of the path values, which would answer a different question and could allow positive and negative portions to cancel. It also differs from threshold-based over-representation analysis: a thresholded analysis asks about membership in a selected subset, whereas the running-sum score uses positions across the ranked list. The ES is a descriptive statistic of that ordering, not a probability, adjusted p-value, or proof of a biological mechanism.

The score also compresses information. Two different hit patterns can produce the same ES, even though their full running-sum paths differ. Rank ties, gene-to-set mapping decisions, and the precise tested-gene universe can change the path. In addition, the equal hit increment used in this tutorial is one particular running-sum construction; alternative contributions change the path calculation and must be stated explicitly. Regardless of the chosen construction, the score should be interpreted according to the running sum actually defined and computed.

Key takeaway: The GSEA enrichment score is the running sum's largest signed excursion from zero: the positive maximum when it dominates in magnitude, or the negative minimum when that dominates. Always calculate and inspect the complete path before selecting the score.

Check Your Understanding

Answer each question, then compare your work with the solutions below.

A list has $N=6$ genes and a set of $N_H=2$ genes at ranks 2 and 5. Calculate the running sum at each rank and find ES.
For a running sum with $R_{\max}=0.42$ and $R_{\min}=-0.51$, what is ES, and what does its sign indicate?
Why must the final running sum be zero under the stated unweighted definition?
A student reports the largest positive running-sum value without checking the negative minimum. What information might be lost, and how should the score be selected?

Solutions. (1) Each hit adds $1/2$, and each of four misses subtracts $1/4$. Thus $R(1)=-0.25$, $R(2)=0.25$, $R(3)=0$, $R(4)=-0.25$, $R(5)=0.25$, and $R(6)=0$. The positive and negative extremes have equal magnitude, so under the stated tie convention $ES=0.25$. (2) Since $0.51>0.42$, $ES=-0.51$; the greatest-magnitude excursion is downward, indicating that set members lag the proportional accumulation benchmark at that extreme. (3) All $N_H$ hits add $N_H(1/N_H)=1$, and all $N-N_H$ misses subtract $(N-N_H)/(N-N_H)=1$, so their total change is zero. (4) It could miss a larger-magnitude negative excursion and therefore misstate both the score's magnitude and direction. Compare $R_{\max}$ with $|R_{\min}|$ and retain the larger-magnitude signed extreme.

Rank \(k\)	Gene status	Running sum \(R(k)\)
0	Start	0
1	Hit	\(1/3=0.3333\)
2	Hit	\(2/3=0.6667\)
3	Miss	\(2/3-1/5=0.4667\)
4	Hit	\(2/3-1/5+1/3=0.8000\)
5	Miss	\(0.8000-0.2000=0.6000\)
6	Miss	\(0.6000-0.2000=0.4000\)
7	Miss	\(0.4000-0.2000=0.2000\)
8	Miss	\(0.2000-0.2000=0\)