Why the Ranking Metric Matters
Ranked enrichment methods use the order of genes, not just a list of genes that pass a significance threshold. The ranking metric therefore determines which genes appear near the top and bottom of the list, and how strongly those positions reflect effect size, uncertainty, and direction. A pathway can move in the ranking, and its enrichment result can change, when a different defensible metric is used.
The choice is not simply between a “good” and “bad” statistic. A log fold change emphasizes estimated magnitude. A test statistic combines estimated effect with its uncertainty. A transformed p-value emphasizes evidence against a gene-level null hypothesis. Each can be useful, but each answers a somewhat different ranking question.
Define the Contrast and the Gene-Level Score
For each gene \(i\), suppose an expression analysis estimates a contrast \(\hat{\beta}_i\), such as the difference in mean log expression between treatment and control. The sign convention must be stated: here, \(\hat{\beta}_i>0\) means higher expression in treatment than control, and \(\hat{\beta}_i<0\) means lower expression in treatment. The standard error \(SE_i\) measures the estimated uncertainty in \(\hat{\beta}_i\).
Three common choices are:
- Log fold change: rank by \(\hat{\beta}_i\), often a log-base-2 fold change. This emphasizes estimated direction and magnitude, but does not directly account for uncertainty.
- Signed test statistic: rank by a statistic such as \(T_i=\hat{\beta}_i/SE_i\), with an appropriate \(t\)- or normal reference distribution under the gene-level null. It combines effect size and precision.
- Signed transformed p-value: rank by \(\operatorname{sign}(\hat{\beta}_i)[-\log_{10}(p_i)]\), where \(p_i\) is a two-sided gene-level p-value. This retains direction while expressing strength of evidence on a logarithmic scale.
These definitions assume that the genes are tested for the same stated contrast, that the sign is aligned across genes, and that the expression model and its standard errors are appropriate for the data. A test statistic is only as reliable as its model-based uncertainty estimate. A p-value transformation does not correct a misspecified model, and a fold change does not become precise merely because it is large.
What Each Metric Emphasizes
| Metric | What it emphasizes | Main risk when used alone |
|---|---|---|
| Log fold change | Estimated effect magnitude and direction | Can rank noisy, extreme estimates highly |
| Signed test statistic | Effect relative to its estimated uncertainty | Can favor small effects estimated very precisely |
| Signed transformed p-value | Evidence against the gene-level null, with direction retained | Does not directly communicate effect magnitude; can be sensitive to p-value calculation and numerical limits |
A fold change is often intuitive because it describes estimated magnitude on the expression scale. If the contrast is a log-base-2 fold change, a value of 1 corresponds to an estimated twofold expression ratio. But genes measured with different precision can have the same estimated fold change and very different evidence. Conversely, a very small effect can be estimated precisely enough to have a large test statistic.
For two-sided tests with the same reference distribution, the transformed p-value and the absolute test statistic produce the same ordering by strength of evidence: larger \(|T_i|\) gives a smaller p-value and a larger \(-\log_{10}(p_i)\). Adding the sign of \(\hat{\beta}_i\) makes the transformed score increase in the positive direction and decrease in the negative direction. When genes have different degrees of freedom or different testing procedures, transformed p-values and test statistics need not have identical orderings.
Worked Example: Precision Changes the Order
Suppose two hypothetical genes are tested for the same treatment-minus-control contrast. Use a normal reference distribution for illustration. Gene A has \(\hat{\beta}_A=0.80\) log-base-2 units and \(SE_A=0.50\); gene B has \(\hat{\beta}_B=0.60\) log-base-2 units and \(SE_B=0.15\).
By fold change, gene A ranks above gene B because \(0.80>0.60\). Their test statistics are:
Thus, by the signed test statistic, gene B ranks above gene A. For two-sided normal tests, the p-values are \(p_A=2[1-\Phi(1.60)]\approx0.1095986\) and \(p_B=2[1-\Phi(4.00)]\approx0.00006334\). The signed transformed scores are approximately \(+0.960\) and \(+4.198\), respectively, since \(-\log_{10}(0.1095986)\approx0.960195\) and \(-\log_{10}(0.00006334)\approx4.198\).
The conclusions differ because gene A has the larger estimated effect, while gene B’s smaller effect is estimated much more precisely. Neither ordering is automatically the right one for every question: the analyst should decide whether the ranking is intended to emphasize estimated magnitude or evidence relative to uncertainty.
Direction Must Be Preserved Deliberately
A two-sided p-value is unsigned: it is small for strong evidence in either direction. Ranking genes by \(-\log_{10}(p_i)\) alone therefore places strongly up-regulated and strongly down-regulated genes together near the top. That ordering can be appropriate for an undirected question about perturbation, but it does not distinguish enrichment at the positive end from enrichment at the negative end.
For directional enrichment, preserve the sign. With a contrast defined as treatment minus control, positive signed scores indicate higher expression in treatment and negative scores indicate lower expression. Reversing the contrast reverses all signs. The same convention must be used for every gene in the list; otherwise the ends of the ranking have no consistent biological interpretation.
Worked Example: Making a Two-Sided p-Value Directional
Consider two hypothetical genes tested using a normal reference distribution. Gene C has \(z_C=2.00\), and gene D has \(z_D=-3.00\). Their two-sided p-values are approximately \(0.0455003\) and \(0.0026998\).
The transformed evidence scores are:
If the scores are unsigned, gene D appears above gene C because \(2.569>1.342\), but the ranking does not show that D is negative. Applying the effect signs gives approximately \(+1.342\) for C and \(-2.569\) for D. When genes are sorted from largest to smallest signed score, C lies toward the positive end and D toward the negative end. The sign says which direction the estimated contrast points; the magnitude reflects evidence, not the size of the expression change.
A Short Justification for the Transformed Score
The direction and ordering properties of the signed transformed p-value follow from the two-sided normal p-value. For a gene with statistic \(z_i\), the p-value is \(p_i=2[1-\Phi(|z_i|)]\). The standard normal cumulative distribution function \(\Phi\) increases with its argument, so \(p_i\) decreases as \(|z_i|\) increases. Since \(-\log_{10}(p_i)\) also increases as \(p_i\) decreases, the transformed score increases with the magnitude of the statistic.
For positive \(z_i\), increasing \(z_i\) makes \(S_i\) larger. For negative \(z_i\), increasing \(|z_i|\) makes \(S_i\) more negative. Thus, sorting \(S_i\) from largest to smallest orders positive evidence toward the top and negative evidence toward the bottom. This is an ordering property, not a claim that the score is an effect estimate or a probability that the gene is truly differentially expressed.
Magnitude, Uncertainty, and Rank Position
Worked Example: Fold Change and Test Statistic Answer Different Questions
Suppose two other hypothetical genes have the same treatment-minus-control contrast. Gene M has a log-base-2 fold change of \(1.50\) with \(SE_M=0.75\); gene N has a fold change of \(0.90\) with \(SE_N=0.225\). Use normal statistics and two-sided tests.
Fold-change ranking places M above N because \(1.50>0.90\). The standardized statistics are:
The corresponding p-values are approximately \(0.0455003\) and \(0.00006334\), giving signed transformed scores of \(+1.342\) and \(+4.198\). The test-statistic and transformed-p rankings therefore place N above M. Both estimated effects are positive, but N has a smaller effect with substantially lower uncertainty.
If N belongs to a pathway and M does not, changing from fold-change ranking to test-statistic ranking can move a pathway hit closer to the positive end of the list. That shift can affect the running-sum enrichment score described in earlier material on GSEA. It does not, by itself, establish that the pathway has a causal role or that its member genes have large effects.
Diagnostics, Alternatives, and Limitations
A ranking should be checked before it is used for pathway interpretation. Compare the top and bottom genes under at least one alternative defensible metric. Large changes in gene order can identify genes for which precision, rather than estimated magnitude, is driving the result. Inspect whether the signs agree with the contrast definition, whether a few extreme values dominate the ends, and whether many genes are tied or missing. If a metric is based on p-values, ensure those p-values are finite and were calculated from the intended gene-level tests.
Extreme transformed scores require particular care. A computer may report a p-value as zero when it is too small to represent numerically. Then \(-\log_{10}(0)\) is infinite, which is not a meaningful finite ranking value. Use a numerically stable tail-probability calculation or the test statistic itself rather than treating zero as an exact p-value. Ties can also result from rounded p-values or discrete tests; arbitrary tie-breaking can change local rank order, so report the rule used if ties matter.
Some differential-expression procedures moderate or shrink gene-level estimates or variances to stabilize noisy values. Such methods can be useful when uncertainty estimates are unstable, but they do not remove the need to state the resulting ranking metric and its sign convention. A raw fold change is especially vulnerable to extreme estimates from genes with little information. A signed test statistic can be dominated by very precise small effects. A transformed p-value emphasizes evidence and can obscure whether the estimated effect is biologically substantial.
No ranking metric solves every problem. In particular, a ranked enrichment analysis should use the full set of genes tested in the expression analysis, not only genes that passed an arbitrary significance cutoff; filtering first can remove relevant genes and alter ranks and enrichment. The selected metric should match the scientific emphasis, and sensitivity to reasonable alternatives should inform interpretation. The enrichment result remains dependent on the ranking, the gene set, and the underlying expression model.
Common Mistake
Check Your Understanding
Answer the questions, then compare your reasoning with the solutions below.
- Two genes have estimated effects of \(0.70\) and \(0.50\), with standard errors \(0.35\) and \(0.10\), respectively. Which ranks higher by effect and which by the signed test statistic?
- Why can ranking genes by an unsigned \(-\log_{10}(p_i)\) be unsuitable for directional enrichment?
- A gene has a treatment-minus-control estimate of \(-0.40\) and a two-sided p-value of \(0.01\). What is its signed transformed score, approximately?
- What should be done if software returns a p-value of zero for a gene?
Solutions. (1) The first gene ranks higher by effect because \(0.70>0.50\). Its statistic is \(0.70/0.35=2.00\), while the second gene’s is \(0.50/0.10=5.00\), so the second ranks higher by signed test statistic. (2) The unsigned score treats strong evidence in either direction as positive, so it does not separate up-regulation from down-regulation. (3) The magnitude is \(-\log_{10}(0.01)=2\); the negative estimate gives a signed score of \(-2\). (4) Do not treat zero as an exact p-value or assign an infinite score. Calculate the tail probability with a numerically stable method or use the corresponding test statistic for ranking.