Direction and Magnitude in Pathway-Level Results

Direction, Magnitude, and Evidence Answer Different Questions

A pathway result is not a single conclusion. Its direction describes which end of a comparison the pathway’s genes tend toward; its magnitude describes the size of an effect summary on a specified scale; and its p-value measures how incompatible the observed statistic is with a stated null model. These quantities are related, but none can stand in for the others.

This distinction matters especially in genomic studies. With many samples, a small but consistent expression shift may produce strong statistical evidence. Conversely, a pathway may contain genes with substantial changes in both directions, so a single signed summary near zero conceals meaningful heterogeneity. Interpretation must therefore begin by identifying the comparison, the statistic, its direction convention, and the null hypothesis.

Definition: A pathway-level result has a direction when its statistic or summary has a sign tied to an explicitly defined contrast; a magnitude is the size of that statistic or summary on its own scale; and a p-value is the probability, under the specified null model, of obtaining a statistic at least as extreme as the observed one. A p-value is not an effect-size measure and is not the probability that the null hypothesis is true.

For a ranked-list analysis, state how the list was oriented before interpreting a sign. If genes are ranked from the largest positive treatment-minus-control statistic to the largest negative, a positive GSEA enrichment score (ES) indicates that pathway members tend to occur toward the treatment-higher end of that list; a negative ES indicates concentration toward the control-higher end. Reversing the contrast reverses that interpretation. The sign does not, by itself, establish that a pathway is biologically activated or inhibited.

A normalized enrichment score (NES) is a version of ES adjusted relative to a null distribution, with the normalization depending on the analysis procedure. NES is useful for interpreting enrichment relative to that procedure’s null, but it is not in the original units of gene expression. In particular, an NES of 2 is not twice the biological effect of an NES of 1, and NES values from different procedures need not be directly comparable as effect sizes.

Summaries That Preserve More Than One Feature

When gene-level effects are available, several descriptive summaries can clarify what a pathway contains. Let \(\hat{\beta}_g\) be the estimated effect for gene \(g\), such as a treatment-minus-control log fold change, and let \(S\) be a pathway with \(m\) measured genes. The average signed effect is \(\bar{\beta}_S\); the proportion with positive estimated effects describes directional balance; and a root-mean-square (RMS) effect summarizes unsigned magnitude.

\bar{\beta}_S=\frac{1}{m}\sum_{g\in S}\hat{\beta}_g,\qquad \hat{p}_{+}=\frac{1}{m}\sum_{g\in S} I(\hat{\beta}_g>0),\qquad \operatorname{RMS}_S=\sqrt{\frac{1}{m}\sum_{g\in S}\hat{\beta}_g^2}.

Here \(I(\hat{\beta}_g>0)\) equals 1 when the estimated effect is positive and 0 otherwise. The average signed effect can be small because individual effects are small, because positive and negative effects cancel, or both. The RMS is nonnegative and avoids cancellation, but it does not show which direction the changes take. These are descriptive summaries; they are not automatically formal tests, and their uncertainty depends on the study design and estimation procedure.

Key identity: If the pathway’s effects are treated as a finite collection, then \(\operatorname{RMS}_S^2=\bar{\beta}_S^2+\frac{1}{m}\sum_{g\in S}(\hat{\beta}_g-\bar{\beta}_S)^2\). Thus a large RMS can reflect a large common shift, substantial dispersion around a small mean, or both.

Worked Example: A Small Average Shift With a Small p-Value

Suppose a hypothetical study compares treatment with control and reports a pathway analysis based on phenotype permutations. The ranked list is oriented so positive gene statistics mean higher expression under treatment. For a pathway of 80 genes, the mean estimated treatment-minus-control effect is \(0.08\) log2 fold-change units, and the reported pathway p-value is \(0.001\). Assume the study design supports the permutations used and that the reported p-value is for the stated pathway statistic.

The signed summary is \(+0.08\) log2 fold-change units, so its direction is treatment-higher on average. The size of the average shift is \(0.08\) on the log2 fold-change scale. The p-value supplies separate evidence: under the specified permutation null, a result at least as extreme as the observed statistic occurred with probability \(0.001\). It does not say the pathway’s effect is large.

A small p-value can accompany this modest average if the pathway’s genes show a consistent ranking pattern and the study provides substantial precision. To show the scale of the average shift, convert it to a fold-change ratio: \(2^{0.08}\approx 1.057\), or about a 5.7% increase in the geometric expression ratio. This describes the average log2 effect, not necessarily every gene’s change. The correct conclusion is that the analysis found strong evidence against its null for a pathway-level pattern, while the average estimated shift was modest in magnitude.

Worked Example: Large Changes in Both Directions

Consider a hypothetical six-gene pathway with treatment-minus-control log2 fold-change estimates \(1.2,\ 1.0,\ 0.8,\ -0.9,\ -0.8,\ -0.7\). The positive estimates correspond to higher expression under treatment; the negative estimates correspond to lower expression.

The signed mean is \((1.2+1.0+0.8-0.9-0.8-0.7)/6=0.6/6=0.10\) log2 fold-change units. The positive proportion is \(3/6=0.50\). The RMS is \(\sqrt{(1.44+1.00+0.64+0.81+0.64+0.49)/6}=\sqrt{5.02/6}\approx 0.915\) log2 fold-change units.

The mean is close to zero because effects in opposite directions cancel, not because every effect is close to zero. The RMS and directional proportion reveal that changes are substantial and evenly split by sign. A single positive pathway label would therefore be misleading. A self-contained test of a null that all pathway effects are zero could, in principle, find evidence of changes without implying a common direction; its conclusion would depend on the test’s precise statistic and assumptions. This example’s summaries alone do not provide a p-value.

Why the Sign and the Null Must Be Explicit

An ES sign is meaningful only in relation to the ranking convention. The running-sum statistic compares the accumulation of pathway members with the accumulation of genes outside the set as the ranked list is traversed. Its signed maximum deviation indicates which end of the list shows greater concentration. If the list is reversed, the direction associated with a positive score reverses too. A reader should therefore report the contrast and which end represents positive effects, rather than reporting “positive enrichment” without context.

The null model also matters. A competitive test asks whether pathway genes show a more extreme pattern than genes outside the pathway, whereas a self-contained test asks whether the pathway itself shows evidence against a specified no-effect null. These tests address different questions, so their p-values should not be described as interchangeable evidence for a single biological claim. As established in the earlier material on competitive and self-contained tests, the conclusion follows from the null being tested—not merely from the word “enrichment.”

For GSEA, ES and NES are pathway-level statistics derived from the ranked list; a gene-level mean effect is a different summary. Agreement among the sign of ES, the sign of the pathway’s average effect, and the proportion of positive gene estimates is reassuring but not guaranteed. Differences can arise because these summaries weight genes differently or emphasize different parts of the distribution. The leading-edge genes can help locate the genes contributing strongly to the ranked-list signal, but they do not turn an enrichment statistic into a direct estimate of a pathway-wide biological effect.

Worked Example: Reading a Signed GSEA Result

Suppose a hypothetical analysis ranks genes by treatment-minus-control statistics, with positive values at the top. A pathway has \(ES=0.62\), \(NES=1.85\), and permutation \(p=0.006\). Assume the ranking and permutation scheme match the study design, and the p-value is calculated for the specified enrichment statistic.

Because ES is positive and the top of the list represents treatment-higher expression, pathway members are more concentrated toward the treatment-higher end. The positive NES has the same directional interpretation under this procedure’s normalization. The p-value indicates that under the specified null, a statistic at least as extreme as the observed one was uncommon. It is evidence about the enrichment statistic, not a statement that the pathway has a 1.85-fold expression effect.

Without gene-level effect summaries, this result does not quantify the average log2 fold change or the fraction of pathway genes that increased. Nor does it prove coordinated activation, establish causation, or show that every pathway member changed in the same direction. A careful report would give the contrast orientation, ES or NES, p-value, and the inferential null, and would describe the result as enrichment toward the treatment-higher end of the ranked list.

Diagnostics, Comparisons, and Limitations

Before interpreting a pathway-level result, check that gene identifiers were mapped consistently, the pathway genes were actually represented in the ranked or measured set, and the contrast orientation was documented. Examine the distribution of gene-level effect estimates when available: a mean near zero with a large RMS suggests cancellation or heterogeneity, while a mean and most gene signs pointing together supports a more coherent directional description. These checks explain a summary; they do not replace the inferential method.

Compare the available summaries according to the question they answer:

Quantity	What it describes	What it does not establish by itself
ES sign	Which end of the oriented ranked list has pathway-member concentration	A common gene-level effect or pathway activation
NES	Enrichment relative to a procedure-specific null normalization	An effect in expression units or a universally comparable magnitude
Mean signed gene effect	Average estimated change in the effect’s stated units	Whether the average is statistically distinguishable from zero
RMS gene effect	Unsigned size of the collection of gene estimates	The direction of the changes or their statistical significance
p-value	Compatibility of a statistic with a specified null model	Effect magnitude or the probability that the null is true

These summaries have limitations. A mean can conceal opposing changes; a directional fraction ignores effect sizes; RMS can be dominated by a few large estimates; and ES or NES depends on the ranking, pathway membership, and analysis procedure. Estimates can also be noisy, and genes within a pathway may be correlated. A pathway-level result is not automatically a causal or mechanistic result. Interpret it in the context of the design, the null, and the specific summary, and avoid claiming more than those quantities support.

Common Mistakes

“The p-value is small, so the pathway effect must be large.” A p-value measures evidence relative to a null under a model; it is not on an effect-size scale. Report a magnitude summary in stated units as well.
“Positive enrichment means activation.” Positive enrichment means concentration toward whichever end of the ranked list was defined as positive. It does not by itself establish pathway activation.
“The average is near zero, so nothing changed.” Opposing effects can cancel. Inspect directional balance and an unsigned summary before concluding that changes are uniformly small.
“The pathway p-value tests that all genes move in one direction.” That conclusion depends on the test statistic and null. State what was tested, and do not infer common direction from significance alone.
“An NES is a fold change.” NES is a normalized enrichment statistic, not an expression effect in fold-change units.

Key takeaway: Interpret direction relative to the contrast and ranking convention, magnitude on an explicitly stated scale, and significance relative to a stated null. When gene effects may be mixed, report a signed summary together with information that reveals their distribution.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions below.

A ranked list places positive treatment-minus-control statistics at the top. What does a negative ES indicate, and what happens if the contrast is reversed?
A pathway has a small p-value and an average estimated log2 fold change of \(0.03\). Which result describes evidence and which describes magnitude?
For effects \(0.6,\ 0.4,\ -0.5,\ -0.3\), calculate the signed mean and positive proportion. What feature does the mean conceal?
Why does an NES of 2 not mean twice the biological effect of an NES of 1?

Solutions. (1) A negative ES indicates concentration toward the negative end, here the control-higher end. Reversing the contrast reverses the end’s biological interpretation. (2) The p-value describes evidence relative to the specified null; \(0.03\) log2 fold-change units is the estimated average magnitude. A small p-value does not make that effect large. (3) The mean is \((0.6+0.4-0.5-0.3)/4=0.2/4=0.05\); the positive proportion is \(2/4=0.50\). The near-zero mean conceals effects in both directions. (4) NES is a procedure-normalized enrichment statistic, not a measurement in biological effect units, so its numerical ratio is not a ratio of pathway effects.