From a Differential-Expression List to a Pathway Test
Over-representation analysis (ORA) asks whether a pathway contains more genes from a selected list than would be expected under a specified chance model. In a differential-expression analysis, the list is usually formed by applying a decision rule—such as an adjusted-p-value threshold—to genes that were tested. The choice of which genes were eligible for selection is therefore part of the analysis, not a bookkeeping detail.
The central comparison is between the selected genes in a pathway and the selected genes outside it. ORA does not ask whether any gene in the pathway is associated with the outcome, nor does it by itself estimate the pathway’s direction or magnitude of biological activity. Its standard hypergeometric test is a competitive test: it compares the pathway with genes outside the pathway in the defined universe.
Before calculating a p-value, the universe must represent genes that could actually have entered the selected list. In differential-expression analysis this ordinarily means genes that passed the study’s measurement and testing requirements—not every gene appearing in an annotation database. The pathway membership count \(K\) must also be calculated after restricting membership to that same universe. Each gene should be represented consistently, with duplicate or ambiguous identifiers handled before counting.
The Contingency Table and Hypergeometric Tail
The overlap can be displayed in a 2-by-2 table. The rows indicate pathway membership, and the columns indicate whether a gene was selected. The table makes clear that the comparison is against the genes outside the pathway, not against an abstract list of all genes in biology.
| Selected | Not selected | Total | |
|---|---|---|---|
| In pathway | \(x\) | \(K-x\) | \(K\) |
| Outside pathway | \(n-x\) | \(N-K-n+x\) | \(N-K\) |
| Total | \(n\) | \(N-n\) | \(N\) |
For over-representation, the alternative is that the overlap is unusually large. Under the null model, the probability of exactly \(j\) selected genes in the pathway is the number of ways to choose \(j\) genes from the \(K\) pathway genes and \(n-j\) genes from the \(N-K\) genes outside it, divided by the number of ways to choose any \(n\) genes from the universe.
The p-value is the probability, under this null model, of an overlap at least as large as the observed one. It is not the probability that the null hypothesis is true. The same one-sided calculation can be obtained from a one-sided Fisher exact test on the table. A two-sided Fisher test answers a different question and is not the usual test for over-representation.
Worked Example: An Exact Over-Representation Test
Suppose a hypothetical differential-expression analysis tests \(N=20\) genes and selects \(n=5\) using a prespecified threshold. A hypothetical DNA-repair pathway contains \(K=4\) of the tested genes, of which \(x=3\) are selected. Assume the identifiers are unique and the pathway membership and selection rule were fixed before this calculation.
The table is:
| Selected | Not selected | Total | |
|---|---|---|---|
| In pathway | 3 | 1 | 4 |
| Outside pathway | 2 | 14 | 16 |
| Total | 5 | 15 | 20 |
Under the uniform-selection null, \(X\) can equal 3 or 4 for an overlap at least this large. There are \(\binom{20}{5}=15{,}504\) possible selected sets. The number with exactly three pathway genes is \(\binom{4}{3}\binom{16}{2}=4(120)=480\); the number with all four pathway genes is \(\binom{4}{4}\binom{16}{1}=1(16)=16\). Thus
This overlap is more extreme than expected under the specified random-selection model, with a raw one-sided p-value of about 0.0320. It is not, by itself, evidence that DNA repair is activated, that the pathway caused the measured phenotype, or that the result remains significant after testing many pathways.
Adjusting Across Pathways
An analysis typically tests many pathways, so a small raw p-value must be interpreted in light of the full family of tests. The number of tests is the number of pathways actually evaluated under the stated filtering and mapping rules; it is not just the number eventually reported as interesting. For controlling the false discovery rate (FDR), the Benjamini–Hochberg procedure orders the \(m\) raw p-values, compares them with rank-specific thresholds, and identifies the largest rank \(i\) satisfying \(p_{(i)}\le i\alpha/m\). The hypotheses at ranks 1 through \(i\) are rejected. Its usual FDR guarantee applies under independence and also under certain forms of positive dependence; overlapping pathways can make dependence relevant.
An equivalent reporting convention is to give Benjamini–Hochberg adjusted p-values, often called BH q-values. For a p-value at rank \(i\), the adjusted value is the smallest of \(m p_{(j)}/j\) over ranks \(j\ge i\), capped at 1. These values make it possible to state which pathways meet a chosen FDR threshold without confusing an unadjusted p-value with a multiplicity-adjusted result.
Worked Example: Adjusting Five Pathway Tests
Consider a hypothetical analysis in which five prespecified pathways were tested on a common universe. Suppose their exact ORA p-values, in increasing order, are \(0.004\), \(0.018\), \(0.032\), \(0.200\), and \(0.700\). The third value is the rounded p-value from the preceding example. At FDR level \(\alpha=0.05\), the Benjamini–Hochberg thresholds are \(i(0.05)/5\): \(0.010\), \(0.020\), \(0.030\), \(0.040\), and \(0.050\).
The first two comparisons pass: \(0.004\le0.010\) and \(0.018\le0.020\). The third fails because \(0.032>0.030\), and the later comparisons fail as well. The largest passing rank is therefore 2, so the procedure rejects the first two pathway hypotheses at this FDR level.
For the adjusted p-values, calculate \(5p_{(i)}/i\), then enforce a nondecreasing sequence from the largest rank backwards by taking the minimum over that rank and all larger ranks. The results are approximately \(0.020\), \(0.045\), \(0.0533\), \(0.250\), and \(0.700\). Thus the first two pathways meet the 0.05 adjusted threshold, while the third does not. These are hypothetical pathway-level results; the adjustment does not correct a poorly defined gene universe or an invalid gene-level selection model.
Assumptions, Diagnostics, and Interpretation
The hypergeometric calculation is exact for its stated random-selection model, but that does not make the model automatically appropriate for a real differential-expression list. Its key exchangeability assumption is that, under the null, every gene in \(U\) has the same chance of being among the \(n\) selected genes. If genes differ in their probability of being measured, tested, or selected—because of expression level, gene length, annotation, or other gene-level biases—uniform selection may be implausible. The earlier material on gene-level biases in ORA addresses this concern; a carefully computed hypergeometric p-value cannot repair unequal selection probabilities.
The standard calculation also treats the selected list as a fixed-size subset and does not model dependence among genes. Correlated expression can make selection events dependent, affecting calibration relative to the simple null. In addition, pathway overlap means pathway-level test statistics and p-values are generally dependent. These limitations should be considered when interpreting both raw and adjusted results; they are not resolved by changing the arithmetic in the 2-by-2 table.
Several practical checks can reveal an analysis that is going wrong:
- Verify that the selected genes are a subset of the tested universe and that \(0\le x\le\min(K,n)\).
- Check that the table margins equal \(N\), \(K\), and \(n\), and that no gene was inadvertently counted more than once.
- Confirm that the pathway size \(K\) is its size within the tested universe, not its database-wide size.
- Record the selection threshold and pathway collection, and count all pathways included in the multiple-testing adjustment.
- Inspect whether the list or universe is dominated by genes with characteristics that affect selection probability.
A frequent mistake is to use every annotated gene as the background because it seems like the most comprehensive reference. If some annotated genes could not be measured or tested, they could not have entered the selected list. Including them changes the expected overlap and can change the p-value without representing the study’s selection process. Use the genes eligible for selection, and state how eligibility was defined.
Worked Example: The Consequence of an Inappropriate Universe
Return to the hypothetical example with 20 tested genes, 5 selected genes, 4 pathway genes in the tested universe, and an overlap of 3. Now suppose the pathway database contains 16 additional genes that were not tested. A mistaken analysis includes those 80 untested genes in the background as well, producing \(N=100\) and \(K=20\), while leaving \(n=5\) and \(x=3\) unchanged.
That calculation gives
The numerical result differs from \(0.0320\), but this is not a valid alternative analysis of the same selection process: the 80 added genes had no opportunity to be selected. The correct universe remains the 20 tested genes. The example shows why universe definition must precede pathway counting and p-value calculation.
ORA is useful when the scientific question concerns a clearly defined selected list and its pathway composition. Its thresholded list discards information about genes just above or below the cutoff, however, and its usual competitive null is not a self-contained test of whether genes in a pathway show any association with the phenotype. The earlier material on competitive and self-contained gene-set tests distinguishes those null hypotheses. A ranked gene-set analysis is another approach when retaining the ordering of gene-level evidence is central; it answers a related but not identical question.
Biological interpretation should follow the statistical result rather than exceed it. A small adjusted p-value supports a statement that a pathway is over-represented among the selected genes relative to the specified tested-gene background and null model. It does not establish pathway activation, identify a causal mechanism, or show that every member contributes in the same direction. Report the universe, selection rule, pathway source, overlap counts, raw p-values, and adjustment method so that the result can be assessed in context.
Check Your Understanding
Answer each question using the definitions and calculations above.
- If \(N=500\), \(K=25\), and \(n=40\), what is the expected overlap under the uniform-selection null?
- For a pathway with \(K=12\) genes and an observed overlap of \(x=2\), which tail of the hypergeometric distribution tests over-representation?
- Why is the number of all genes in an annotation database not automatically the correct universe?
- Five pathways have ordered p-values \(0.006\), \(0.021\), \(0.040\), \(0.30\), and \(0.80\). At \(\alpha=0.05\), how many are rejected by Benjamini–Hochberg?
- What does a significant ORA result establish, and what does it not establish about pathway activity?
Solutions. 1. The expected overlap is \(nK/N=40(25)/500=2\) genes. 2. Use the upper tail, \(P(X\ge2)\), because over-representation means an overlap at least as large as observed. 3. Genes that could not be measured or selected were not eligible outcomes of the selection process; including them changes the null model. 4. The thresholds are \(0.010\), \(0.020\), \(0.030\), \(0.040\), and \(0.050\). The first passes, the second fails, and all later ones fail, so the largest passing rank is 1 and one pathway is rejected. 5. It supports over-representation among the selected genes relative to the specified universe and null model; it does not by itself establish activation, causation, or a common direction of effect for pathway genes.