Selection Thresholds and Their Effects on Over-Representation

Why the Selection Threshold Matters

Over-representation analysis (ORA) asks whether a selected list of genes contains more members of a gene set than would be expected under a specified null model. The selection threshold is therefore part of the analysis: it determines which genes enter that list. A stringent differential-expression cutoff can produce a short list with few pathway members; a more permissive cutoff can add both pathway and non-pathway genes. Either change can alter the pathway’s overlap count and enrichment p-value.

A pathway result is consequently conditional on how the list was selected. It does not establish that the pathway is biologically active, nor does it show that every pathway gene is differentially expressed. It summarizes an overlap for a particular selected list and a particular tested-gene universe.

Definition: Let the universe contain \(M\) genes eligible for selection, let a pathway contain \(K\) of those genes, and let a differential-expression rule select \(n\) genes. If \(k\) selected genes belong to the pathway, the overlap count is \(k\). In cutoff-based ORA, the rule might require an adjusted p-value at or below a chosen threshold, an absolute effect estimate at or above a chosen minimum, or both.

The universe should generally comprise genes that could have been selected by the differential-expression analysis, not every gene in an annotation database. For example, genes that were not measured or were excluded before testing cannot enter the selected list and should not be treated as eligible non-selected genes. The universe and pathway membership must use a consistent gene identifier and annotation scheme.

How a Cutoff Enters the ORA Calculation

Under the hypergeometric null model, the \(n\) selected genes are treated as a subset drawn uniformly at random from the \(M\) eligible genes, conditional on the observed list size. If \(X\) is the number of pathway genes in that random subset, then

P(X=x)=\frac{\binom{K}{x}\binom{M-K}{n-x}}{\binom{M}{n}}.

For an enrichment test, the p-value is the upper-tail probability of observing at least the actual overlap \(k\):

P(X\geq k)=\sum_{x=k}^{\min(K,n)} \frac{\binom{K}{x}\binom{M-K}{n-x}}{\binom{M}{n}}.

The selection cutoff affects \(n\), \(k\), or both. It does not change the definition of the pathway-level p-value; it changes the data supplied to that test. A smaller list is not automatically better: it may concentrate pathway genes, or it may discard them. A larger list may capture more pathway genes while also adding many genes outside the pathway.

Why the Expected Overlap Changes with List Size

The expected overlap under this null model can be justified directly. For each of the \(K\) pathway genes, define an indicator that equals 1 if the gene is selected and 0 otherwise. Each gene has probability \(n/M\) of appearing in a uniformly chosen subset of size \(n\). The overlap \(X\) is the sum of those \(K\) indicators, so linearity of expectation gives:

E(X)=\sum_{g\ \mathrm{in\ pathway}}P(g\ \mathrm{selected}) =K\frac{n}{M} =\frac{nK}{M}.

This expectation is a useful reference, not an enrichment p-value. The p-value also depends on the variability of \(X\) under the hypergeometric model and on the observed overlap. Increasing \(n\) raises the expected overlap, but whether the observed overlap is unusually large depends on how quickly \(k\) grows relative to that expectation.

Worked Example: A More Permissive Cutoff Weakens Enrichment

Consider a hypothetical analysis with \(M=20\) eligible genes, of which \(K=5\) belong to one pathway. Three nested differential-expression cutoffs produce lists of \(n=3\), \(n=4\), and \(n=8\) genes, with pathway overlaps \(k=3\), \(k=3\), and \(k=4\), respectively. These counts are hypothetical; the lists are nested because relaxing the cutoff only adds genes.

At the strictest cutoff, all 3 selected genes are pathway members. The enrichment p-value is:

P(X\geq 3)=\frac{\binom{5}{3}\binom{15}{0}}{\binom{20}{3}} =\frac{10}{1140}=0.008772.

At the intermediate cutoff, one non-pathway gene has been added, so \(n=4\) but \(k\) remains 3:

P(X\geq 3) =\frac{\binom{5}{3}\binom{15}{1}+\binom{5}{4}\binom{15}{0}} {\binom{20}{4}} =\frac{150+5}{4845} =0.0320.

At the most permissive cutoff, the list has \(n=8\) genes and includes 4 pathway members:

P(X\geq 4) =\frac{\binom{5}{4}\binom{15}{4}+\binom{5}{5}\binom{15}{3}} {\binom{20}{8}} =\frac{6825+455}{125970} =0.0578.

The observed pathway count increases from 3 to 4 as the cutoff relaxes, but the enrichment p-value becomes less small. The larger list also makes a larger overlap plausible by chance. In this example, the strict list yields stronger evidence against the hypergeometric null; that does not justify choosing the strict cutoff after seeing which cutoff gives the smallest p-value.

Different Cutoffs Can Reveal or Obscure a Pathway

The reverse pattern is also possible. Suppose genes with moderate differential-expression evidence include several members of a pathway. A very strict cutoff may omit those genes, while a more permissive cutoff includes them and makes the overlap more exceptional relative to the enlarged list. Thus, threshold sensitivity is not a one-way trend: relaxing a cutoff can strengthen or weaken enrichment.

Worked Example: A Permissive Cutoff Captures More Pathway Members

In a second hypothetical analysis, \(M=12\) genes are eligible and \(K=4\) belong to a pathway. A strict cutoff selects \(n=3\) genes, of which \(k=2\) are pathway members. A more permissive cutoff produces a nested list with \(n=6\) and \(k=4\): it adds the other two pathway genes and one non-pathway gene.

For the strict list, an overlap of 2 or more has probability:

P(X\geq 2) =\frac{\binom{4}{2}\binom{8}{1}+\binom{4}{3}\binom{8}{0}} {\binom{12}{3}} =\frac{48+4}{220} =0.2364.

For the permissive list, all 4 pathway genes are selected. The upper tail contains only \(X=4\):

P(X\geq 4) =\frac{\binom{4}{4}\binom{8}{2}}{\binom{12}{6}} =\frac{28}{924} =0.0303.

Here, relaxing the cutoff makes the pathway overlap more compelling under the null, because the added genes include all remaining pathway members. The expected overlap rises from \(3(4/12)=1\) to \(6(4/12)=2\), while the observed overlap rises from 2 to 4. The change in the p-value reflects both the larger list and its composition.

Specify the Selection Rule and Check Its Consequences

A cutoff should be stated precisely enough that another analyst could reconstruct the selected list. For example, “selected genes have adjusted p-values at or below 0.05” specifies a gene-level significance rule. A rule that also requires an estimated absolute log-fold change of at least a stated amount is a different selection rule and can produce a different list. The adjusted p-value cutoff is not the pathway-level significance threshold: the former selects genes, while the latter assesses pathway overlap. If many pathways are tested, pathway-level p-values also require appropriate multiplicity control.

Before interpreting a result, report the universe size \(M\), pathway size \(K\) within that universe, list size \(n\), overlap \(k\), the gene-selection rule, and the pathway-level result. For a practical sensitivity check, compare a small set of scientifically defensible cutoffs specified before examining which one gives the strongest enrichment. Track both the pathway overlap count and the enrichment result; a p-value alone hides how its underlying counts changed.

Define the eligible universe.
Use genes that could have entered the selected list after the analysis’s measurement and testing filters. Keep this universe consistent across the cutoff comparisons.

Write down the selection rule.
Specify the gene-level statistic or adjusted p-value, any effect-size requirement, the direction if relevant, and the exact cutoff values.

Recalculate the list and overlap.
For each rule, record \(n\) and \(k\). The same pathway can have different overlaps and different null distributions across lists.

Compare results without selecting a winner.
Look for findings that persist across reasonable cutoffs, and describe pathway results that depend strongly on one threshold as threshold-sensitive.

Direction, Diagnostics, and Failure Modes

If the scientific question concerns increases versus decreases, construct and assess up-regulated and down-regulated lists separately using a clearly stated direction convention. Combining both directions can conceal an opposing pattern or produce an overlap that does not have a coherent direction.

Worked Example: Direction-Specific Lists Give Different Results

Suppose \(M=12\) genes are eligible and \(K=4\) belong to a pathway. Under a hypothetical strict cutoff, the up-regulated list contains 3 genes, all pathway members. Its enrichment p-value is:

P(X\geq 3) =\frac{\binom{4}{3}\binom{8}{0}}{\binom{12}{3}} =\frac{4}{220} =0.0182.

At a more permissive up-regulated cutoff, the list contains \(n=6\) genes but still only \(k=3\) pathway members; the other three added genes are outside the pathway. The corresponding p-value is:

P(X\geq 3) =\frac{\binom{4}{3}\binom{8}{3}+\binom{4}{4}\binom{8}{2}} {\binom{12}{6}} =\frac{224+28}{924} =0.2727.

In a separate hypothetical down-regulated list of \(n=3\), suppose \(k=1\). Then:

P(X\geq 1) =1-\frac{\binom{8}{3}}{\binom{12}{3}} =1-\frac{56}{220} =0.7455.

The strict up-regulated list shows a smaller unadjusted enrichment p-value than either the more permissive up list or the down list. These calculations do not establish pathway activation or causation, and they do not account for testing many pathways. They describe the overlap evidence for each specified list.

Several diagnostics help identify fragile or misleading conclusions. Check that the universe does not change silently between cutoffs, that the reported \(n\) and \(k\) agree with the lists, and that pathway membership is counted only among eligible genes. Inspect whether a small change in the cutoff adds many genes, whether the overlap is driven by only one or two genes, and whether the direction-specific results are consistent with the question being asked. A large collection of nearly identical pathways can also create many related tests; pathway-level multiplicity adjustment remains important.

A common mistake is to try many cutoffs and report only the one with the smallest pathway p-value as if it had been chosen in advance. This uses the same data both to choose the rule and to claim evidence under that rule, so the reported p-value does not reflect the cutoff search. Instead, pre-specify a primary selection rule, report its results, and use a small sensitivity analysis to show how other defensible rules affect the conclusion. If the strongest result appears only at one finely tuned cutoff, describe it as exploratory and cutoff-sensitive.

Cutoff-based ORA is easy to communicate, but it discards information about genes just above or below the boundary and can be sensitive to that boundary. Its conclusions also depend on the hypergeometric null model’s assumption that, conditional on list size, selected genes are exchangeable with respect to pathway membership. Differences in gene detectability or dependence among genes can make that assumption questionable. These limitations are reasons to report the analysis conditions carefully, not reasons to interpret a thresholded p-value as a direct measure of pathway importance.

Key takeaway: Changing a differential-expression cutoff changes the selected list, and therefore can change both the observed pathway overlap and the null distribution used by ORA. Report the selection rule, universe, and overlap counts, and assess whether conclusions persist across defensible thresholds.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions.

A pathway has \(K=6\) eligible genes in a universe of \(M=30\). A selected list has \(n=5\). What is the expected overlap under the hypergeometric null?
When a cutoff is relaxed, the selected list grows from 10 to 18 genes and the pathway overlap grows from 4 to 5. Can you determine from these counts alone whether the enrichment p-value must decrease? Explain.
Why should genes that were not eligible for differential-expression testing generally be excluded from the ORA universe?
An analyst tries 20 cutoffs and reports only the one with the smallest pathway p-value. What is the interpretive problem, and what is a better practice?

Solutions. (1) \(E(X)=nK/M=5(6)/30=1\) pathway gene. (2) No. Both the observed overlap and the null distribution change with \(n\); the p-value depends on the full upper-tail probability, not only on the overlap count. (3) Such genes could not have entered the selected list, so treating them as eligible non-selected genes misstates the sampling universe. (4) The cutoff was selected using the observed enrichment results, so the reported p-value ignores that search. Specify a primary rule in advance and report sensitivity across a limited set of defensible alternatives.