Gene-Level Biases in Over-Representation Analysis

Why Gene-Level Selection Bias Matters

Over-representation analysis (ORA) asks whether a selected list contains more members of a gene set than expected from a specified background. The usual hypergeometric calculation treats each eligible gene as having the same chance of selection under the null. In expression studies, however, genes can have very different chances of passing a differential-expression or detection threshold, even when there is no gene-set-specific effect. If pathway membership is associated with those gene-level characteristics, ordinary ORA can report apparent enrichment that reflects selection bias rather than a pathway-level signal.

Important sources include gene length, the number of probes assigned to a gene, transcript abundance, and unequal probability of detecting or quantifying a gene. The direction and size of bias depend on the measurement platform and selection procedure. A bias diagnosis therefore begins with the actual rule used to create the gene list and the set of genes that could have entered it.

The Equal-Probability Assumption in Ordinary ORA

Definition: Let the analysis universe contain $N$ eligible genes, of which $K$ belong to a specified gene set. Let a selected list contain $n$ genes, with $x$ gene-set members. Under the ordinary ORA null, the $n$ genes are treated as a simple random sample without replacement from the $N$ genes. Thus every subset of $n$ genes is equally likely, and the number of gene-set members in the list has a hypergeometric distribution: $$ X \sim \operatorname{Hypergeometric}(N,K,n), \qquad P(X=x)=\frac{\binom{K}{x}\binom{N-K}{n-x}}{\binom{N}{n}}. $$ For an over-representation test, the p-value is $P(X\geq x)$.

The expectation under this model is $E(X)=nK/N$. One justification is to define $I_i=1$ if gene $i$ is selected, and $I_i=0$ otherwise, for the $K$ genes in the set. Under simple random sampling, $P(I_i=1)=n/N$. Since $X$ is the sum of those indicators, linearity of expectation gives:

E(X)=\sum_{i=1}^{K}E(I_i)=K\frac{n}{N}=\frac{nK}{N}.

This expected count is a useful reference, but it does not establish that the hypergeometric model is appropriate. The model’s key assumption is equal selection probability for eligible genes under the null, not merely that the list and universe have been counted correctly. When some genes are more likely to be selected, the relevant null distribution must reflect that unequal probability.

Worked Example: A Length-Driven Apparent Enrichment

Consider a hypothetical analysis with 100 eligible genes. A gene set contains 20 genes, all classified as long. The other 80 genes comprise 20 long genes and 60 short genes. A selection procedure returns 22 genes: 8 of the 20 long set members, 8 of the 20 long genes outside the set, and 6 of the 60 short genes outside the set.

The ordinary ORA expected count is:

E(X)=\frac{nK}{N}=\frac{22(20)}{100}=4.4 \text{ genes}.

The observed count is 8, so the observed-to-expected ratio is $8/4.4=1.818$. Ordinary ORA would compare 8 with the hypergeometric distribution for $N=100$, $K=20$, and $n=22$. But the list’s selection rates reveal a competing explanation: $8/20=0.40$ for long genes outside the set, compared with $6/60=0.10$ for short genes. The set’s 20 members are all long, and their selection rate is also $8/20=0.40$. Thus the set members are not selected more often than comparable long genes; their apparent excess relative to the whole universe arises from their length composition.

This example diagnoses a confounding pattern, not proof that length caused selection in any real study. The data are hypothetical, and the ordinary ORA calculation has not isolated a gene-set effect from the length-related selection probability.

How Common Gene-Level Features Change Selection

Gene length. In RNA sequencing, longer genes can produce more reads at the same per-base expression level. Depending on the count model, filtering rule, and effect size, this can affect the precision of estimated expression differences and the probability of passing a significance or detection threshold. Length is not guaranteed to increase selection in every pipeline; the relevant question is whether the observed selection probability varies with length under the analysis used.

Probe count. In a probe-based assay, a gene represented by several probes may have more opportunities to produce an extreme probe-level statistic if a gene is declared selected whenever any probe passes a threshold. If probe results are then combined or mapped to genes differently, the resulting selection probability depends on that rule. Probe correlation and downstream multiple-testing procedures change the size of the effect, so the independence calculation below is illustrative rather than a universal model.

Abundance and detection. Highly abundant transcripts may be measured more reliably, while low-abundance transcripts may be filtered, recorded as undetected, or have noisier estimates. The resulting probability of entering a selected list can vary with abundance, detection rate, or other quality-related features. Filtering also changes the eligible universe: genes that could not have been selected should not ordinarily be treated as if they were eligible controls.

Worked Example: More Probes, More Chances to Pass a Threshold

Suppose, hypothetically, that every probe-level p-value for a null gene is uniformly distributed between 0 and 1, probes are independent, and the gene is selected if at least one probe has a p-value below 0.05. For a gene with one probe, the selection probability is $0.05$. For a gene with five probes, the chance that none passes is $0.95^5$, so:

P(\text{gene selected}\mid 5\text{ probes}) =1-(1-0.05)^5 =1-0.95^5 =1-0.7738 =0.2262.

Under these assumptions, the five-probe gene has a 22.62% selection probability, compared with 5% for a one-probe gene, despite both genes being null. If a gene set contains more genes with many probes than the background does, selecting genes by “any probe passes” can therefore raise its observed count. Real probes may be correlated, and a well-designed pipeline may combine probes or control multiplicity, so the independent-probe result should not be applied without checking the actual procedure.

Diagnosing Bias and Choosing an Alternative

Begin by defining the analysis universe precisely: the unique genes that were measured, passed relevant filters, and could have been included in the selected list. Then examine whether selection frequency changes across values or groups of length, probe count, abundance, or detection probability. Plots of selection rate against binned covariates, or a model of selection as a function of those covariates, can reveal patterns that a single ORA table cannot.

A useful check is to compare gene-set members with nonmembers across the same covariate ranges. Strongly different length distributions, for example, flag a possible bias, but do not by themselves prove that length changes selection. Conversely, similar average lengths do not rule out bias if selection probability varies nonlinearly or if another feature, such as abundance, is imbalanced. Also check that the universe uses the same gene identifiers and aggregation rules as the selected list and the gene-set annotation.

Bias-aware alternatives: A probability-weighting approach estimates each gene’s selection probability from a relevant covariate, such as gene length, and calculates enrichment against a null that accounts for unequal selection. GOseq is a commonly used approach for length-biased RNA-sequencing selection. Alternatively, a regression model can predict selection from gene-set membership and covariates; the gene-set coefficient then assesses membership conditional on those covariates. Stratification or a permutation scheme that preserves the relevant covariate structure can be useful when its assumptions are defensible. A ranked gene-set analysis avoids a single hard selection threshold, but it still requires an appropriate ranking and does not automatically remove every source of bias.

These methods answer related but not identical questions. Probability weighting targets the selection process through estimated gene-specific probabilities. Regression estimates a conditional association between gene-set membership and selection, given included covariates. Stratification compares within covariate groups, but can be unstable when groups are sparse. Ranked methods use information beyond a selected-versus-unselected indicator; their suitability depends on how the ranking was constructed. None repairs an incorrectly defined universe or unmeasured selection mechanisms automatically.

Worked Example: Comparing Naive and Length-Weighted Expectations

Suppose a hypothetical universe has 100 genes. A set contains 10 long and 10 short genes; the 80 nonmembers comprise 30 long and 50 short genes. The observed selected counts are 4 set members and 12 nonmembers among long genes, and 1 set member and 5 nonmembers among short genes. Thus $N=100$, $K=20$, $n=22$, and the observed set count is $x=5$.

Ordinary ORA gives the expected count:

E_{\mathrm{ordinary}}(X)=\frac{nK}{N}=\frac{22(20)}{100}=4.4.

The observed-to-ordinary-expected ratio is $5/4.4=1.136$. Now estimate selection rates within the two length groups from the hypothetical counts. In the long group, $16$ of $40$ genes are selected, giving $16/40=0.40$. In the short group, $6$ of $60$ are selected, giving $6/60=0.10$. Applying these rates to the set’s composition gives a length-adjusted expected count:

E_{\mathrm{length}}(X) =10(0.40)+10(0.10) =4+1 =5 \text{ genes}.

The observed count equals this illustrative length-adjusted expectation. This comparison shows why the ordinary ratio alone can be misleading: the set has the same overall selection rate as the background once its long-versus-short composition is taken into account. A formal probability-weighted test would estimate selection probabilities using an appropriate model and account for the list-generation scheme; the simple arithmetic here illustrates the adjustment, not a complete inferential procedure.

Assumptions, Diagnostics, and Common Pitfalls

A bias-aware analysis also needs assumptions. For probability weighting, the covariate model should describe the main systematic variation in selection probability, the estimated probabilities should be adequate over the relevant range, and the gene universe should be appropriate. A regression adjustment requires a correctly specified relationship between selection and covariates, sufficient overlap in covariates between set members and nonmembers, and a model suited to the binary selection outcome. If nearly all pathway genes are long and nearly all nonmembers are short, the data may not support a reliable comparison at common lengths.

Diagnostics should therefore include selection rates across covariate ranges, the covariate distributions of members and nonmembers, and checks for sparse or empty groups. Compare conclusions under plausible universe definitions and selection rules. If changing a reasonable background or adjusting for a strong selection predictor substantially changes the result, report that sensitivity rather than presenting the ordinary ORA as definitive. Gene-level selection probabilities may also be dependent: genes that co-vary in expression or share technical features are not necessarily independent. A model that only adjusts for gene length does not address such dependence or other omitted sources of bias.

Mistake: “The hypergeometric p-value is small, so selection was equally likely for every gene.” The p-value is calculated under the equal-probability sampling model; it does not verify that assumption. Check selection patterns against relevant gene-level features.
Mistake: “All genes in the genome are the background.” Genes that were unmeasured or ineligible for selection could not enter the list. Use an eligible universe aligned with measurement, filtering, identifier mapping, and list construction.
Mistake: “Length adjustment proves the pathway is biologically active.” Adjustment addresses a specified selection mechanism. Enrichment remains an association between the selected genes and the set under the chosen analysis; it does not establish causation or biological activity by itself.
Mistake: “A ranked method automatically eliminates bias.” Ranking avoids one particular hard cutoff but can still reflect length, abundance, or detection effects. Evaluate how the ranking statistic behaves across those features.

The practical goal is not to adjust for every available gene annotation indiscriminately. Include features with a defensible connection to the selection mechanism, check whether the data support adjustment, and state what remains unaccounted for. The direction of a bias cannot be inferred from a feature name alone: it depends on how that feature relates to selection and to gene-set membership in the analysis universe.

Check Your Understanding

Answer each question using the definitions and assumptions above. Short solutions follow.

An ORA has $N=200$ eligible genes, $K=30$ set members, and a list of $n=20$. What is the ordinary expected set count, and what assumption makes that reference appropriate?
Under independent null probe p-values and a rule selecting a gene if any probe has $p<0.05$, calculate the selection probability for three probes.
In a hypothetical universe, a set has 5 long and 5 short genes. Estimated selection probabilities are 0.30 for long and 0.08 for short genes. What is the probability-weighted expected set count?
Why can a measured gene that failed selection be part of the ORA universe, while a gene that could not be measured should generally be excluded?
Name one diagnostic and one limitation of covariate-adjusted regression for selection bias.

Solutions. (1) $E(X)=nK/N=20(30)/200=3$ genes. This reference assumes the selected list is a simple random sample without replacement from the eligible universe under the null, so eligible genes have equal selection probability. (2) $1-0.95^3=1-0.857375=0.142625$, or about 14.26%. (3) $5(0.30)+5(0.08)=1.50+0.40=1.90$ expected genes. (4) A measured, eligible gene could have entered the list and is a relevant comparison; an unmeasured gene had no opportunity to enter and can distort the sampling background. (5) A diagnostic is checking overlap in covariate distributions and selection rates across covariate values. A limitation is that sparse or absent overlap can make the conditional comparison unstable or unsupported.