Functional Annotation Coverage Bias in Pathway Tests

Why Annotation Coverage Can Distort a Pathway Test

Pathway tests compare gene-level results with gene-set membership, but both can depend on how much is known about each gene. A well-studied gene may have more recorded functional annotations, more chances to be assigned to a pathway, and more evidence available for prioritization. A sparsely studied gene may be just as relevant biologically yet appear to have no pathway membership or little functional support. The resulting enrichment can partly reflect unequal documentation rather than a pathway-specific pattern in the association results.

The concern is not that every well-annotated gene is biased or that every annotation is unreliable. It is that annotation coverage can be associated with both the pathway information used in the test and the gene-level evidence being tested. A pathway result is more interpretable when its stability has been checked against plausible differences in annotation completeness, evidence quality, and gene eligibility.

Three related mechanisms deserve separate attention. First, coverage can affect whether a gene is labelled as a member of a pathway. Second, it can affect gene-level scores or prioritization when those scores use functional evidence. Third, coverage can affect which genes enter the analysis at all—for example, if eligibility requires a gene to have a recorded annotation. These mechanisms can coexist, but a diagnostic should identify which one is plausible rather than treating “annotation bias” as a single adjustment.

How Coverage Creates Apparent Enrichment

Let \(G_i=1\) indicate that gene \(i\) belongs to the pathway under the membership definition being used, and let \(Y_i=1\) indicate that it meets a prespecified gene-level selection criterion. Let \(C_i\) denote an annotation-coverage category. A basic competitive comparison asks whether selected genes are more common in the pathway than outside it, within a defined eligible universe. If selection is more likely among well-covered genes, and pathway genes have a different coverage distribution from other eligible genes, the marginal comparison can be misleading.

The reason can be written directly. Suppose selection depends on coverage but, within each coverage category, not on pathway membership. If \(q_g=P(C_i=\text{high}\mid G_i=g)\), \(p_H=P(Y_i=1\mid C_i=\text{high})\), and \(p_L=P(Y_i=1\mid C_i=\text{low})\), then:

When \(q_1\ne q_0\) and \(p_H\ne p_L\), the selection rates can differ between pathway and nonpathway genes even though there is no pathway difference within either coverage category. This is a form of confounding by coverage. It is also possible for incomplete pathway labels to misclassify true members, which changes the gene set itself rather than just the selection rates.

Diagnosing the Coverage Mechanism

Start by defining the eligible gene universe and the pathway-membership source. Then summarize coverage for pathway and nonpathway genes, and for selected and unselected genes. Useful summaries include the proportion with at least one qualifying annotation and the distribution of annotation counts. If available, distinguish coverage by source, evidence code, tissue or assay context, and date of curation. The measure should be specified before comparing results; changing its definition after seeing the enrichment can make a sensitivity analysis difficult to interpret.

A simple diagnostic is to compare selected and unselected genes within coverage strata. If the pathway association is substantially weaker after stratification or adjustment, coverage may explain part of the marginal result. A regression can represent the same diagnostic: for a binary selection outcome, one possible model is \( \operatorname{logit}P(Y_i=1)=\beta_0+\beta_GG_i+\beta_CC_i \), where \(C_i\) is a prespecified coverage measure. The coefficient \(\beta_G\) describes the pathway association conditional on the included coverage term, under the model assumptions. It does not prove that coverage caused the original signal or remove bias from unmeasured features. For ranked gene-level statistics, the corresponding diagnostic can compare ranks or scores within coverage strata or include coverage as a covariate in an appropriate model.

Adjustment is informative only when the data support the comparison. There must be pathway and nonpathway genes at comparable coverage levels; otherwise the conditional pathway contrast is poorly identified. A coverage variable may also be measured with error, and an annotation count may combine evidence of very different relevance. Report the coverage measure and strata, the eligible universe, and the size of groups being compared. Do not describe a change after adjustment as a definitive correction if these conditions are weak.

Evidence Filters and Gene Eligibility

Evidence-code filters change which annotations count as pathway membership. Restricting to experimental evidence can reduce reliance on indirect or computationally inferred assignments, but it also tends to reduce coverage. A result that disappears under a stringent filter may be sensitive to evidence type, to the smaller resulting set, or both. A result that persists is more robust to that particular filter, but is not thereby proven biologically correct. When possible, report pathway size, selected-member count, and the coverage distribution alongside each filtered result.

Gene eligibility defines the comparison population and therefore the estimand. A gene should generally be eligible because its association statistic could have been observed and analyzed under the study design, not merely because it has a pathway annotation. For example, eligibility might require a valid gene-level statistic under the specified mapping and quality rules. Including genes that could not have been selected, or excluding genes because they lack functional documentation, changes the background and can distort the comparison.

A Practical Sensitivity Workflow

A compact assessment can be organized as a set of prespecified comparisons. Keep the gene-level statistics and selection rule fixed while changing one annotation or eligibility choice at a time; otherwise, the source of a change becomes hard to identify.

Common Mistakes and Interpretation

• Wrong reasoning: “Genes with no recorded annotation have no relevant function, so they should be treated as nonmembers.” Correction: Lack of recorded evidence is not evidence of biological absence. Treat unannotated membership as uncertain or assess it through a defensible completeness sensitivity analysis.
• Wrong reasoning: “A pathway result survives an experimental-evidence filter, so annotation bias is ruled out.” Correction: The filter changes both evidence composition and coverage. Persistence is a robustness check for that filter, not proof that all coverage-related bias is absent.
• Wrong reasoning: “Using every gene in the genome gives the fairest background.” Correction: The background should reflect genes that could have contributed to the analysis. Genes without an analyzable statistic may not be valid comparison genes.
• Wrong reasoning: “Adjustment makes pathway membership independent of annotation coverage.” Correction: Adjustment only addresses the measured coverage features represented in the model, and only where there is adequate overlap. Unmeasured coverage differences and membership misclassification may remain.
• Wrong reasoning: “A larger odds ratio after adding annotations proves the pathway is biologically enriched.” Correction: The result depends on which members were added, the gene universe, and the selection rule. An odds ratio describes a contrast; it does not establish a causal mechanism.