Tutorials › Statistical Genetics › Encoding Functional Annotations for Variance Partitioning

Partitioned Genetic Heritability · Tutorial 538 of 989

Encoding Functional Annotations for Variance Partitioning

Represent each annotation as a column of zeroes and ones, while preserving variants that belong to several functional categories.

Advanced 9 min read

What You'll Learn

  • Define a variant universe and annotation sets before constructing indicators
  • Build and read a variant-by-annotation binary matrix
  • Represent variants belonging to multiple annotations without assigning them to only one group
  • Calculate annotation sizes and pairwise overlaps from indicator columns
  • Distinguish annotation membership from genotype dosage and estimated genetic effect
  • Check that variant identifiers and annotation rules are consistent

Why Encode Functional Annotations?

A functional annotation labels variants according to a defined feature, such as whether a variant lies in a specified genomic region. In a variance-partitioning analysis, these labels need a precise representation: every variant must have a recorded membership status for every annotation being considered. That representation lets a joint model use several annotation categories at once without silently discarding variants that satisfy more than one category.

The key is to represent each category with its own binary column. A zero means that a variant is not in that annotation under the stated rules; a one means that it is. If a variant is both conserved and in an enhancer, its row contains a one in both columns. Overlap is recorded, not resolved by choosing a single label.

Definition: Let the analyzed variant universe contain \(m\) variants, indexed by \(j=1,\ldots,m\), and let there be \(K\) annotations, indexed by \(k=1,\ldots,K\). Annotation \(k\) is a set of variants, denoted \(A_k\). Its binary indicator for variant \(j\) is \(c_{jk}=1\) if variant \(j\in A_k\), and \(c_{jk}=0\) otherwise. The \(m\)-by-\(K\) matrix \(C=(c_{jk})\) is the variant-by-annotation indicator matrix.

Each row of \(C\) corresponds to one variant, and each column corresponds to one annotation. The annotation names and the rule that defines membership must be established before interpreting the matrix. For example, “enhancer” could mean overlap with a specified enhancer set from a particular data source and genome build; it is not a universal label with one fixed membership rule.

The indicator is about annotation membership, not an allele count, genotype dosage, effect estimate, or probability. A value of one does not say that a variant is causal or has a large effect. It says only that the variant meets the operational definition of that annotation.

Building the Indicator Matrix

Start by fixing the variant universe: the variants eligible for the analysis, with identifiers that can be matched consistently to the annotation data. Then define each annotation as a membership rule and evaluate that rule for every variant. The result is one binary entry per variant-annotation pair.

$$ c_{jk} = \begin{cases} 1, & \text{if variant } j \text{ meets the rule for annotation } k,\\ 0, & \text{otherwise.} \end{cases} $$

The rule should specify details that can change membership. These may include the genome build, the genomic interval boundaries, how variants are assigned to genes or regulatory regions, and the source or version of the annotation. The variant identifiers used in the analysis must be aligned to the identifiers used to make those assignments. If the annotation file lacks a record for a variant, that absence should be handled according to a documented rule; it should not automatically be treated as a zero unless missing records are known to mean “not in the annotation.”

When a variant satisfies several annotation rules, it receives a one in each corresponding column. The columns therefore need not form a disjoint partition. In a joint model, keeping these columns separate preserves the original membership information. Recoding each variant into just one category would instead change the annotations and discard overlap.

Worked Example: Encoding Three Overlapping Annotations

Consider six hypothetical variants and three annotations: promoter, enhancer, and conserved. Suppose the annotation rules yield the memberships shown below. The table is illustrative only; no real genomic data are being represented.

VariantPromoterEnhancerConserved
v1YesNoYes
v2NoYesYes
v3YesYesNo
v4NoNoYes
v5NoYesNo
v6NoNoNo

Replace Yes by 1 and No by 0, retaining the same row and column order. The resulting indicator matrix is

$$ C = \begin{pmatrix} 1&0&1\\ 0&1&1\\ 1&1&0\\ 0&0&1\\ 0&1&0\\ 0&0&0 \end{pmatrix}. $$

For example, row 1 is \((1,0,1)\): v1 is in the promoter and conserved annotations but not the enhancer annotation. Row 3 is \((1,1,0)\), so v3 belongs to both promoter and enhancer. It must not be assigned to only one of those columns merely to make the categories exclusive.

The column sums give the number of variants in each annotation. Here they are \(2\) promoter variants, \(3\) enhancer variants, and \(3\) conserved variants. The row sums give the number of annotations met by each variant: the six values are \(2,2,2,1,1,0\). Thus three variants meet two annotations, two meet one, and one meets none.

What Overlap Means in the Matrix

An annotation is a set, and the indicator column records membership in that set. Two annotations overlap when at least one variant belongs to both sets. Their intersection can be counted directly from the corresponding columns: for each variant, the product of the two indicators equals one only when both indicators equal one.

$$ |A_a\cap A_b| = \sum_{j=1}^{m} c_{ja}c_{jb}. $$

This formula is a bookkeeping identity, not a model for genetic variance. It follows from the binary values: \(c_{ja}c_{jb}=1\) exactly when variant \(j\) is in both annotations, and equals zero otherwise. Summing over all variants therefore counts the members of the intersection.

In the six-variant example, promoter and enhancer overlap only at v3. The product of their columns is \(1\) for v3 and \(0\) for every other row, so the intersection contains one variant. Promoter and conserved overlap only at v1, also giving an intersection size of one. These pairwise counts need not sum to the total number of variants: one variant may contribute to several pairwise intersections.

Worked Example: Counting Memberships and Overlap

Use the six-variant matrix above. The promoter column is \((1,0,1,0,0,0)\), and the enhancer column is \((0,1,1,0,1,0)\). To count variants in both annotations, multiply entries row by row and add:

$$ |A_{\text{promoter}}\cap A_{\text{enhancer}}| = (1)(0)+(0)(1)+(1)(1)+(0)(0)+(0)(1)+(0)(0) = 1. $$

The only shared member is v3. The separate annotation sizes are the column sums: promoter has \(1+0+1+0+0+0=2\) members, and enhancer has \(0+1+1+0+1+0=3\) members. Their union has \(2+3-1=4\) variants, namely v1, v2, v3, and v5. The subtraction is necessary because v3 was counted in both annotation sizes.

This calculation shows why the sum of column sizes need not equal the number of distinct annotated variants. Here the column sizes sum to \(2+3+3=8\), while only five distinct variants belong to at least one of the three annotations. Counting a variant once in each annotation it meets is appropriate for column membership; it is not a count of distinct variants across all annotations.

Encoding Membership From Explicit Rules

A reproducible matrix comes from applying explicit rules, not from assigning labels by intuition after seeing results. Each rule should specify how membership is determined for the chosen variant unit. For example, a rule might define a variant as a member when its genomic position falls within an interval. Another might define membership by an externally supplied variant list. Whatever the rule, the same rule must be applied to every variant.

Worked Example: Applying a Boundary Rule

Suppose four hypothetical variants have positions \(104\), \(120\), \(151\), and \(180\) on the same chromosome and genome build. Define annotation A as positions from 100 through 150, inclusive, and annotation B as positions from 120 through 180, inclusive. Because both intervals include their endpoints, a position equal to 120 belongs to both, and a position equal to 150 belongs to A.

Apply the rules in order. Position 104 is inside A and outside B, so its row is \((1,0)\). Position 120 is inside both intervals, so its row is \((1,1)\). Position 151 is outside A but inside B, giving \((0,1)\). Position 180 is outside A and on the included endpoint of B, giving \((0,1)\). The complete encoding is

$$ C = \begin{pmatrix} 1&0\\ 1&1\\ 0&1\\ 0&1 \end{pmatrix}. $$

The column sums are \(2\) for A and \(3\) for B. Their intersection has size \(1\), because the product of the two columns is \((0,1,0,0)\). Their union has size \(2+3-1=4\), so all four variants meet at least one rule. If the interval endpoints had instead been defined as excluded, boundary variants could receive different indicators; specifying inclusivity is therefore part of defining the annotation.

Using the Encoding in a Joint Analysis

For a joint analysis, place the selected annotation indicators together in the same variant-by-annotation matrix. “Joint” means that the model is supplied the columns together, not that the underlying categories must be mutually exclusive. A row such as \((1,1,0)\) remains \((1,1,0)\) when the annotations are considered jointly.

This encoding keeps two questions distinct. The first is descriptive: which variants satisfy each annotation rule? The matrix answers that directly. The second is inferential: how should variation associated with those categories be represented or estimated in a statistical model? The encoding supplies the membership information needed for that next step, but it does not itself estimate variance, heritability, or a causal effect.

Overlapping columns also mean that a variant may contribute to multiple annotation-specific quantities in a later model, according to that model’s specification. That is not a data error. It is the intended record of multiple memberships. By contrast, forcing every variant into exactly one label creates a new, exclusive classification that may not match the original functional definitions.

Common Mistakes and Checks

  • Forcing overlap into exclusive categories. Assigning a variant to “promoter” instead of “conserved” loses one of its memberships when it meets both rules. Keep a separate indicator column for each annotation.
  • Treating a one as an effect estimate. A one encodes membership only. It does not imply a nonzero effect, a causal role, or a particular effect size.
  • Confusing annotation counts with distinct-variant counts. Adding column sums counts memberships, so a variant in two columns is counted twice. Use the union or intersection calculation appropriate to the question.
  • Leaving the variant universe or rule ambiguous. Membership depends on the selected variants, genome build, and annotation definition. State these choices and ensure identifiers are aligned.
  • Silently converting missing annotation records to zero. A missing record is not necessarily evidence of non-membership. Establish whether missingness means “not annotated” before assigning a zero.

Before passing an indicator matrix to a joint analysis, check that its rows correspond to the intended variants, its columns correspond to the intended annotation definitions, and every entry is binary. A simple row check is also useful: a row of all zeroes means the variant meets none of the included annotation rules, while a row with several ones records multiple memberships. Neither pattern should be altered simply to make the matrix look like a partition.

Key takeaway: Encode annotation \(k\) with a separate indicator \(c_{jk}\) for every variant \(j\). Preserve multiple ones in a row when annotations overlap; binary columns record membership, not mutually exclusive categories or genetic effects.

Check Your Understanding

Answer using the definitions and encoding rules in this tutorial.

  1. A variant belongs to a promoter annotation and a conserved annotation, but not an enhancer annotation. What is its row in a matrix whose columns are ordered promoter, enhancer, conserved?
  2. Why can the sum of annotation column totals exceed the number of distinct variants in the analysis?
  3. For two indicator columns \(a\) and \(b\), explain why summing their row-wise products counts variants in both annotations.
  4. A variant has no entry in an annotation file. What information is needed before encoding its indicator as zero?
  5. Does an indicator value of one imply that the variant is causal or has a large genetic effect? Explain what it records instead.
Clinical Biostats