Quality Metrics for Rare Variant Calls

Why Rare Calls Need More Than a Genotype Label

A rare-variant call can be consequential even when it is supported by only one or a few people. A genotype label such as \(0/1\) is therefore not enough to describe the evidence behind it. The reads must be sufficiently numerous, their placement must be credible, and the observed reference and alternate reads should be compatible with the called genotype. Evidence may also look suspicious if alternate-supporting reads are concentrated on one sequencing strand.

These quality metrics answer different questions. Depth describes how much read evidence is available. Genotype quality describes uncertainty about a sample’s genotype call under a caller’s model. Mapping quality concerns where reads are placed in the genome. Allele balance and strand-bias summaries describe patterns in the reads supporting the alleles. None alone proves that a rare call is true or false. The interpretation depends on what was measured, at what level, and under which caller convention.

Depth and Genotype Quality

Depth is the number of sequencing reads contributing to evidence at a position or genotype, according to the caller’s counting rules. A sample-level depth for one genotype is often denoted DP in a variant file. It is not necessarily the same as the number of all reads that overlap the genomic position: quality filters, duplicate handling, and allele-specific counting rules can affect which reads contribute. A site-level depth may instead aggregate evidence over multiple samples.

More depth can provide more information, but it does not guarantee a correct call. Many reads can share the same technical artifact, and reads that are poorly placed can create misleading support. Conversely, a plausible genotype supported by few reads remains uncertain because there is little evidence. Depth should therefore be read alongside metrics about genotype uncertainty and read properties.

Genotype quality, commonly called GQ, is a Phred-scaled measure of uncertainty in the genotype assigned to one sample at one site. In a common interpretation, \(GQ=-10\log_{10}(P(\text{called genotype is wrong}))\), with probability calculated under the caller’s model. Higher GQ means less estimated probability of a genotype error. Actual implementations can use priors and may cap or otherwise define GQ differently, so the file specification governs the exact interpretation.

Genotype likelihoods provide a related but distinct quantity. For a diploid biallelic site, the possible genotypes are \(0/0\), \(0/1\), and \(1/1\), where 0 denotes the reference allele and 1 the alternate allele. A PL field usually reports normalized Phred-scaled likelihoods: the best-supported genotype has PL 0, and larger PL values indicate lower likelihood relative to that best genotype. PLs are likelihoods, not generally posterior probabilities; GQ is a genotype uncertainty summary and need not equal a simple PL difference in every caller.

Mapping Quality: Confidence in Read Placement

Mapping quality, often written MAPQ for an individual read, concerns the confidence that the read has been assigned to the correct genomic location. A common Phred interpretation is \(MAPQ=-10\log_{10}(P(\text{read is incorrectly mapped}))\). Thus a MAPQ of 20 corresponds approximately to an estimated mapping error probability of \(10^{-2}=0.01\), while MAPQ 60 corresponds approximately to \(10^{-6}\). These are mapping probabilities under the aligner’s model, not probabilities that the variant genotype is wrong.

A read can have high mapping quality and still contain a sequencing error. Likewise, a genotype can have high GQ even though some of its supporting reads are poorly mapped, depending on how evidence and annotations are calculated. These metrics address different uncertainties and should not be substituted for one another.

A variant file may report a site-level MQ summary derived from the mapping qualities of reads at that site. For example, one common convention reports a root-mean-square summary, \(MQ=\sqrt{\sum_i MAPQ_i^2/N}\), where \(N\) is the number of contributing reads. Other fields or callers can use different definitions. The field specification is necessary before interpreting a site-level number as an average or comparing it across files.

Allele Balance and Strand Bias

For a sample with a heterozygous genotype, allele balance is commonly calculated as the fraction of reference- and alternate-supporting reads that support the alternate allele:

Under a simple model of an unbiased diploid heterozygote, each allele has probability \(1/2\) of being represented by a read. If \(D\) independent informative reads are observed, the alternate count \(X\) then follows \(X\sim\operatorname{Binomial}(D,1/2)\), and \(AB=X/D\) is expected to be 0.5. This expectation is not a requirement that every observed sample have exactly equal counts. Sampling variation, allelic imbalance, read filtering, and technical effects can move the observed fraction away from one half. The simple model also does not capture dependence among reads, such as that introduced by amplification.

Strand bias refers to an uneven distribution of reference- or alternate-supporting reads across forward and reverse strands. If alternate support occurs almost entirely on one strand while reference support is more evenly distributed, that pattern can be a warning sign. Some callers report strand-bias annotations as a statistical summary; for example, a Fisher-strand field may be a Phred-scaled Fisher exact test result based on a two-by-two table of allele and strand counts. The precise field definition and direction of its scale are caller-specific. The counts themselves are useful context for any summary.

Which Level Does a Metric Describe?

The same site can have different evidence in different people. For example, one carrier may have a high-depth, high-GQ heterozygous call, while another sample has low depth and no confident genotype. A sample-level metric preserves that distinction. In contrast, a variant-level summary may combine information over reads or samples and can conceal individual differences if treated as though it described every genotype.

A practical reading order is to identify the field’s level and definition first, then inspect the value in context. For a sample genotype, check its depth and genotype quality, then examine allele counts and read-level evidence. For site-level summaries, ask which reads or samples were aggregated. Where the file convention is unclear, the abbreviation alone is not enough to establish the calculation.

Short Justification: Why Allele Balance Centers on One Half

The one-half expectation follows from the simple sampling model for a true heterozygote. Suppose each informative read independently comes from either homolog with equal probability. Let \(X\) be the number of alternate reads among \(D\) informative reads. Then \(E(X)=D(1/2)\). Since \(AB=X/D\), its expectation is

The derivation explains the reference point, not a rigid cutoff. Real sequencing data can depart from independent, equally likely sampling; consequently, a measured balance different from 0.5 should be interpreted alongside depth, read properties, and the caller’s model.

Common Mistakes

• Calling every quality score “confidence in the variant.” GQ concerns a sample’s genotype under a caller model; MAPQ concerns read placement. Neither has the same meaning as the other.
• Assuming more depth guarantees a valid call. A large number of reads can still be affected by systematic errors or poor mapping. Depth measures quantity, not the independence or correctness of the evidence.
• Treating allele balance of 0.5 as a mandatory rule. One half is the expectation under a simple unbiased heterozygote model. Finite counts and technical effects create variation.
• Interpreting a site summary as a sample-specific result. A variant-level MQ or strand-bias annotation may aggregate reads and cannot automatically be assigned to every individual genotype.
• Reading PL values without their genotype order. For a diploid biallelic site, the stated order matters. For example, PL values \((42,0,120)\) in \(0/0,0/1,1/1\) order support \(0/1\) as the best-likelihood genotype; \((0,42,120)\) instead place the minimum at \(0/0\).