Choosing QC Thresholds Without Overfiltering

Why Threshold Choice Requires Judgment

Genotyping QC produces measurements such as sample call rate, variant call rate, genotype quality, and depth. A threshold turns one of these measurements into a decision: retain or exclude a sample, a variant, or sometimes an individual genotype call. The decision matters in both directions. A threshold that is too permissive may retain unreliable data; one that is too strict may discard useful data, reduce the number of samples or variants available for analysis, and change which data remain.

There is rarely one cutoff that is automatically correct for every platform, study, or intended analysis. A defensible choice begins by examining the observed distribution, then asking whether its features make technical sense. Candidate cutoffs should be compared by their consequences, and important decisions should be checked for sensitivity to reasonable alternatives.

Sample call rate is the fraction of eligible genotypes that are called for a sample; variant call rate is the fraction of eligible samples with a called genotype at a variant. A threshold on one is not interchangeable with a threshold on the other. Other metrics may use a different rule: for example, a maximum threshold might be appropriate for a metric where unusually high values are concerning. The direction and meaning of each rule must follow the metric, rather than being inferred from a general preference for higher or lower values.

The denominator also matters. Call rates depend on which samples, variants, chromosomes, or genotype calls are considered eligible. If the eligible set changes after exclusions, the metric may change too. Record the denominator and the order in which rules are applied so that a reported threshold has a reproducible meaning.

Reading a QC Distribution Before Choosing a Cutoff

Start with a plot or a clear numerical summary of the metric across samples or variants. Look for the overall range, clusters, long tails, gaps, and isolated outliers. Then examine whether notable patterns align with technical information, such as assay design, sequencing depth, genotyping batch, or sample handling. A visible gap can suggest a practical boundary, but it does not prove that observations on one side are unreliable. A smooth distribution may offer no natural dividing point at all.

The distribution should be considered at the level relevant to the threshold. A pooled sample-call-rate distribution can conceal a problem concentrated in one batch. A pooled variant-call-rate distribution can similarly obscure uneven performance across technical groups. Stratifying a diagnostic plot does not by itself determine the correct decision; it helps reveal whether a single pooled cutoff is an adequate description of the data.

Technical context helps distinguish a plausible exclusion rule from a convenient one. For example, a low call rate could reflect generally poor sample performance, or it could be concentrated in a technically distinct group. A low variant call rate could reflect weak assay performance, or a region with known measurement limitations. A threshold should not be chosen solely because it removes a desired number of observations or produces a visually tidy dataset.

Compare Candidate Rules, Not Just Candidate Numbers

A useful threshold comparison reports what each candidate would remove and what technical evidence motivates the decision. The retained percentage is important, but it is not a quality metric: retaining more data does not establish that those data are reliable, and removing more data does not establish that the remainder is unbiased or error-free.

For a minimum threshold, raising the cutoff can only maintain or reduce the retained set, provided the same metric and eligible set are used. This simple fact is useful when checking calculations and explaining trade-offs. The cutoff should then be assessed against independent technical evidence where available, such as call-quality distributions or known assay limitations, rather than justified only by the number it excludes.

Technical Context and the Denominator

A threshold may appear reasonable in a pooled summary but behave differently when data are examined by technical group. Consider a hypothetical variant called in 248 of 250 samples in batch A and 230 of 250 in batch B. Its pooled call rate is

A pooled minimum of 0.95 would let this variant pass. Yet its batch-specific rates are \(248/250=0.992\) in batch A and \(230/250=0.920\) in batch B. The pooled pass therefore does not show that performance is comparable across batches. This pattern calls for technical investigation; it is not a reason to assume that the variant is either reliable everywhere or unusable everywhere. The threshold decision should be made with the observed pattern and the study's technical context in view.

Sample and variant exclusions can also affect one another because call-rate denominators change. Suppose a hypothetical variant is called in 95 of 100 samples. Ten samples are subsequently excluded under a sample-level rule; 7 of those ten had a call at this variant and 3 were missing. The remaining data have 88 calls among 90 samples, so the recalculated variant call rate is \(88/90=0.9778\), rather than \(95/100=0.9500\). The increase follows from removing samples, including three that were missing at this variant. It is a consequence of the changed denominator, not independent evidence that the variant's calls became more accurate.

Sensitivity Analysis: Is the Decision Fragile?

A sensitivity analysis repeats the relevant summaries or downstream checks under a small set of defensible alternative thresholds. The aim is not to search for the cutoff that gives a preferred result. It is to learn whether conclusions depend strongly on a judgment call and whether stricter or more permissive choices change the data in technically important ways.

For example, investigators could compare sample call-rate thresholds of 0.95, 0.97, and 0.99, as in the earlier calculation. If the stricter rule removes many additional samples but does not correspond to a distinct technical concern, that is a reason to scrutinize the stricter choice. If the added exclusions are supported by independent evidence of poor genotyping, a loss in sample count may be justified. Neither outcome can be decided by retention percentages alone.

A Short Justification for Monotonicity

The way a minimum threshold changes retention follows directly from the pass rule. Let \(R(t)\) be the set of samples whose call rates are at least \(t\). If \(t_2>t_1\), any sample with \(r_i\geq t_2\) must also satisfy \(r_i\geq t_1\). Therefore every sample passing the stricter rule also passes the more permissive one:

This proves that raising a minimum cutoff cannot add retained samples; it can only leave the set unchanged or remove samples. The same argument applies to variants using a minimum variant call-rate threshold. It does not prove that the stricter rule improves the validity of an analysis. That conclusion requires evidence about the quality of the removed data and the consequences of their removal.

Common Mistakes and Practical Interpretation

• Treating a familiar number as a universal standard. A cutoff used in one dataset is not automatically suitable for another platform or study. Describe the metric distribution and technical rationale for the chosen rule.
• Choosing a cutoff only to retain a target number. A desired sample or variant count does not establish data quality. Compare the candidates' consequences with technical evidence.
• Assuming a threshold pass certifies accuracy. A call-rate rule addresses missingness, not every possible genotype error. Passing one threshold does not override concerns from other QC metrics.
• Using pooled metrics without checking technical patterns. A pooled rate can pass despite poor performance in a subgroup. Inspect relevant technical strata before interpreting the pooled result.
• Ignoring changes in the denominator. Removing samples can change variant call rates. Record which samples and variants were eligible when each metric was calculated.
• Calling a sensitivity check a search for the best-looking result. Alternative thresholds are for assessing robustness, not for selecting the rule that produces a preferred scientific conclusion.

Threshold choice balances two legitimate aims: removing data that are insufficiently reliable for the intended use and retaining data that remain informative. The balance should be made explicit. A clear record of candidate cutoffs, their retention consequences, the technical evidence considered, and the sensitivity of relevant conclusions makes the decision interpretable and reproducible.