Inverse-Probability Correction for Variant Ascertainment

Why Correct for Variant Ascertainment?

A frequency summary calculated from selected variants describes those variants as observed, not necessarily the full set of variants of interest. If selection favors variants that are common in a discovery sample, for example, the selected set may have a different average frequency from the target set. When each variant’s probability of selection is known, inverse-probability weighting can correct the summary for unequal selection chances.

The correction has a clear target: a summary over a defined set of variants. It does not recover variants that had no chance of selection, and it does not remove bias caused by using the same noisy frequency estimate both to select variants and to estimate their frequencies.

The target mean gives every variant in the defined set equal importance. The weight \(1/\pi_i\) compensates for the fact that a variant with a smaller selection probability represents more variants like itself in the target set. The denominator is the known number \(M\), not the number selected. This makes the estimator an estimate of a fixed-set mean rather than a mean normalized by the realized selected sample.

Conditions for the Correction

The estimator’s basic unbiasedness statement relies on a sampling design, not just on the presence of weights. First, the target set and its size \(M\) must be specified. Second, every variant in that set must have a positive, known selection probability: \(\pi_i>0\). Third, the selection probabilities used in the calculation must describe the actual ascertainment process.

For the simple variance formulas below, assume variants are selected independently, so the \(S_i\) are independent Bernoulli variables with probabilities \(\pi_i\). Also assume that \(\tilde{p}_i\) is unbiased for \(p_i\), with variance \(\tau_i^2\), and is obtained from data independent of the selection event. Errors in the frequency estimates for different variants are taken to be independent for these formulas. If variants are selected dependently or frequency errors are correlated, covariance terms are needed.

A usual way to support the independence condition is to use one data set to ascertain variants and an independent validation sample to estimate their frequencies. If selection depends on a noisy discovery estimate and that same estimate is then reported as \(\tilde{p}_i\), the selected estimates can be systematically high or otherwise distorted. Weighting by \(\pi_i\) alone does not generally repair this selection-induced dependence.

Why the Estimator Is Unbiased

For a fixed variant \(i\), selection occurs with probability \(\pi_i\). Under the stated assumptions, its frequency estimate has expectation \(p_i\) and is independent of selection. Therefore:

Averaging these unbiased contributions over the \(M\) variants gives \(E(\hat{\theta}_{HT})=M^{-1}\sum_i p_i=\theta\). This is a design-based statement: the frequencies \(p_i\) are fixed, while which variants are selected is random. If frequency estimates are also random but unbiased and independent of selection, the same expectation still holds.

Variance and the Cost of Small Probabilities

For independent selection and independent frequency-estimation errors, the contribution for variant \(i\) has variance

The first term is variation from whether the variant is selected. The second is variation from estimating its frequency, amplified because that estimate contributes only when the variant is selected. Thus the variance of the mean estimator is

The formula assumes independent selections and errors across variants. If those assumptions do not hold, covariances between variant contributions must also be included. In particular, linkage or shared sampling effects can make frequency-estimation errors correlated, so the sum of individual variances alone may not describe total uncertainty.

Rare Selection Probabilities and Sensitivity

A small \(\pi_i\) creates a large weight. When a rare-probability variant is selected, its contribution to the estimated mean is \( \tilde{p}_i/(M\pi_i)\), which can be large even if its frequency is modest. When it is not selected, it contributes zero. This contrast creates high variability across repeated ascertainment samples.

Interpretation and Practical Checks

The inverse-probability estimate is most useful when the target variant set is meaningful, selection probabilities are credible, and no target variant has zero probability of inclusion. It corrects unequal representation under the specified ascertainment mechanism; it does not correct genotype-call errors, population mismatch, or an incorrectly defined target set. Allele coding must also be consistent across variants and populations for the frequency summary to have the intended interpretation.

Very large weights signal that the estimate may be sensitive to a few selected variants. Report the target set, how the probabilities were obtained, the weighted estimate, and an uncertainty calculation that matches the selection design. A sensitivity analysis can show how estimates change when uncertain selection probabilities are varied within plausible ranges. Truncating or capping weights may reduce variance, but it generally changes the estimator and can introduce bias; it is not a cost-free repair.

• Common mistake: “The selected variants have an average frequency of 0.12, so that is the average frequency in the full target set.” This ignores unequal selection probabilities and confuses the selected-set mean with the target-set mean.
• Correction: Define the target set and weight each selected estimate by \(1/\pi_i\), using the known frame size \(M\) for the Horvitz–Thompson mean. Then assess uncertainty, especially for small \(\pi_i\).
• Another limitation: If \(\pi_i=0\) for a target variant, no observed data can represent that variant through inverse weighting. The target set or ascertainment design must change.