Transmission Likelihoods with Uncertain Genotype Calls

Why Genotype Uncertainty Matters

A genotype inferred from sequencing reads is not always certain. At a biallelic locus, a diploid individual has genotype $0$, $1$, or $2$, where the number is the count of copies of the alternate allele. Reads provide evidence for these possibilities, but even a genotype with the largest read likelihood may not be the true genotype. In a parent-offspring trio, the offspring’s genotype is also constrained by Mendelian inheritance. A suitable likelihood must combine both sources of information without first replacing uncertain genotypes with single calls.

The key operation is summation over possible latent genotypes. For each possible pair of parental genotypes, Mendelian transmission assigns probabilities to the offspring’s genotypes. The read likelihood for each offspring genotype is weighted by that transmission probability. If the parental genotypes are uncertain too, their possibilities are weighted by their read evidence and any specified genotype prior, then summed as well.

Genotype Likelihoods and Transmission Probabilities

Definition: Let $D_i$ denote the reads observed for individual $i$, and let $G_i\in\{0,1,2\}$ denote that individual’s genotype at a diploid biallelic locus. The read-based genotype likelihood is $L_i(g)=P(D_i\mid G_i=g)$. It measures how compatible the reads are with genotype $g$; it is not, by itself, the probability that the genotype is $g$.

Likelihoods need not sum to one across genotypes. They may be reported on any common proportional scale: multiplying all three likelihoods for one individual by the same positive constant does not change their relative support. A genotype posterior probability would additionally require a genotype prior and normalization.

For a parent with genotype $g$, the probability that a randomly transmitted allele is alternate is $g/2$. Assuming ordinary Mendelian transmission at this locus, the mother and father each transmit one allele. The offspring genotype $g_c$ is the sum of the two transmitted alternate-allele indicators.

Definition: The Mendelian transmission probability $T(g_c\mid g_m,g_f)$ is the probability that the offspring has genotype $g_c$, conditional on maternal genotype $g_m$ and paternal genotype $g_f$. With $a_m=g_m/2$ and $a_f=g_f/2$, $$ T(0\mid g_m,g_f)=(1-a_m)(1-a_f),\qquad T(1\mid g_m,g_f)=a_m(1-a_f)+(1-a_m)a_f,\qquad T(2\mid g_m,g_f)=a_ma_f. $$

These probabilities sum to one because they cover the four possible combinations of transmitted alleles: both reference, one alternate, or both alternate, with the two ways to transmit exactly one alternate included in the middle probability. This model assumes diploid inheritance at a non-sex-linked locus, no mutation in transmission, and no transmission distortion. Read errors can still be represented in $L_i(g)$; they do not alter the Mendelian transmission probabilities.

Likelihood When Parental Genotypes Are Known

If parental genotypes are treated as known, only the offspring genotype needs to be summed over. Conditional on the parental genotypes, the probability of the offspring’s reads is $$ P(D_c\mid g_m,g_f)=\sum_{g_c=0}^{2}T(g_c\mid g_m,g_f)L_c(g_c). $$ This is a likelihood for the offspring read data under the specified parental genotypes. If parental reads are also included and are conditionally independent given their genotypes, their likelihood contributions multiply this sum.

Worked Example: Summing Over an Uncertain Offspring Genotype

Suppose the mother is known to have genotype $1$, and the father is known to have genotype $0$. The offspring read likelihoods are $L_c(0)=0.10$, $L_c(1)=0.80$, and $L_c(2)=0.20$. Calculate the likelihood of the offspring reads under these parental genotypes.

The mother transmits the alternate allele with probability $1/2$; the father, who has genotype $0$, transmits a reference allele with probability $1$. Therefore $T(0\mid1,0)=1/2$, $T(1\mid1,0)=1/2$, and $T(2\mid1,0)=0$. The likelihood is $$ \sum_{g_c=0}^{2}T(g_c\mid1,0)L_c(g_c) =(0.50)(0.10)+(0.50)(0.80)+(0)(0.20) =0.05+0.40+0 =0.45. $$

The offspring reads have likelihood $0.45$ under these parental genotypes. The genotype $2$ read likelihood contributes nothing here because this parental pair cannot produce genotype $2$ under the stated transmission model. The result is not the probability that the offspring has genotype $1$; it is the likelihood of the reads after accounting for transmission.

Summing Over Uncertain Parental Genotypes

When parental genotypes are uncertain, they too must remain latent in the calculation. One convenient form uses parental genotype weights $w_m(g_m)$ and $w_f(g_f)$. These can be posterior genotype probabilities based on each parent’s reads and a specified prior. If the parents’ genotypes are independent given their own data and the model, the offspring read evidence conditional on the parental reads is $$ \sum_{g_m=0}^{2}\sum_{g_f=0}^{2}w_m(g_m)w_f(g_f) \sum_{g_c=0}^{2}T(g_c\mid g_m,g_f)L_c(g_c). $$

This is a weighted average of the offspring read likelihood across parental genotype pairs. If the weights are posterior probabilities based on parental reads, this quantity is the offspring read evidence conditional on those parental reads. A joint likelihood for all three individuals also includes the marginal likelihood contributions from the parents’ reads.

Worked Example: Averaging Across Two Possible Genotypes per Parent

Suppose the mother’s genotype weights are $w_m(0)=0.70$ and $w_m(1)=0.30$, and the father’s are $w_f(0)=0.40$ and $w_f(1)=0.60$. For this example, the weights assign zero mass to genotype $2$. The offspring read likelihoods are $L_c(0)=0.20$, $L_c(1)=0.60$, and $L_c(2)=0.10$. Find the offspring read evidence conditional on the parental reads.

For parental genotypes $(0,0)$, the offspring must be genotype $0$, giving read likelihood $0.20$. For $(0,1)$ or $(1,0)$, the offspring has genotype $0$ or $1$ with probability $1/2$ each, giving $$ (0.50)(0.20)+(0.50)(0.60)=0.40. $$ For $(1,1)$, transmission probabilities are $1/4,1/2,1/4$ for offspring genotypes $0,1,2$, so the read likelihood is $$ (0.25)(0.20)+(0.50)(0.60)+(0.25)(0.10) =0.05+0.30+0.025 =0.375. $$

The parental pair weights are $0.70(0.40)=0.28$ for $(0,0)$, $0.70(0.60)=0.42$ for $(0,1)$, $0.30(0.40)=0.12$ for $(1,0)$, and $0.30(0.60)=0.18$ for $(1,1)$. Weighting each offspring read likelihood by its parental pair weight gives $$ (0.28)(0.20)+(0.42)(0.40)+(0.12)(0.40)+(0.18)(0.375) =0.056+0.168+0.048+0.0675 =0.3395. $$

Thus the conditional offspring read evidence is $0.3395$ on the stated likelihood scale. The calculation averages over all parental genotype pairs with nonzero weight; it does not select the most likely genotype for each parent and treat those selections as certain.

The Full Trio Likelihood

A full trio likelihood includes the read data from both parents and the offspring. Let $D_m,D_f,D_c$ denote their respective reads. Specify a joint prior probability $\pi(g_m,g_f)$ for the parental genotypes. For unrelated founder parents, one common assumption is that their genotypes are independent under the chosen population model, so $\pi(g_m,g_f)=\pi_m(g_m)\pi_f(g_f)$. If that assumption is not appropriate, a joint prior can be used instead.

Definition: Under conditional independence of the three individuals’ reads given their genotypes, the trio likelihood is $$ L_{\mathrm{trio}} =\sum_{g_m=0}^{2}\sum_{g_f=0}^{2}\sum_{g_c=0}^{2} \pi(g_m,g_f)\, L_m(g_m)L_f(g_f)L_c(g_c)\, T(g_c\mid g_m,g_f). $$

The offspring has no separate genotype prior in this expression: conditional on the parental genotypes, its genotype distribution is supplied by $T$. A population genotype prior for the parents is needed if their genotypes are to be averaged over in a joint likelihood. The prior may depend on model parameters, in which case that dependence is part of the likelihood.

Why the Summation Is the Correct Operation

The formula follows from the law of total probability applied to the unobserved genotypes. For a particular triple of genotypes, the joint probability of the reads and genotypes is the probability of the parental genotypes, multiplied by the read likelihoods and the conditional transmission probability. The reads’ likelihood is obtained by adding these contributions over all possible triples.

Specify a possible genotype triple.
For each $(g_m,g_f,g_c)$, assign parental probability $\pi(g_m,g_f)$ and transmission probability $T(g_c\mid g_m,g_f)$.

Attach the read evidence.
Under conditional independence of reads given genotypes, multiply by $L_m(g_m)L_f(g_f)L_c(g_c)$.

Sum over latent possibilities.
Because the genotype triple is not observed with certainty, add the contribution for every possible triple, giving the trio likelihood.

The resulting expression is a sum, not a product of sums that ignores inheritance. The transmission term links the offspring genotype to the parental genotypes. Without it, the model would treat the three genotypes as unrelated.

Worked Example: Averaging Over All Three Genotypes

Suppose the maternal posterior genotype weights based on her reads are $(0.50,0.30,0.20)$ for genotypes $0,1,2$, and the paternal weights are the same. The offspring read likelihoods are $(0.70,0.20,0.10)$. Calculate the offspring read evidence conditional on these parental reads, assuming the parental genotype weights are independent.

For either parent, the probability of transmitting an alternate allele averaged over that parent’s genotype weights is $$ q=(0.50)(0/2)+(0.30)(1/2)+(0.20)(2/2) =0+0.15+0.20 =0.35. $$ The maternal and paternal transmitted alleles are independent under the stated assumptions. Thus, after averaging over all parental genotype pairs, the offspring genotype probabilities are $(1-q)^2$, $2q(1-q)$, and $q^2$. Substitution gives $$ P(G_c=0)=0.65^2=0.4225,\quad P(G_c=1)=2(0.35)(0.65)=0.4550,\quad P(G_c=2)=0.35^2=0.1225. $$

Weight these probabilities by the offspring read likelihoods: $$ (0.4225)(0.70)+(0.4550)(0.20)+(0.1225)(0.10) =0.29575+0.09100+0.01225 =0.3990. $$

The conditional offspring read evidence is $0.3990$. The simplification through $q$ is valid here because the parental weights are independent and each parent contributes one transmitted allele. It is a way to evaluate the sum over all parental and offspring genotypes, not a replacement for the transmission model.

Assumptions and a Common Pitfall

The calculations rely on a specified inheritance model and on a clear interpretation of the read likelihoods. In particular, the read data are assumed conditionally independent across individuals given their genotypes; the parental genotype prior must reflect the intended population or pedigree assumptions; and the transmission probabilities must match the locus and inheritance pattern being analyzed. At a locus with mutation, copy-number variation, or nonstandard inheritance, the transmission model must be changed rather than silently using the diploid formula above.

Pitfall: choosing hard calls first. A common but incorrect shortcut is to select the highest-likelihood genotype for each individual, then calculate the trio likelihood as if those calls were certain. This discards uncertainty and can give an overly small likelihood when an alternative genotype combination has appreciable read support and is compatible with transmission.
Correction: Keep each genotype as a latent possibility. Multiply its read likelihood by the appropriate prior or transmission probability, and sum across possibilities.
Pitfall: treating genotype likelihoods as probabilities. The three values $L_i(0),L_i(1),L_i(2)$ are not automatically probabilities that sum to one.
Correction: Use likelihoods as likelihoods. To obtain genotype posterior probabilities, combine them with a prior and normalize; to form the trio likelihood, use the specified parental prior and transmission probabilities in the full sum.

Key takeaway: Read evidence describes how well each genotype explains the reads; Mendelian transmission connects the trio’s genotypes. The trio likelihood combines these ingredients by weighting each possible genotype configuration and summing over configurations that were not observed with certainty.

Check Your Understanding

Use the definitions and assumptions in this tutorial to answer the following questions.

If both parents have genotype $0$, what offspring genotype probabilities does the diploid Mendelian model assign?
Why is a read-based genotype likelihood not automatically a posterior probability for that genotype?
When parental genotypes are uncertain, what quantities weight the offspring read likelihood under each parental genotype pair?
What conditional-independence assumption allows the three individuals’ read likelihoods to be multiplied in the full trio likelihood?
Give one reason the diploid Mendelian transmission formula might need to be changed for a particular locus.