Multinomial Likelihoods for Genotype Counts

Introduction

When genotypes are recorded for many individuals, the data may be summarized as three counts: the numbers with genotypes $AA$, $Aa$, and $aa$. A likelihood for these counts describes how plausible the observed composition is under specified probabilities for the three categories. Writing it correctly requires accounting for both the probability of each genotype and the number of different individual-level arrangements that produce the same counts.

This tutorial treats the three genotype probabilities as free parameters. It does not impose a relationship among them, such as one arising from Hardy–Weinberg equilibrium. The goal is to construct the multinomial likelihood, simplify it for inference about those probabilities, and interpret the resulting estimates.

The Three-Category Model

Definition: Suppose $n$ individuals are sampled independently, and each individual belongs to exactly one of the three diploid genotype categories $AA$, $Aa$, or $aa$. Let their probabilities be $q_{AA}$, $q_{Aa}$, and $q_{aa}$, where $$ q_{AA}\geq 0,\qquad q_{Aa}\geq 0,\qquad q_{aa}\geq 0,\qquad q_{AA}+q_{Aa}+q_{aa}=1. $$ If the observed category counts are $n_{AA}$, $n_{Aa}$, and $n_{aa}$, then $n=n_{AA}+n_{Aa}+n_{aa}$, and the count vector has a multinomial distribution with sample size $n$ and these three category probabilities.

The assumptions are important. The sample size is fixed; each individual is classified in exactly one category; and the sampled individuals have the same category probabilities and are independent. The model concerns genotype counts, not a genotype call with uncertainty. If individuals are sampled in a way that changes the genotype composition—for example, through selection based on genotype—then the probabilities describe that sampled population unless the sampling mechanism is modeled separately.

The parameter vector $(q_{AA},q_{Aa},q_{aa})$ lies on the probability simplex: the set of nonnegative triples that sum to one. There are only two freely varying probabilities because the third is determined by the sum constraint. This constraint is part of the model, not an additional genetic assumption.

From Individual Outcomes to a Count Likelihood

For one particular ordered sequence of genotypes, independence gives a product of category probabilities. For example, a sequence with 4 individuals of genotype $AA$, 3 of genotype $Aa$, and 2 of genotype $aa$ has probability $q_{AA}^{4}q_{Aa}^{3}q_{aa}^{2}$. But the counts alone do not specify the order in which those genotypes occurred. The multinomial coefficient counts how many ordered sequences have those same counts.

Likelihood: For observed counts $n_{AA},n_{Aa},n_{aa}$, the likelihood as a function of the genotype probabilities is $$ L(q_{AA},q_{Aa},q_{aa}) = \frac{n!}{n_{AA}!\,n_{Aa}!\,n_{aa}!} q_{AA}^{n_{AA}}q_{Aa}^{n_{Aa}}q_{aa}^{n_{aa}}. $$ Here $n=n_{AA}+n_{Aa}+n_{aa}$. A factor with exponent zero is defined to equal 1, including when its probability is zero.

The coefficient $n!/(n_{AA}!n_{Aa}!n_{aa}!)$ counts the distinct arrangements of the observed categories. It is needed when the likelihood is the probability of the count vector itself. For fixed observed counts, this coefficient does not depend on the probabilities, so it can be omitted when maximizing the likelihood over those probabilities. Keeping it in the count likelihood makes clear what data event is being modeled and preserves the probability of the observed counts.

The likelihood is zero if any category has a positive observed count but is assigned probability zero. A category with count zero contributes a factor of one, regardless of its probability, because no observation from that category was made. In log-likelihood expressions, the corresponding convention is that a zero count contributes zero; a positive count paired with zero probability gives log-likelihood negative infinity.

Worked Example: Writing the Count Likelihood

Suppose a hypothetical sample contains 16 individuals: 8 with genotype $AA$, 5 with genotype $Aa$, and 3 with genotype $aa$. Write the probability of these counts when the genotype probabilities are $q_{AA}=0.5$, $q_{Aa}=0.3$, and $q_{aa}=0.2$.

The counts sum to $8+5+3=16$, and the proposed probabilities are nonnegative and sum to $0.5+0.3+0.2=1$, so they satisfy the model constraints. The likelihood is $$ L = \frac{16!}{8!\,5!\,3!}(0.5)^8(0.3)^5(0.2)^3. $$

The coefficient is $$ \frac{16!}{8!\,5!\,3!} = \binom{16}{8}\binom{8}{5} = 12870(56) = 720720. $$ The probability product is $$ (0.5)^8(0.3)^5(0.2)^3 = 0.00390625(0.00243)(0.008) = 0.0000000759375. $$ Therefore, $$ L=720720(0.0000000759375)=0.054729675. $$

Under this hypothetical model, the probability of observing this particular count vector is about $0.05473$. The corresponding probability of one particular ordered sequence with these counts is only $0.00000007594$; the larger count probability includes all 720,720 possible arrangements.

Maximum-Likelihood Estimates

For a positive sample size, the maximum-likelihood estimate (MLE) is the probability vector that makes the observed count vector most likely, among all vectors on the simplex. The result is the observed fraction in each genotype category.

Theorem: If $n>0$, the multinomial likelihood over all nonnegative genotype probabilities summing to one is maximized at $$ \hat q_{AA}=\frac{n_{AA}}{n},\qquad \hat q_{Aa}=\frac{n_{Aa}}{n},\qquad \hat q_{aa}=\frac{n_{aa}}{n}. $$ A category with zero observed count has MLE zero. If $n=0$ is allowed, all three counts are zero and the likelihood is 1 for every probability vector on the simplex; every such vector is an MLE, and the count ratios are undefined.

The condition $n>0$ matters. With a positive sample size, at least one genotype category has a positive count, so the empirical fractions are defined and sum to one. With no sampled individuals, there are no observed fractions from which to estimate the category probabilities.

Why the Estimates Maximize the Likelihood

A short log-likelihood argument proves the result and also covers observed zero counts. Let $O$ be the set of categories whose observed counts are positive. For each category $i$ in $O$, write its count as $n_i$, its probability as $q_i$, and its observed fraction as $\hat q_i=n_i/n$. If any $q_i=0$ for $i\in O$, then the likelihood is zero, so such a vector cannot improve on the empirical vector, which gives positive probability to every observed category.

For a vector assigning positive probability to each observed category, the log-likelihood difference between that vector and the empirical vector is $$ \ell(q)-\ell(\hat q) = \sum_{i\in O} n_i\log\left(\frac{q_i}{\hat q_i}\right) = n\sum_{i\in O}\hat q_i\log\left(\frac{q_i}{\hat q_i}\right). $$ For $x>0$, $\log(x)\leq x-1$. Applying this inequality to each ratio gives $$ \ell(q)-\ell(\hat q) \leq n\sum_{i\in O}\hat q_i\left(\frac{q_i}{\hat q_i}-1\right) = n\left(\sum_{i\in O}q_i-1\right) \leq 0. $$ The last inequality follows because probabilities in the unobserved categories, if any, are nonnegative, and all three probabilities sum to one. Thus no admissible probability vector has a larger log-likelihood than the empirical vector. Equality requires the probabilities of the observed categories to equal their observed fractions and any unobserved categories to have probability zero. This establishes the stated MLE, including when one or more counts are zero.

Taking logarithms also shows why the coefficient is often left out during maximization: $$ \ell(q)= \log\left(\frac{n!}{n_{AA}!\,n_{Aa}!\,n_{aa}!}\right) +n_{AA}\log(q_{AA})+n_{Aa}\log(q_{Aa})+n_{aa}\log(q_{aa}). $$ The first term is constant in $q$. It is part of the likelihood for the counts, but it cannot change which probability vector maximizes that likelihood.

Worked Example: Estimating Probabilities When One Count Is Zero

A hypothetical sample has 12 individuals with genotype $AA$, none with genotype $Aa$, and 4 with genotype $aa$. Estimate all three genotype probabilities by maximum likelihood.

The sample size is $n=12+0+4=16>0$. Therefore, the MLE theorem applies: $$ \hat q_{AA}=\frac{12}{16}=0.75,\qquad \hat q_{Aa}=\frac{0}{16}=0,\qquad \hat q_{aa}=\frac{4}{16}=0.25. $$ These estimates are nonnegative and sum to $0.75+0+0.25=1$.

The likelihood at this estimate is $$ L(\hat q) = \frac{16!}{12!\,0!\,4!}(0.75)^{12}(0)^0(0.25)^4 = 1820(0.031676352)(1)(0.00390625) \approx 0.2252. $$ The zero count contributes a factor of 1, not zero. Assigning positive probability to $Aa$ would leave less probability available for the two observed categories and would lower the likelihood; the maximizing estimate places zero probability on the unobserved category.

The estimate $\hat q_{Aa}=0$ is the MLE in this sample, not proof that $Aa$ is impossible in the population. It records that no $Aa$ individuals appeared among these 16 sampled individuals.

Worked Example: The Empty Sample

Consider the boundary case in which no individuals were sampled. Then $n=0$ and the counts are $n_{AA}=n_{Aa}=n_{aa}=0$. What does the likelihood say about the genotype probabilities?

Using the count likelihood, with $0!=1$ and each zero-exponent factor equal to 1, $$ L(q_{AA},q_{Aa},q_{aa}) = \frac{0!}{0!\,0!\,0!} q_{AA}^{0}q_{Aa}^{0}q_{aa}^{0} = 1. $$ This is true for every nonnegative probability triple summing to one. For example, $(1,0,0)$, $(0.2,0.5,0.3)$, and $(0,0,1)$ all have likelihood 1.

Thus every point on the simplex is an MLE when $n=0$; the data provide no information that distinguishes among the probability vectors. The ratios $n_{AA}/n$, $n_{Aa}/n$, and $n_{aa}/n$ are undefined because their denominators are zero, so the positive-sample-size formula must not be applied.

Interpreting the Fitted Probabilities

For $n>0$, $\hat q_{AA}$, $\hat q_{Aa}$, and $\hat q_{aa}$ are the observed genotype proportions and are MLEs of the corresponding probabilities in the modeled population. They are dimensionless proportions, not counts and not odds. For example, $\hat q_{Aa}=0.25$ means that one quarter of the sampled individuals had genotype $Aa$; under the model, it is also the maximum-likelihood estimate of the probability that a sampled individual has that genotype.

The three estimates are constrained to sum to one, so they should not be treated as three unrelated parameters. They describe genotype-category probabilities directly. The model does not say how those probabilities relate to allele frequencies, and it does not require a particular pattern across the three genotype categories. Such restrictions would define a different model.

Common Mistakes and Why the Model Matters

Dropping the multinomial coefficient when reporting a count probability. The product of powers is the probability of one ordering, not of the whole count vector. Include $n!/(n_{AA}!n_{Aa}!n_{aa}!)$ for the count likelihood.
Thinking the coefficient changes the MLE. It does not, because it is constant in the probabilities for fixed observed counts. It still belongs in the likelihood for the count data.
Setting a zero-count factor to zero. A category observed zero times contributes $q^0=1$, including at $q=0$. This is why the MLE for that category can be zero.
Using count ratios when $n=0$. The ratio is undefined. With no sampled individuals the likelihood is constant, and every probability vector is an MLE.
Interpreting the estimates as evidence for an unstated genetic constraint. The unrestricted multinomial model estimates three genotype probabilities subject only to summing to one. It does not by itself establish a relationship among them.

The multinomial formulation matters because it distinguishes a model for category probabilities from the observed count pattern. It provides a precise likelihood for genotype composition, handles zero counts coherently, and makes explicit which assumptions support interpreting the observed proportions as estimates of population probabilities.

Check Your Understanding

Answer each question using the three-category multinomial model.

For counts $n_{AA}=6$, $n_{Aa}=2$, and $n_{aa}=4$, write the likelihood, including the multinomial coefficient.
Why does the multinomial coefficient not affect the MLE when the observed counts are fixed?
If $n_{Aa}=0$ but the total sample size is positive, what is the MLE of $q_{Aa}$, and what does that estimate mean?
State the assumptions that allow the genotype count vector to have a multinomial distribution.
When $n=0$, what is the likelihood for each probability vector, and why are the empirical-fraction estimates undefined?