Estimating Global Ancestry in Admixed Samples

Why Estimate Global Ancestry?

An admixed person may have genetic contributions associated with more than one reference population. A global ancestry estimate summarizes those contributions across the genome using observed genotypes and allele frequencies in selected reference panels. The calculation is useful because it turns many marker-level comparisons into a small set of interpretable proportions. But those proportions are not direct measurements of a person’s family tree: they depend on the reference populations and statistical model used.

Here the goal is a supervised likelihood estimate. “Supervised” means that the reference groups and their allele frequencies are specified in advance. The method then finds the mixture weights that make the observed genotypes most likely under that model. It does not decide which populations should be used, and it does not locate ancestry along particular chromosome segments.

Definition: For an individual modeled using \(K\) reference populations, the global ancestry proportions are \(\theta_1,\ldots,\theta_K\), where \(\theta_k\) is the modeled fraction of ancestry assigned to reference population \(k\). They satisfy \(\theta_k \geq 0\) and \(\sum_{k=1}^K \theta_k=1\). They are estimated mixture weights, conditional on the chosen references and model.

Genotype Probabilities and the Likelihood

At marker \(j\), let \(p_{kj}\) be the frequency of a specified allele in reference population \(k\). Let \(g_j\) be the target individual’s genotype dosage for that allele: 0, 1, or 2 copies. Given ancestry proportions \(\boldsymbol{\theta}\), the model’s allele frequency at that marker is the mixture

q_j(\boldsymbol{\theta})=\sum_{k=1}^K \theta_k p_{kj}.

To turn this allele frequency into a genotype probability, the basic model treats the two allele copies at a marker as independent draws from that mixture. Thus the dosage has a binomial distribution with two trials and success probability \(q_j\). This is a Hardy–Weinberg form for the modeled mixture at that marker; it is an assumption of the likelihood, not a conclusion established by fitting it.

P(g_j\mid \boldsymbol{\theta})= \binom{2}{g_j}q_j(\boldsymbol{\theta})^{g_j} \left[1-q_j(\boldsymbol{\theta})\right]^{2-g_j}.

For a set of markers treated as independent, multiply these probabilities to obtain the likelihood. Equivalently, maximize the log-likelihood, which is easier to calculate:

L(\boldsymbol{\theta})=\prod_{j=1}^M P(g_j\mid\boldsymbol{\theta}), \qquad \ell(\boldsymbol{\theta})= \sum_{j=1}^M\left[ \log\binom{2}{g_j} +g_j\log q_j +(2-g_j)\log(1-q_j) \right].

The estimate \(\hat{\boldsymbol{\theta}}\) is a value in the simplex—that is, it obeys the nonnegative and sum-to-one constraints—that maximizes this likelihood. The binomial coefficient does not depend on the proportions, so it can be omitted when finding the maximizing values. The reference allele frequencies are treated as fixed inputs in this calculation.

Model assumptions: The reference groups and their allele frequencies are appropriate inputs for the target; each marker’s genotype is represented by the dosage of a specified allele; the two allele copies at a marker follow the stated binomial mixture model; and the marker likelihoods are multiplied as if the included markers were independent. In practice, linkage disequilibrium can violate the last assumption, so analyses may use a set of markers chosen to limit redundancy or treat the product as a composite likelihood. Genotype uncertainty and uncertainty in estimated reference frequencies are not represented by this basic likelihood.

Worked Examples

Worked Example: One Marker With Two Reference Groups

Suppose a hypothetical individual is heterozygous at one marker, so \(g_1=1\). The specified allele has frequency \(0.20\) in reference group A and \(0.80\) in reference group B. Let \(\theta_A=\theta\), so \(\theta_B=1-\theta\). The modeled allele frequency is

q_1(\theta)=0.20\theta+0.80(1-\theta)=0.80-0.60\theta.

For a heterozygote, the likelihood is \(2q_1(1-q_1)\). This is maximized at \(q_1=0.50\): the product \(q_1(1-q_1)\) is largest when its two factors are equal. Solving for the mixture weight gives

0.80-0.60\hat{\theta}=0.50, \qquad \hat{\theta}=\frac{0.80-0.50}{0.60}=0.500.

The fitted proportions are 0.500 for group A and \(1-0.500=0.500\) for group B. This single-marker example illustrates the likelihood calculation, not a reliable genome-wide analysis: one genotype provides very little information, and its estimate is highly dependent on the specified reference frequencies.

Worked Example: Combining Two Markers

Consider a second hypothetical individual with a heterozygous genotype at each of two independent markers. At marker 1, the allele frequencies are \(0.20\) in group A and \(0.80\) in group B; at marker 2, they are \(0.80\) in group A and \(0.20\) in group B. Again, write \(\theta\) for the proportion assigned to A. Then

q_1=0.80-0.60\theta, \qquad q_2=0.20+0.60\theta.

Because both dosages equal 1, the likelihood is

L(\theta)=\left[2q_1(1-q_1)\right]\left[2q_2(1-q_2)\right].

At \(\theta=0.500\), both modeled allele frequencies are \(0.500\). Each heterozygote then has probability \(2(0.5)(0.5)=0.500\), giving \(L(0.500)=0.500\times0.500=0.250\). To check that this is the maximum, note that \(q_2=1-q_1\), so the two heterozygote probabilities are equal. For any \(q\) between 0 and 1, \(q(1-q)\leq 0.25\), since \((q-0.5)^2\geq0\). Therefore \(L(\theta)\leq 0.25\), with equality when both frequencies are 0.5. That occurs at \(\hat{\theta}=0.500\).

The opposing reference-frequency patterns make the two markers informative in complementary ways. In a real analysis, the same principle applies across many markers, while linkage between markers and the suitability of the references must also be considered.

Optimization and Identifiability

With more markers, the maximum generally has no simple closed-form expression. Numerical optimization searches over the simplex for the proportions giving the largest log-likelihood. The boundary is part of the allowed parameter space: an estimate can be zero for a reference group if the likelihood is maximized there. A fitted vector must always be nonnegative and sum to one.

The likelihood has a useful shape. Each \(q_j\) is a linear function of the proportions, and the log-likelihood contribution from a marker is \(g_j\log q_j+(2-g_j)\log(1-q_j)\), apart from a constant. For \(0<q_j<1\), its second derivative with respect to \(q_j\) is

-\frac{g_j}{q_j^2}-\frac{2-g_j}{(1-q_j)^2}\leq 0.

So that contribution is concave in \(q_j\), and therefore concave in the proportions because \(q_j\) is linear in them. A sum of concave functions is concave. Consequently, this likelihood has no isolated inferior local maximum within the feasible region; the global maximum can be sought by constrained optimization. Concavity does not guarantee a unique estimate. If the reference groups have indistinguishable allele frequencies at the analyzed markers, different proportions can yield the same likelihood.

The following example uses that connection between marker-level allele frequencies and mixture weights. It is a small, constructed data set for demonstrating the calculation, not a realistic genome-wide panel.

Worked Example: Fitting Three Ancestry Proportions

Suppose ten hypothetical individuals have been genotyped at each of three independent markers, and their combined dosages at the markers are 10, 6, and 10 copies. The corresponding sample allele frequencies are \(10/20=0.50\), \(6/20=0.30\), and \(10/20=0.50\), because there are two allele copies per individual. The specified-allele frequencies in three reference groups, A, B, and C, are:

Marker	Group A	Group B	Group C	Observed allele frequency
1	0.80	0.20	0.20	0.50
2	0.10	0.60	0.35	0.30
3	0.50	0.40	0.65	0.50

At each marker, the genotype likelihood is maximized when its modeled frequency equals the observed allele frequency. To see this, if the total dosage is \(G\) among \(n\) diploid individuals, the log-likelihood as a function of \(q\), apart from constants, is \(G\log q+(2n-G)\log(1-q)\). Setting its derivative to zero gives \(G/q-(2n-G)/(1-q)=0\), hence \(q=G/(2n)\). Here the three target frequencies are therefore \(0.50\), \(0.30\), and \(0.50\).

Let the proportions be \(\theta_A,\theta_B,\theta_C\). The first marker requires

0.80\theta_A+0.20\theta_B+0.20\theta_C=0.50.

Using \(\theta_A+\theta_B+\theta_C=1\), the left side is \(0.20+0.60\theta_A\), so \(0.20+0.60\theta_A=0.50\), giving \(\theta_A=0.500\). The second marker requires

0.10(0.500)+0.60\theta_B+0.35\theta_C=0.30, \qquad \theta_C=0.500-\theta_B.

Substituting the second expression into the first gives \(0.050+0.60\theta_B+0.35(0.500-\theta_B)=0.30\). Thus \(0.225+0.25\theta_B=0.30\), so \(\theta_B=0.300\) and \(\theta_C=0.200\). The third marker checks the solution: \(0.50(0.500)+0.40(0.300)+0.65(0.200)=0.250+0.120+0.130=0.500\), matching its observed frequency.

The resulting fitted proportions are 0.500 for A, 0.300 for B, and 0.200 for C. Because these values make each marker’s modeled frequency equal its likelihood-maximizing frequency, they maximize the product likelihood in this constructed example. The markers distinguish the groups sufficiently to determine these weights here; in practical data, many markers are used, and the result depends on how well the chosen references represent the relevant variation.

What the Proportions Mean—and Do Not Mean

A result such as \((0.50,0.30,0.20)\) means that, under the specified reference panel and genotype likelihood, the best-fitting mixture assigns those weights to the three reference groups. It is not a statement that exactly 50% of a person’s ancestors came from a named present-day population, nor that exactly 50% of their DNA can be traced to a particular set of genealogical ancestors. Reference groups are statistical representations of sampled populations, not timeless or perfectly separated biological categories.

The estimates are also global: they summarize across the markers in the analysis. They do not specify where on the chromosomes a modeled contribution occurs. A value can change if the reference groups, markers, or model assumptions change. Thus, interpret the fitted proportions in context and report which reference groups were used. A numerical result with several decimal places does not remove this dependence.

Common Mistakes

Calling a fitted proportion an exact genealogical fraction. The likelihood estimates mixture weights relative to specified reference allele frequencies; it does not reconstruct a complete family tree.
Ignoring the sum-to-one constraint. Optimizing each group’s weight separately can produce invalid values. The fitted proportions must be nonnegative and add to 1.
Assuming all markers contribute independent evidence. Multiplying marker likelihoods as if independent can overstate information when markers are correlated. Marker selection or a likelihood that accounts for dependence is needed for that issue.
Treating reference frequencies as error-free facts. The basic calculation plugs them in as fixed values. The resulting estimate does not automatically include uncertainty in those frequencies.
Reading a global estimate as a chromosome-by-chromosome description. A genome-wide proportion does not reveal the locations of ancestry segments.

Key takeaway: Supervised likelihood estimation chooses nonnegative ancestry weights summing to one to make the observed genotypes most likely under a mixture of reference allele frequencies. Interpret the result as a model-dependent global summary, not as an exact genealogical fraction.

Check Your Understanding

Use the genotype-mixture model and assumptions described above to answer the following questions.

For two reference groups with allele frequencies 0.30 and 0.70, write the modeled allele frequency when the first group’s proportion is \(\theta\).
Why must estimated ancestry proportions be nonnegative and sum to one?
At a marker, 12 diploid individuals have a total dosage of 9 copies of the specified allele. What modeled allele frequency maximizes the marker’s likelihood?
What does it mean if two reference groups have the same allele frequencies at every analyzed marker?
Give one reason why a fitted global ancestry proportion should not be described as an exact fraction of genealogical ancestors.