Genotype Covariance Under Population Mixture

Why Population Mixture Creates Genotype Covariance

Two loci can be independent within each ancestry group and still show covariance when people from those groups are analyzed together. The reason is that ancestry can affect the expected genotype at both loci. If individuals with more ancestry from a group tend to have higher expected dosage at both loci, their dosages tend to vary together across the pooled sample. If the ancestry groups have opposing allele-frequency patterns at the two loci, the resulting covariance can instead be negative.

This is a consequence of mixing populations, not evidence by itself that the loci are physically linked or associated within each population. The key calculation separates variation in genotypes within ancestry strata from variation in their conditional means between strata.

To describe population mixture, let \(Z\) denote a latent ancestry group. Write \(p_{jz}\) for the frequency of the dosage allele at locus \(j\) in group \(z\). Suppose that, conditional on \(Z=z\), the two loci are independent and each allele copy has frequency \(p_{jz}\) at locus \(j\). Then the conditional mean dosage is \(E[G_j\mid Z=z]=2p_{jz}\). Independence across loci implies \(\operatorname{Cov}(G_j,G_k\mid Z=z)=0\).

The Total-Covariance Decomposition

The law of total covariance divides the pooled covariance into two parts: the average covariance remaining within ancestry groups, and the covariance between the group-specific expected dosages. This decomposition makes the source of mixture-induced covariance explicit.

Under conditional independence, the first term is zero. Since the conditional means are \(2p_{jZ}\) and \(2p_{kZ}\), the remaining covariance is

Thus the sign depends on how allele frequencies at the two loci vary across the ancestry groups represented in the sample. Similar frequency patterns tend to produce positive covariance; opposing patterns tend to produce negative covariance. If one locus has the same frequency in every group, the mixture term is zero, even if ancestry proportions vary.

Varying Ancestry Proportions

A person's ancestry need not be represented by a single group label. Let \(Q\) be the proportion of ancestry from source A in a two-source mixture, so the proportion from source B is \(1-Q\). Let \(p_{jA}\) and \(p_{jB}\) be the dosage-allele frequencies at locus \(j\) in the two sources. Under a simple global-ancestry model, the allele frequency conditional on \(Q\) is

Suppose that, conditional on \(Q\), the genotypes at the two loci are independent and have mean dosages \(2p_j(Q)\) and \(2p_k(Q)\). The total-covariance decomposition then gives

The last equality follows because \(p_j(Q)=p_{jB}+Q(p_{jA}-p_{jB})\), so its only varying part is \(Q(p_{jA}-p_{jB})\); applying the same expression at locus \(k\) gives the product of the two frequency differences times \(\operatorname{Var}(Q)\). This form shows that mixture-induced covariance is zero if ancestry proportion does not vary, or if either locus has the same frequency in both sources. Its sign is positive when the frequency differences have the same sign, and negative when they have opposite signs.

Derivation and Assumptions

For completeness, the total-covariance identity follows by adding and subtracting conditional means. Let \(m_j(Z)=E[G_j\mid Z]\) and \(m_k(Z)=E[G_k\mid Z]\). Then

and the corresponding expression holds for \(G_k-E[G_k]\). Multiplying the two expressions and taking expectations gives four terms. The term involving the product of the two within-group deviations is \(E[\operatorname{Cov}(G_j,G_k\mid Z)]\). Each cross-term is zero: for example, conditional on \(Z\), \(m_k(Z)-E[G_k]\) is fixed and \(E[G_j-m_j(Z)\mid Z]=0\). The remaining term is \(\operatorname{Cov}(m_j(Z),m_k(Z))\). This proves the decomposition.

The simple mixture formula is useful only when its assumptions are clear. To interpret the covariance as arising solely from ancestry variation, the model needs the following conditions:

• The ancestry variable captures the relevant structure. \(Z\) or \(Q\) must distinguish the ancestry-related differences in allele frequencies that affect the loci under study.
• Conditional means are correctly specified. For a group model, the expected dosage must be \(2p_{jz}\); for the two-source model, the assumed mixture frequency must be \(Qp_{jA}+(1-Q)p_{jB}\).
• There is no residual cross-locus covariance under the conditioning. Conditional independence makes \(E[\operatorname{Cov}(G_j,G_k\mid Z)]=0\). If loci remain correlated within groups, that term need not vanish.
• The allele-frequency and sampling model is appropriate. For example, using binomial genotype variation within a group assumes the two allele copies have the stated frequency and are sampled independently at the locus. That assumption is also what gives conditional genotype variance \(2p_{jz}(1-p_{jz})\).

If conditional independence fails, the full decomposition still holds, but the between-group term is no longer the whole pooled covariance. It must be added to the average within-group covariance. This distinction matters because linkage disequilibrium is covariance or association between alleles at different loci in a specified population, whereas the between-group term here can appear solely because groups with different allele frequencies have been pooled.

What Pooled Covariance Can and Cannot Show

A nonzero pooled covariance is compatible with multiple explanations. It can arise from variation in ancestry proportions under the model above; it can also reflect within-group linkage disequilibrium, relatedness, assortative mating, or other unmodeled structure. Conversely, ancestry variation need not produce covariance at every pair of loci: if their allele frequencies do not vary together across the represented groups, the between-group term can be small or zero.

The decomposition is therefore a statement about a model, not a method for identifying ancestry from a covariance alone. It explains how latent structure can generate covariance and specifies conditions under which that explanation accounts for all of it. A covariance calculation by itself does not establish which source is responsible.

Common Mistake

• Wrong reasoning: “The loci have positive covariance in the pooled sample, so they must be linked or correlated within each ancestry group.”
• Correction: Pooled covariance includes both within-group covariance and covariance between group-specific means. Even when the within-group term is zero, different allele-frequency patterns across groups can produce positive or negative pooled covariance. Check which population is being analyzed and whether conditional independence is plausible before assigning the covariance to a particular mechanism.