Why Genotype Encoding Matters
A genotype matrix is useful only if each entry has a consistent meaning. A value of zero may encode a reference allele, a homozygous-reference genotype, or—in a poorly designed import—a missing value. These meanings are not interchangeable. If a missing entry is read as a reference genotype, counts of observed genotypes are wrong; if a real reference genotype is discarded as missing, they are wrong in the opposite direction.
This tutorial focuses on diploid genotype entries: how two alleles are represented for a sample at a variant, how missingness can be encoded, and how to count the resulting entry types without ambiguity. The examples use explicitly hypothetical data.
Reading a Diploid Genotype Entry
At a biallelic variant, the two alleles are conventionally indexed 0 and 1: 0 denotes the reference allele and 1 denotes the alternate allele. The three possible complete genotypes are \(0/0\), \(0/1\), and \(1/1\), representing reference homozygous, heterozygous, and alternate homozygous genotypes, respectively. The slash indicates that the alleles are unphased: their order is not being used to assert which allele came from which parent.
A vertical bar, as in \(0|1\), denotes a phased entry: the order of the alleles is retained, typically to represent their assignment to haplotypes. At a single locus, \(0|1\) and \(1|0\) contain the same unordered pair of alleles, but their phase assignments differ. Do not silently discard phase information if the analysis depends on haplotypes. For simple counts of complete diploid genotypes, either separator indicates two called alleles.
The allele-index convention extends to sites with more than one alternate allele. If a site has one reference allele and two alternate alleles, for example, the indices may be 0, 1, and 2. A complete diploid entry could then be \(1/2\). The indices identify alleles in the variant record; they do not mean that 2 is a larger amount of alternate allele than 1.
The two-allele assumption is about ploidy, not about how many allele labels a site has. It is appropriate only when the sample is expected to have two chromosome copies at that locus. Some genomic regions or samples have different expected ploidy. Such entries should be handled according to the study's ploidy and data-format conventions rather than forced into diploid categories.
Worked Example: Classifying Calls at a Biallelic Variant
In a hypothetical data set, five diploid samples have the following entries at one biallelic site: \(0/0\), \(0/1\), \(1/1\), \(./.\), and \(0/.\). Classify each entry.
The first three entries are complete calls because each specifies two alleles: \(0/0\) is reference homozygous, \(0/1\) is heterozygous, and \(1/1\) is alternate homozygous. The entry \(./.\) is fully missing: neither allele is specified. The entry \(0/.\) is partial: one reference allele is specified, but the second allele is missing.
Thus, among these five entries, there are 3 complete calls, 1 fully missing entry, and 1 partial entry. The check is \(3+1+1=5\), equal to the number of samples. The partial entry must not be counted as a complete diploid genotype.
Matrix Layout and Missing-Value Categories
A common representation is an \(n \times m\) matrix with samples in rows and variants in columns. The entry in row \(i\), column \(j\) describes the genotype recorded for sample \(i\) at variant \(j\). For a fixed variant \(j\), each of the \(n\) sample entries should be assigned to exactly one of three categories: complete, fully missing, or partial. This partition assumes that every matrix cell contains one entry and that the data have not been dropped or duplicated.
The distinction in that rule matters. A partial entry is incomplete, but it is not fully missing. If partial entries are tracked separately, they belong in \(P_j\), not in \(M_j\). If an analysis deliberately groups all incomplete entries together, it can count them in \(U_j\), provided that definition is stated and applied consistently.
In Variant Call Format (VCF), genotype entries are commonly stored in a GT field. A diploid fully missing entry can appear as \(./.\), and a partial entry as \(0/.\). A phased fully missing entry may appear as \(.|.\). Text formats may also use a single dot or another missing token in contexts where the ploidy or field structure makes its meaning clear. The token alone is not enough to determine how to count an entry; interpret it using the field definition and expected ploidy.
Other data layouts use numerical codes or software-specific missing-value markers. A numeric value such as \(-1\) may be reserved to indicate missingness, but it is safe only if the encoding specification says so and the value cannot be confused with a valid genotype or dosage. In a diploid allele-count representation, values 0, 1, and 2 can mean counts of alternate alleles. Treating 0 as missing would erase valid reference-homozygous observations. Conversely, treating a missing sentinel as a valid number creates a false genotype category.
Worked Example: Checking Counts Across a Matrix
Consider a hypothetical matrix with six samples and two variants. The entries are:
| Sample | Variant A | Variant B |
|---|---|---|
| 1 | 0/0 | 0/1 |
| 2 | 0/1 | ./. |
| 3 | 1/1 | 0/. |
| 4 | ./. | 1/1 |
| 5 | 0/. | 0/0 |
| 6 | 0/0 | ./. |
For Variant A, the six entries are \(0/0\), \(0/1\), \(1/1\), \(./.\), \(0/.\), and \(0/0\). The four complete calls are \(0/0\), \(0/1\), \(1/1\), and \(0/0\); there is one fully missing entry and one partial entry. Therefore \(C_A=4\), \(M_A=1\), \(P_A=1\), and the check is \(4+1+1=6\).
For Variant B, the entries are \(0/1\), \(./.\), \(0/.\), \(1/1\), \(0/0\), and \(./.\). Three are complete calls (\(0/1\), \(1/1\), \(0/0\)), two are fully missing, and one is partial. Thus \(C_B=3\), \(M_B=2\), and \(P_B=1\); the check is \(3+2+1=6\).
If the question instead asks for the number of incomplete entries at Variant B, combine the two fully missing and one partial entry: \(U_B=M_B+P_B=2+1=3\). Then \(C_B+U_B=3+3=6\). The two counting conventions agree on the total only when their categories are defined clearly.
Deriving the Count Check and Auditing Encodings
The counting identity is a bookkeeping result, not a statistical model. For a fixed variant, give each sample entry one indicator for each category: \(I_C\) equals 1 for a complete call and 0 otherwise; define \(I_M\) and \(I_P\) similarly. Under the stated classification rule, exactly one indicator equals 1 for every sample. Therefore, for each sample \(i\), \(I_{C,i}+I_{M,i}+I_{P,i}=1\). Summing over the \(n\) samples gives:
This equality is useful as an audit: if the counts do not sum to the number of matrix entries, the categories may overlap, a category may have been omitted, or the matrix may have an unexpected shape. The identity does not establish that the genotype calls are biologically correct. It verifies only that the entries have been accounted for under the chosen definitions.
A practical audit proceeds from the data specification to the counts. First, confirm the expected ploidy and the allele-index convention. Next, inspect the actual stored values, including separators and missing tokens. Then classify complete, fully missing, and partial entries, and check their sum against the number of samples. For any numeric representation, verify that valid genotype codes and missing sentinels are disjoint. Finally, compare a small set of matrix entries with the original genotype records when possible.
Confirm what each allele index means and how many allele copies are expected for the samples and locus.
Determine which tokens represent fully missing entries and which represent partially specified genotypes.
Count complete calls, fully missing entries, and partial entries separately unless a stated summary combines the incomplete categories.
Verify \(C_j+M_j+P_j=n\), or \(C_j+U_j=n\) when \(U_j\) explicitly includes every incomplete entry.
Encoding Choices, Diagnostics, and Limitations
Different encodings serve different purposes. Allele-pair strings such as \(0/1\) retain the two allele indices and can distinguish phase if the separator is preserved. A dosage representation may instead store the number of alternate alleles, usually 0, 1, or 2 for a complete diploid biallelic genotype. Dosage is compact and useful for some analyses, but by itself it does not preserve the distinction between \(0|1\) and \(1|0\). Missing dosage still requires an unambiguous missing-value convention.
A useful diagnostic is to look for unexpected values and category totals before using the matrix. Examples include a missing sentinel appearing as a genotype, a partial call being counted as fully missing without that being the stated rule, or a genotype with more allele copies than the expected ploidy. For a biallelic diploid allele-count encoding, a value outside the valid complete range of 0, 1, and 2 deserves investigation. These checks can reveal parsing or conversion problems, but they cannot determine whether a complete call is biologically accurate.
Neither a complete call nor a missing marker reveals why the value was recorded that way. A missing entry may result from low-quality evidence, a technical failure, or a processing decision; the encoding alone does not identify the cause. Similarly, a syntactically complete call may still be wrong. Quality-control counts describe the stored data under a specified rule, not the certainty or truth of each genotype.
The most important comparison is between preserving the original allele-level entry and simplifying it to a single count. Simplification can make summaries easier, but it loses information such as phase and, depending on the representation, the identities of alleles at a multiallelic site. Choose the representation to match the intended analysis, retain the original data when possible, and document how partial and fully missing entries are handled. A proportion of calls can be calculated from counts only after defining its numerator and denominator; the basic counts should be checked first.
Worked Example: Auditing a Numeric Encoding
A hypothetical import stores alternate-allele counts for four diploid samples at a biallelic site. The specification says that 0, 1, and 2 are valid complete genotype counts and that \(-1\) denotes a missing genotype. The stored values are \(0, 1, -1, 2\).
The value 0 represents a complete reference-homozygous genotype; it is not missing. The values 1 and 2 are also complete calls, representing one and two alternate alleles. The value \(-1\) is the single missing entry under the stated specification. Thus the count of complete calls is \(C=3\), the count of missing entries is \(M=1\), and there are no separately represented partial entries, so \(P=0\). The check is \(3+1+0=4\), the number of samples.
If an import routine incorrectly treated 0 as missing, it would instead report only two complete calls and two missing entries. The total would still be four, so the sum check alone would not detect this semantic error. The valid-code specification and inspection of the data are essential complements to the arithmetic check.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions below.
- A hypothetical variant has entries \(0/1\), \(./.\), \(1/.\), \(1/1\), and \(0/0\). How many complete, fully missing, and partial entries are there?
- Why is it incorrect to count \(0/.\) as a complete reference-homozygous call?
- In a diploid alternate-allele-count encoding, the valid complete values are 0, 1, and 2, while \(-9\) is the declared missing sentinel. Classify the entries \(2,0,-9,1,0\), and check the total.
- A data table has 12 samples at one variant, with \(C=9\) complete calls and \(M=1\) fully missing entry. If partial entries are counted separately, how many are there? What is the count of all incomplete entries?
Solutions. For question 1, \(0/1\), \(1/1\), and \(0/0\) are complete, \(./.\) is fully missing, and \(1/.\) is partial. The counts are \(C=3\), \(M=1\), and \(P=1\), with \(3+1+1=5\). For question 2, \(0/.\) specifies only one of the two alleles required for a complete diploid call, so its correct category is partial; it does not specify \(0/0\). For question 3, the four values \(2,0,1,0\) are complete calls and \(-9\) is missing, giving \(C=4\), \(M=1\), \(P=0\), and \(4+1+0=5\). For question 4, \(P=12-9-1=2\). All incomplete entries total \(M+P=1+2=3\), and the check is \(9+1+2=12\).