Preparing Molecular Phenotypes For QTL Mapping

Why phenotype preparation matters

A molecular measurement is not automatically a suitable QTL phenotype. RNA-sequencing counts depend on sequencing depth, protein or metabolite abundances may be strongly skewed, and low-signal measurements may be dominated by noise. Before comparing molecular measurements across donors, analysts must decide what to filter, how to normalize, whether to transform, and whether to scale. Each choice can change the numerical genotype effect and the population of measurements to which it refers.

These steps have different purposes. Normalization adjusts measurements for systematic technical differences, such as differing sequencing depth. A transformation changes the measurement scale, often to reduce skew or stabilize variability. Scaling expresses a phenotype in units such as standard deviations. Filtering removes measurements or features that meet prespecified quality or signal criteria. Outlier handling investigates unusually extreme observations and decides how to treat them. None of these operations, by itself, removes biological confounding or establishes a causal effect.

Normalization, transformation, and scaling are not interchangeable

For count data, the number of reads assigned to a gene depends partly on the total reads obtained for that donor. A library-size adjustment aims to make samples comparable in sequencing depth. Counts per million (CPM) divides a gene's count by the total mapped count in that sample and multiplies by one million. More elaborate methods, such as median-ratio size factors or trimmed mean of M-values (TMM), estimate sample-specific scaling factors while attempting to reduce the influence of highly abundant features or broad compositional differences.

A simple depth adjustment assumes that the chosen scaling factors adequately represent technical sampling differences and that most features, or a suitable reference subset, do not undergo a coordinated biological shift that is mistaken for depth. This assumption can fail if a small number of transcripts dominate the library or if most genes truly change in one direction. A global scaling method cannot distinguish a genome-wide biological shift from a technical shift without additional information. For targeted panels or protein assays, normalization may instead use total signal, internal standards, reference proteins, or other assay-specific controls; the choice must match how the measurements were generated.

Normalization does not usually make the distribution symmetric. A transformation such as \(\log_2(\mathrm{CPM}+c)\), with a stated positive constant \(c\), can compress large values and make differences at lower abundance less dominated by the largest measurements. The constant is important: it affects values near zero and therefore can affect estimated effects for weakly expressed features. A log transformation also changes interpretation: an additive difference on a log scale corresponds approximately to a multiplicative contrast on the original scale, but the exact relationship depends on the transformation and any offset.

Scaling often means subtracting a mean and dividing by a standard deviation, so that values are expressed in standard-deviation units. It may help compare effects across features, but it does not correct sequencing depth, remove skew, or make an inappropriate phenotype model appropriate. If the same positive affine transformation is applied to every donor for one feature, a mean difference between genotype groups is divided by the scale factor; its sign and test evidence in a simple group comparison are unchanged. Scaling separately within genotype groups would be different: it would remove the between-group contrast and is not a valid way to prepare a phenotype for estimating that contrast.

Filtering defines which features can be mapped

Filtering can exclude features that are unmeasured, fail assay quality criteria, or have too little signal for reliable comparisons. For example, a gene with almost all counts equal to zero provides little information for distinguishing genotype groups. A filter might require a minimum count or abundance in a specified number or proportion of samples. The threshold should be chosen without using the observed genotype association, and it should be reported alongside the number of features retained.

Filtering changes the set of molecular traits entering QTL analysis. A signal-based filter can also change which donors contribute usable measurements if missing or below-detection values are handled by sample exclusion. If detectability is related to genotype, phenotype level, or a technical factor correlated with genotype, the retained observations may not represent the intended donor set. Filtering should therefore be based on assay properties and defined consistently, with attention to whether exclusions differ across genotype groups. It is not a remedy for a weak or inconvenient result.

For a particular feature, genotype effects can also depend on whether the phenotype is analyzed as a raw value, a normalized value, or a transformed value. Such estimates answer related but distinct questions. An effect of 10 CPM is an additive difference in CPM units; a difference of 1 on a \(\log_2\) scale is a difference in transformed units, corresponding to a twofold ratio only when the transformation is exactly \(\log_2\) without an offset and the comparison is otherwise defined on that scale.

Outliers and scaling: protect the estimate, not the preferred result

An extreme molecular value may reflect a technical failure, a sample mix-up, a genuine rare biological state, or ordinary variation. A value is not an error merely because it is far from the other observations. Outlier review should use prespecified quality measures where possible, inspect the underlying assay and sample metadata, and consider whether the unusual observation is plausible. Removing a point only because it weakens a genotype association makes the result selection-dependent and can exaggerate evidence.

A useful diagnostic is to compare the estimated effect with and without a flagged observation, while clearly labeling such a comparison as a sensitivity analysis rather than silently replacing the primary result. Robust summaries can reduce sensitivity to extremes, but they target different quantities: a difference in means and a difference in medians need not be equal. Winsorizing, transforming, or deleting values also has consequences and should be stated rather than treated as neutral cleanup.

A principled preparation sequence

Preparation is most defensible when decisions are made in an order that separates assay quality from association results. The genotype should not be used to select a transformation, threshold, or exclusion rule that makes its estimated effect look stronger. A practical sequence is:

Why the preparation choices matter for QTL interpretation

The genotype effect is meaningful only together with its phenotype definition. An effect reported in raw counts may reflect sequencing depth; an effect on log abundance is not an additive change in the original units; and a standardized effect is relative to the standard deviation used for scaling. If different features are standardized separately, their effects can be compared in standard-deviation units, but those units conceal differences in absolute abundance and variability.

Preparation can also alter the evidence available for mapping. Filtering reduces the number of tested features but may exclude weakly measured traits. Transformation can reduce the influence of large values while changing the effect estimand. An outlier can dominate a mean contrast, whereas excluding it can change the analyzed sample. These decisions should therefore be reported with enough detail for another analyst to reconstruct the phenotype: the filtering rule, normalization method, transformation and offset, scaling reference, and outlier policy.