Why the phenotype scale matters
A molecular QTL analysis relates genetic variation to a molecular phenotype, such as a transcript count, a detection indicator, or a measured abundance. Linear regression is appropriate when the response and its conditional error distribution are reasonably described by a continuous, approximately normal model. Counts and binary outcomes do not have that structure: counts are nonnegative integers, while binary outcomes take only two values. Applying an ordinary linear model without considering those features can give implausible fitted values or misleading standard errors.
The model should reflect both the recorded phenotype and the scientific contrast of interest. A count model can estimate a relative change in expected count, a logistic model can estimate an odds ratio, and a model of a transformed continuous trait can estimate a difference on the transformed scale. These are different effect estimands. As with other molecular QTL models, the genetic coefficient describes an association conditional on the included covariates; it does not by itself establish a causal effect.
Choosing a model for the phenotype
For a count outcome, a Poisson generalized linear model uses the log link and assumes that, conditional on genotype and covariates, the count has a Poisson distribution. A negative binomial model also uses a log link but allows the conditional variance to exceed the conditional mean. If counts arise over different library sizes or other known exposure amounts, include the log exposure as an offset: its coefficient is fixed at one rather than estimated.
In this expression, \(E_i\) is the exposure, such as library size on a suitable scale. Conditional on the covariates, \(\exp(\beta_G)\) is the ratio of expected counts per unit exposure for a one-unit increase in \(G_i\). If \(G_i\) is coded as allele dosage 0, 1, or 2, that is a per-allele rate ratio under the model’s assumed linear effect on the log scale. If genotype is instead coded as a binary contrast, the coefficient compares those two groups.
A Poisson model requires more than nonnegative integer outcomes: observations must be independent conditional on the model structure, and the Poisson mean-variance relationship must be adequate. Residual variation substantially above that predicted by the fitted means is called overdispersion. A negative binomial model may be more suitable when that extra variation is genuine and can be represented by its dispersion parameter. Neither model automatically solves dependence among related donors or repeated measurements; the dependence structure must be handled with an appropriate model.
For a binary molecular phenotype, such as whether a feature is detected, logistic regression models the probability of the outcome being one. Its conditional distribution is Bernoulli, and its logit link maps probabilities between zero and one to the real line. The genetic coefficient is a log odds ratio, not a change in probability.
The model assumes the log odds are linear in the included predictors, conditional on the other terms. It also requires enough information to estimate the coefficients. If a genotype group perfectly predicts detection status, maximum-likelihood estimates can diverge; this is separation, and ordinary Wald inference is then unreliable.
For positive, continuous, right-skewed measurements, a logarithmic transformation may make residuals more nearly normal and stabilize their variance. A linear model on the transformed outcome estimates differences on that transformed scale. This is not interchangeable with a count model: it answers a different question and requires the transformed residuals, rather than the raw measurements, to meet the linear model assumptions. For a bounded continuous outcome, a beta regression can be considered when values lie strictly between zero and one and its mean-variance structure is appropriate. For count-derived proportions with extra variability, a binomial model may be inadequate; a beta-binomial model is one possible extension.
| Phenotype recorded | Common model choice | Genetic effect scale | Key issue to assess |
|---|---|---|---|
| Nonnegative integer count | Poisson or negative binomial GLM | Rate ratio with a log link | Exposure differences and overdispersion |
| Binary detection indicator | Logistic regression | Odds ratio | Separation and adequate model fit |
| Positive, skewed continuous abundance | Linear model of a log-transformed outcome, if diagnostics support it | Difference on log scale; ratio after exponentiation | Residual distribution and variance |
| Continuous proportion strictly between zero and one | Beta regression, where suitable | Depends on the specified link and parameterization | Boundaries, mean-variance form, and fit |
Worked examples
Worked Example: A Poisson Rate-Ratio QTL
Suppose a hypothetical study measures a molecular feature in 20 donors in each of two genotype groups. Let \(G=0\) and \(G=1\) label the groups. Each donor contributes one equal unit of exposure. The total counts are 60 for \(G=0\) and 90 for \(G=1\). Fit a Poisson log-link model with genotype as the only predictor, and estimate the group rate ratio.
The estimated rates are \(60/20=3.00\) counts per exposure unit for \(G=0\), and \(90/20=4.50\) for \(G=1\). Under this two-group log-linear model, the estimated genotype coefficient and rate ratio are
Using the independent-Poisson approximation for these group totals, the standard error of the log rate ratio is
The Wald statistic for \(H_0:\beta_G=0\) is \(z=0.4055/0.1667=2.433\), giving a two-sided normal-approximation \(p\)-value of about 0.0150. A 95% interval on the log scale is \(0.4055\pm1.96(0.1667)=[0.0788,0.7321]\). Exponentiating both endpoints gives an approximate 95% interval of \([1.082,2.080]\) for the rate ratio.
Under the stated model, the estimated expected count per exposure unit is 1.50 times as high in the \(G=1\) group as in the \(G=0\) group. The interval is an approximate confidence interval for that rate ratio, not a 95% probability statement about this particular interval. The calculation relies on the Poisson variance and independence assumptions. Group totals alone do not reveal whether individual donor counts are more variable than those assumptions allow, so donor-level fit and dispersion checks remain important.
Worked Example: A Logistic QTL for Detection
In a hypothetical study, a molecular feature is detected in 4 of 20 donors with \(G=0\) and 10 of 20 donors with \(G=1\). Let \(Y=1\) denote detection. Estimate the unadjusted logistic-regression genotype effect and its Wald uncertainty.
The odds of detection are \(4/16=0.25\) in the \(G=0\) group and \(10/10=1.00\) in the \(G=1\) group. Thus the estimated odds ratio and log odds ratio are
The estimated standard error of the log odds ratio from the four cell counts is
The Wald statistic is \(z=1.386/0.7159=1.936\), with an approximate two-sided \(p\)-value of 0.0529. The 95% Wald interval for the log odds ratio is \(1.386\pm1.96(0.7159)=[-0.0169,2.7894]\), which becomes approximately \([0.983,16.27]\) after exponentiation. The data estimate four times the odds of detection in the \(G=1\) group, but the interval is wide and includes an odds ratio of one. The result is not strong evidence against a zero log odds ratio at the 0.05 level using this test.
All four outcome-by-group cells are nonzero, so these counts do not show complete separation. Nevertheless, the sample is small, and the Wald approximation may be imprecise. Logistic regression models odds; the observed detection proportions, \(4/20=0.20\) and \(10/20=0.50\), are not themselves odds and their ratio is not the estimated odds ratio.
Worked Example: A Log-Transformed Abundance Trait
A hypothetical assay gives positive abundance values of 2, 4, 4, and 8 for four \(G=0\) donors, and 4, 8, 8, and 16 for four \(G=1\) donors. Consider a two-group linear model for \(Z=\log_2(Y)\), assuming independent normal residuals with a common variance.
The transformed observations are \(1,2,2,3\) in the \(G=0\) group and \(2,3,3,4\) in the \(G=1\) group. Their means are \(\bar{Z}_0=2\) and \(\bar{Z}_1=3\), so the estimated difference is \(1.00\) log-base-2 unit. In each group the squared deviations from the mean sum to \(2\); the pooled residual sum of squares is therefore \(4\), with \(df=4+4-2=6\). The pooled variance estimate is \(s^2=4/6=0.6667\).
The standard error of the difference is
With 6 degrees of freedom, the two-sided \(p\)-value is approximately 0.134. Using \(t_{0.975,6}=2.447\), the 95% interval for the transformed difference is \(1.00\pm2.447(0.5774)=[-0.413,2.413]\) log-base-2 units. Exponentiation to base 2 gives an abundance ratio of \(2.00\), with an interval of approximately \([0.751,5.32]\). This interval includes a ratio of one.
The model estimates a difference in mean log abundance, which corresponds to a ratio of geometric means on the original scale under this transformation. Its inference relies on the transformed residuals being reasonably normal with a common variance. The exact doubling pattern in these hypothetical observations does not establish those assumptions; with real data, residual plots and sensitivity to influential observations should be examined.
Checking assumptions and interpreting tests
A model choice is not justified solely by the fact that software can fit it. Check that the outcome scale matches the proposed distribution, that the genetic predictor and covariates are encoded as intended, and that any exposure offset represents a known proportional contribution to the mean. Then assess the conditional fit: examine residual patterns and fitted values, compare observed variation with model-implied variation, and look for influential observations. For a count model, a variance much larger than the mean suggests that a simple Poisson model may understate uncertainty. Excess zeros can arise for several reasons and do not by themselves prove that a zero-inflated model is needed; the proposed mechanism and model fit should support such an extension.
For a binary model, inspect fitted probabilities and check for sparse genotype-outcome combinations or separation. For a transformed continuous outcome, check whether the transformation improves the residual distribution and variance pattern rather than relying only on the raw phenotype’s skewness. If assumptions are poor, change the model or use a justified inferential alternative; do not interpret a familiar-looking coefficient as reliable merely because the fitting procedure returned a number.
The usual Wald test divides an estimated coefficient by its standard error and compares the result to a normal reference distribution for a large-sample test. The test evaluates \(H_0:\beta_G=0\), equivalent to a rate ratio or odds ratio of one for the respective log-link or logit-link examples. A small \(p\)-value measures how incompatible the observed statistic is with that null model under the test assumptions; it is not the probability that the null hypothesis is true. Confidence intervals describe uncertainty in the effect parameter under the model and its repeated-sampling procedure. When counts or binary cells are sparse, normal approximations may be poor even if the model form is otherwise reasonable.
Distinguish integer counts, binary indicators, continuous measurements, and bounded proportions before choosing a likelihood or transformation.
Specify genotype coding, covariates, link function, and any exposure offset. State whether the target is a rate ratio, odds ratio, or transformed-scale difference.
Check dispersion, residual behavior, sparse cells, and dependence structure; then interpret the coefficient, interval, and test conditional on the adequacy of those assumptions.
Common mistake
Wrong reasoning: “The phenotype is a count, so Poisson regression is automatically correct; a significant genotype coefficient proves the variant changes expression.” The count scale alone does not guarantee the Poisson mean-variance relationship, independence, or correct handling of exposure. Nor does an association test establish causation.
Correction: Choose the model from the outcome’s support and distribution, include the appropriate exposure and covariates, and assess whether its assumptions are credible. Interpret the coefficient on the model’s scale—for example, as a conditional rate ratio—and describe it as an association. If overdispersion, separation, or poor transformed-residual behavior undermines the usual test, address that issue before drawing a conclusion.
Check Your Understanding
Try these questions before reviewing the worked examples.
- In a Poisson log-link model with an exposure offset, what does \(\exp(\beta_G)\) represent for a one-unit increase in genotype dosage?
- Why can overdispersion make a Poisson Wald test unreliable, and what model is one possible alternative?
- A logistic-regression coefficient is 0.7. What is the corresponding odds ratio, and why is it not a 0.7 increase in detection probability?
- What assumptions are required to interpret a difference from a linear model of log-transformed abundance?
- Why does a small QTL association \(p\)-value not establish that the genetic variant causally changes the molecular phenotype?