Quasi-Likelihood Tests For RNA-seq

Why Add Quasi-Likelihood Inference?

A negative binomial generalized linear model (GLM) describes each gene’s mean count using the experimental design and allows count variance to exceed the mean. Even when that model is appropriate on average, the observed variability for a particular gene may be greater than its negative binomial variance predicts. Quasi-likelihood inference addresses this uncertainty when testing a specified coefficient contrast. It does not test whether a gene is excessively variable; the hypothesis remains a hypothesis about the contrast.

The distinction matters in small RNA-seq experiments. If the variance estimate is too small, a contrast can appear more precise than the data justify. Quasi-likelihood (QL) testing adds a variance multiplier and accounts for the uncertainty in estimating it. This often produces more cautious inference than a likelihood-ratio test (LRT) that relies on the negative binomial likelihood alone.

Here \(\phi_g\) describes the negative binomial mean-variance relationship, while \(q_g\) describes additional variation relative to that relationship. A value \(q_g=1\) corresponds to the negative binomial variance as specified; \(q_g>1\) means the variance is larger. The two dispersions are not interchangeable. Earlier material on dispersion estimation and shrinkage concerns \(\phi_g\); QL inference adds estimation and moderation of \(q_g\).

A standard setup assumes a correct log-mean model, such as \(\log(\mu_{gj})=o_{gj}+x_j^\mathsf{T}\beta_g\), where \(o_{gj}\) is a known offset and \(x_j\) is the design row. It also assumes independent samples conditional on the model, a suitable negative binomial variance function, and enough residual information to estimate variability. The design, offsets, and negative binomial dispersion must be handled appropriately: a QL multiplier cannot repair a systematically wrong mean model or an omitted covariate.

What the Test Tests

Let \(\beta_g\) be the coefficient vector for gene \(g\), and let \(C\) specify the contrast or set of contrasts of interest. The null hypothesis is \(H_0:C\beta_g=0\); the alternative is that at least one tested contrast is nonzero. For example, \(C\) might select the treatment coefficient in a two-condition design, or select a set of coefficients for a joint test.

Let \(\widehat{\beta}_g\) be the fitted coefficients, and let \(V_{0,g}\) be their estimated covariance matrix under the negative binomial model with \(q_g=1\). If there are \(r\) tested contrasts, a schematic QL statistic is

The fitted contrast and its negative-binomial covariance form the numerator; the estimated QL dispersion scales the statistic to account for extra variability. The numerator degrees of freedom \(r\) equal the number of independent restrictions tested. The denominator degrees of freedom \(\nu_g\) reflect residual information and, when dispersion estimates are moderated, information borrowed across genes. Exact implementations can use adjustments to the statistic and degrees of freedom, so this expression is a useful schematic rather than a universal software formula.

For one contrast, the quadratic form reduces to a squared standardized contrast. The QL F statistic is then equivalent to a squared t-like statistic, with \(F_{1,\nu}=t_\nu^2\). The p-value is the upper-tail probability under the relevant F reference distribution. A small p-value is evidence against the specified coefficient null, given the model and variance estimation; it is not the probability that the null is true.

Why the Variance Multiplier Changes Inference

A short variance argument shows the role of \(q_g\). For one contrast estimate \(\widehat{\theta}\), suppose its variance under the negative binomial model alone is \(v_0\). Under the QL model it is approximately \(q_gv_0\). Therefore its standard error is multiplied by \(\sqrt{q_g}\), and its squared standardized value is divided by \(q_g\):

Thus, holding the fitted contrast and baseline covariance fixed, a larger QL dispersion reduces the test statistic. But \(q_g\) is estimated rather than known. Treating it as known would understate uncertainty, particularly with few residual degrees of freedom. QL procedures therefore use an F-type reference distribution, and commonly moderate gene-wise QL dispersion estimates toward a shared trend or prior. This makes the inference reflect both the estimated variance level and the limited information available to estimate it.

Estimating and Moderating Quasi-Likelihood Dispersion

In practice, a gene’s residual variation provides information about \(q_g\). A rough moment-based illustration uses residual deviance divided by residual degrees of freedom. This ratio is not an exact universal estimator: practical implementations may adjust residual quantities for leverage and use their own estimation procedures. Its purpose here is to show why gene-wise dispersion estimates can be unstable when a gene has few residual degrees of freedom.

Moderation stabilizes such estimates by combining gene-specific information with information from a distribution or trend shared across genes. Under a simple chi-square approximation, suppose an unmoderated estimate \(\widehat{q}_g\) has \(d\) residual degrees of freedom and a prior value \(q_0\) has prior degrees of freedom \(d_0\). A transparent weighted estimate is

This is an illustrative moderation formula, not a claim that every implementation uses this exact calculation. The principle is that a noisy gene-wise estimate receives more shrinkage toward the shared value than a precise estimate does. Moderation also informs the effective denominator degrees of freedom used for testing.

Assumptions, Diagnostics, and Alternatives

QL inference is useful only if the mean model and variance structure are credible. Check that the design represents the experimental comparisons, offsets account for library sizes, and the negative binomial dispersion estimates are reasonable. Residual patterns, unusually influential counts, and markedly elevated QL dispersions can flag lack of fit or extra variability. These signs call for investigation; a large \(q_g\) alone does not identify the cause. It could reflect real biological variation, an outlying observation, an omitted design feature, or another model failure.

A QL adjustment cannot make dependent samples independent, correct confounding, or compensate for an inappropriate mean relationship. Very few residual degrees of freedom also limit gene-specific variance estimation, even when moderation helps. Conversely, an estimated \(q_g\) below 1 is possible; QL testing is not guaranteed to be more conservative for every gene. Its aim is calibrated inference accounting for variance estimation, not uniformly larger p-values.

The Wald test evaluates a coefficient contrast using an estimated standard error, typically with a normal or t reference approximation. The LRT compares fitted likelihoods under the full and null models and commonly uses a chi-square reference distribution. Both are appropriate tools under their assumptions, but the QL test explicitly accommodates extra variance beyond the negative binomial variance and the uncertainty in estimating that extra component. It is not simply a test of whether \(q_g\) exceeds 1, and it does not replace the negative binomial mean model. Which test is preferable depends on the design, sample size, and variance-estimation procedure.