Why Test Choice Depends on the Design
A differential-expression test is not selected just because a count table has two columns or because one procedure is familiar. The choice depends on the scientific hypothesis, the design matrix, and how uncertainty in the count model is handled. A test that answers whether one specified condition contrast is nonzero is not interchangeable with a test of whether any of several condition effects are nonzero.
The methods considered here use the negative binomial generalized linear model (NB GLM) for gene-level counts. They differ in how they test model parameters and how they represent uncertainty. In broad terms, Wald tests use an estimated coefficient and its standard error; likelihood-ratio tests compare fitted likelihoods for nested models; quasi-likelihood tests add an extra variance component to the GLM test; and exact tests are specialized to a two-group comparison.
The coefficient vector \(\boldsymbol{\beta}\) describes effects on the log-mean scale. A contrast is a linear combination \(c^\mathsf{T}\boldsymbol{\beta}\); for example, a condition coefficient can represent the log fold change between two conditions. The design, contrast, and null hypothesis should be written down before choosing the test. Otherwise, it is easy to calculate a valid statistic for the wrong scientific question.
What Each Test Uses
A Wald test evaluates a specified coefficient or contrast using its estimate and estimated standard error. For a one-dimensional contrast, the statistic is the estimate divided by its standard error and is compared with a standard normal distribution under the null, usually as a large-sample approximation. A confidence interval for the contrast is built from the same estimated uncertainty. This makes the Wald test convenient when the target is a particular effect, including one contrast from a model with covariates.
A likelihood-ratio test (LRT) compares two nested models fitted to the same response data, with the same offset and dispersion treatment. The reduced model imposes the null restrictions; the full model allows the effects being tested. The statistic is twice the increase in maximized log likelihood. Under the null and regularity conditions, it is compared with a chi-square distribution whose degrees of freedom equal the number of independent restrictions added by the full model. An LRT can therefore test several coefficients jointly, not just one contrast.
A quasi-likelihood (QL) test starts with the NB GLM mean model but allows gene-wise variability beyond the fitted NB variance through an additional quasi-likelihood dispersion. Its test is commonly reported as an \(F\) statistic, with numerator degrees of freedom for the tested restrictions and denominator degrees of freedom reflecting residual information. The extra dispersion affects the uncertainty: if observed variability exceeds what the NB model alone predicts, the QL test can be less significant than a test that treats the NB dispersion estimate as fully known. Some implementations also stabilize or moderate dispersion estimates using information across genes.
An exact test is designed for a two-group count comparison under a specified count model. In its usual conditional construction, it conditions on the total count for a gene and evaluates how unusual the division of that total between the groups would be under the null, accounting for group exposures and dispersion. In the Poisson limit with equal exposures, that conditional distribution is binomial. With overdispersion, the corresponding conditional distribution is not generally binomial. The exact test avoids relying on a large-sample normal or chi-square approximation for that two-group comparison, but it does not provide a general substitute for a GLM with covariates or a complex design.
A Practical Selection Sequence
First define the null hypothesis in the model. For a single prespecified condition contrast, a Wald test is direct; a QL test is often preferred when the analysis uses quasi-likelihood dispersion to account for uncertainty beyond the NB fit. For a joint hypothesis, such as no condition effect across several levels, compare nested models with an LRT or use the corresponding multi-parameter QL test when that is the chosen inferential framework. For a simple two-group comparison with no covariate structure, an exact test may be appropriate, especially when discrete counts and small samples make asymptotic approximations a concern.
Specify whether the target is one contrast or a joint test of several effects, and write its null hypothesis in terms of the model coefficients.
Determine whether the comparison is an unadjusted two-group design or whether covariates, multiple groups, or other modeled factors are required.
Use a Wald test for a specified contrast, an LRT for nested-model improvement, a QL test when extra variability is explicitly incorporated, or an exact test for a suitable two-group count comparison.
Check the test statistic, degrees of freedom, dispersion assumptions, and whether the approximation or conditional test matches the design and sample size.
Worked Example: A Prespecified Contrast With a Covariate
Suppose a hypothetical RNA-seq experiment compares treatment with control, includes a batch covariate, and has independent biological samples in each treatment-by-batch combination. For one gene, the fitted NB GLM gives an adjusted treatment contrast \(\hat{\beta}=0.6931\) on the natural-log scale, with estimated standard error \(SE=0.2500\). The target is specifically the treatment-versus-control effect after adjusting for batch.
A Wald test is a direct choice because the question concerns one specified contrast in a fitted model that includes the needed covariate. Under \(H_0:\beta=0\), the statistic is $$ z=\frac{0.6931}{0.2500}=2.7724. $$
The two-sided standard normal p-value is approximately \(0.0056\). A 95% Wald interval on the log scale is $$ 0.6931\pm1.96(0.2500)=[0.2031,\ 1.1831]. $$ Exponentiating gives a fold-change interval of approximately \([1.225,\ 3.265]\). Under the fitted model, the estimated treatment mean is about \(e^{0.6931}=2.00\) times the control mean, adjusted for batch. The interval is a confidence interval for the model-based fold change, not a probability statement about where a fixed parameter lies.
The independent-sample assumption and the inclusion of batch in the design are part of the setup. The Wald calculation also relies on the estimated standard error and a sufficiently reliable normal approximation. If the question were whether treatment has any effect across several treatment coefficients, a single Wald contrast would not answer that joint question.
Worked Example: A Joint Test Across Three Conditions
Consider hypothetical independent samples from three conditions, with batch included in both models. The reduced model includes an intercept and batch; the full model additionally includes the two coefficients needed to represent the three-level condition factor. Suppose their maximized log likelihoods for one gene are \(-125.5\) and \(-120.0\), respectively. The null is that the condition has no effect, which imposes two restrictions.
The models are nested, use the same observations and offset, and differ only by the condition terms. Therefore the LRT statistic is $$ G^2=2\{-120.0-(-125.5)\}=11.0. $$
The degrees of freedom are \(2\), the number of added independent condition coefficients. Comparing with a chi-square distribution gives \(P(\chi^2_2\geq11.0)=e^{-11/2}\approx0.004087\). This is evidence against the joint null that all condition effects are zero. It does not by itself identify which condition pairs differ; that requires appropriately specified contrasts.
The chi-square calibration is an asymptotic result, and the likelihood comparison requires the same dispersion treatment in both fits. If a condition term were omitted from the reduced model but the designs were not nested, this LRT calculation would not be justified.
Worked Example: Why Extra Quasi-Likelihood Variation Matters
Suppose, hypothetically, a one-degree-of-freedom condition contrast has estimate \(0.600\) log units. The NB GLM covariance calculation gives contrast variance \(0.060\), while the fitted quasi-likelihood dispersion multiplier for this gene is \(1.50\). For illustration, the QL variance used for the contrast is therefore \(1.50(0.060)=0.090\). Assume the residual denominator degrees of freedom are \(8\).
The QL \(F\) statistic is the squared estimate divided by its QL variance: $$ F=\frac{0.600^2}{0.090}=4.00. $$
With numerator and denominator degrees of freedom \(1\) and \(8\), respectively, the upper-tail p-value is approximately \(0.0805\). The estimated fold change is \(e^{0.600}=1.822\), but this test does not provide strong evidence against a zero contrast at the 0.05 level. The variance multiplier matters: ignoring it would use \(0.060\) and produce a larger statistic, \(0.600^2/0.060=6.00\). This arithmetic illustrates why the QL uncertainty estimate can change the inferential result. It assumes the stated QL variance estimate is appropriate; the example does not establish that it would be appropriate for every gene or design.
Worked Example: An Exact Test in the Poisson Limit
Suppose a hypothetical two-group experiment has equal effective exposures and, for one gene, counts of \(22\) in group A and \(8\) in group B. In the Poisson-limit case of the conditional two-group test, under equal expression the total \(30\) is fixed and the group-A count \(X\) has distribution \(X\sim\operatorname{Binomial}(30,0.5)\). This example has no covariates and assumes the equal-exposure, Poisson-limit model.
For the symmetric two-sided test, outcomes at least as far from \(15\) as the observed \(22\) are \(X\geq22\) or \(X\leq8\). Thus $$ P(X\geq22\text{ or }X\leq8) =2\sum_{k=0}^{8}{30\choose k}(0.5)^{30} =\frac{17{,}313{,}874}{1{,}073{,}741{,}824} \approx0.01613. $$
The observed split is unusual under this conditional null. The result is specific to the two-group comparison and its assumptions. In a genuinely overdispersed setting, the conditional distribution must reflect that dispersion; the binomial calculation above should not be used automatically.
Comparison, Diagnostics, and Limitations
| Test | Best matched question | Main uncertainty or calibration | Important limitation |
|---|---|---|---|
| Wald | Is one specified coefficient or contrast nonzero? | Estimated contrast standard error; normal approximation | Can be unreliable when estimates or standard errors are unstable, particularly with sparse information. |
| Likelihood-ratio | Does a larger nested model improve fit, possibly for several terms? | Likelihood difference; asymptotic chi-square reference | Requires correctly nested models and the same data and dispersion treatment. |
| Quasi-likelihood | Is a coefficient or set of coefficients nonzero with extra variability represented? | QL variance estimate and an \(F\) reference with residual degrees of freedom | Depends on estimating the extra variability adequately; small residual degrees of freedom can limit precision. |
| Exact | Is a two-group count split unusual under the conditional null? | Conditional count distribution under the specified model | Not a general covariate-adjusted or multi-factor test; discrete p-values can be conservative. |
Selection should be revisited if the design or fitted model gives warning signs. A contrast that is not estimable, a rank-deficient design, very few biological replicates, or a dispersion estimate poorly supported by the data can undermine the intended test. For an LRT, check that the models truly are nested and differ only in the terms under test. For a Wald test, check that the coefficient and standard error are finite and that the contrast is the one intended. For a QL test, the extra variability estimate and denominator degrees of freedom are central to interpreting the result. For an exact test, check the group structure, exposures, and dispersion assumptions rather than treating “exact” as synonymous with “assumption-free.” These checks are about whether the model and test are suitable; they do not replace formal model diagnostics.
The tests can yield different p-values without any arithmetic error because they need not use the same estimate of uncertainty or reference distribution. A Wald test focuses on a fitted contrast; an LRT asks whether the larger model gains enough likelihood; a QL test incorporates additional variability; and an exact test uses a conditional distribution for a restricted design. None is universally most powerful or appropriate. In particular, an exact test is not automatically preferable whenever counts are small, and a likelihood-ratio test is not automatically preferable whenever there are multiple groups.
Common Mistake
Wrong reasoning: “There are two conditions, so I should use an exact test; adding batch would only complicate the calculation.” The correction is that the scientific design determines the model. If batch is a relevant covariate and the target is an adjusted condition effect, omitting batch can change the comparison and confound the interpretation. Use a GLM test that represents the design, such as a Wald or QL test for the adjusted contrast, or an LRT for a suitable nested-model hypothesis. The exact test is suited to its two-group conditional setup, not to every experiment that happens to have two condition labels.
Check Your Understanding
For each problem, choose and justify a test. Solutions are included so the reasoning can be checked.
- A two-condition study includes a batch covariate and asks for the adjusted treatment-versus-control effect. Which test is a direct choice?
- A three-condition study asks whether condition has any effect, with batch included in both models. Which test can answer this joint question?
- In a two-group design with equal exposures and Poisson-limit assumptions, the total count is \(20\). Under the null, what is the conditional distribution of the count in group A?
- Why does a QL test sometimes give a larger p-value than a Wald test from the NB GLM?
- What must be checked before using an LRT to test whether a group of coefficients improves a model?
Solutions. (1) A Wald test is direct for the specified adjusted contrast, assuming the GLM and its standard-error approximation are suitable; a QL test is also a natural choice if the analysis uses QL uncertainty. (2) Use an LRT comparing a reduced model without condition terms to a full model with the condition factor, or a corresponding joint QL test; a single pairwise contrast would not test the whole null. (3) \(X\sim\operatorname{Binomial}(20,0.5)\), because conditioning on the total under equal exposure and the Poisson limit gives equal allocation probability under the null. (4) The QL test can use a larger variance when extra variability beyond the NB fit is estimated, reducing the test statistic relative to a calculation that uses only the NB covariance. (5) Confirm that the models are nested, use the same observations, offsets, and dispersion treatment, and differ only by the terms whose joint contribution is being tested.