Why Read a Dispersion-Mean Plot?
A negative binomial model allows count variability to exceed the Poisson variance. In RNA-seq, the amount of extra variability is summarized by a dispersion parameter, and its estimate affects the uncertainty used in differential-expression tests. A dispersion-mean plot makes it possible to inspect how that variability behaves across genes: whether estimates follow a smooth pattern, scatter widely around it, or include genes that stand apart.
The plot is a diagnostic summary, not a test of differential expression and not a substitute for examining a particular gene’s fitted model. Its main purpose is to show whether the dispersion assumptions used for inference look plausible across the range of expression. The earlier material on negative binomial GLMs and dispersion estimation defines the model and estimation procedures; here the focus is how to interpret their plotted results.
Axes, Points, and Trend
The horizontal coordinate is commonly the gene’s mean normalized count across samples, or a related estimate of its average expression. Normalization accounts for differences in library size so that the means are more comparable across samples. The vertical coordinate is an estimated dispersion. Both axes are often shown on logarithmic scales because expression means and dispersion estimates can span several orders of magnitude. Always check the plot’s labels and legend: software and analysis stages can differ in whether points show raw gene-wise, adjusted, or final estimates.
The trend summarizes how dispersion is expected to change with mean expression. It may decline at low means and level off at higher means, but its exact shape depends on the data and the fitted trend procedure. The trend is not a line that every gene is expected to sit exactly on. Gene-wise estimates are noisy, especially when counts are low or the number of replicates is limited. Their scatter around the trend is therefore part of what the plot displays.
A point well above the trend has a gene-wise dispersion estimate larger than the trend predicts at that mean. A point well below it has a smaller estimate. These are relative descriptions: neither position alone establishes a biological cause, a data error, or a significant differential-expression effect. A dispersion outlier is also not the same thing as an extreme count observation in one sample. The former concerns a gene-level variability estimate relative to a trend; the latter concerns an individual observation and requires its own diagnostics.
Worked Example: Reading Three Points Against a Trend
Suppose a hypothetical plot has a fitted trend of \(0.20\) at mean normalized count \(100\), and \(0.10\) at mean normalized count \(1{,}000\). Three genes have the following gene-wise estimates: Gene A has mean \(100\) and dispersion \(0.24\); Gene B has mean \(100\) and dispersion \(0.80\); Gene C has mean \(1{,}000\) and dispersion \(0.07\). Interpret their positions.
At mean \(100\), the trend value is \(0.20\). Gene A’s estimate is \(0.24\), only \(0.04\) above that value, so it is close to the trend relative to Gene B. Gene B’s estimate is \(0.80\), which is \(0.80-0.20=0.60\) above the trend and four times the trend value. It is conspicuously above the trend and merits attention as a possible dispersion outlier.
At mean \(1{,}000\), the trend is \(0.10\). Gene C’s estimate is \(0.07\), or \(0.03\) below the trend. It is below-trend, but that alone does not show that the gene is misfit or that its test is invalid. The plot identifies relative dispersion patterns; further checks are needed before assigning a cause or deciding that a result is unreliable.
Why Mean Expression and Dispersion Matter for Variance
The negative binomial variance formula explains why both axes matter. Dividing the variance by \(\mu^2\) gives $$ \frac{\operatorname{Var}(Y)}{\mu^2}=\frac{1}{\mu}+\alpha. $$ Thus, at fixed dispersion, the relative contribution of Poisson sampling variability, \(1/\mu\), decreases as the mean increases. The dispersion contributes \(\alpha\) to this relative variance at every mean. A dispersion-mean trend describes how that latter contribution is estimated to vary across genes; it does not mean that the raw count variance is constant.
This also helps explain why low-mean genes often have unstable gene-wise dispersion estimates. With few counts, the data contain limited information to distinguish the Poisson component from extra-Poisson variation. A small number of samples makes that problem more acute. Consequently, wide scatter among low-mean estimates is not automatically evidence of a collection of biological outliers.
Worked Example: Comparing Variability at Two Means
Consider two hypothetical genes with the same dispersion estimate, \(\alpha=0.15\), but means \(\mu=20\) and \(\mu=200\). Use the negative binomial model to compute their variances and squared coefficients of variation.
For the gene with mean \(20\), $$ \operatorname{Var}(Y)=20+0.15(20)^2=20+60=80, $$ so its standard deviation is \(\sqrt{80}=8.944\). Its squared coefficient of variation is \(80/20^2=0.20\), also obtained from \(1/20+0.15=0.20\).
For the gene with mean \(200\), $$ \operatorname{Var}(Y)=200+0.15(200)^2=200+6{,}000=6{,}200, $$ so its standard deviation is \(\sqrt{6{,}200}=78.74\). Its squared coefficient of variation is \(6{,}200/200^2=0.155\), equal to \(1/200+0.15=0.155\).
The higher-mean gene has much larger count variance in absolute units, while its relative variance is smaller. This distinction prevents a common misreading: a downward trend in relative dispersion does not imply that higher-expression genes have smaller raw count variance.
How Dispersion Patterns Affect Fitted Tests
For a fitted negative binomial GLM, estimated dispersion contributes to the variance of the counts and therefore to the uncertainty of fitted coefficients. In general, a larger dispersion means less precise coefficient estimates, all else equal. The precise standard error also depends on the design matrix, fitted means, sample sizes, offsets, and contrasts; a dispersion-mean plot alone cannot determine a gene’s test statistic or p-value.
A simple approximation illustrates the link. Suppose two independent groups each have \(r\) replicate samples, with equal expected count \(\mu\) in each sample under the null and a common dispersion \(\alpha\). For a group sample mean \(\bar{Y}\), independence gives \(\operatorname{Var}(\bar{Y})=(\mu+\alpha\mu^2)/r\). At sufficiently large means for the delta method to be reasonable, $$ \operatorname{Var}(\log \bar{Y})\approx \frac{\operatorname{Var}(\bar{Y})}{\mu^2} =\frac{1/\mu+\alpha}{r}. $$ The two group means are independent, so the approximate variance of their log difference is $$ \operatorname{Var}(\log\bar{Y}_1-\log\bar{Y}_2) \approx \frac{2(1/\mu+\alpha)}{r}. $$ This is a simplified calculation, not a replacement for the GLM coefficient covariance calculation. It does show why larger dispersion tends to increase uncertainty, and why low means and small replicate counts also make estimates less precise.
Worked Example: Dispersion and Approximate Log-Ratio Uncertainty
Suppose a hypothetical balanced experiment has \(r=4\) replicates per group and an expected normalized count of \(\mu=100\) per sample under the null. Compare the approximate standard error of the log mean difference at dispersions \(0.05\) and \(0.20\), using the expression above.
When \(\alpha=0.05\), $$ \operatorname{Var}(\log\bar{Y}_1-\log\bar{Y}_2) \approx\frac{2(1/100+0.05)}{4} =\frac{2(0.06)}{4}=0.030, $$ giving an approximate standard error of \(\sqrt{0.030}=0.1732\).
When \(\alpha=0.20\), $$ \operatorname{Var}(\log\bar{Y}_1-\log\bar{Y}_2) \approx\frac{2(1/100+0.20)}{4} =\frac{2(0.21)}{4}=0.105, $$ giving an approximate standard error of \(\sqrt{0.105}=0.3240\). Under these simplifying assumptions, the larger dispersion yields a larger standard error. The calculation does not predict a particular gene’s p-value; it illustrates why dispersion estimates materially affect test precision.
Reading Scatter and Outliers Carefully
Interpret the plot by looking first for the broad trend, then for the scatter around it, and finally for points that are unusually far from it. Ask whether the trend follows the main cloud across the full mean range, whether scatter is concentrated at low means, and whether a few points are separated from the cloud. A gradual change in scatter or a handful of remote points may be expected in finite samples; a systematic bend or widespread displacement can indicate that a single smooth trend does not describe the estimates well.
A gene far above the trend may have genuinely elevated biological variability, an influential sample, an unmodeled design feature, or simply a noisy gene-wise estimate. The plot itself cannot distinguish these explanations. Check the gene’s counts across samples, the experimental design, and relevant model diagnostics. The residual-diagnostic material addresses observation-level model checks; the present plot contributes a complementary gene-level view.
Likewise, a point below the trend is not automatically an opportunity to claim unusually precise inference. The final dispersion used by a method may be moderated or otherwise adjusted rather than equal to the raw gene-wise estimate. Test reliability depends on the procedure actually used, the model assumptions, and whether the fitted model represents the experiment.
Common and Alternative Dispersion Choices
A common-dispersion approach assigns one value across genes. It is simple, but can be too restrictive if dispersion changes systematically with expression. A trended approach uses the fitted mean-dispersion relationship, allowing expected dispersion to depend on mean. Gene-wise estimates allow each gene to have its own value, but can be noisy, especially at low counts. Shrinkage combines gene-wise information with the trend or other shared information to stabilize estimates; a plot may show these stages separately, so its legend matters.
These are modeling choices, not visual decorations. A trend that misses a broad pattern may yield poorly calibrated dispersion values in parts of the mean range. Unmoderated gene-wise values can be highly variable and may give unstable test uncertainties. Conversely, borrowing information toward a trend can be misleading for a genuine outlier if the procedure does not handle such cases appropriately. The plot is useful for checking the pattern and identifying where assumptions deserve scrutiny, but it does not by itself choose the correct method.
| Dispersion approach | What it represents | Interpretive concern |
|---|---|---|
| Common | One dispersion shared across genes | May miss a systematic mean-dependent pattern |
| Trended | Dispersion varies according to a fitted mean relationship | A misspecified trend can misrepresent regions of the plot |
| Gene-wise | A separate estimate for each gene | Estimates can be noisy, particularly with few samples or low counts |
| Shrunk or moderated | Gene-wise estimates stabilized using shared information | Inspect the method and how it treats genes far from the trend |
Diagnostics, Pitfalls, and Limitations
A useful reading should include the scale, the legend, and the displayed estimation stage. Logarithmic axes can make equal visual distances represent multiplicative rather than additive differences. A point that looks far from the trend may not be as extreme on the original scale, while a visually modest vertical gap on a log scale can correspond to a large ratio. Do not compare distances without checking the scale.
- Do not call every high point a data error. It is an estimate above the trend, not a diagnosis of its cause.
- Do not confuse dispersion outliers with single-sample count outliers. They refer to different levels of variation and require different checks.
- Do not infer differential expression from vertical position. Dispersion describes variability; differential expression concerns a modeled condition contrast.
- Do not infer test reliability from one plot alone. The design, fitted means, sample size, model diagnostics, and dispersion procedure all matter.
The plot has further limits: a clean-looking cloud cannot prove that the negative binomial model is correct, and a smooth trend cannot establish that all genes share the same biological variability pattern. It summarizes estimated dispersion against mean expression across genes; it does not display all sample-level structure or explain why a gene has its particular estimate.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions below.
- At mean \(50\), the trend is \(0.30\), while one gene’s estimate is \(0.90\). What does the point’s position establish, and what does it not establish?
- For \(\mu=40\) and \(\alpha=0.10\), calculate the negative binomial variance and squared coefficient of variation.
- Why can gene-wise dispersion estimates be especially scattered at low mean expression?
- Under the approximation in this tutorial, calculate the standard error of a two-group log mean difference for \(r=5\), \(\mu=80\), and \(\alpha=0.10\).
- Name one limitation of using the dispersion-mean plot to judge the reliability of a specific gene’s test.
Solutions. (1) The estimate is \(0.90-0.30=0.60\) above the trend and three times its value, so it is a conspicuous above-trend dispersion estimate. The plot alone does not establish its cause, whether it is a data error, or whether the gene is differentially expressed. (2) \(\operatorname{Var}(Y)=40+0.10(40)^2=40+160=200\). The squared coefficient of variation is \(200/40^2=0.125\), equivalently \(1/40+0.10=0.125\). (3) Low counts provide limited information for separating Poisson sampling variation from extra-Poisson variation, so gene-wise estimates are less stable. (4) The approximate variance is \(2(1/80+0.10)/5=2(0.1125)/5=0.045\), and the standard error is \(\sqrt{0.045}=0.2121\). (5) The plot does not show all sample-level structure or fully determine the coefficient uncertainty; the design, fitted means, model diagnostics, and dispersion procedure also affect the test.