Why Dependence Matters for Empirical Bayes Calibration
A genomic analysis may produce thousands of test statistics, but those statistics are rarely independent: tests on nearby genomic features can share biological signal, and all tests may use the same subjects and experimental conditions. Dependence matters when interpreting a pooled histogram, estimating mixture parameters, and judging whether a calibration diagnostic shows a real mismatch or ordinary sampling variation.
The central distinction is between a marginal model for each test statistic and a joint model for the entire vector of statistics. A marginal mixture describes the distribution of a statistic chosen from the collection. It does not, by itself, assert that the statistics are independent. Independence is an additional assumption about their joint distribution.
Under this model, the marginal local false discovery rate at \(z\) is \( \operatorname{lfdr}(z)=P(H_i=0\mid Z_i=z)=\pi_0 f_0(z)/f(z) \), where the densities and \(\pi_0\) describe the marginal model. The formula is a statement about the posterior probability for a feature given its own statistic. It is not automatically the posterior probability given every statistic in the data set.
What Dependence Changes in Estimation
Many empirical Bayes procedures estimate mixture parameters by maximizing a product of marginal densities, such as \(\prod_{i=1}^m f(Z_i;\theta)\). When the statistics are dependent, this product is not generally the true joint density. It can still be used as a composite likelihood or marginal fitting criterion. Under correct marginal specification, identifiability, and suitable conditions such as a stationary, ergodic sequence with a law of large numbers for the fitting criterion, its maximizer can estimate the marginal parameters consistently. Those conditions are substantive: arbitrary dependence does not guarantee that the empirical distribution converges to the claimed marginal distribution in a single data set.
Even when marginal estimates remain meaningful, dependence changes their sampling variability. For a pooled quantity \(Y_i\), the variance of its sample average is \( \operatorname{Var}(\bar{Y})=m^{-2}\sum_i\sum_j\operatorname{Cov}(Y_i,Y_j) \). The independence calculation drops every off-diagonal covariance. If the covariances are positive, that calculation typically understates uncertainty. A product likelihood's usual independent-observation standard errors can likewise be too small, even when its point estimate is a useful marginal fit.
For an exchangeable block of \(b\) observations with common variance \(\sigma_Y^2\) and pairwise correlation \(\rho\), the block's average has variance \(\sigma_Y^2[1+(b-1)\rho]/b\). The multiplier \(1+(b-1)\rho\) is often called a design effect. It summarizes the extra variance relative to independent observations in this particular block model; it is not a universal correction for arbitrary dependence.
Worked Example: Dependence and a Threshold-Based Estimate of the Null Proportion
Suppose a hypothetical analysis has \(m=1{,}000\) test statistics. For a chosen threshold, let \(Y_i=1\) if statistic \(i\) exceeds it and \(Y_i=0\) otherwise. Assume the marginal exceedance probabilities are \(q_0=0.10\) for null features and \(q_1=0.50\) for alternatives, and observe \(\hat q=0.46\). Treat \(q_0\) and \(q_1\) as known for this illustration.
The marginal exceedance probability satisfies \(q=\pi_0q_0+(1-\pi_0)q_1\). Solving for the null proportion gives \(\pi_0=(q_1-q)/(q_1-q_0)\). Substituting the observed proportion, \( \hat\pi_0=(0.50-0.46)/(0.50-0.10)=0.04/0.40=0.10 \). Equivalently, the alternative proportion is \((0.46-0.10)/0.40=0.90\). These are plug-in estimates from the stated marginal probabilities, not estimates that account for every possible source of uncertainty.
Now suppose the 1,000 tests fall into 250 blocks of four. Within each block, assume the indicators have common pairwise correlation \(\rho=0.40\); assume different blocks are uncorrelated. With marginal \(q=0.46\), each indicator has variance \(q(1-q)=0.46(0.54)=0.2484\). The block design effect is \(1+(4-1)(0.40)=2.20\), so \( \operatorname{Var}(\hat q)=0.2484(2.20)/1{,}000=0.00054648 \). Thus \(SE(\hat q)=\sqrt{0.00054648}\approx0.02338\). Since \(\hat\pi_0=(0.50-\hat q)/0.40\), its standard error under these assumptions is \(0.023376\ldots/0.40\approx0.05844\).
For comparison, treating all tests as independent gives \(SE(\hat q)=\sqrt{0.46(0.54)/1{,}000}\approx0.01576\), and \(SE(\hat\pi_0)=0.01576/0.40\approx0.03940\). The point estimate is the same in both calculations because the marginal equation is unchanged; the estimated uncertainty is larger under the specified positive block dependence. The calculation does not establish that the block model describes real data.
Dependence and Calibration Diagnostics
A calibration diagnostic asks whether the assumed null distribution and fitted mixture provide a reasonable account of the observed statistics. Dependence does not necessarily change the expected marginal null distribution: if each null statistic has the specified \(f_0\), each still has that marginal distribution. It does, however, affect how much the empirical histogram, mean, or Q-Q plot can fluctuate around that expectation. Treating correlated tests as independent can make ordinary collective deviations look far more surprising than they are.
Worked Example: A Q-Q Plot With a Shared Random Shift
Consider a hypothetical collection of \(m=1{,}000\) all-null statistics with \(Z_i=\sqrt{\rho}\,U+\sqrt{1-\rho}\,\epsilon_i\), where \(U\) and the \(\epsilon_i\) are independent standard normal variables and \(\rho=0.02\). Each statistic has marginal distribution \(N(0,1)\), since its mean is zero and its variance is \(0.02+0.98=1\). The statistics are nevertheless correlated: for distinct \(i,j\), their covariance is \(0.02\), so their correlation is \(0.02\).
The sample mean is \(\bar Z=\sqrt{0.02}\,U+\sqrt{0.98}\,\bar\epsilon\). Therefore \( \operatorname{Var}(\bar Z)=0.02+0.98/1{,}000=0.02098 \), and its standard deviation is \(\sqrt{0.02098}\approx0.1448\). Under independence with the same marginal \(N(0,1)\) distribution, the standard deviation would be \(1/\sqrt{1{,}000}\approx0.03162\). A common shift can therefore move the whole empirical distribution together, producing a noticeable offset in a Q-Q plot despite correct one-feature null marginals.
The diagnostic still has value: a systematic pattern may point to a null-model problem, as discussed in the earlier material on null-model misspecification. But an independent-sample reference band is too narrow for this dependent example. A Q-Q plot is evidence about fit, not a proof of independence or a calibrated test of the null model by itself.
Useful checks include comparing the fitted marginal distribution with the empirical distribution, examining whether residual patterns recur across genomic blocks, and checking whether uncertainty changes materially under plausible dependence structures. If dependence is known or can be summarized by meaningful blocks, cluster-robust variance estimates or a block-based bootstrap may be appropriate. The method must reflect the actual dependence unit; arbitrarily grouping features does not correct uncertainty.
Marginal Local FDR and Joint Information
Worked Example: A Marginal Local FDR Is Not a Joint Posterior
Suppose \(H_i=0\) has prior probability \(\pi_0=0.80\), with \(Z_i\mid H_i=0\sim N(0,1)\) and \(Z_i\mid H_i=1\sim N(2,1)\). At \(z=1\), the ratio of the alternative to null density is \( f_1(1)/f_0(1)=\exp\{-(1-2)^2/2+(1-0)^2/2\}=1 \).
Consequently, \(f(1)=0.80f_0(1)+0.20f_1(1)=f_0(1)\), and the marginal local false discovery rate is \( \operatorname{lfdr}(1)=0.80f_0(1)/f(1)=0.80 \). This says that, under the specified marginal model, the posterior null probability for a feature with statistic 1, using that feature's statistic, is 0.80.
If neighboring statistics carry information about shared latent states or common effects, then \(P(H_i=0\mid Z_1,\ldots,Z_m)\) can differ from \(P(H_i=0\mid Z_i)\). The marginal formula above does not compute that joint posterior. Using it remains coherent when the target is the one-statistic marginal local FDR; interpreting it as a posterior conditional on the entire dependent vector requires a joint model.
Comparing Modeling Choices and Recognizing Failure
| Approach | What it models | Main caution |
|---|---|---|
| Marginal mixture fit | The one-feature density \(f(z)\) and marginal parameters | Independent-observation standard errors may be wrong under dependence |
| Joint dependence model | The full distribution of the statistic vector, including dependence | Requires a defensible dependence structure and can be difficult to fit |
| Marginal fit with dependence-aware uncertainty | The same marginal target, with variance estimation reflecting dependence | Requires suitable blocks, covariance estimates, or other dependence information |
Warning signs include standard errors that change sharply after accounting for blocks, calibration plots whose deviations cluster by genomic region, and parameter estimates that vary substantially across plausible ways of thinning or grouping correlated features. Such checks are diagnostic, not automatic repairs: thinning may discard information, and a fitted dependence correction can be wrong if its structure is misspecified.
- Mistake: “The marginal mixture is correct, so the product likelihood is the true joint likelihood.” Correction: The mixture specifies one-feature distributions. The product is the joint density only under the required independence assumptions.
- Mistake: “Dependence changes every marginal local FDR.” Correction: Dependence alone does not alter the marginal formula based on \(Z_i\); it can change uncertainty and the posterior conditional on the full vector.
- Mistake: “A Q-Q departure proves the null distribution is wrong.” Correction: Dependence can increase the variability of empirical diagnostics. Assess the departure with a dependence-appropriate reference and consider null misspecification as one possibility.
Check Your Understanding
Try each question, then compare your reasoning with the solutions.
- If each test statistic has marginal density \(f(z)=\pi_0f_0(z)+(1-\pi_0)f_1(z)\), does that imply the statistics are independent? Explain.
- For blocks of size 5 with within-block correlation \(0.20\), calculate the design effect for a block average.
- With \(\pi_0=0.70\), \(f_1(z)/f_0(z)=2\), calculate the marginal local FDR at \(z\).
- Why might an independent-sample Q-Q reference band be misleading when statistics share a common factor?
Solutions. (1) No. The formula gives the marginal density of a statistic; independence is a separate joint-distribution assumption. (2) The design effect is \(1+(5-1)(0.20)=1.80\), so the variance of the block average is 1.80 times the independent-observation variance for the same sample size. (3) The mixture density at \(z\) is \(\pi_0f_0(z)+(1-\pi_0)f_1(z)=0.70f_0(z)+0.30(2f_0(z))=1.30f_0(z)\). Thus the local FDR is \(0.70f_0(z)/(1.30f_0(z))\approx0.5385\). (4) A shared factor can move many statistics together, making empirical deviations more variable than under independence; the independent reference band may therefore be too narrow.