Dependence Patterns Among Genomic Tests

Why the Joint Pattern of P-Values Matters

A genomic analysis may produce thousands of p-values, one for each feature, variant, or other prespecified test. Each p-value is intended to summarize evidence for its own null hypothesis. But a list of individually valid p-values does not describe how those values behave together. Tests may be independent, or small p-values may tend to occur together; in other settings, one test’s small p-value may make another less likely to be small.

That distinction matters because multiple-testing procedures operate on a collection of p-values. Their joint behavior affects such features as the number of small p-values in a dataset and the variability of that number. Here the focus is on describing dependence patterns, not on establishing the guarantees of a particular correction procedure.

Definition: Test statistics \(T_1,\ldots,T_m\) are independent if their joint probability distribution factors into the product of their marginal distributions. Their p-values \(P_1,\ldots,P_m\) are independent if the same factorization holds for the p-values. Dependence means that the joint distribution does not factor in this way. Under a continuous, correctly calibrated null test, its p-value is uniform on \([0,1]\); a valid p-value for a discrete test is typically super-uniform, meaning \(P(P_i\leq t)\leq t\) for \(0\leq t\leq1\). Neither marginal property implies independence across tests.

The index \(i\) identifies a test, and \(m\) is the number of tests in the family. Dependence can arise because tests use overlapping samples, share measured variables or covariates, or respond to related biological signals. These mechanisms are reasons to investigate joint behavior, not proof of any one dependence pattern. Also, dependence can be present among test statistics, p-values, or both; the transformation from a statistic to a p-value can change how the association appears.

Four Useful Dependence Patterns

The following labels are descriptive. They refer to how tests tend to behave jointly, especially in the tails relevant to small p-values; they should not be treated as interchangeable formal assumptions.

Pattern	Typical joint behavior	Important qualification
Independence	Knowing one test’s result does not change the distribution of another test’s result.	Zero correlation alone does not establish independence.
Positive dependence	Small p-values tend to occur together more often than they would under independence.	A positive correlation of raw statistics does not by itself establish a particular positive-dependence condition for p-values.
Negative dependence	A small p-value for one test tends to coincide with a less-small p-value for another.	Negative dependence is not captured fully by one correlation coefficient.
Block dependence	Tests are dependent within groups or blocks, while dependence between blocks is weaker or absent.	Dependence may differ across blocks and need not be identical within them.

In genomic settings, a shared sample or a shared nuisance estimate can induce dependence across many tests. Biological or measurement structure can also make some subsets of tests more related than others. A block description is useful when tests fall into clusters with stronger within-cluster than between-cluster dependence. It is a simplified representation: real data need not divide neatly into independent blocks.

Worked Example: Independent Null P-Values

Suppose two hypothetical tests are both true null hypotheses, each uses a continuous calibrated test, and their p-values \(P_1\) and \(P_2\) are independent. Each p-value is therefore uniform on \([0,1]\). What is the probability that both p-values are at most \(0.05\)?

For independent events, the probability of their intersection is the product of their probabilities. Each event has probability \(0.05\), so

P(P_1\leq 0.05,\ P_2\leq 0.05) =P(P_1\leq 0.05)P(P_2\leq 0.05) =0.05\times0.05=0.0025.

Thus, under these assumptions, both null p-values are at most \(0.05\) with probability \(0.0025\), or \(0.25\%\). This calculation relies on independence as well as the uniform marginals. The marginal statement alone gives the two probabilities \(0.05\); it does not determine their intersection.

Small-P-Value Co-Occurrence

A simple way to describe joint tail behavior is to define an indicator for whether test \(i\) has a p-value below a threshold \(t\): \(I_i(t)=1\) if \(P_i\leq t\), and \(I_i(t)=0\) otherwise. If both tests have uniform null p-values, then \(P(I_i(t)=1)=t\). Their joint tail behavior is summarized by

\operatorname{Cov}(I_1(t),I_2(t)) =P(P_1\leq t,\ P_2\leq t)-t^2.

Under independence this covariance is zero. A positive value means that both p-values fall below \(t\) more often than \(t^2\), while a negative value means they do so less often. This is a threshold-specific description; it does not fully characterize the joint distribution or prove a stronger formal dependence condition.

Worked Example: Extreme Positive and Negative Dependence

For a hypothetical pair of true null tests, let \(U\) be uniform on \([0,1]\). In one model, set \(P_1=U\) and \(P_2=U\). In a second model, set \(P_1=U\) and \(P_2=1-U\). In both models, each p-value is individually uniform, but the pair behaves differently. Calculate the chance both p-values are at most \(0.05\).

Same-uniform model. The events \(P_1\leq0.05\) and \(P_2\leq0.05\) are both the event \(U\leq0.05\). Therefore,

P(P_1\leq0.05,\ P_2\leq0.05) =P(U\leq0.05)=0.05.

This is twenty times the independent value \(0.0025\). Conditional on \(P_1\leq0.05\), \(P_2\leq0.05\) occurs with probability \(1\) in this deliberately extreme construction.

Complementary-uniform model. Both p-values can be at most \(0.05\) only if \(U\leq0.05\) and \(1-U\leq0.05\), the latter requiring \(U\geq0.95\). These conditions cannot both hold, so

P(P_1\leq0.05,\ P_2\leq0.05)=0.

Each model has the same valid uniform marginal p-values, but their joint probabilities range from zero to \(0.05\) in these examples, with independence giving \(0.0025\). The constructions are illustrative extremes rather than claims about typical genomic data. They demonstrate why marginal calibration is not a description of joint behavior.

Dependence in Test Statistics and P-Values

When a statistic is converted to a p-value, the conversion depends on the test. For a one-sided test whose p-value decreases as its statistic increases, large values of two positively associated statistics may yield small p-values together. But the relationship is not always that direct. For a two-sided test, p-values are small for statistics with large absolute values in either direction. Consequently, statistics with negative correlation can still produce small two-sided p-values at the same time.

For example, suppose \(Z_1\) and \(Z_2\) are standard normal test statistics with correlation \(\rho\), and both tests use two-sided p-values \(P_i=2\Phi(-|Z_i|)\), where \(\Phi\) is the standard normal cumulative distribution function. A large \(|Z_1|\) and a large \(|Z_2|\) can occur together whether the signs match or differ. The sign of \(\rho\) alone therefore does not tell us the direction or strength of dependence between the two p-values. The testing direction and the statistic-to-p-value transformation must be considered.

Worked Example: A Block-Dependent Collection

Consider four hypothetical true null tests with uniform p-values. Let tests 1 and 2 share a common p-value \(U\), and let tests 3 and 4 share another common p-value \(V\). Assume \(U\) and \(V\) are independent uniforms on \([0,1]\). Thus dependence is perfect within each pair, while the two blocks are independent. At threshold \(t=0.10\), find the probability of at least one p-value below the threshold, the expected number below it, and the variance of that number.

Let \(B_U=I(U\leq0.10)\) and \(B_V=I(V\leq0.10)\). The number \(D\) of p-values at most \(0.10\) is \(D=2B_U+2B_V\), because each block contributes either zero or two. Each indicator is Bernoulli with probability \(0.10\), and the two indicators are independent.

At least one p-value is below the threshold exactly when at least one block is below it. Therefore,

$$ P(D>0)=1-P(B_U=0,B_V=0) =1-(0.90)(0.90)=0.19. $$

The expected number is

$$ E(D)=2E(B_U)+2E(B_V) =2(0.10)+2(0.10)=0.40. $$

Since a Bernoulli variable with success probability \(0.10\) has variance \(0.10(0.90)=0.09\), independence between blocks gives

\operatorname{Var}(D)=4\operatorname{Var}(B_U)+4\operatorname{Var}(B_V) =4(0.09)+4(0.09)=0.72.

For comparison, if all four p-values were independent uniforms, the expected number below \(0.10\) would still be \(4(0.10)=0.40\), but the probability of at least one would be \(1-(0.90)^4=0.3439\), and the variance of the count would be \(4(0.09)=0.36\). The block structure changes the probability and variability of simultaneous events, even though every marginal probability and the expected count remain the same.

What Dependence Changes—and What It Does Not

For valid marginal p-values, the expected number \(D_t\) of p-values at most \(t\) is determined by the marginal probabilities:

E(D_t)=E\left(\sum_{i=1}^{m} I(P_i\leq t)\right) =\sum_{i=1}^{m}P(P_i\leq t).

This identity uses linearity of expectation and does not require independence. For \(m\) independent, exactly uniform null p-values, it reduces to \(E(D_t)=mt\). Dependence instead changes joint probabilities and generally affects the variance and distribution of \(D_t\). In the block example, the expected count stayed fixed while its variance doubled relative to four independent tests.

This distinction is useful when interpreting a genomic p-value list. A large cluster of small p-values may reflect several related signals, dependence among tests, or both. It should not automatically be read as the same number of independent pieces of evidence. Conversely, dependence does not make each test’s marginal p-value invalid by itself; calibration of individual tests and assumptions about their joint behavior are separate questions.

How to Assess Dependence and Recognize Pitfalls

Dependence is not diagnosed by a single summary. Useful exploratory checks include plots of paired test statistics or p-values for selected tests, correlation summaries across tests, and counts of joint small-p-value events at prespecified thresholds. These checks should be interpreted with the test design in mind. A correlation coefficient can miss nonlinear or tail-specific dependence; an overall average can hide strong dependence in a subset of tests. Dependence may also be hard to estimate precisely when the number of samples is small relative to the number of tests.

When using resampling to explore dependence, the resampling scheme must preserve the relevant sample structure under the intended null. Breaking paired observations or other design relationships can erase the dependence one is trying to assess. A diagnostic based on simulated or resampled data is only as informative as the assumptions and structure retained by that procedure.

Mistake: “Every p-value is valid, so the p-values are independent.” Marginal validity controls each test’s null p-value distribution; it does not specify intersections such as \(P(P_1\leq t,P_2\leq t)\). Check joint behavior separately.
Mistake: “Positive statistic correlation means positive p-value dependence.” The transformation to a one-sided or two-sided p-value matters. Examine the testing direction and the joint tail events of interest.
Mistake: “Zero correlation proves independence.” Zero correlation only describes a particular linear association. Dependence can remain in nonlinear relationships or tail events.
Mistake: “All tests have one common dependence pattern.” A collection may contain independent tests, positively associated clusters, and other relationships at once. A block description can be more informative than a single label.

These descriptions set up the question of which formal dependence assumptions are sufficient for particular multiple-testing guarantees. Those assumptions are more specific than simply saying that p-values are “positively correlated.” Do not infer a procedure’s guarantee from a descriptive pattern alone.

Key takeaway: Marginal p-value validity describes each test separately; independence or another dependence pattern describes how tests behave jointly. Compare joint small-p-value probabilities, not just marginal distributions, and be cautious about translating dependence in statistics directly into dependence in p-values.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions.

Two independent true-null tests have uniform p-values. What is the probability both are at most \(0.02\)?
In the same-uniform construction \(P_1=U,\ P_2=U\), what is the probability both p-values are at most \(0.02\), and why does it differ from the independent case?
Why can the expected number of p-values below a threshold be unchanged by dependence while the variance changes?
For two-sided tests, why does a negative correlation between test statistics not necessarily imply that small p-values cannot occur together?
What is one reason a single correlation coefficient may be insufficient for describing dependence among genomic tests?

Solutions. (1) Independence gives \(0.02\times0.02=0.0004\). (2) The joint event is \(U\leq0.02\), so its probability is \(0.02\); the p-values are identical rather than independent. (3) Linearity of expectation depends only on the marginal event probabilities, whereas the variance includes covariances between the indicators. (4) Two-sided p-values are small for large absolute statistics, so statistics with opposite signs can still both have large absolute values and small p-values. (5) A correlation coefficient can miss nonlinear or tail dependence, and dependence can vary across subsets or blocks of tests.