Dependence as a Source of Extra Variation

Why Dependence Matters for Dispersion

A binomial or Poisson GLM specifies a conditional mean and a corresponding variance benchmark. A dispersion check asks whether observed variation is larger than that benchmark. The check can signal extra variation even when the mean model is plausible, if observations that the model treats as independent are actually dependent.

This occurs naturally when several measurements come from the same person, patients share a clinic, or counts are recorded in neighboring time periods. Knowing the predictors may explain the expected response at each observation without removing the association between responses. The key question is therefore not only whether each observation has the right marginal variance, but also whether the joint variation matches the independence assumption.

For simplicity, the derivations below condition on the covariates and treat the model means and variance parameters as known. This isolates the effect of dependence; estimating regression coefficients changes the exact distribution of fitted-model diagnostics.

How Covariance Changes a Cluster Total

Suppose \(Y_1,\ldots,Y_m\) are responses in one cluster, with conditional means \(\mu_j\), variances \(V_j\), and pairwise covariances \(C_{jk}\). The cluster total is \(T=\sum_{j=1}^m Y_j\). Its mean is the sum of the means, but its variance also contains every pairwise covariance.

Proof. Write \(T-E[T]=\sum_j(Y_j-E[Y_j])\), square this expression, and take expectations. Each squared term has expectation \(V_j\). For each pair \(j<k\), the two cross-products have total expectation \(2E[(Y_j-E[Y_j])(Y_k-E[Y_k])]=2C_{jk}\). Summing these terms gives the formula. \(\square\)

Under conditional independence, all \(C_{jk}=0\), so the variance of a total is the sum of its marginal variances. A dispersion benchmark based on that sum will be too small when the actual within-cluster covariances are positive. The effect depends on the size and pattern of the covariances, not just on the number of observations.

Binomial Responses Within Clusters

Consider \(m\) binary responses \(Y_j\) with the same conditional success probability \(p\). Each response has variance \(p(1-p)\). Suppose each distinct pair has the same conditional correlation \(\rho\), so its covariance is \(\rho p(1-p)\). The total \(K=\sum_jY_j\) has mean \(mp\). If the responses were independent, \(K\) would have the binomial variance \(mp(1-p)\). With the stated dependence, its variance is multiplied by a factor called the design effect.

Derivation. There are \(m\) marginal variance terms and \(m(m-1)/2\) distinct pairs. Applying the variance-of-a-sum identity, \[ \operatorname{Var}(K)=mp(1-p)+2\frac{m(m-1)}{2}\rho p(1-p) =mp(1-p)\{1+(m-1)\rho\}. \] Dividing by the independent-binomial variance \(mp(1-p)\) gives the design effect \(1+(m-1)\rho\). For positive \(\rho\), even a modest pairwise correlation can matter when clusters are large. \(\square\)

For grouped binomial data, the Pearson contribution for a group is the squared difference between the observed group count and its fitted mean, divided by the independent-binomial variance. If the probability is known and used as the fitted value, the expected numerator is the true variance of the group count. In the example, the expected Pearson contribution is therefore \(1.344/0.84=1.60\), rather than 1. A collection of such groups can consequently show a high dispersion ratio even when the probability model for each individual is plausible.

Dependent Poisson Counts

The same covariance accounting applies to counts. Suppose \(Y_j\) has conditional Poisson mean \(\mu_j\), so its marginal variance is also \(\mu_j\). Under independence, a total has variance \(\sum_j\mu_j\), matching its mean. Under dependence, the sum of pairwise covariances adds to that variance. A positive covariance can therefore make a total’s variance exceed its Poisson mean without changing the individual marginal means.

Clustering and temporal dependence need not take the form of shared events. The examples illustrate the general point: whenever nearby or grouped counts tend to rise and fall together, their sum varies more than the independent-count calculation predicts. If some covariances are negative, the effect can go in the other direction; dependence does not guarantee overdispersion.

Dependence and the Calibration of Dispersion Checks

A crucial distinction is between a group total and a sum of individual Pearson contributions. For individual observations with known means and correctly specified marginal variances, dependence alone does not change the expected sum of those contributions. It can, however, change the contribution sum’s variance and distribution. Thus a high realization may be more likely than an independence-based calibration suggests, even when the expected value has not increased.

Proof. By linearity of expectation, \[ E[Q]=\sum_{j=1}^m\frac{E[(Y_j-\mu_j)^2]}{V_j} =\sum_{j=1}^m\frac{\operatorname{Var}(Y_j)}{V_j} =\sum_{j=1}^m 1=m. \] No cross-products occur in \(Q\), so the pairwise covariances do not enter this expectation. They can still affect the distribution of \(Q\), because its summands may be associated. \(\square\)

This distinction explains why dependence can produce apparent extra variation in more than one way. For grouped totals, positive covariance directly increases the expected squared residual relative to an independence-based binomial or Poisson variance. For individual-level Pearson sums, dependence can instead undermine the usual reference distribution by making the statistic more variable or changing its tail probabilities. A large observed ratio is evidence of a mismatch with the independence-based benchmark, but it does not identify the cause by itself.

Practical Interpretation and a Common Mistake

When measurements are clustered, repeated, or ordered in time, first ask which responses could remain associated after conditioning on the predictors in the mean model. Then identify the unit used in the diagnostic: an individual response or a cluster total. A single global dispersion ratio cannot reveal whether the issue is dependence, a wrong mean structure, or another source of variation. Residual patterns across the recorded groupings or time order can help make the dependence concern visible.

• Common mistake: “Every response has the right binomial or Poisson marginal variance, so the usual dispersion diagnostic is calibrated.”
• Correction: Marginal means and variances do not specify the joint distribution. Check whether the observations can be treated as conditionally independent; for totals, include the covariance terms, and for individual contributions, remember that dependence can change the diagnostic’s distribution.

Do not conclude that every positive association must increase every dispersion summary. The variance of a total rises with positive covariance, while the expected individual Pearson sum in the known-mean result remains fixed. Dependence changes the calibration in ways that depend on how the statistic is constructed.