Negative Dependence and Hypergeometric Variance

Why Sampling Without Replacement Changes Variance

When items are drawn without replacement, the result of one draw changes what remains available for the next. If the first draw is a success, there is one fewer success in the population still to be drawn from. Thus distinct draw indicators are negatively dependent. The mean of a hypergeometric count can be found by adding the mean contributions of its draw indicators; its variance also depends on how those indicators co-vary.

That covariance has a useful consequence: relative to the variance \(np(1-p)\) suggested by \(n\) independent indicators with the same success probability, sampling without replacement reduces the variance. The exact reduction is the finite-population correction.

Setup and Indicator Covariance

Every draw position has the same marginal success probability, \(P(Y_i=1)=p\), and hence \(\operatorname{Var}(Y_i)=p(1-p)\). The crucial step is to calculate the probability that two distinct draws are both successes. For \(i\ne j\), the first draw is a success with probability \(K/N\); conditional on that success, the second draw is a success with probability \((K-1)/(N-1)\).

Since \(E[Y_i]E[Y_j]=p^2=K^2/N^2\), subtracting the product of the means gives the covariance.

To verify the covariance formula directly, put both terms over the common denominator \(N^2(N-1)\):

For \(0<K<N\), each indicator has variance \(p(1-p)>0\), so dividing the covariance by that variance gives the correlation \(-1/(N-1)\). If \(K=0\) or \(K=N\), all indicators are constant and their correlation is undefined, although the covariance formula remains valid and equals zero. The negative covariance expresses the fixed total number \(K\) of successes: observing a success leaves fewer successes among the remaining items.

Deriving the Hypergeometric Variance

The variance of a sum includes the individual variances and the covariances between every distinct pair. In this setting all draw indicators have the same variance, and every distinct pair has the same covariance. There are \(n\) individual terms and \(n(n-1)\) ordered cross-terms in the expanded variance.

Proof. Since \(X=\sum_{i=1}^{n}Y_i\), linearity of variance for a finite sum gives

The last equality follows because \((N-1)-(n-1)=N-n\). Substituting \(p=K/N\) gives the second expression. This argument uses only a finite sum, so no limit or interchange condition is needed. \(\square\)

The formula also handles \(n=0\), \(n=N\), \(K=0\), and \(K=N\), provided \(N>1\). For a census, \(n=N\), it gives variance zero: the total number of successes is then known to be exactly \(K\). If \(N=1\), the displayed formula has a zero denominator; the only possible population and sample counts are deterministic, so the variance is zero and must be handled directly.

The Finite-Population Correction

The factor \((N-n)/(N-1)\) is the exact variance correction for the hypergeometric count relative to \(np(1-p)\). For standard deviations, the corresponding multiplier is its square root. The correction reflects the negative covariance: the cross-terms subtract from the sum of the individual variances. As the sample approaches a census, the correction approaches zero.

For the sample proportion \(\bar Y=X/n\), when \(n>0\), the variance is

A common finite-population convention expresses this in terms of the population variance. Assign each population item a value \(z_a\), equal to 1 for a success and 0 otherwise, for \(a=1,\ldots,N\). Its mean is \(\bar z=p\), and define its finite-population variance with denominator \(N-1\):

Among the \(K\) successes, \(z_a-\bar z=1-p\); among the \(N-K\) failures, \(z_a-\bar z=-p\). Therefore,

Substituting this expression into the sample-proportion variance gives the familiar finite-population form:

Here the standard deviation of the sample mean is \(S/\sqrt n\) multiplied by \(\sqrt{1-n/N}\). The two correction expressions are consistent: \((N-n)/(N-1)\) applies when the baseline variance is written as \(np(1-p)\), whereas \(1-n/N\) applies when it is written using \(S^2\) with denominator \(N-1\).

Common Mistake and Why the Dependence Matters

A frequent error is to treat the indicators as independent merely because each has the same marginal success probability. That reasoning would set every covariance to zero and give \(\operatorname{Var}(X)=np(1-p)\). Equal marginal probabilities do not imply independence. Once a success has been drawn, the next-draw success probability is \((K-1)/(N-1)\), not \(K/N\); for \(0<K<N\), the difference is

The missing covariance terms are therefore negative, and omitting them overstates the variance unless there is only one draw or the indicators are degenerate. The correction is especially consequential when the sample is a substantial fraction of the population; at a full census, it correctly reduces the variance to zero.