Why Sampling Without Replacement Creates Dependence
When items are selected from a finite population without replacement, learning the result of one draw changes the composition of the items available for later draws. If the first item is a success, there is one fewer success left; if it is a failure, the success proportion among the remaining items rises. This dependence can be summarized by the covariance between success indicators at two different draw positions.
The calculation applies to a simple random ordering of a finite population: every ordering of the items is equally likely, and each selected item is removed before the next draw. It does not assume that successive outcomes are independent. The population contains a fixed number of successes and failures throughout the sampling process, even though the number remaining changes from draw to draw.
Each indicator is Bernoulli: it takes the value 1 for a success and 0 for a failure. Because every population item is equally likely to occupy a particular position, the success probability at each position is \(p=K/N\). The key distinction is that this shared marginal probability does not make the indicators independent.
Deriving the Covariance
For two distinct positions, both indicators equal 1 precisely when both items at those positions are successes. The first position is a success with probability \(K/N\). Given that it is a success, \(K-1\) successes remain among \(N-1\) items, so the conditional probability of a success at the other position is \((K-1)/(N-1)\). Thus the probability of two successes is the product of these probabilities.
The equality \(E[I_rI_s]=P(I_r=1,I_s=1)\) holds because the product of two zero-one indicators is 1 exactly when both are 1, and is 0 otherwise. Also, \(E[I_r]=E[I_s]=K/N\). Substituting these quantities into the product-moment identity for covariance gives the result.
For two distinct positions, \(E[I_rI_s]=\dfrac{K(K-1)}{N(N-1)}\).
Each indicator has mean \(p=K/N\), so \(E[I_r]E[I_s]=\dfrac{K^2}{N^2}\).
The difference is \(\dfrac{K(K-1)}{N(N-1)}-\dfrac{K^2}{N^2}=-\dfrac{K(N-K)}{N^2(N-1)}\).
To verify the simplification, the joint probability minus the product of the marginal probabilities is \[ \frac{K(K-1)}{N(N-1)}-\frac{K^2}{N^2} =\frac{K^2N-KN-K^2(N-1)}{N^2(N-1)} =\frac{K^2N-KN-K^2N+K^2}{N^2(N-1)} =-\frac{K(N-K)}{N^2(N-1)}. \] Since \(K(N-K)\ge 0\), this covariance is never positive. It is strictly negative when the population contains at least one success and at least one failure. If all items have the same outcome, both indicators are constant and their covariance is zero.
Worked Calculations
Worked Example: Two Defective Components in a Sample
A hypothetical inspection tray contains 10 components, exactly 4 of which are defective. Two components are selected sequentially without replacement. Let \(I_1\) and \(I_2\) indicate whether the first and second components are defective. Calculate their covariance, and verify it using the joint probability.
Method and model. The tray is a finite population with \(N=10\), \(K=4\), and \(p=4/10=0.4\). Since the selections are without replacement, use the distinct-position covariance formula.
Formula calculation. \[ \operatorname{Cov}(I_1,I_2) =-\frac{p(1-p)}{N-1} =-\frac{(0.4)(0.6)}{9} =-\frac{0.24}{9} =-0.026666\ldots \] To four decimal places, the covariance is \(-0.0267\).
Joint-probability check. The probability that both components are defective is \[ P(I_1=1,I_2=1)=\frac{4}{10}\frac{3}{9} =\frac{12}{90} =\frac{2}{15} =0.1333\ldots. \] The product of the means is \[ E[I_1]E[I_2]=(0.4)(0.4)=0.16. \] Therefore, \[ E[I_1I_2]-E[I_1]E[I_2] =0.1333\ldots-0.16 =-0.026666\ldots, \] which rounds to \(-0.0267\). The two calculations agree. The common mistake avoided here is replacing the joint probability with \(0.4^2\), which would incorrectly treat the draws as independent.
Worked Example: Nonadjacent Draw Positions
A hypothetical shipment contains 12 sensor units, 5 of which fail a specified test. A technician randomly orders the units and examines them without replacement. Let \(I_1\) indicate a failure at position 1 and \(I_4\) a failure at position 4. Find the covariance.
Method and model. The positions are distinct, and every ordering is equally likely. Thus \(N=12\), \(K=5\), and both indicators have mean \(p=5/12\). The separation between positions does not change the formula.
The covariance is \[ \operatorname{Cov}(I_1,I_4) =-\frac{(5/12)(7/12)}{11} =-\frac{35}{144\cdot 11} =-\frac{35}{1584} =-0.0220959\ldots. \] To four decimal places, \(\operatorname{Cov}(I_1,I_4)=-0.0221\).
Check using the joint probability. The probability of failures in both specified positions is \[ P(I_1=1,I_4=1) =\frac{5}{12}\frac{4}{11} =\frac{20}{132} =\frac{5}{33}. \] The product of the means is \[ E[I_1]E[I_4]=\left(\frac{5}{12}\right)^2=\frac{25}{144}. \] Using the common denominator \(1584\), \[ \frac{5}{33}-\frac{25}{144} =\frac{240}{1584}-\frac{275}{1584} =-\frac{35}{1584} =-0.0220959\ldots. \] This confirms the result. A common error is to use the probability of success on the immediately following draw without checking what event is being considered. The joint event here concerns positions 1 and 4; equal likelihood of random orderings makes the two-position formula valid for them just as it is for positions 1 and 2.
Worked Example: Covariance and Correlation in a Small Population
A hypothetical batch has 15 packages, 6 of which are underweight. Two distinct positions in a random ordering are selected for review. Let their underweight indicators be \(I_r\) and \(I_s\). Calculate the covariance and correlation, and state whether the sign is consistent with the sampling mechanism.
Method and model. Here \(N=15\), \(K=6\), and \(p=6/15=0.4\). The indicators refer to distinct positions sampled without replacement.
Their covariance is \[ \operatorname{Cov}(I_r,I_s) =-\frac{(0.4)(0.6)}{14} =-\frac{0.24}{14} =-0.0171428\ldots. \] To four decimal places, it is \(-0.0171\).
Each indicator has variance \(p(1-p)=(0.4)(0.6)=0.24\), so each standard deviation is \(\sqrt{0.24}\). Because both standard deviations are positive, the correlation is \[ \operatorname{Corr}(I_r,I_s) =\frac{-0.24/14}{\sqrt{0.24}\sqrt{0.24}} =\frac{-0.24/14}{0.24} =-\frac{1}{14} =-0.071428\ldots. \] To four decimal places, the correlation is \(-0.0714\). The negative sign is consistent: finding an underweight package leaves fewer underweight packages for the other position. The check is that the covariance and correlation have the same sign when both variances are positive.
Interpretation, Limits, and an Exam Trap
The covariance is negative because a success at one sampled position slightly reduces the chance of a success at another. Its size depends on both the population composition and population size. When the success proportion is near 0 or 1, \(p(1-p)\) is small, so the covariance is close to zero. For a fixed success proportion, increasing \(N\) also reduces the magnitude through the factor \(1/(N-1)\).
For \(0<K<N\), each indicator has variance \(p(1-p)\). Dividing the covariance by the product of the standard deviations therefore gives a useful related result: \[ \operatorname{Corr}(I_r,I_s) =\frac{-p(1-p)/(N-1)}{p(1-p)} =-\frac{1}{N-1}. \] This correlation formula requires \(0<K<N\), since if \(K=0\) or \(K=N\), the indicators have variance zero and correlation is undefined. The covariance formula itself remains valid in those degenerate cases and gives zero.
The formula also requires sampling without replacement from a fixed finite population, with the positions selected according to a uniformly random ordering. If draws are made with replacement, each draw has success probability \(p\) and the draws are independent under that model; the covariance between distinct success indicators is then zero. Do not use the without-replacement formula for that different sampling design.
A reliable calculation begins by identifying \(N\), \(K\), and whether the sampling is with or without replacement. Then confirm that the indicators refer to two distinct positions, find their common mean \(K/N\), and use the joint probability \(K(K-1)/[N(N-1)]\). Finally, check that the covariance is nonpositive and agrees with the joint-probability calculation when one is available. These checks make the result aligned with the SOA Exam P (Probability) syllabus and guard against the most common modeling error.
Check Your Understanding
For each problem, assume uniform random sampling without replacement unless stated otherwise. Give covariance or correlation to four decimal places where needed.
- A sample is drawn from a population of 8 items, 3 of which are successes. Find the covariance between the success indicators at two distinct positions. (Answer: \(-0.0335\))
- A population has 14 items, exactly 7 of which are successes. Find the covariance between the indicators at positions 2 and 9. (Answer: \(-0.0192\))
- A population contains 10 items, 2 successes, and 8 failures. Let \(I_1\) and \(I_3\) denote success at positions 1 and 3. Calculate their joint success probability and covariance. (Answer: joint probability \(1/45=0.0222\); covariance \(-4/225=-0.0178\))
- A population has 18 items, 6 successes, and 12 failures. Find the correlation between success indicators at two distinct positions. (Answer: \(-1/17=-0.0588\))
- A container has 10 marked and 4 unmarked tokens. Two tokens are drawn independently, with replacement. Find the covariance between the indicators that the first and second draws are marked. (Answer: \(0\))