Why the Replacement Rule Matters
Suppose a container holds red and blue objects, and two objects are drawn in succession. The color of the first draw can affect the chance of each color on the second draw—but only if the first object is not returned. A simulation that fails to represent this rule is modeling a different chance process, even if its individual draws look random.
The key is to distinguish the physical objects from their colors. If there are several red objects, drawing one red object without replacement removes that particular object, not every red object. The number of red objects still available decreases by one, while the total number available also decreases by one.
For the basic model in this tutorial, assume the collection starts with \(N\) distinct physical objects, of which \(R\) are red and \(B=N-R\) are blue. Each object available at a draw is equally likely to be selected. Draws are made one at a time, and a new repetition of the whole experiment starts again with the original collection. The replacement rule determines whether the available collection changes between draws within a repetition.
Representing the Draws in a Simulation
A useful simulation labels every physical object, even when several objects have the same color. For example, a container with two red objects and one blue object can be labeled R1, R2, and B1. The labels ensure that the simulation can remove one specific object while leaving the other objects of the same color available.
To simulate a draw with replacement, choose one of the \(N\) object labels at random and return it before choosing again. To simulate a draw without replacement, choose one of the labels still available, record its color, and remove that label from the available list. At the start of the next repetition, restore the entire list. In either case, the random choice at each step must give equal probability to each object currently available.
Give each physical object its own label and record its color. This preserves the difference between, for example, R1 and R2.
With replacement, choose among all \(N\) objects each time. Without replacement, choose only among the objects that remain.
Record the colors or event of interest. Then restore the original collection before starting a new repetition of the experiment.
If software selects a random integer rather than a label, the integer range must match the available objects. With replacement, a draw can use an integer from 1 to \(N\) every time. Without replacement, the first draw uses 1 to \(N\), the second uses 1 to \(N-1\), and so on; each new integer is mapped to one of the objects still available. Reusing the original range without removing the selected object would not simulate drawing without replacement.
Worked Example: Encoding a Two-Draw Simulation
A hypothetical container holds R1, R2, and B1. Design a simulation for two draws without replacement, and calculate the exact probability of drawing two red objects as a check on the model.
Set up the random choices. For the first draw, generate an equally likely integer from 1 to 3 and use it to select R1, R2, or B1 in a fixed order. Remove the selected object. For the second draw, generate an equally likely integer from 1 to 2 and use it to select from the two remaining objects in a fixed order. Thus there are \(3\times2=6\) equally likely ordered pairs of choices.
Check the exact probability. There are two ways to draw two reds: R1 followed by R2, or R2 followed by R1. Therefore,
The same calculation follows from the changing numbers of available objects: the first draw is red with probability \(2/3\), and after a red is removed, one red remains among two objects. Hence \(P(\text{two red})=(2/3)(1/2)=1/3\).
For illustration, suppose six simulation repetitions produce the integer pairs (1,1), (3,1), (2,2), (1,2), (2,1), and (3,2), using the fixed-order procedure above. The corresponding color pairs are RR, BR, RB, RB, RR, and BR. Two of the six repetitions have two reds, so the observed proportion is \(2/6=1/3\). This small illustrative run happens to equal the exact probability; a finite simulation run need not do so.
How Replacement Changes the Probabilities
Let \(A\) be the event that the first draw is red, and let \(C\) be the event that the second draw is red. In either model, \(P(A)=R/N\). The second-draw probability, however, depends on what happened first when objects are not replaced.
With replacement, the collection is restored after the first draw. There are still \(R\) red objects among \(N\), so \(P(C)=R/N\) regardless of the first color. Since each draw is made from the same unchanged collection, the color outcomes of the two draws are independent: the probability of any pair of colors is the product of the probabilities for those colors on each draw.
Without replacement, if the first draw is red, only \(R-1\) red objects remain among \(N-1\) objects. If the first draw is blue, \(R\) red objects remain among \(N-1\). So the second-draw probability changes with the first result. In particular, for two red draws,
These products use the probability for the first draw and then the appropriate probability for the second draw given the first result. This is a direct way to calculate the probability of an ordered sequence; the second factor is not generally the original red proportion.
Worked Example: Two Red Draws With and Without Replacement
A hypothetical container has three red objects and two blue objects. Two objects are drawn in order. Find the probability that both are red under each replacement rule.
With replacement. The first red draw is followed by returning that object, so the second draw again has three red objects among five:
Without replacement. After the first red object is removed, two red objects remain among four:
The probability is lower without replacement because a first red draw leaves fewer red objects available. The difference is due to the changing composition of the collection, not to a change in how fairly each available object is selected.
Dependence Between Draws
Two events are independent when the probability that both occur equals the product of their separate probabilities. For the events \(A\) and \(C\), this means \(P(A\cap C)=P(A)P(C)\). With replacement, the color on the first draw does not change the probabilities on the second, so the draw outcomes are independent under the stated model.
Without replacement, the draws are generally dependent: learning the first color changes the composition of the remaining collection, and therefore changes the probability of the second color. For the red-red event, the size and direction of the change can be shown directly by comparing the joint probability with the product of the marginal probabilities. When \(N\geq2\),
Since \(N-R=B\), the right-hand side is nonpositive. It is strictly negative when \(0<R<N\) and \(N\geq2\). If \(R=0\), both draws must be blue; if \(R=N\), both must be red. In either endpoint case the red indicators are constant and their covariance is zero, not strictly negative. This qualification matters: “without replacement” does not imply strict dependence in every possible collection.
The same result can be described using indicator variables. Let \(I_1=1\) when the first draw is red and 0 otherwise, and define \(I_2\) similarly for the second draw. Their covariance is \(E(I_1I_2)-E(I_1)E(I_2)\), which equals the difference just calculated. It is negative when both colors are present, because a red on the first draw slightly reduces the chance of a red on the second.
Worked Example: Exactly One Red Object
Use the same hypothetical container with three red and two blue objects. Find the probability of exactly one red in two draws, first with replacement and then without replacement. The order can be red-blue or blue-red.
With replacement. Each ordered sequence has the same probability as its reverse:
Without replacement. For red then blue, there are two blue objects among four after the red is removed. For blue then red, there are still three red objects among four:
The probability is larger without replacement. A first red makes a second red less likely, while a first blue makes a second red more likely. Those changes make a mixed-color pair more likely than under replacement.
A Practical Check for a Correct Model
Before running or interpreting a simulation, check that the simulated mechanism matches the physical rule. In particular, ask whether each draw selects a physical object or merely a color, whether the selected object returns before another draw, and whether the available set is restored before a new repetition. A correct random-number generator cannot fix an incorrectly updated set of outcomes.
| Feature | With replacement | Without replacement |
|---|---|---|
| Objects available on the next draw | All original objects | Only objects not already selected |
| Red probability after a red draw | \(R/N\) | \((R-1)/(N-1)\) |
| Are successive color draws independent? | Yes, under the stated model | Generally no when both colors are present |
| Collection at the start of a new repetition | Restore the original collection | Restore the original collection |
Common Mistake
- Wrong reasoning: “The first draw is random, so the second draw has the same probabilities and is independent, even without replacement.” Random selection alone does not make draws independent. Removing an object changes which objects remain and can change the next draw's probabilities.
- Correction: Track the available objects after every draw. Without replacement, after a red draw the red proportion is \((R-1)/(N-1)\), not \(R/N\). With replacement, the original collection is restored and the red proportion remains \(R/N\).
- Another coding error: Removing an entire color after selecting one object of that color. Remove only the selected physical object; any other objects of the same color remain available.
Check Your Understanding
Answer each question by tracking the objects available at each draw.
- A container has four red and three blue objects. What is the probability of two red draws with replacement? What is the probability without replacement?
- Why must two physical objects of the same color have separate labels in a simulation without replacement?
- A container has only red objects. Are two draws without replacement strictly dependent as red-versus-blue outcomes? Explain.
- In a simulation without replacement, what should happen to the available object list after the final draw of one repetition and before the next repetition?
- For a container with \(0<R<N\) and \(N\geq2\), why is the probability of two red draws without replacement less than the product of the two marginal red probabilities?