Multi-Stage Experiments Without Independence Assumptions

Introduction

A multi-stage experiment produces more than one observation, and the order of those observations can matter. Two coin tosses, two successive draws, and two measurements on the same patient all produce pairs of outcomes. To describe such an experiment, we need to specify what can happen at each stage and how a complete ordered sequence is represented.

The central distinction is between which sequences are possible and how probability is assigned to them. A product sample space records the stage-by-stage structure. By itself, it does not say that stages are independent, equally likely, or even governed by the same probability law.

Ordered Sequences and Product Sample Spaces

The coordinate position matters: \((x_1,x_2)\) says that \(x_1\) occurred first and \(x_2\) second. Even if the same set is used at both stages, the first coordinate and second coordinate refer to different times or roles. Thus two coin tosses have sample space \(\{H,T\}\times\{H,T\}\), whose outcomes are \(HH,HT,TH,TT\). The outcomes \(HT\) and \(TH\) are different sequences.

The definition uses a product when every combination of stage outcomes is possible. A product is a convenient starting point, but it may include tuples that the experiment cannot actually produce. In that case, the sample space must retain only the admissible sequences.

To see why the count holds, first choose a stage-1 outcome in \(|S_1|\) ways. For each such choice, there are \(|S_2|\) choices at stage 2, giving \(|S_1||S_2|\) pairs. Each pair can be extended by any of the \(|S_3|\) stage-3 outcomes, so there are \(|S_1||S_2||S_3|\) triples. Continuing stage by stage gives the stated product. This count assumes that every combination is permitted; it cannot be applied unchanged when earlier outcomes restrict later choices.

When Earlier Results Restrict Later Choices

In some experiments, the possible outcomes at a later stage depend on the history so far. A fixed product of stage-specific sets can then contain impossible tuples. One can still describe a complete outcome as an ordered tuple, but the sample space consists only of tuples that satisfy the experiment's restrictions.

This definition concerns possibility, not probability. For example, if a card is removed after it is drawn, drawing that same card again is not an admissible second-stage result. The set of second-stage possibilities depends on the first draw, so the complete sample space is naturally a restricted subset of the product of the original card set with itself.

The subset claim follows directly from the definitions: every admissible sequence has a stage-\(i\) outcome in \(S_i\), so it is an element of \(S_1\times\cdots\times S_k\). If every product tuple is admissible, the two sets contain exactly the same tuples and are equal. If at least one product tuple is not admissible, the admissible sample space omits it and is a proper subset. \(\square\)

When a later choice depends on previous outcomes, it is often clearest to organize the experiment as a branching tree. A branch records one choice at each stage, and a complete path from the start to the end records one admissible sequence. The tree makes restrictions visible without changing the fact that the final outcome is an ordered sequence.

Sample-Space Structure Does Not Imply Independence

A product notation is about the form of outcomes, not about a probability rule. Even when the sample space is a full Cartesian product, probabilities can favor some sequences, and the stages can be statistically dependent. In a finite model, independence of two recorded stages means that each pair's probability factors into the product of the probabilities of its two component outcomes. Nothing in the definition of \(S_1\times S_2\) requires that factorization.

For example, on \(\{H,T\}\times\{H,T\}\), assigning probability \(1/4\) to each pair gives one possible probability model. Assigning probabilities \(0.4,0.1,0.1,0.4\) to \(HH,HT,TH,TT\), respectively, gives another. Both use the same product sample space. In the second model, each stage has probability \(0.5\) for \(H\): for the first stage, \(0.4+0.1=0.5\), and for the second, \(0.4+0.1=0.5\). But the probability of \(HH\) is \(0.4\), not \(0.5\cdot0.5=0.25\). Thus those two stages are not independent under this assignment.

Common Mistake: Reading Independence into a Product

A common mistake is to argue, “The sample space is \(S_1\times S_2\), so the two stages are independent.” This confuses the mathematical product of sets with a product rule for probabilities. The set product says that outcomes are ordered pairs and, if it is the full sample space, that all combinations are possible. Independence is an additional property of the probability assignment.

The reverse confusion also occurs: dependence does not necessarily mean that some pair is impossible. In the repeated-measurement example, all four pairs have positive probability, yet the stages are dependent. Likewise, a restricted sample space such as drawing without replacement describes a restriction on possible sequences; it does not by itself provide a complete probability assignment. The experiment's probability model supplies that additional information.

Why the Distinction Matters

This distinction is essential in sampling and measurement. Sampling with replacement may allow every repeated label sequence, while sampling without replacement excludes repeats. Repeated measurements may each have the same possible values but exhibit a joint pattern that differs from what independence would predict. In each case, a correct analysis begins by recording the stages and their admissible sequences, then uses the probability model appropriate to the mechanism.

A useful check is to ask two separate questions: What ordered sequences can occur? and What probability does the model assign to each sequence? A product sample space answers the first question when all combinations are possible. It never answers the second question on its own.