Independence of Random Blocks

Why Grouping Independent Variables Works

An independent family can be organized into larger pieces. For example, from four independent coordinates one might form the vector of the first two and the vector of the last two. The natural question is whether these vectors are independent, not merely whether each coordinate is independent of each coordinate in the other vector.

The key is to consider everything that can be observed from each block. The events determined by one block form a sigma-algebra. If the blocks use disjoint coordinates from an independent family, these sigma-algebras are independent. Consequently, any measurable summaries of the separate blocks are independent as well.

Independence and Blocks

A finite family of real-valued random variables $X_1,\ldots,X_n$ is independent if, for every choice of Borel sets $B_1,\ldots,B_n\subseteq\mathbb R$,

\mathbb P(X_1\in B_1,\ldots,X_n\in B_n) = \prod_{i=1}^{n}\mathbb P(X_i\in B_i).

This formulation also covers events involving fewer than $n$ variables: choose $B_i=\mathbb R$ for the unused coordinates.

Definition: Sigma-algebras $\mathcal F_1,\ldots,\mathcal F_k$ are independent if, for every choice of events $A_j\in\mathcal F_j$, the probability of their intersection equals the product of their probabilities: $$ \mathbb P\left(\bigcap_{j=1}^{k} A_j\right) = \prod_{j=1}^{k}\mathbb P(A_j). $$

For a set of indices $I\subseteq\{1,\ldots,n\}$, write $\sigma(X_i:i\in I)$ for the sigma-algebra generated by those variables. It consists of the events that can be determined from the coordinates indexed by $I$. If $I=\varnothing$, this is the trivial sigma-algebra $\{\varnothing,\Omega\}$.

A block is a group of coordinates selected by an index set. Several blocks are disjoint when no coordinate belongs to more than one of their index sets. They need not be consecutive: for instance, $\{1,3\}$ and $\{2,4\}$ are disjoint blocks.

The Block Independence Theorem

Theorem: Let $X_1,\ldots,X_n$ be an independent finite family of real-valued random variables. Let $I_1,\ldots,I_k$ be pairwise disjoint subsets of $\{1,\ldots,n\}$. Then the sigma-algebras $$ \sigma(X_i:i\in I_1),\ldots,\sigma(X_i:i\in I_k) $$ are independent.

The idea of the proof is to begin with events that restrict each coordinate separately. Independence of the original variables gives the required factorization for those events. A pi-lambda argument then extends the factorization to all events determined by each block.

Proof: For each block $I_j$, let $\mathcal C_j$ be the collection of events of the form $$ \bigcap_{i\in I_j}\{X_i\in B_i\}, $$ where each $B_i$ is Borel in $\mathbb R$. For an empty block, take $\mathcal C_j=\{\Omega\}$. Each $\mathcal C_j$ contains $\Omega$, is closed under finite intersections, and generates $\sigma(X_i:i\in I_j)$. For $C_j\in\mathcal C_j$, independence of $X_1,\ldots,X_n$, together with the disjointness of the index sets, gives $$ \mathbb P\left(\bigcap_{j=1}^{k} C_j\right) = \prod_{j=1}^{k}\mathbb P(C_j). $$ Coordinates outside the blocks can be assigned the set $\mathbb R$ in the definition of independence. We extend this identity one block at a time. Suppose it holds when the events in the first $r$ blocks are arbitrary elements of their generated sigma-algebras and the events in the remaining blocks belong to their respective $\mathcal C_j$. Fix such events in all blocks except block $r+1$. Denote their intersection by $Q$, and let $$ c=\left(\prod_{j=1}^{r}\mathbb P(A_j)\right) \left(\prod_{j=r+2}^{k}\mathbb P(C_j)\right), $$ where $A_j\in\sigma(X_i:i\in I_j)$ for $j\le r$ and $C_j\in\mathcal C_j$ for $j\ge r+2$. Consider $$ \mathcal D=\{B\in\sigma(X_i:i\in I_{r+1}):\mathbb P(Q\cap B)=c\,\mathbb P(B)\}. $$ The induction assumption, applied with the event $\Omega$ in block $r+1$, gives $\mathbb P(Q)=c$. Thus $\Omega\in\mathcal D$. If $B\in\mathcal D$, then $$ \mathbb P(Q\cap B^c)=\mathbb P(Q)-\mathbb P(Q\cap B) =c-c\,\mathbb P(B) =c\,\mathbb P(B^c), $$ so $B^c\in\mathcal D$. For pairwise disjoint $B_m\in\mathcal D$, countable additivity gives $$ \mathbb P\left(Q\cap\bigcup_{m=1}^{\infty}B_m\right) =\sum_{m=1}^{\infty}\mathbb P(Q\cap B_m) =c\sum_{m=1}^{\infty}\mathbb P(B_m) =c\,\mathbb P\left(\bigcup_{m=1}^{\infty}B_m\right). $$ Therefore $\mathcal D$ is a lambda-system. By the induction assumption, it contains $\mathcal C_{r+1}$. The pi-lambda theorem now implies that it contains the sigma-algebra generated by $\mathcal C_{r+1}$, namely $\sigma(X_i:i\in I_{r+1})$. This extends the factorization to arbitrary events in the first $r+1$ blocks, while retaining cylinder events in the remaining blocks. Starting with $r=0$ and repeating the argument through $r=k-1$ proves the desired factorization for arbitrary events in all $k$ sigma-algebras. Hence the sigma-algebras are independent. $\square$

From Sigma-Algebras to Random Vectors

For a nonempty block $I_j=\{i_{j,1},\ldots,i_{j,m_j}\}$, list its indices in increasing order and form the random vector $$ Z_j=(X_{i_{j,1}},\ldots,X_{i_{j,m_j}})\in\mathbb R^{m_j}. $$ The sigma-algebra generated by $Z_j$ is exactly $\sigma(X_i:i\in I_j)$: the vector is measurable with respect to the coordinate sigma-algebra, and each coordinate can be recovered as a measurable coordinate projection of the vector.

Corollary: Under the hypotheses of the block independence theorem, the random vectors $Z_1,\ldots,Z_k$ from the nonempty blocks are independent. In particular, for Borel sets $A_j\subseteq\mathbb R^{m_j}$, $$ \mathbb P(Z_1\in A_1,\ldots,Z_k\in A_k) = \prod_{j=1}^{k}\mathbb P(Z_j\in A_j). $$

Proof: For each $j$, the event $\{Z_j\in A_j\}$ belongs to $\sigma(Z_j)=\sigma(X_i:i\in I_j)$. The block independence theorem therefore factors the probability of the intersection of these events into the product of their probabilities. This is the defining event-factorization criterion for independence of the random vectors. $\square$

The conclusion also applies to measurable functions of the vectors. If $f_j:\mathbb R^{m_j}\to S_j$ is measurable for each $j$, then $f_j(Z_j)$ is measurable with respect to the sigma-algebra generated by block $j$. Thus the resulting random elements are independent. The functions may combine coordinates within their own blocks in any measurable way; the essential condition is that the blocks themselves do not share coordinates.

Worked Examples

Worked Example: Two Blocks of Fair Bernoulli Variables

Let $X_1,X_2,X_3,X_4$ be independent Bernoulli random variables with $\mathbb P(X_i=1)=\mathbb P(X_i=0)=1/2$. Form the blocks $Z_1=(X_1,X_2)$ and $Z_2=(X_3,X_4)$. Consider the events that the first block contains exactly one success and the second block contains no successes.

The first event has probability $2/4=1/2$, since its possible outcomes are $(1,0)$ and $(0,1)$. The second event has probability $1/4$, since its only outcome is $(0,0)$. The blocks are disjoint, so the theorem gives $$ \mathbb P(X_1+X_2=1,\ X_3+X_4=0) = \frac12\cdot\frac14 = \frac18. $$ Directly, the joint event requires two specified outcomes for the first pair and one specified outcome for the second pair. It therefore consists of two equally likely outcomes among the $2^4=16$ possible outcomes, also giving $2/16=1/8$.

Worked Example: Noncontiguous Coordinates and a Block Transformation

Let $X_1,X_2,X_3$ be independent standard normal random variables. Use the disjoint blocks $I_1=\{1,3\}$ and $I_2=\{2\}$, and define $$ U=(X_1+X_3,\ X_1-X_3),\qquad V=X_2. $$ The vector $(X_1,X_3)$ is independent of $X_2$ by the block theorem. Since $U$ is a measurable function of $(X_1,X_3)$, $U$ and $V$ are independent.

For a concrete event calculation, $X_1+X_3$ is a centered normal random variable with variance $2$, so symmetry and continuity give $\mathbb P(X_1+X_3>0)=1/2$. Also $\mathbb P(X_2>0)=1/2$. Therefore, $$ \mathbb P(U_1>0,\ V>0) = \mathbb P(U_1>0)\mathbb P(V>0) = \frac12\cdot\frac12 = \frac14. $$ The blocks need not appear consecutively in the original list; only disjointness matters.

Worked Example: Why Overlap Matters

Let $X_1,X_2$ be independent fair Bernoulli variables, but now form $U=X_1$ and $V=(X_1,X_2)$. The sets of coordinates are $\{1\}$ and $\{1,2\}$, which overlap. The events $\{U=1\}$ and $\{V_1=1\}$ are the same event, so $$ \mathbb P(U=1,V_1=1)=\frac12, \qquad \mathbb P(U=1)\mathbb P(V_1=1)=\frac12\cdot\frac12=\frac14. $$ These quantities differ, so the two random elements are not independent. This does not contradict the theorem: its disjointness hypothesis fails, and the shared coordinate is visible in both blocks.

What the Result Does—and Does Not—Say

The theorem is a statement about blocks, not a claim that every pair of quantities built from an independent family is independent. A function of one block may combine its coordinates and create dependence within that block. For instance, two different functions of $X_1$ can share information about $X_1$. The theorem concerns independence between separate blocks whose coordinate sets do not overlap.

A useful way to apply the result is to check the index sets first. If the blocks are disjoint, the generated sigma-algebras are independent, and measurable summaries taken separately from those blocks remain independent. If the blocks overlap, the theorem gives no conclusion; one must test independence directly or use other information about the distribution.

This block viewpoint is useful whenever a joint random vector is reorganized into groups of coordinates. It lets us establish independence of whole vectors without checking every possible pair of events individually: independence of the original family supplies the factorization on coordinate events, and the pi-lambda theorem extends it to all events determined by each block.

Check Your Understanding

Use the definitions and theorem to answer the following questions.

If $X_1,\ldots,X_6$ are independent, are the blocks indexed by $\{1,4\}$, $\{2,5\}$, and $\{3,6\}$ independent? Why?
What is the role of the cylinder-event collections $\mathcal C_j$ in the proof?
Why does the factorization for cylinder events alone not yet establish independence of the generated sigma-algebras?
If $Z_1$ and $Z_2$ are independent block vectors, what condition on measurable functions $f_1$ and $f_2$ ensures that $f_1(Z_1)$ and $f_2(Z_2)$ are independent?
Give an example of two blocks that overlap and explain how their shared coordinate can prevent independence.

What You'll Learn