One-Time Distributions and Dependence
A process is a collection of random variables indexed by time. Looking at one time at a time tells us how the process is distributed at each individual index, but not how its values at different times fit together. For instance, two processes can each take the values \(0\) and \(1\) with equal probability at every time, while one process changes independently from one time to the next and another repeats its first value. Their one-time distributions agree, but their joint distributions do not.
For clarity, the examples below use a process \((X_t)_{t\in T}\) on a probability space \((\Omega,\mathcal{F},\mathbb{P})\), with discrete time index set \(T=\{0,1\}\) and finite state space \(S=\{0,1\}\). The same definitions apply at any pair of time indices and to any finite or countable state space. For states \(a,b\in S\), the two-time joint probability is \(\mathbb{P}(X_0=a,X_1=b)\). The one-time marginal probabilities are obtained by summing this joint probability over the other coordinate.
These sums are not merely a convenient way to arrange numbers. The events \(\{X_0=a,X_1=b\}\), as \(b\) varies, split the event \(\{X_0=a\}\) into disjoint possibilities. The next result formalizes that observation.
Proof. Fix \(a\in S\). For distinct \(b\in S\), the events \(\{X_0=a,X_1=b\}\) are disjoint, and their union is \(\{X_0=a\}\), since \(X_1\) takes some value in \(S\). Countable additivity therefore gives \(\mathbb{P}(X_0=a)=\sum_{b\in S}\mathbb{P}(X_0=a,X_1=b)\). Fixing \(b\) and partitioning \(\{X_1=b\}\) by the value of \(X_0\) gives the other identity. Every entry is nonnegative because it is a probability. Summing the first identity over \(a\) gives \(\sum_{a\in S}\sum_{b\in S}q(a,b)=\sum_{a\in S}\mathbb{P}(X_0=a)=1\). Thus the joint table has total \(1\), and its indicated sums are the marginals. \(\square\)
Marginalization goes from the joint distribution to the individual distributions. In general, the reverse direction is not determined: knowing the row and column sums does not tell us how probability is allocated among the cells. Independence is one particular way to fill in a joint distribution from its marginals.
Proof. If \(X_0\) and \(X_1\) are independent, apply the definition with \(A=\{a\}\) and \(B=\{b\}\) to obtain \(q(a,b)=\mathbb{P}(X_0=a)\mathbb{P}(X_1=b)\). Conversely, suppose these singleton equations hold. For any \(A,B\subseteq S\), the events indexed by \((a,b)\in A\times B\) are disjoint, so countable additivity and the assumed equations give \(\mathbb{P}(X_0\in A,X_1\in B)=\sum_{a\in A}\sum_{b\in B}q(a,b)=\sum_{a\in A}\sum_{b\in B}\mathbb{P}(X_0=a)\mathbb{P}(X_1=b)=\mathbb{P}(X_0\in A)\mathbb{P}(X_1\in B)\). The last equality follows by multiplying the two marginal sums. Thus the variables are independent. \(\square\)
Three Joint Laws with the Same Marginals
We now compare three processes with the same state space and time indices. In each example, both \(X_0\) and \(X_1\) have probability \(\tfrac12\) of being \(0\) and probability \(\tfrac12\) of being \(1\). The difference is in the joint distribution.
Worked Example: Independent Values at Two Times
Let \(\Omega=\{(0,0),(0,1),(1,0),(1,1)\}\), let \(\mathcal{F}\) contain all subsets of \(\Omega\), and assign probability \(\tfrac14\) to each outcome. Define \(X_0(u,v)=u\) and \(X_1(u,v)=v\). This gives a process on the probability space \((\Omega,\mathcal{F},\mathbb{P})\), with \(T=\{0,1\}\) and \(S=\{0,1\}\).
Each value of \(X_0\) occurs in two of the four equally likely outcomes. Hence \(\mathbb{P}(X_0=0)=\tfrac14+\tfrac14=\tfrac12\) and \(\mathbb{P}(X_0=1)=\tfrac14+\tfrac14=\tfrac12\). The same count gives \(\mathbb{P}(X_1=0)=\tfrac12\) and \(\mathbb{P}(X_1=1)=\tfrac12\). The joint table is
Every joint entry is the product of the relevant marginal probabilities: for example, \(\mathbb{P}(X_0=1,X_1=0)=\tfrac14=\tfrac12\cdot\tfrac12\). The same product calculation holds in all four cells, so the independence criterion applies. As a conditional check, \(\mathbb{P}(X_1=1\mid X_0=1)=\mathbb{P}(X_0=1,X_1=1)/\mathbb{P}(X_0=1)=(\tfrac14)/(\tfrac12)=\tfrac12\); the denominator is positive, as required.
Check: The row sums and column sums are each \(\tfrac12,\tfrac12\), and the four joint entries add to \(\tfrac14+\tfrac14+\tfrac14+\tfrac14=1\).
Worked Example: Repeating the Same Random Value
Let \(\Omega=\{0,1\}\), let \(\mathcal{F}\) contain all subsets of \(\Omega\), and assign probability \(\tfrac12\) to each outcome. Define \(X_0(\omega)=\omega\) and \(X_1(\omega)=\omega\). Thus the process has the same value at both time indices on every outcome.
At each time, \(\mathbb{P}(X_t=0)=\tfrac12\) and \(\mathbb{P}(X_t=1)=\tfrac12\). The two possible process values are \((0,0)\) and \((1,1)\), each with probability \(\tfrac12\). Therefore its joint table is
The marginals agree with those in the independent example, but the joint law is different. In particular, \(\mathbb{P}(X_0=1,X_1=1)=\tfrac12\), whereas the product of the marginals is \(\tfrac12\cdot\tfrac12=\tfrac14\). The variables are not independent. Also, since \(\mathbb{P}(X_0=1)=\tfrac12>0\), the conditional probability is defined and equals \(\mathbb{P}(X_1=1\mid X_0=1)=(\tfrac12)/(\tfrac12)=1\).
Check: Each row sum and column sum is \(\tfrac12\), and the joint entries add to \(\tfrac12+0+0+\tfrac12=1\). The probability of equal values is \(\tfrac12+\tfrac12=1\), as the construction requires.
Worked Example: Equal Marginals with Partial Dependence
Let \(\Omega=\{(0,0),(0,1),(1,0),(1,1)\}\), with probabilities \(\tfrac13,\tfrac16,\tfrac16,\tfrac13\), respectively. Take \(\mathcal{F}\) to be all subsets of \(\Omega\), and again let \(X_0\) and \(X_1\) be the first and second coordinates. These probabilities are nonnegative and sum to \(\tfrac13+\tfrac16+\tfrac16+\tfrac13=1\), so they define a probability space.
For \(X_0\), the probability of \(0\) is \(\tfrac13+\tfrac16=\tfrac12\), and the probability of \(1\) is \(\tfrac16+\tfrac13=\tfrac12\). For \(X_1\), the probability of \(0\) is \(\tfrac13+\tfrac16=\tfrac12\), and the probability of \(1\) is \(\tfrac16+\tfrac13=\tfrac12\). The joint table is
This pair is dependent because \(\mathbb{P}(X_0=1,X_1=1)=\tfrac13\), not \(\mathbb{P}(X_0=1)\mathbb{P}(X_1=1)=\tfrac14\). Its conditional probability, with positive denominator \(\mathbb{P}(X_0=1)=\tfrac12\), is \(\mathbb{P}(X_1=1\mid X_0=1)=(\tfrac13)/(\tfrac12)=\tfrac23\). Thus observing \(X_0=1\) changes the probability that \(X_1=1\), but it does not force \(X_1\) to equal \(1\).
Check: The four entries sum to \(\tfrac13+\tfrac16+\tfrac16+\tfrac13=1\), and all four marginal sums are \(\tfrac12\). The probability of matching values is \(\mathbb{P}(X_0=0,X_1=0)+\mathbb{P}(X_0=1,X_1=1)=\tfrac13+\tfrac13=\tfrac23\).
What the Joint Distribution Adds
The examples show exactly what is lost when we keep only the one-time marginals. All three processes have the same distribution at time \(0\) and the same distribution at time \(1\). Yet their probabilities of matching values are different: the independent pair matches with probability \(\tfrac12\), the repeated pair matches with probability \(1\), and the third pair matches with probability \(\tfrac23\). Their conditional probabilities \(\mathbb{P}(X_1=1\mid X_0=1)\) are also different: \(\tfrac12\), \(1\), and \(\tfrac23\), respectively.
A joint distribution can answer questions that the marginals alone cannot, such as the probability of a particular pair of values or the probability that the process changes value between the two times. For the independent example, \(\mathbb{P}(X_0\ne X_1)=\tfrac14+\tfrac14=\tfrac12\). For the repeated example, this probability is \(0+0=0\). The marginal probabilities alone do not distinguish these answers.
Common mistake. It is incorrect to multiply marginal probabilities automatically. For example, in the repeated-variable example, the calculation \(\mathbb{P}(X_0=1,X_1=1)=\mathbb{P}(X_0=1)\mathbb{P}(X_1=1)=\tfrac14\) wrongly assumes independence. The actual joint probability is \(\tfrac12\), obtained from the two outcomes in the probability space: only \(\omega=1\) gives the pair \((1,1)\), and that outcome has probability \(\tfrac12\). Multiplying marginals is justified precisely when independence has been established.
When reading or constructing a joint table, check both its total and its margins: the entries must be nonnegative, their total must be \(1\), and summing across rows or columns must recover the stated one-time distributions. These checks verify consistency, but they do not by themselves imply independence. Independence requires the additional product condition for every pair of states.
Check Your Understanding
Use joint tables and marginalization to answer the following questions. When using a conditional probability, first check that the conditioning event has positive probability.
- A joint table on \(\{0,1\}^2\) has entries \(\tfrac18,\tfrac38,\tfrac38,\tfrac18\), in the order \((0,0),(0,1),(1,0),(1,1)\). Find both marginals and decide whether the variables are independent.
- In the repeated-variable example, find \(\mathbb{P}(X_1=0\mid X_0=0)\). State why this conditional probability is defined.
- Can two processes have the same marginal distribution at each of two times but different probabilities of changing value between those times? Explain using the examples.
- For a joint table on a finite state space, what two sums give the time-0 and time-1 marginal probabilities?
- Suppose \(X_0\) and \(X_1\) are independent and each takes \(0\) and \(1\) with probabilities \(\tfrac12,\tfrac12\). Compute \(\mathbb{P}(X_0=1,X_1=0)\) and identify the condition that justifies the multiplication.