One Random Input, Many Process Values
A process can be formed by reusing the same random variable at every time. Its values may look like a sequence, but they are all functions of one underlying source of randomness. This construction makes joint distributions especially direct to compute, and it also creates dependence that can be missed if each time is considered only through its marginal distribution.
Work on a probability space \((\Omega,\mathcal{F},\mathbb{P})\), and let \(Y\) be a random variable taking values in a measurable space \(E\). For a discrete-time process \((X_n)_{n\ge0}\) with state space \(S\), a shared-input construction has the form \(X_n=f_n(Y)\), where each \(f_n:E\to S\) is measurable. A finite-dimensional distribution is the joint law of \(X_{n_1},\ldots,X_{n_k}\) at a finite collection of times. In this construction, that joint law is determined by the one variable \(Y\), not by independent draws at the different times.
Proof. For each \(r\), the event \(\{X_{n_r}\in A_r\}\) equals \(\{f_{n_r}(Y)\in A_r\}\), which equals \(\{Y\in f_{n_r}^{-1}(A_r)\}\) by the definition of a preimage. Intersecting these event equalities over \(r=1,\ldots,k\) gives \(\{X_{n_1}\in A_1,\ldots,X_{n_k}\in A_k\}=\{Y\in\bigcap_{r=1}^k f_{n_r}^{-1}(A_r)\}\). Taking probabilities proves the formula. \(\square\)
The proposition gives a practical method: translate each requested condition on a process value into a condition on \(Y\), intersect those conditions, and evaluate the probability. It does not say that the coordinates are independent. In fact, when multiple coordinates use the same \(Y\), their events may overlap in ways that make them dependent.
Constant Processes
A basic shared-input construction is the constant process \(X_n=Y\) for every \(n\ge0\). “Constant” here means constant along each sample path: once \(Y(\omega)\) is chosen, \(X_n(\omega)\) has that same value at every time. The process need not be deterministic, because the common level \(Y\) can be random.
Proof. Strict stationarity in discrete time means that for every integer \(k\ge1\), every choice of indices \(n_1,\ldots,n_k\ge0\), and every shift \(h\ge0\), the vectors \((X_{n_1},\ldots,X_{n_k})\) and \((X_{n_1+h},\ldots,X_{n_k+h})\) have the same distribution. Under \(X_n=Y\), both vectors equal \((Y,\ldots,Y)\) on every outcome \(\omega\in\Omega\). They therefore have the same distribution. This also identifies the finite-dimensional law as the law of the stated diagonal vector. \(\square\)
Worked Example: A Bernoulli Level Repeated at Every Time
Let \(\Omega=\{0,1\}\), \(\mathcal{F}\) be all subsets of \(\Omega\), \(\mathbb{P}(\{1\})=\tfrac25\), and \(\mathbb{P}(\{0\})=\tfrac35\). Define \(Y(\omega)=\omega\) and the process \(X_n=Y\) for \(n\ge0\), with state space \(S=\{0,1\}\). Find the joint law at any two distinct times, and determine whether the two coordinates are independent.
For distinct \(r,s\ge0\), \(X_r=X_s=Y\). Thus the pair \((X_r,X_s)\) is \((0,0)\) when \(Y=0\), with probability \(\tfrac35\), and is \((1,1)\) when \(Y=1\), with probability \(\tfrac25\). The other two pairs cannot occur. In table form, the joint probabilities are \(\mathbb{P}(X_r=0,X_s=0)=\tfrac35\), \(\mathbb{P}(X_r=0,X_s=1)=0\), \(\mathbb{P}(X_r=1,X_s=0)=0\), and \(\mathbb{P}(X_r=1,X_s=1)=\tfrac25\).
Each marginal has \(\mathbb{P}(X_r=1)=\mathbb{P}(X_s=1)=\tfrac25\). If the coordinates were independent, their joint probability of both being 1 would be \(\tfrac25\cdot\tfrac25=\tfrac4{25}\). The actual probability is \(\tfrac25=\tfrac{10}{25}\), which is different, so they are dependent. As a numerical check, the four joint probabilities sum to \(\tfrac35+0+0+\tfrac25=1\), and each marginal probability of 1 agrees with the joint table.
In general, if \(Y\) is Bernoulli with \(\mathbb{P}(Y=1)=p\) and \(X_n=Y\), then \(\mathbb{E}[X_n]=p\) and \(\operatorname{Var}(X_n)=p(1-p)\). For distinct times \(r,s\), \(X_rX_s=Y^2=Y\), since \(Y\) takes only the values 0 and 1. Consequently, \[ \operatorname{Cov}(X_r,X_s)=\mathbb{E}[X_rX_s]-\mathbb{E}[X_r]\mathbb{E}[X_s]=p-p^2=p(1-p). \] For \(0<p<1\), this covariance is positive and the coordinates are dependent. At \(p=0\) or \(p=1\), both coordinates are deterministic, and they are independent; this is why the nondegenerate condition \(0<p<1\) matters for that dependence conclusion.
Two Different Functions of the Same Input
Sharing an input does not force coordinates to be equal. Different functions of the input can produce distinct values and either positive or negative dependence. To find the dependence correctly, calculate the joint outcomes, rather than multiplying marginal probabilities as if the coordinates had been generated independently.
Worked Example: A Two-Time Vector from Six Equally Likely Inputs
Let \(\Omega=\{1,2,3,4,5,6\}\), let every singleton have probability \(\tfrac16\), and define two \(\{0,1\}\)-valued random variables by \(X_0=\mathbf{1}_{\{2,4,6\}}(Z)\) and \(X_1=\mathbf{1}_{\{4,5,6\}}(Z)\), where \(Z(\omega)=\omega\). This specifies a two-coordinate vector, not a claim about a process at all times. Find its joint law and covariance.
For \(Z=1,2,3,4,5,6\), the resulting pairs \((X_0,X_1)\) are, in order, \((0,0),(1,0),(0,0),(1,1),(0,1),(1,1)\). Therefore \((0,0)\) occurs twice, \((1,0)\) once, \((0,1)\) once, and \((1,1)\) twice. Their probabilities are respectively \(\tfrac26=\tfrac13\), \(\tfrac16\), \(\tfrac16\), and \(\tfrac26=\tfrac13\). Each marginal equals 1 with probability \(\tfrac36=\tfrac12\).
Now \(\mathbb{E}[X_0X_1]=\mathbb{P}(X_0=1,X_1=1)=\tfrac13\), while \(\mathbb{E}[X_0]\mathbb{E}[X_1]=\tfrac12\cdot\tfrac12=\tfrac14\). Hence \(\operatorname{Cov}(X_0,X_1)=\tfrac13-\tfrac14=\tfrac1{12}\). The coordinates are dependent because independence would require \(\mathbb{P}(X_0=1,X_1=1)=\tfrac14\), not \(\tfrac13\). As checks, the four joint probabilities sum to \(\tfrac13+\tfrac16+\tfrac16+\tfrac13=1\); the first marginal probability of 1 is \(\tfrac16+\tfrac13=\tfrac12\), and the second is also \(\tfrac16+\tfrac13=\tfrac12\).
This calculation concerns exactly the two stated coordinates. A two-coordinate distribution alone does not define a process at every time, so it cannot establish whether a process is strictly stationary. For stationarity, the process must be specified at the relevant time indices, and its shifted finite-dimensional distributions must be compared.
Random Phase and Strict Stationarity
A periodic sequence with a fixed starting phase usually changes its values when shifted in time. If the phase is instead chosen uniformly at random, a time shift merely permutes the possible phases. This gives a useful construction of a strictly stationary process with dependent coordinates.
Proof. Fix \(k\ge1\), indices \(n_1,\ldots,n_k\ge0\), and a shift \(h\ge0\). The unshifted vector is \((a_{(n_1+U)\bmod m},\ldots,a_{(n_k+U)\bmod m})\). The shifted vector is \((a_{(n_1+h+U)\bmod m},\ldots,a_{(n_k+h+U)\bmod m})\). Let \(V=(U+h)\bmod m\). Addition by \(h\) modulo \(m\) is a bijection of \(\{0,\ldots,m-1\}\), so \(V\) is uniform on that set: each possible value has probability \(\tfrac1m\). The shifted vector therefore has the same distribution as \((a_{(n_1+V)\bmod m},\ldots,a_{(n_k+V)\bmod m})\), which has the same distribution as the unshifted vector because \(V\) and \(U\) are both uniform on the same finite set. This holds for every choice of \(k\), indices, and shift, proving strict stationarity. \(\square\)
Worked Example: A Three-Position Pattern with a Uniform Phase
Let \(\Omega=\{0,1,2\}\) with uniform probability, define \(U(\omega)=\omega\), and set \(S=\{0,1\}\), \(a_0=0\), \(a_1=0\), and \(a_2=1\). Define \(X_n=a_{(n+U)\bmod3}\) for \(n\ge0\). Compute the marginal and the joint law of \((X_0,X_1)\), then check stationarity and dependence.
For phases \(U=0,1,2\), the ordered pair \((X_0,X_1)\) is respectively \((a_0,a_1)=(0,0)\), \((a_1,a_2)=(0,1)\), and \((a_2,a_0)=(1,0)\). Each outcome has probability \(\tfrac13\), and \((1,1)\) has probability 0. In particular, \(\mathbb{P}(X_0=1)=\tfrac13\). The next coordinate also has probability \(\tfrac13\) of being 1: the three possible values of \(X_1\) are \(0,1,0\).
For stationarity, take any finite list of times \(n_1,\ldots,n_k\) and any shift \(h\ge0\). Replacing \(U\) by \((U+h)\bmod3\) permutes the three equally likely phases. Thus the shifted vector has the same distribution as the unshifted vector, in accord with the theorem. The two coordinates are nevertheless dependent: \(\mathbb{P}(X_0=1,X_1=1)=0\), whereas the product of their marginal probabilities is \(\tfrac13\cdot\tfrac13=\tfrac19\). Their covariance is \(0-\tfrac19=-\tfrac19\). As checks, the joint probabilities \(\tfrac13,\tfrac13,\tfrac13,0\) sum to 1, and the joint table gives each marginal probability of 1 as \(\tfrac13\).
For contrast, if the phase were fixed at 0 rather than chosen uniformly, the sequence would be \(0,0,1,0,0,1,\ldots\). Its time-0 value is always 0, while its time-2 value is always 1, so these one-dimensional distributions differ. That fixed-phase process is not strictly stationary. Randomizing the phase is what makes the distribution invariant under shifts.
Common Pitfall: Multiplying Marginals
A frequent error is to calculate a joint probability by multiplying marginals without first establishing independence. For example, let \(Y\) be Bernoulli with \(\mathbb{P}(Y=1)=\tfrac13\), and define \(X_0=X_1=Y\). The incorrect calculation \(\mathbb{P}(X_0=1,X_1=1)=\mathbb{P}(X_0=1)\mathbb{P}(X_1=1)=\tfrac13\cdot\tfrac13=\tfrac19\) treats the two coordinates as independent. The correct calculation uses their shared definition: \(\{X_0=1,X_1=1\}=\{Y=1\}\), so the joint probability is \(\tfrac13\). The correction is not a small adjustment to the product rule; the product rule applies only when independence has been justified.
It is also important to keep three statements separate. Equal marginal distributions at different times do not by themselves establish strict stationarity, which requires equality of all finite-dimensional distributions under time shifts. Strict stationarity does not imply independence, as the random-phase example shows. And dependence does not rule out stationarity: a process may have a shift-invariant joint law while its coordinates remain dependent.
Check Your Understanding
Use the joint-law formula and the definitions to answer each question.
- If \(X_n=Y\) for every \(n\), where \(\mathbb{P}(Y=1)=p\) for \(0<p<1\), find \(\mathbb{P}(X_2=1,X_7=0)\). Are \(X_2\) and \(X_7\) independent?
- For the six-input example, recompute the probability of \((X_0,X_1)=(1,1)\) by listing the values of \(Z\) that produce that pair. Why is multiplying the marginals invalid?
- For a uniform random phase on a pattern of length \(m\), explain why adding a fixed shift modulo \(m\) preserves the distribution of the phase.
- For the pattern \((0,0,1)\) with a uniform random phase, find \(\mathbb{P}(X_0=0,X_1=0)\) and compare it with \(\mathbb{P}(X_0=0)\mathbb{P}(X_1=0)\).
- Why does a two-time joint distribution, by itself, not establish strict stationarity of a process?