Stationarity Through Time-Shifted Distributions

Why Time Shifts Matter

A time series is a sequence of random variables, \(X_t\), indexed by time \(t\). A process is stationary when its probabilistic behavior does not depend on the particular time origin: if we shift every time index by the same amount, the joint distribution remains unchanged. This is a statement about distributions, not a claim that the observed values must be constant or that every realized sequence looks alike.

The central test is therefore not simply whether the values appear to fluctuate around a level. It is whether the probability law for a collection of observations at specified times is the same as the law for that collection after a common shift. This distinction matters because checking the distribution at each time separately can miss changes in how observations are related across time.

Definition: A process \(\{X_t\}\) is strictly stationary if, for every positive integer \(k\), every set of time indices \(t_1,\ldots,t_k\), and every shift \(h\), the random vectors \((X_{t_1},\ldots,X_{t_k})\) and \((X_{t_1+h},\ldots,X_{t_k+h})\) have the same joint distribution. Equivalently, their joint cumulative distribution functions agree at every point.

The times in a vector need not be consecutive. For example, the definition applies to \(X_2\) and \(X_7\) just as it applies to \(X_2\) and \(X_3\). The shift \(h\) must be applied to every time in the vector, while the spacing between those times stays the same. A stationary process can have substantial variation and dependence; what must remain stable is its full probabilistic behavior under these shifts.

How to Check the Definition

A direct check compares joint distributions at a collection of times with those at shifted times. If even one choice of times and shift produces different joint distributions, the process is not strictly stationary. Conversely, to establish strict stationarity, the equality must hold for every finite collection and every allowable shift.

Choose a finite collection of times.
For example, compare observations at \(t_1=2\) and \(t_2=5\), not just one observation in isolation.

Choose a common shift.
Replace each selected time \(t_j\) by \(t_j+h\). Keep the time gaps unchanged.

Compare the joint distributions.
Compare probabilities of corresponding events or compare the joint distribution functions. A difference disproves strict stationarity.

There is a useful short justification for why independent observations with one fixed distribution form a strictly stationary process. Suppose the \(X_t\) are independent and each has the same cumulative distribution function \(F\). For any times \(t_1,\ldots,t_k\), independence gives the joint distribution function as the product of the marginal distribution functions. Shifting the times does not change any factor, so it does not change the product:

P(X_{t_1}\le x_1,\ldots,X_{t_k}\le x_k) =\prod_{j=1}^{k}F(x_j) =P(X_{t_1+h}\le x_1,\ldots,X_{t_k+h}\le x_k).

This argument proves equality for every finite collection and every shift, which is exactly the definition. Independence is sufficient in this example, but it is not part of the definition of stationarity. Stationary processes may have dependence across time, provided their joint distributions are invariant under shifts.

Worked Example: Checking a Stable Independent Process

Suppose \(X_t\) are independent Bernoulli random variables with \(P(X_t=1)=0.3\) and \(P(X_t=0)=0.7\) at every integer time \(t\). Compare the probability of the pattern \((1,0,1)\) at times \((2,5,8)\) with its probability at times shifted by \(h=4\). Decide whether this check supports stationarity, and state what establishes the full result.

Method. Use independence to calculate each pattern probability, then use the fixed common distribution to check the general finite-dimensional condition.

At times \((2,5,8)\), the requested event is \(X_2=1,\ X_5=0,\ X_8=1\). Independence gives

$$ P(X_2=1,X_5=0,X_8=1) =(0.3)(0.7)(0.3)=0.063. $$

The shifted times are \((6,9,12)\). The event is \(X_6=1,\ X_9=0,\ X_{12}=1\), so its probability is

P(X_6=1,X_9=0,X_{12}=1) =(0.3)(0.7)(0.3)=0.063.

The two probabilities agree. More generally, for any \(k\) distinct times, the probability of a specified pattern with \(r\) ones and \(k-r\) zeros is \(0.3^r0.7^{k-r}\). That expression does not depend on the selected times, so every finite joint distribution is invariant under a shift. The process is strictly stationary. Common mistake avoided: a single matching probability is not enough to prove stationarity; the general argument works for every finite pattern.

Stable Marginals Do Not Guarantee Stationarity

The definition includes joint distributions, not only the distribution of \(X_t\) at one time. Thus, having the same marginal distribution at every time is necessary for strict stationarity, but it is not sufficient. A process can have an unchanged one-time distribution while the dependence between observations changes with their location in time.

Worked Example: Equal Marginals but Changing Joint Behavior

Let \(Z_k\), for integer \(k\), be independent Bernoulli random variables with \(P(Z_k=1)=0.5\). Define a process by \(X_{2k}=Z_k\) and \(X_{2k+1}=Z_k\). Check the marginal distribution at every time, then compare the joint distributions at times \((0,1)\) and \((1,2)\).

Step 1: Check each marginal. Every \(X_t\) is equal to one of the \(Z_k\), so

P(X_t=1)=0.5,\qquad P(X_t=0)=0.5

for every \(t\). The distribution of one observation is therefore unchanged over time.

Step 2: Examine the first pair. Since \(X_0=Z_0\) and \(X_1=Z_0\), the two values are always equal. In particular,

$$ P(X_0=1,X_1=1)=P(Z_0=1)=0.5. $$

Step 3: Shift the pair by one time unit. Now \(X_1=Z_0\) and \(X_2=Z_1\). These are independent, so

$$ P(X_1=1,X_2=1)=P(Z_0=1)P(Z_1=1)=(0.5)(0.5)=0.25. $$

The joint probabilities differ: \(0.5\ne0.25\). Therefore the process is not strictly stationary, despite having the same marginal distribution at every time. Common mistake avoided: checking only \(P(X_t=1)\) would miss the time-dependent joint behavior.

Changing Distributions: A Trend Example

A systematic change in level can also make the distribution depend on time. Consider a process formed by adding a deterministic trend to random variation. The random part can have the same distribution at every time, yet the trend shifts the distribution of the total observation.

Worked Example: Detecting a Time-Dependent Level

Let \(E_t\) be independent Bernoulli random variables with \(P(E_t=1)=0.5\), and define \(X_t=2t+E_t\) for integer \(t\). Compare the distributions at \(t=0\) and \(t=1\), and determine whether the process is strictly stationary.

Step 1: Find the distribution at time zero. Since \(X_0=E_0\),

P(X_0=0)=0.5,\qquad P(X_0=1)=0.5.

Step 2: Find the distribution one time unit later. Since \(X_1=2+E_1\),

P(X_1=2)=0.5,\qquad P(X_1=3)=0.5.

These marginal distributions are different. For instance, \(P(X_0\le1)=1\), whereas \(P(X_1\le1)=0\). The shift \(h=1\) already fails the stationarity condition for a one-element vector, so the process is not strictly stationary. Check: the random component has the same Bernoulli distribution at both times, but adding \(2t\) moves the possible values, producing a time-dependent distribution.

Common Mistakes and Why the Distinction Matters

Stationarity is a property of the probability model for a process, not a visual label assigned to one recorded sequence. A finite observed series cannot by itself display all possible joint distributions. Plots and sample summaries can suggest that behavior changes, but the definition asks a precise probability question about every finite collection of time points.

Checking only the average or one-time distribution. A process can have identical marginal distributions but different joint distributions after a shift, as the pair-block example shows.
Shifting only some indices. The comparison uses the same shift \(h\) for every selected time. Changing the spacing between observations is a different comparison.
Assuming stationarity means no movement. A stationary series may fluctuate substantially. The requirement is invariance of its distributions, not constancy of its values.
Assuming uncorrelated observations are automatically stationary. Stationarity is a statement about full joint distributions. A limited dependence summary alone does not establish it.
Using one matching event as proof. A mismatch disproves stationarity, but a match for one event does not establish equality of all finite-dimensional distributions.

The distinction is useful because statistical models for time series often rely on a stable probabilistic structure to interpret patterns across time. If the distribution changes with the time origin, then conclusions drawn from one period may not describe another. This tutorial’s distributional definition is the strict form of stationarity; related stationarity conditions are treated separately in the time-series sequence.

Key takeaway: To test strict stationarity, compare the joint distribution of any finite collection of observations with the joint distribution after shifting every time index by the same amount. Stable marginals alone are not enough.

Check Your Understanding

For each problem, focus on what changes under a common shift of the selected time indices.

Suppose \(X_t\) are independent Poisson random variables with parameter \(2\) at every integer time. Is the process strictly stationary? What is \(P(X_1=0,X_4=1)\), and what is the corresponding probability at times \((6,9)\)? (Answer: Yes; both probabilities are \(2e^{-4}\), approximately \(0.0366\).)
Let \(U_t\) be independent and uniformly distributed on \((-1,1)\), and define \(X_t=5+U_t\). Is the process strictly stationary? State its marginal distribution. (Answer: Yes; each \(X_t\) is uniform on \((4,6)\), and independence with a common distribution makes all finite joint distributions shift-invariant.)
In the pair-block process \(X_{2k}=Z_k,\ X_{2k+1}=Z_k\), with independent \(Z_k\sim\mathrm{Bernoulli}(0.5)\), find \(P(X_3=1,X_4=1)\) and compare it with \(P(X_2=1,X_3=1)\). Does the comparison show stationarity? (Answer: The probabilities are \(0.25\) and \(0.5\), respectively; no, the process is not stationary.)
Let \(U\) be uniformly distributed on \((0,1)\), and set \(X_t=U+t\) for integer \(t\). Compare the distributions of \(X_0\) and \(X_1\). Is the process strictly stationary? (Answer: \(X_0\) is uniform on \((0,1)\), while \(X_1\) is uniform on \((1,2)\); no.)
Suppose every \(X_t\) has the same Bernoulli \((0.5)\) distribution, but \(P(X_0=X_1)=1\) and \(P(X_1=X_2)=0.5\). Can the process be strictly stationary? Explain using the definition. (Answer: No; the joint distributions at \((0,1)\) and its shift \((1,2)\) differ.)

What You'll Learn