What Stationarity Means for an FCS Chain
An FCS algorithm produces a sequence of completed-data states: at each sweep, it updates missing entries using the specified conditional models while keeping observed values fixed. The sequence can keep changing even when it has reached a stable distribution. Stationarity describes stability of the distribution across sweeps, not stability of the particular completed data set.
To define this precisely, treat the collection of missing entries after sweep \(t\) as a random state \(Z_t\). The observed entries are fixed, so they need not be included as changing coordinates in \(Z_t\). If the procedure also carries parameters from one sweep to the next, those parameters belong in the state; if parameter draws are made within a sweep and then integrated out, their randomness is part of the transition rule. A sweep-level transition kernel gives the probability law for the next state conditional on the current one.
The kernel refers to a complete sweep, not just one variable update. With a fixed update order, the sweep kernel is the composition of the update kernels in that order. If the order is randomized using the same probability rule at every sweep, the sweep kernel averages over that randomization. In either case, the transition rule must be the same from sweep to sweep for the ordinary time-homogeneous definition above to apply. If the schedule changes over time, the successive sweep kernels may differ; one cannot assume a single invariant-kernel equation describes the sequence without further formulation.
In a finite state space, write the probability of moving from state \(i\) to state \(j\) as \(K_{ij}\). If \(\pi_i\) is the probability assigned to state \(i\), then stationarity requires $$ \pi_j=\sum_i \pi_i K_{ij} $$ for every destination state \(j\). This equation says that the probability flowing into each state, averaged over the current distribution, equals the probability already assigned to it.
Worked Examples of Invariance
Worked Example: Checking a Stationary Distribution for a Binary FCS Sweep
Suppose two missing binary entries, \(A\) and \(B\), have four possible completed-data states, ordered as \(00,01,10,11\). Consider a fixed sweep that first draws \(A\mid B\), then draws \(B\mid A\) using the newly drawn \(A\). For this hypothetical example, the conditional draw probabilities are \(P(A=1\mid B=0)=0.2\), \(P(A=1\mid B=1)=0.8\), \(P(B=1\mid A=0)=0.2\), and \(P(B=1\mid A=1)=0.8\). These are the conditionals of the hypothetical joint state probabilities \((0.4,0.1,0.1,0.4)\) in the stated order.
For example, starting at \(00\), the sweep produces \(00\) with probability \(0.8\times0.8=0.64\): \(A\) is drawn as 0 with probability 0.8, then \(B\) is drawn as 0 with probability 0.8. It produces \(01\) with probability \(0.8\times0.2=0.16\), \(10\) with probability \(0.2\times0.2=0.04\), and \(11\) with probability \(0.2\times0.8=0.16\). Applying the same reasoning from each starting state gives the sweep transition matrix
The second and fourth rows apply when the starting value of \(B\) is 1; the first and third apply when it is 0. To check the proposed distribution, multiply it by the matrix:
For instance, the probability of state \(00\) after a sweep is \(0.4(0.64)+0.1(0.16)+0.1(0.64)+0.4(0.16)=0.400\). For state \(01\), it is \(0.4(0.16)+0.1(0.04)+0.1(0.16)+0.4(0.04)=0.100\). The corresponding calculations for \(10\) and \(11\) give 0.100 and 0.400. Thus the distribution is invariant under this sweep kernel. This check establishes stationarity for the specified hypothetical kernel; it is not a general guarantee for arbitrary FCS conditional models.
Worked Example: Stationary Does Not Mean Motionless
Consider a hypothetical reduced state description with two categories, \(L\) and \(R\), and sweep transition probabilities \(P(L\to R)=0.2\) and \(P(R\to L)=0.3\). The transition matrix, with rows and columns ordered \(L,R\), is
Let the probability of \(R\) under a candidate stationary distribution be \(q\), so that the probability of \(L\) is \(1-q\). After one sweep, the probability of \(R\) is \((1-q)(0.2)+q(0.7)\). Equating this to \(q\) gives
The stationary distribution is therefore \((0.6,0.4)\). Under this distribution, some transitions still occur: the probability of moving from \(L\) to \(R\) in a sweep is \(0.6(0.2)=0.12\), and the probability of moving from \(R\) to \(L\) is \(0.4(0.3)=0.12\). The two flows balance, so the category frequencies remain the same in distribution even though individual states can change.
Worked Example: A Starting State Is Not Automatically Stationary
Use the four-state matrix from the first example, but start every chain at state \(00\). The distribution before the first sweep is \((1,0,0,0)\). Multiplication by \(K\) gives the distribution after that sweep:
This differs from the invariant distribution \((0.4,0.1,0.1,0.4)\). Therefore, the chain does not begin in stationarity merely because its sweep rule has a stationary distribution. This one-step calculation says nothing by itself about how many sweeps, if any, are needed for the distribution to become close to invariant; that is a separate question from defining stationarity.
Why Invariance Matters and What It Does Not Establish
Stationarity gives a precise meaning to the idea that the distribution of completed-data states is unchanged across sweeps. When a chain is initialized with a stationary distribution, every sweep has that same marginal state distribution. This is useful for reasoning about the long-run target of an iterative procedure and for understanding why individual saved imputations may differ even when the chain’s distribution is stable.
The central justification is short. Suppose \(Z_t\sim\pi\), and suppose \(\pi\) is invariant for the sweep kernel. For any set \(A\),
The first equality averages the conditional transition probabilities over the current-state distribution. The second uses the definition of the kernel, and the third uses invariance. Hence \(Z_{t+1}\sim\pi\). Repeating the same argument shows that each subsequent sweep also has marginal distribution \(\pi\). This proves the claimed preservation of the distribution; it does not assert that \(Z_{t+1}=Z_t\), or that two successive states are independent.
A familiar sufficient setting is a Gibbs sampler whose updates use full conditional distributions from a common joint distribution: updating one coordinate by its conditional distribution preserves that joint distribution, and so does a sequence of such updates. The finite-state example illustrates this kind of preservation. In FCS, however, conditional models are specified separately, and one must not assume that an invariant joint distribution exists or that it is the intended target merely because the algorithm can be run. The existence and properties of a common joint distribution require additional analysis.
Stationarity also does not imply that the chain started in the invariant distribution. Nor does a stable-looking trace establish the invariance equation. Whether a chain approaches an invariant distribution, and how to distinguish that behavior from stationarity itself, are separate issues. In practical work, plots and summaries across sweeps can reveal obvious changes in location, spread, or category frequencies, but finite simulation can fluctuate even under stationarity. Such checks are evidence about a run, not a proof of the invariant distribution.
Common Pitfalls and Practical Checks
- “Stationary means the completed data set stops changing.” This confuses a distribution with a state. A stationary chain can move every sweep; it is the probabilities assigned to states that are preserved.
- “The algorithm ran for many sweeps, so it must be stationary.” The number of sweeps alone does not verify \(\pi K=\pi\), and a chain’s current distribution need not equal an invariant distribution just because the iteration has been repeated.
- “Each conditional update is stochastic, so a stationary distribution is guaranteed.” Random updates define transitions, but stochasticity alone does not establish a useful invariant distribution or identify one.
- “The transition kernel is fixed even though the implementation changes.” A changing update order, changing conditional model, or changing transition rule can make the sweep kernels vary over time. Check whether the same sweep-level rule is actually being applied.
When inspecting an implementation, first specify what counts as the state, which values are fixed as observed, and whether parameter draws are included or integrated into the sweep kernel. Then identify the complete-sweep transition rule. For a finite state space, verify invariance directly by checking \(\pi K=\pi\). For a large or continuous state space, the same definition applies, but direct matrix multiplication is unavailable; empirical stability summaries do not replace a mathematical invariance argument.
Compared with a direct joint-model approach, FCS defines its sweep through a sequence of conditional updates rather than by drawing from one specified joint distribution. That difference matters when asking what distribution, if any, the completed states preserve. A transition-matrix calculation can verify invariance for a particular finite example, while an appeal to a common joint model can supply a preservation argument under its conditions. Neither calculation should be generalized to every FCS specification without justification.
The main limitation of stationarity as a concept is that it is a property of a transition rule and a distribution, not a certificate of scientific adequacy. Even when an invariant distribution is established, that fact alone does not show that the chosen conditional models represent the intended imputation problem or that a finite run reflects the invariant distribution. Keep the question narrow: stationarity asks whether one sweep leaves a distribution unchanged.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- In a finite-state FCS chain, what equation must a candidate stationary distribution satisfy?
- A chain has stationary distribution \((0.6,0.4)\) on states \(L,R\), but it can transition from \(L\) to \(R\). Is that a contradiction? Explain.
- For the matrix \(\begin{pmatrix}0.8&0.2\\0.3&0.7\end{pmatrix}\), verify that \((0.6,0.4)\) is stationary by calculating the product.
- Why is starting a chain at one completed-data state not enough to claim it is stationary?
Solutions. (1) It must satisfy \(\pi K=\pi\), with the entries of \(\pi\) nonnegative and summing to 1. (2) No. Stationarity concerns the distribution, not an unchanging state; transitions can occur while probability flows balance. (3) The product is \((0.6,0.4)\begin{pmatrix}0.8&0.2\\0.3&0.7\end{pmatrix}=(0.6(0.8)+0.4(0.3),\,0.6(0.2)+0.4(0.7))=(0.60,0.40)\). (4) A single starting state gives a point-mass initial distribution, which need not satisfy the invariance equation. Stationarity requires the current state distribution to be invariant under the sweep kernel.