Invariant Distributions And Detailed Balance

Why Invariance Matters

A Markov chain can be used to approximate posterior summaries only if its transitions are compatible with the posterior it is intended to explore. For example, if the target is the posterior distribution of a treatment effect, the chain should preserve that distribution: when the parameter currently has the posterior distribution, one transition should leave its distribution unchanged. This property is called invariance.

Invariance is a precise target-preservation condition, not a guarantee that a chain started at an arbitrary value will quickly reach the target or explore it well. This distinction matters in clinical trial analyses: a transition rule may preserve the desired posterior mathematically, yet fail to move between important regions of the parameter space. The earlier material on posterior expectations as Monte Carlo averages explains why draws from the target distribution support posterior summaries; here we establish a condition on transitions that makes the posterior an appropriate stationary distribution.

Transition Kernels and Invariant Distributions

Let $\theta$ denote a parameter, with state space $\mathcal{S}$, and let $K(\theta,A)$ be the probability that the next state belongs to a measurable set $A\subseteq\mathcal{S}$, given that the current state is $\theta$. A valid Markov transition kernel assigns probabilities between 0 and 1, has $K(\theta,\mathcal{S})=1$ for every $\theta$, and has the required measurability in $\theta$. In a discrete state space, it is represented by a transition matrix whose entries are nonnegative and whose rows sum to 1.

Definition: A probability distribution $\pi$ on $\mathcal{S}$ is invariant for $K$ if, when the current state has distribution $\pi$, the next state also has distribution $\pi$. Equivalently, for every measurable set $A$, $$ \pi(A)=\int_{\mathcal{S}} K(\theta,A)\,\pi(d\theta). $$

For a finite or countable state space, write $\pi_i=P(X=i)$ and $P_{ij}=P(X_{t+1}=j\mid X_t=i)$. Invariance then means $\pi_j=\sum_i\pi_iP_{ij}$ for every state $j$, or, in matrix notation, $\pi=\pi P$. If the target is a posterior distribution, $\pi$ denotes that posterior, for example $\pi(\theta)=p(\theta\mid y)$, where $y$ is the observed trial data. The posterior must be a proper probability distribution for this interpretation as an invariant distribution to apply.

Detailed Balance: A Sufficient Condition

Invariance compares the total probability flowing into each set with the probability assigned to that set. A stronger condition balances probability flow in both directions between states or sets.

Definition: A distribution $\pi$ and a transition kernel $K$ satisfy detailed balance if, for all measurable sets $A,B$, $$ \int_A K(\theta,B)\,\pi(d\theta) = \int_B K(\theta,A)\,\pi(d\theta). $$ For a discrete state space this is $\pi_iP_{ij}=\pi_jP_{ji}$ for every pair $i,j$. When densities with respect to a common measure exist, the corresponding condition is $\pi(\theta)k(\theta,\theta')=\pi(\theta')k(\theta',\theta)$ almost everywhere.

Detailed balance says that, under $\pi$, the probability flowing from $A$ to $B$ in one step equals the probability flowing from $B$ to $A$. A kernel satisfying detailed balance with respect to $\pi$ is called reversible with respect to $\pi$. Reversibility is sufficient for invariance, but is not necessary.

Why Detailed Balance Implies Invariance

The justification uses the fact that a valid transition kernel assigns total probability 1 to the whole state space. Fix any measurable set $A$. By detailed balance, take $B=\mathcal{S}$. Then

\begin{aligned} \int_{\mathcal{S}} K(\theta,A)\,\pi(d\theta) &= \int_A K(\theta,\mathcal{S})\,\pi(d\theta) \\ &= \int_A 1\,\pi(d\theta) \\ &= \pi(A). \end{aligned}

The left-hand side is the probability that the next state is in $A$ when the current state is distributed as $\pi$. The right-hand side is the original probability of $A$. This is exactly the definition of invariance. The validity condition $K(\theta,\mathcal{S})=1$ is essential to the argument; detailed-balance equations for a rule that loses probability mass would not establish invariance.

Worked Example: A Reversible Kernel for Two Posterior States

Suppose a hypothetical trial analysis uses a discrete approximation with two possible values for a treatment-effect parameter: state $L$, a lower effect, and state $H$, a higher effect. Assume the posterior probabilities are $\pi_L=0.70$ and $\pi_H=0.30$. Consider the transition matrix

P= \begin{pmatrix} 0.80 & 0.20\\ 7/15 & 8/15 \end{pmatrix},

where rows are current states $L,H$ and columns are next states $L,H$. The entries are nonnegative, and each row sums to 1: $0.80+0.20=1$ and $7/15+8/15=1$. Thus this is a valid transition matrix.

Check the off-diagonal detailed-balance equation. The posterior flow from $L$ to $H$ is $0.70(0.20)=0.14$. The reverse flow is $0.30(7/15)=0.14$. The diagonal equations hold automatically, so detailed balance holds. Directly checking invariance gives

\begin{aligned} (\pi P)_L &= 0.70(0.80)+0.30(7/15)=0.56+0.14=0.70,\\ (\pi P)_H &= 0.70(0.20)+0.30(8/15)=0.14+0.16=0.30. \end{aligned}

One transition therefore preserves the stated posterior probabilities. This example checks the target distribution, not how quickly a chain initialized at either state would produce representative draws.

Worked Example: A Continuous Reversible Kernel

Suppose a hypothetical trial model has a parameter $\theta$ whose posterior, after centering and scaling, is $N(0,1)$. Consider the transition rule

\theta'=\rho\theta+\sqrt{1-\rho^2}\,\varepsilon, \qquad \varepsilon\sim N(0,1),

where $|\rho|<1$, and $\varepsilon$ is independent of the current state $\theta$. Given $\theta$, the next state is normal with mean $\rho\theta$ and variance $1-\rho^2$, so the conditional distribution is properly normalized. To check detailed balance, suppose the current state has the target distribution, $\theta\sim N(0,1)$. Then $\theta'$ also has mean 0 and variance $\rho^2+(1-\rho^2)=1$.

More strongly, the joint pair $(\theta,\theta')$ is bivariate normal with mean vector $(0,0)$, variances 1, and covariance $\rho$. Its joint density is

f(\theta,\theta') = \frac{1}{2\pi\sqrt{1-\rho^2}} \exp\left\{-\frac{\theta^2-2\rho\theta\theta'+\theta'^2}{2(1-\rho^2)}\right\}.

Swapping $\theta$ and $\theta'$ leaves this expression unchanged. Thus the joint density of the current and next states is symmetric, which is the density form of detailed balance. The $N(0,1)$ target is consequently invariant. This illustration verifies a kernel for a specific normal target; it does not show that the same transition rule preserves an arbitrary posterior.

Invariance Does Not Require Detailed Balance

Detailed balance is a useful sufficient condition, not the definition of invariance itself. For example, consider three states with a uniform target distribution, $\pi_i=1/3$, and the deterministic cycle $1\to2$, $2\to3$, $3\to1$. Each state has exactly one incoming transition with probability 1, so the next state is uniform whenever the current state is uniform. Hence $\pi$ is invariant. But detailed balance fails: $\pi_1P_{12}=1/3$, while $\pi_2P_{21}=0$. This is a nonreversible kernel that still preserves its target.

The distinction helps when assessing a proposed transition rule. If detailed balance is established, the invariance proof is immediate. If it is not, that fact alone does not show that the target is wrong; invariance must instead be checked directly. In discrete problems, calculate each component of $\pi P$. In continuous problems, verify the integral defining invariance or establish another valid argument for it.

Failure Modes and Practical Interpretation

Several conditions should be checked before treating a posterior as invariant for a proposed chain. The transition rule must be a valid kernel, including appropriate behavior at boundaries or constraints of the parameter space. Its transitions must not assign probability to states outside the space in a way that invalidates the rule. The target must be a proper posterior distribution, and the detailed-balance or invariance calculation must use that same target—not an unnormalized expression as though it were already a probability distribution.

Even when invariance is correct, it does not by itself establish convergence from the chosen initial state. For instance, the identity transition $P_{ii}=1$ for every state satisfies detailed balance with every distribution, including a nonuniform posterior: both sides of each detailed-balance equation are zero for distinct states, and equal for identical states. Yet the chain never moves. If initialized at state $i$, it remains there forever, so its draws do not explore a non-degenerate target. In a finite state space, irreducibility (the ability to reach every state from every other state) and aperiodicity, together with invariance, provide standard conditions for convergence to the invariant distribution. More general state spaces require appropriately stronger conditions.

A practical audit therefore asks two separate questions: does the kernel preserve the posterior? and can the resulting chain explore that posterior from its initialization? Transition-row sums and detailed-balance checks address the first question, while reducibility, immobility, and other mixing failures concern the second. Invariance is essential for posterior Monte Carlo sampling, but it is not a complete assessment of sampling quality.

Direct independent posterior sampling, when available, avoids dependence between successive draws and can be preferable. When the posterior cannot be sampled directly, a Markov chain may be useful if its kernel preserves the target and the chain explores it adequately. These are different approaches: a proof of invariance does not make dependent draws independent, and exact posterior calculations do not require a Markov chain. The method chosen should fit the posterior computation problem rather than treating detailed balance as a guarantee of efficiency.

Key takeaway: Invariance means that one transition leaves the target distribution unchanged. Detailed balance equates probability flow in opposite directions and guarantees invariance for a valid kernel, but neither reversibility nor invariance alone guarantees that a chain will explore the target effectively.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions below.

In a finite-state chain, what equation states that $\pi$ is invariant?
A valid kernel satisfies detailed balance with respect to a proper posterior. What conclusion follows, and what is the key step in the justification?
Can an invariant chain fail to satisfy detailed balance? Give a reason based on the three-state cycle.
Why does an identity transition matrix demonstrate that invariance alone does not ensure useful posterior sampling?

Solutions. (1) In vector notation, $\pi=\pi P$, or componentwise $\pi_j=\sum_i\pi_iP_{ij}$. (2) The posterior is invariant. Set one set in detailed balance to the entire state space and use $K(\theta,\mathcal{S})=1$. (3) Yes. The cycle preserves the uniform distribution because each state has one incoming transition, but flow from state 1 to 2 is positive while reverse flow is zero. (4) Every state remains fixed, so the chain can satisfy detailed balance with the target yet never leave its initial state or explore other posterior states.