Target Distributions and Markov Chain Sampling

Targets and the Sampling Goal

Many statistical computations ask for a probability under a distribution that is easy to specify up to a constant but difficult to normalize or sum over. A posterior distribution is a common example: its relative weights come from the prior and likelihood, while its normalizing constant may be an intractable sum or integral. Markov chain Monte Carlo (MCMC) addresses this computational setting by producing a dependent sequence of states intended to represent the target distribution. The basic estimate of an event probability is then the fraction of sampled states in that event.

A target distribution is the probability law whose probabilities or expectations are the quantities of interest. On a finite or countable state space \(S\), it can be specified by nonnegative weights \(w(i)\) with finite positive total \(Z=\sum_{i\in S}w(i)\). The resulting probabilities are \( \pi(i)=w(i)/Z \). The weights need not sum to one; the normalized probabilities do. States with zero weight have target probability zero. If \(Z=0\), these weights do not define a probability distribution.

For a continuous state space, the target may be a density with respect to a stated reference measure, such as Lebesgue measure. Its probabilities are integrals of that density over sets. Multiplying all unnormalized weights or density values by the same positive constant leaves the normalized target unchanged. This scale freedom is useful: computations can use \(w\) without first calculating \(Z\).

In Bayesian inference, if \(\theta\) is a parameter, \(y\) is observed data, \(p(\theta)\) is a prior density or mass function, and \(L(y\mid\theta)\) is the likelihood, the posterior is proportional to \(p(\theta)L(y\mid\theta)\). Thus an unnormalized posterior can be taken as \(w(\theta)=p(\theta)L(y\mid\theta)\); the evidence \(Z\) is the sum or integral of these weights over the parameter space. The formulas below require that the prior and likelihood yield \(0<Z<\infty\).

From a Target to an MCMC Estimate

Let \((\Omega,\mathcal{F},\mathbb{P})\) be a probability space, and let \((X_n)_{n\ge0}\) be a discrete-time Markov chain on a finite or countable state space \(S\). A Markov chain Monte Carlo procedure is designed so that the target distribution is an invariant distribution for the chain: if \(X_n\) has that distribution, then \(X_{n+1}\) has the same distribution. This definition states the sampling goal; it does not by itself prove that a particular chain has the desired target or that its output has converged to the target. Verifying invariance and convergence requires additional conditions and methods.

Here \(N\) is a positive integer, so the displayed average is defined. An initial segment of a chain may be omitted in an application; if \(b\ge0\) states are omitted and \(N\ge1\) states retained, the estimate is \(N^{-1}\sum_{n=b+1}^{b+N}\mathbf{1}_{\{X_n\in A\}}\). Omitting initial states does not itself guarantee that the retained sample represents the target. In general, MCMC states are dependent, so this average is not automatically an average of independent draws, nor is it automatically unbiased. Its reliability depends on the chain and its sampling behavior.

Proof. By definition, \(\pi(i)=w(i)/Z\). Countable additivity gives \(\pi(A)=\sum_{i\in A}\pi(i)=\sum_{i\in A}w(i)/Z\). Substituting \(Z=\sum_{i\in S}w(i)\) gives the claimed ratio. The denominator is finite and strictly positive by assumption, so the ratio is defined. \(\square\)

For a continuous target density, the same calculation uses integrals: if \(0<Z=\int_S w(x)\,dx<\infty\), then \(\pi(A)=\int_A w(x)\,dx\,/\,\int_S w(x)\,dx\). This identity can make a probability mathematically well-defined even when evaluating either integral, and therefore \(Z\), is computationally difficult. MCMC estimates the probability by sampling from a chain designed for the target, rather than by evaluating the ratio directly.

Worked Calculations

What the Estimate Does—and Does Not—Say

The empirical average has a direct interpretation: it is the observed fraction of retained states in the event. The cancellation identity explains why an unknown normalizing constant need not prevent estimating a posterior probability. But the identity does not establish that a particular Markov chain samples from the target. That requires checking how the chain was constructed. Nor does a finite sample proportion certify its own accuracy. A sequence that has not explored relevant parts of the state space can give a misleading estimate, even if its intended target is correct.

Common mistake: treating unnormalized event weight as a probability. In the posterior example, the event weight is \(1/3\). Reporting \(1/3\) as the posterior probability ignores the total posterior weight. The correction is to divide by \(Z=13/30\), giving \((1/3)/(13/30)=10/13\). In MCMC, the corresponding correction is different in form but serves the same normalization role: count the sampled states in the event and divide by the total number of retained states. Reporting \(8\) from the sample, rather than \(8/12\), would likewise fail to produce a probability.

Keep three quantities distinct: the unnormalized target \(w\), the normalized target probability \(\pi(A)\), and the sample estimate \(\widehat{\pi}_N(A)\). The first encodes relative mass, the second is the desired theoretical answer, and the third is calculated from a finite chain output. A trustworthy MCMC analysis must connect the chain to the target and assess whether its output adequately represents that target; the sample proportion alone does not supply that justification.