Independence Metropolis With Heavy-Tailed Proposals

Why Use an Independence Proposal?

A Metropolis sampler can propose a new value using a distribution centered on its current value, as in random-walk Metropolis. An independence Metropolis sampler instead draws each candidate from a fixed proposal distribution, regardless of the current state. This can make large moves across the parameter space possible, but it also means the proposal must cover the posterior well on its own.

For a trial parameter such as a treatment log-odds ratio \(\theta\), a proposal with heavier tails than the posterior can assign meaningful probability to values far from the center. This helps avoid a proposal that almost never reaches posterior tail regions. It is not a guarantee of good performance: a proposal that is too diffuse may spend many draws in regions of low posterior density, and a poorly covered region can still create very low acceptance probabilities.

The posterior and proposal densities must be evaluated at both the current value and candidate. In Bayesian computation, the posterior is often known only up to a normalizing constant: if \(\pi(\theta)\) is proportional to \(\tilde{\pi}(\theta)\), that constant cancels in the acceptance ratio. Define the target-to-proposal ratio \(w(\theta)=\tilde{\pi}(\theta)/q(\theta)\). The same rule can then be written \(\alpha(\theta,\theta^\star)=\min\{1,w(\theta^\star)/w(\theta)\}\). The proposal’s independence from the current state does not mean the accept/reject decision is independent of it.

Construction and Assumptions

For an unconstrained real-valued trial effect, one possible heavy-tailed proposal is a Student \(t\) distribution with a modest number of degrees of freedom and a chosen scale. For example, a \(t_4\) proposal centered at zero with scale \(s\) has density proportional to \([1+\theta^2/(4s^2)]^{-5/2}\). Its polynomial tails decay more slowly than the tails of a normal density. The proposal parameters should be selected to cover the important posterior region without being needlessly broad.

The essential support condition is that \(q(\theta)>0\) wherever the posterior has positive density. Otherwise, a region with posterior mass may be unreachable by the chain. The posterior must be proper, and \(q\) must be a normalized density from which candidates can be generated. Given these conditions, the acceptance rule accounts for the fact that the proposal might favor one candidate over another. The proposal need not equal the posterior.

The log calculation is useful when densities are very small: it avoids multiplying quantities that may underflow numerically. Equivalently, draw \(U\) uniformly from \((0,1)\) and accept when \(\log U\) is less than the log ratio, or when the log ratio is positive. A rejected candidate is not discarded from the chain record; the chain’s next value is the previous state.

Why the Acceptance Rule Works

The ratio corrects for the candidate-generation mechanism so that accepted transitions have balanced probability flow under the posterior. For two distinct states \(\theta\) and \(\theta^\star\), the density of accepted movement from \(\theta\) to \(\theta^\star\), weighted by the target density, is

The expression on the right is unchanged when \(\theta\) and \(\theta^\star\) are exchanged. Thus the accepted flow from \(\theta\) to \(\theta^\star\) equals the reverse flow. Rejections add probability of staying at the current state, which does not upset this symmetry. This is detailed balance with respect to \(\pi\), and it implies that the posterior is invariant for the chain. The proposal correction is essential: using only \(\pi(\theta^\star)/\pi(\theta)\) would generally fail to account for the independent proposal’s relative density at the two values.

Worked Calculations

Tail Conditions, Diagnostics, and Limitations

A useful sufficient condition for strong performance is that the normalized ratio \(\pi(\theta)/q(\theta)\) be bounded above, say by \(M<\infty\). For a positive, continuous target with lighter tails than a Student \(t\) proposal, the ratio often approaches zero in the tails; boundedness must also hold over the central region. Under this condition, the independence sampler has a uniform lower bound on its transition probabilities: for any set \(A\),

To see this, write \(w(\theta)=\pi(\theta)/q(\theta)\). The accepted transition density at \(\theta^\star\) is \(q(\theta^\star)\min\{1,w(\theta^\star)/w(\theta)\}\). Since \(w(\theta)\leq M\) and \(w(\theta^\star)\leq M\), this density is at least \(\pi(\theta^\star)/M\). Integrating over \(A\) gives the stated bound. This condition is sufficient, not necessary, and heavier proposal tails alone do not establish it.

In practice, inspect whether the proposal covers the posterior’s central mass and tails, and calculate the target-to-proposal ratio over plausible parameter values. Large variation in \(w(\theta)\) is a warning: when the chain is at a state with a very high ratio, proposed states will often have smaller ratios and be rejected. Repeated states caused by rejection can then dominate stretches of the chain. A proposal that is too wide may have heavy tails yet make inefficient use of draws in the posterior’s central region. Conversely, a proposal with tails that are too light can leave posterior regions inadequately represented.

The main alternative in this setting is random-walk Metropolis, whose proposal depends on the current state and typically makes local moves. Independence proposals can jump far across the parameter space, but depend more strongly on a well-chosen global proposal. If a suitable direct posterior draw is available, it avoids Metropolis rejections; when computation requires MCMC, the independence sampler is one option. These methods are not interchangeable fixes for every model: poor proposal coverage can make an independence sampler inefficient even when its acceptance rule is correct.