Blocked Updates For Correlated Parameters

Why update correlated parameters together?

In a Bayesian trial model, parameters can be strongly dependent in the posterior even when their prior distributions are independent. For a binary endpoint, the control-group event rate informs the baseline log odds, while the treatment-group event rate informs a combination of the baseline and treatment effect. If the data do not sharply distinguish these quantities, a larger baseline value can be offset by a smaller treatment effect, or vice versa. The posterior then has an elongated, tilted shape.

A scalar Metropolis update changes one parameter while holding the others fixed. When the posterior is tilted, such a move is often directed across the narrow dimension of the posterior rather than along its long dimension. Proposals across the narrow dimension tend to land in low-density regions and be rejected; proposals small enough to avoid rejection make little progress along the long dimension. A block update proposes changes to correlated parameters jointly, so that their changes can follow the posterior’s orientation.

Definition: A block Metropolis update proposes a joint value for a selected vector of parameters and accepts or rejects that vector as a unit using the Metropolis-Hastings rule. A rejected proposal leaves every parameter in the block at its current value.

A baseline and treatment-effect block

Consider a two-arm trial with a binary endpoint. Let \(y_0\) of \(n_0\) control participants and \(y_1\) of \(n_1\) treatment participants experience the event. Write \(\alpha\) for the control-group baseline log odds and \(\beta\) for the treatment log odds ratio. The logistic model is

y_0 \sim \operatorname{Binomial}(n_0,\operatorname{logit}^{-1}(\alpha)), \qquad y_1 \sim \operatorname{Binomial}(n_1,\operatorname{logit}^{-1}(\alpha+\beta)), \qquad \operatorname{logit}^{-1}(u)=\frac{1}{1+\exp(-u)}.

Here \(\exp(\beta)\) is the treatment-to-control odds ratio. For example, \(\beta<0\) corresponds to lower event odds in the treatment group under this coding. The likelihood, together with specified priors for \(\alpha\) and \(\beta\), defines a posterior density \(\pi(\alpha,\beta\mid y)\). A block proposal targets this same posterior; it changes the way the chain moves, not the posterior distribution being sampled.

A common proposal is a bivariate normal random walk. From the current vector \(\theta=(\alpha,\beta)^\mathsf{T}\), generate

\theta^\ast=\theta+\varepsilon, \qquad \varepsilon\sim N_2(0,\Sigma_{\mathrm{prop}}).

The covariance matrix \(\Sigma_{\mathrm{prop}}\) controls the scale and direction of proposed joint changes. With a fixed covariance matrix, the random-walk proposal is symmetric: the density of proposing \(\theta^\ast\) from \(\theta\) equals the density of proposing \(\theta\) from \(\theta^\ast\). Therefore, the acceptance probability is

a(\theta,\theta^\ast)= \min\left\{1,\frac{\pi(\theta^\ast\mid y)}{\pi(\theta\mid y)}\right\}.

The unknown normalizing constant of the posterior cancels in this ratio. If a proposal is not symmetric, its forward and reverse proposal densities must also appear in the Metropolis-Hastings ratio; they cannot be omitted merely because the update is blocked.

Worked Example: A Tilted Posterior and Two Proposal Directions

Suppose, for illustration, that the posterior of a centered, standardized pair of parameters \(X\) and \(Y\) is exactly bivariate normal with mean \((0,0)\), marginal variances 1, and correlation \(-0.95\). This hypothetical distribution isolates the geometry: high posterior density lies mainly along a downward-sloping ridge.

For a bivariate normal with this covariance, the density relative to its value at the mean is

\frac{\pi(x,y)}{\pi(0,0)} = \exp\left\{ -\frac{x^2-2\rho xy+y^2}{2(1-\rho^2)} \right\}, \qquad \rho=-0.95.

Compare two symmetric random-walk proposals from \((0,0)\). A scalar-direction move to \((0.5,0)\) has exponent

-\frac{0.5^2}{2(1-0.95^2)} = -\frac{0.25}{0.195} \approx -1.282, \qquad \frac{\pi(0.5,0)}{\pi(0,0)} \approx \exp(-1.282)=0.277.

Its acceptance probability is therefore about 0.277. A coordinated move to \((0.5,-0.5)\), which follows the ridge, has exponent

-\frac{0.25-2(-0.95)(-0.25)+0.25}{0.195} = -\frac{0.025}{0.195} \approx -0.1282, \qquad \frac{\pi(0.5,-0.5)}{\pi(0,0)} \approx \exp(-0.1282)=0.880.

Its acceptance probability is about 0.880. These numbers do not prove that every block proposal will be accepted more often: proposal size matters, and the example uses proposals of the same selected size from the posterior mean. It demonstrates why a move that changes both correlated coordinates in the appropriate direction can have a much better posterior-density ratio than a move that changes just one.

Why the ridge slows scalar updates

The geometry can also be seen from the conditional distribution. If \(X\) and \(Y\) have a bivariate normal distribution with unit marginal variances and correlation \(\rho\), then

X\mid Y=y \sim N(\rho y,1-\rho^2).

For \(\rho=-0.95\), the conditional standard deviation is \(\sqrt{1-0.95^2}=\sqrt{0.0975}\approx0.3122\), much smaller than the marginal standard deviation of 1. Conditional on \(Y\), \(X\) is constrained to a relatively narrow range. A scalar update that changes \(X\) while holding \(Y\) fixed must respect that narrow conditional distribution; then changing \(Y\) while holding \(X\) fixed faces the corresponding constraint. The chain can consequently make many small alternating moves to travel a substantial distance along the negatively sloped ridge.

A block proposal can instead generate a pair of changes with negative covariance. This makes coordinated changes in opposite directions more common. The proposal covariance need not equal the posterior covariance exactly, but its orientation should reflect the important posterior dependence, and its scale should produce changes that are not routinely far outside the region of substantial posterior density.

Worked Example: Computing a Block Metropolis Acceptance Probability

Suppose a hypothetical trial has 18 events among 60 control participants and 30 among 60 treatment participants. Use the logistic model above, with independent priors \(\alpha\sim N(0,2.5^2)\) and \(\beta\sim N(0,2.5^2)\). At the current state \((\alpha,\beta)=(-1.00,0.50)\), consider a symmetric block random-walk proposal to \((\alpha^\ast,\beta^\ast)=(-1.10,0.65)\).

The control linear predictor changes from \(-1.00\) to \(-1.10\); the treatment linear predictor changes from \(-0.50\) to \(-0.45\). Up to constants that cancel in the ratio, the binomial log likelihood is

\ell(\alpha,\beta) = 18\alpha-60\log(1+\exp(\alpha)) + 30(\alpha+\beta)-60\log(1+\exp(\alpha+\beta)).

For the control contribution, the proposed-minus-current difference is \(18(-0.10)-60(0.287335-0.313262)=-1.800+1.556=-0.2444\). For the treatment contribution, it is \(30(0.05)-60(0.493248-0.474077)=1.500-1.150=0.3497\). Thus the log-likelihood difference is approximately \(-0.2444+0.3497=0.1053\).

The independent normal priors contribute a log-density difference of

-\frac{(-1.10)^2+(0.65)^2-\{(-1.00)^2+(0.50)^2\}}{2(2.5^2)} = -\frac{1.6325-1.2500}{12.5} = -0.0306.

The log-posterior difference is therefore \(0.1053-0.0306=0.0747\). Because the proposal is symmetric, the acceptance probability is \(\min\{1,\exp(0.0747)\}=1\): this proposal is accepted. This calculation uses the likelihood and prior at both states; an estimated covariance for the proposal does not replace the posterior-density calculation in the acceptance rule.

Constructing a proposal from posterior geometry

If pilot draws or other preliminary information suggest that the posterior covariance of \((\alpha,\beta)\) is approximately

\widehat{\Sigma}_{\mathrm{post}} = \begin{pmatrix} 0.25 & -0.20\\ -0.20 & 0.25 \end{pmatrix},

then the estimated correlation is \(-0.20/\sqrt{0.25\cdot0.25}=-0.80\). A proposal covariance proportional to this matrix gives joint increments that tend to move in opposite directions. The covariance must be positive definite for a nondegenerate bivariate normal proposal; here its determinant is \(0.25^2-0.20^2=0.0225>0\), so it is positive definite.

Worked Example: Generating a Correlated Proposal Step

For the covariance matrix above, a Cholesky factor is

L= \begin{pmatrix} 0.5 & 0\\ -0.4 & 0.3 \end{pmatrix}, \qquad LL^\mathsf{T} = \begin{pmatrix} 0.25 & -0.20\\ -0.20 & 0.25 \end{pmatrix}.

Let \(z=(1,-0.5)^\mathsf{T}\) be one hypothetical draw of two independent standard normal variates, and choose a proposal multiplier \(c=0.6\). Then

Lz= \begin{pmatrix} 0.5\\ -0.4-0.15 \end{pmatrix} = \begin{pmatrix} 0.5\\ -0.55 \end{pmatrix}, \qquad \varepsilon=cLz= \begin{pmatrix} 0.30\\ -0.33 \end{pmatrix}.

From a current state \((\alpha,\beta)=(-1.00,0.50)\), this construction proposes \((-0.70,0.17)\). Its increments are positively directed for \(\alpha\) and negatively directed for \(\beta\), consistent with the negative posterior correlation. The proposal covariance is \(c^2LL^\mathsf{T}\), or \(\begin{pmatrix}0.09&-0.072\\-0.072&0.09\end{pmatrix}\). The proposal must still be evaluated with the Metropolis-Hastings acceptance rule; alignment with the posterior does not guarantee acceptance.

Assumptions, diagnostics, and alternatives

A block Metropolis update is appropriate when the posterior target is proper, its density can be evaluated up to a common normalizing constant, and the proposal permits movement through the relevant posterior region. A fixed, symmetric normal random walk meets the symmetry condition used in the acceptance formula. If a proposal is restricted by a parameter constraint or uses an asymmetric construction, the proposal-density correction must be included, or the parameters must be represented in an unconstrained space with the corresponding density handled correctly.

Signs that blocking may help include a visibly tilted joint posterior, strong dependence between parameter draws, or scalar updates that make only slow progress despite producing accepted proposals. A scatterplot of paired draws can reveal the orientation of a ridge; a trace of each parameter can reveal prolonged periods of little movement. These are clues, not proof that a particular block is adequate. A proposal covariance with the wrong orientation may perform no better than scalar updates. A very large proposal scale can cause frequent rejection, while a very small scale can yield accepted steps that are too short to move efficiently.

Blocking also has limitations. A covariance matrix approximates overall linear dependence; it may not capture curved, strongly skewed, or multimodal posterior structure. A single block can be unwieldy when it includes many parameters, and grouping parameters that are nearly independent can be unnecessary. For the two-parameter example, it is sensible to consider \((\alpha,\beta)\) together when their posterior dependence is substantial, but not to assume that blocking is beneficial without checking the resulting chain’s movement.

Approach	How it moves	When it can be useful
Separate scalar Metropolis updates	Changes one parameter at a time.	Parameters are weakly dependent or a simple update is effective.
Joint block Metropolis update	Proposes a vector of correlated parameters together.	A block has a clear dependence structure that a joint proposal can reflect.
Gibbs update	Draws from a full conditional distribution.	The required full conditional can be sampled directly and efficiently.

A common mistake is to treat a high scalar acceptance rate as evidence that the chain explores the posterior well. Acceptance alone measures whether proposed moves were retained, not how far the chain travels or whether it follows the posterior’s long direction. The correction is to assess movement as well as acceptance: small, frequently accepted axis-aligned steps can still explore a correlated ridge slowly. Blocking addresses that geometric problem, but it does not by itself establish that the chain has explored every relevant region.

Key takeaway: When a treatment effect and baseline parameter are strongly correlated, propose them as a joint vector with covariance oriented to their posterior dependence. Use the correct Metropolis-Hastings ratio, and judge the update by its actual movement through the posterior, not by acceptance rate alone.

Check Your Understanding

Answer each question, then compare with the solutions below.

In the logistic trial model, what is the treatment log odds ratio, and what does a negative value imply under the stated coding?
A standardized bivariate normal target has correlation \(-0.80\). What is the conditional variance of the first coordinate given the second?
A symmetric block proposal has a proposed-to-current posterior density ratio of \(0.40\). What is its acceptance probability?
Why can a scalar update have a high acceptance rate but still explore a strongly correlated posterior slowly?
Name one limitation of a block proposal based on a single covariance matrix.

Solutions. (1) The treatment log odds ratio is \(\beta\), so \(\beta<0\) means lower event odds in treatment than control under this model. (2) The conditional variance is \(1-\rho^2=1-0.64=0.36\). (3) The acceptance probability is \(\min(1,0.40)=0.40\). (4) Accepted scalar steps can be small and axis-aligned, so they may make little progress along the posterior’s long tilted direction. (5) A single covariance matrix may fail to represent curved, skewed, or multimodal posterior structure.