Metropolis-Within-Gibbs For Logistic Trial Models

Why Logistic Trial Models Need Metropolis-Within-Gibbs

A binary trial endpoint is often modeled by letting a participant’s probability of response depend on treatment assignment and baseline characteristics. With a logistic link, the likelihood is straightforward to evaluate, but the resulting coefficient posteriors generally do not have conjugate full conditional distributions. A practical option is to update one coefficient at a time with a Metropolis step, while treating the remaining coefficients as fixed during that step. Repeating this for every coefficient creates a Metropolis-within-Gibbs sampler.

The method combines two familiar ideas: the conditional updating structure of Gibbs sampling and the accept-or-reject correction of Metropolis-Hastings. The key implementation detail is to evaluate the logistic likelihood at the proposed coefficient value, using the most recently available values of all the other coefficients. In particular, a treatment coefficient changes the linear predictor only for participants whose treatment indicator is one.

Model and Posterior Kernel

For example, a simple two-arm trial can use \(x_i=(1,A_i)^\mathsf{T}\), where \(A_i=1\) denotes assignment to the treatment arm and \(A_i=0\) denotes control. Then \(\beta=(\beta_0,\beta_1)^\mathsf{T}\), the control-arm log odds are \(\beta_0\), and the treatment-versus-control log odds contrast is \(\beta_1\). This interpretation is conditional on the specified model and any covariates included in it.

Assume for this tutorial that the coefficients have independent proper normal priors, \(\beta_j\sim N(m_j,s_j^2)\), with \(s_j^2>0\). Conditional on the observed trial data, the posterior density is proportional to the likelihood times the prior density. For computation, it is useful to work with its log kernel, omitting only constants that do not depend on \(\beta\).

The first term follows because \(P(Y_i\mid x_i,\beta)=p_i^{Y_i}(1-p_i)^{1-Y_i}\). Taking logarithms gives \(Y_i\log(p_i)+(1-Y_i)\log(1-p_i)=Y_i x_i^\mathsf{T}\beta-\log\{1+\exp(x_i^\mathsf{T}\beta)\}\). The second term is the log density of the independent normal priors, with constants omitted. A proper prior makes the posterior proper for this finite binary dataset, including cases in which the likelihood alone has separation.

Coordinate-Wise Metropolis Updates

A sweep updates coefficients in a fixed order. When updating coefficient \(\beta_j\), hold all other coefficients at their current values and propose a candidate \(z\). With independent normal priors, the conditional target for this step is proportional to the likelihood as a function of \(\beta_j\) times the prior density for \(\beta_j\). Prior factors for the other coefficients cancel from the acceptance ratio.

For a symmetric random-walk proposal, such as \(z\sim N(\beta_j^{\mathrm{cur}},\tau_j^2)\), the forward and reverse proposal densities are equal. The log acceptance ratio is therefore the difference between the conditional log-posterior kernels:

Accept the proposal with probability \(\min\{1,\exp(r)\}\). Equivalently, draw \(U\sim\operatorname{Uniform}(0,1)\) and accept when \(\log(U)<r\). If the proposal is not symmetric, its reverse-to-forward proposal-density ratio must also be included. A proposal that is accepted becomes the current value used in subsequent coefficient updates during that sweep.

Why the Sequential Sampler Targets the Joint Posterior

Each coordinate update is a Metropolis-Hastings transition whose target is the corresponding full conditional distribution under the joint posterior. If the other coordinates are held fixed, the acceptance rule corrects for the proposal so that this conditional distribution is invariant for that update. Consequently, when the current vector has the joint posterior distribution, updating one coordinate leaves the joint distribution unchanged. Applying the same reasoning successively to every coordinate shows that a complete sweep also preserves the joint posterior.

This establishes the target distribution, not the speed of convergence from an arbitrary starting point. With proper normal priors and nonzero-variance normal random-walk proposals, each coefficient can move across the real line, supporting exploration of the posterior. Nonetheless, high posterior dependence can make coordinate moves inefficient: changing one coefficient at a time may require many small accepted steps to follow a narrow, tilted region of high posterior density.

Implementation Checks, Alternatives, and Limitations

Evaluate the log posterior rather than exponentiating a likelihood that may be extremely small. For numerical stability, calculate \(\log(1+\exp(\eta))\) with a stable log-one-plus-exponential routine when \(\eta\) could be large. For grouped binomial data, use \(y\eta-n\log(1+\exp(\eta))\); using \(n-y\) in place of \(n\), or omitting the failures’ contribution to the denominator, gives the wrong likelihood. Check the design matrix, arm coding, response counts, and the predictor affected by each proposed coefficient.

A sampler can be mathematically correct yet computationally ineffective. Many consecutive rejections suggest proposals may be too large for the local posterior scale; extremely small moves can be accepted frequently but explore the posterior slowly. Strong posterior dependence can also make single-coordinate movement inefficient. These patterns are reasons to inspect the chain and reconsider the update strategy, not grounds for changing the target density. A proper prior helps address posterior propriety, but it does not repair a misspecified link, unmodeled dependence, incorrect outcome coding, or an inappropriate trial design model.

When full conditional distributions are available in standard forms, ordinary Gibbs sampling draws directly from them and needs no accept-or-reject step; logistic regression with normal coefficient priors generally does not provide those conjugate conditionals. A joint random-walk Metropolis update proposes the entire coefficient vector at once and uses one acceptance decision, which can account for coordinated movement but may be sensitive to the joint proposal. Metropolis-within-Gibbs is a convenient compromise when coordinate-wise conditional targets are easy to evaluate. Its main limitation is that changing coefficients one at a time can mix slowly when posterior dependence is strong.

A practical diagnostic is to compare the proposed and accepted values, not just the acceptance frequency: verify that a hand-calculated update agrees with the code for a small dataset. Also check that rejected proposals leave the coefficient unchanged and that each update uses the latest values from the same sweep. Convergence and mixing require assessment from generated chains; a correct acceptance formula alone does not establish that posterior summaries are reliable.