Prior Predictive and Posterior Diagnostics for BLRM

Why Diagnose a BLRM Fit?

A Bayesian logistic regression model (BLRM) can produce dose-specific toxicity estimates even when observations are sparse. That is useful for dose-finding, but a numerical recommendation is not evidence that the model is credible. The prior can imply implausible curves before any patients are observed, and the likelihood may not sufficiently constrain the intercept, slope, or their joint uncertainty afterward. Diagnostics examine both the model’s implications and the information the data contributed.

Let $Y_i$ be the number of patients with a dose-limiting toxicity (DLT) among $n_i$ patients treated at dose $d_i$. Use the two-parameter logistic form with reference dose $d_{\mathrm{ref}}$:

Y_i\mid \alpha,\beta \sim \operatorname{Binomial}\{n_i,p(d_i)\},\qquad \operatorname{logit}\{p(d)\}=\eta(d)=\alpha+\beta x(d),\qquad x(d)=\log(d/d_{\mathrm{ref}}).

Here, $\alpha$ is the log-odds of DLT at the reference dose, and $\beta$ is the change in log-odds per one-unit increase in log dose. The dose-specific probability is $p(d)=\operatorname{expit}\{\eta(d)\}=1/\{1+\exp[-\eta(d)]\}$. Conditional on the parameters, the binomial observations are assumed independent across dose groups. A positive $\beta$ gives an increasing curve; an unconstrained prior on $\beta$ can also allow decreasing curves.

Definition: A prior predictive distribution describes outcomes or model quantities implied by the prior before observing trial data. A posterior diagnostic checks whether the distribution of parameters and dose-specific risks after observing data is computationally reliable, sufficiently informed, and clinically plausible. Neither diagnostic alone establishes that the model is correct.

Inspecting Prior Predictive Curves

A prior predictive curve is obtained by drawing $(\alpha,\beta)$ from the joint prior, calculating $p(d)$ over the candidate dose range for each draw, and summarizing those curves. It shows more than the curve at the prior mean: it reveals the range of toxicity patterns that the prior regards as plausible. Because the logistic function is nonlinear, $\operatorname{expit}\{E[\eta(d)]\}$ is generally not the prior mean of $p(d)$. It is the probability corresponding to the mean linear predictor, not necessarily the mean probability.

Worked Example: Checking Prior Curves Across the Dose Range

Suppose, hypothetically, $d_{\mathrm{ref}}=100$ mg and independent priors are $\alpha\sim N(-1.20,0.50^2)$ and $\beta\sim N(0.80,0.35^2)$. At 200 mg, $x=\log(2)=0.6931$. The prior mean linear predictor is

E\{\eta(200)\}=-1.20+0.80(0.6931)=-0.6455, \qquad \operatorname{expit}(-0.6455)\approx 0.344.

This is the probability at the mean linear predictor, and also the median probability under these normal priors because the expit function is increasing and $\eta(200)$ is normal. The variance is

\operatorname{Var}\{\eta(200)\} =0.50^2+(0.6931)^2(0.35^2) =0.3089, \qquad SD\{\eta(200)\}\approx 0.5557.

A central 90% interval for the linear predictor is approximately $-0.6455\pm1.645(0.5557)$, or $[-1.560,\,0.269]$. Transforming its endpoints gives a central 90% prior interval for the probability of approximately $[0.174,\,0.567]$. Thus the prior’s median curve at 200 mg is about 0.344, while its uncertainty includes substantially lower and higher risks. A prior predictive plot should make this spread visible at every candidate dose, not just at the reference dose.

Since the prior on $\beta$ is untruncated, it assigns probability $\Phi(-0.80/0.35)\approx0.011$ to a negative slope. That is a small but nonzero prior probability of a decreasing curve. Whether it is acceptable depends on the modeling assumptions and scientific context; it should not pass unnoticed if monotonicity is required.

For a full prior predictive check, inspect complete curves on the actual dose scale and ask whether their levels and shapes are credible. Look for near-zero or near-one probabilities across the entire dose range, abrupt transitions unsupported by prior knowledge, and implausible decreases if toxicity is assumed to rise with dose. An acceptable curve at $d_{\mathrm{ref}}$ does not guarantee acceptable behavior at distant doses: the slope contribution grows with $|x(d)|$.

Posterior Dependence and Weak Identification

The posterior distributions of $\alpha$ and $\beta$ are often dependent. For example, data concentrated around one dose may constrain the log-odds near that dose but allow multiple intercept-slope combinations to fit similarly. A scatterplot of posterior draws can show this as an elongated cloud. The direction and strength of that dependence matter because the uncertainty of the linear predictor at a particular dose depends on both marginal uncertainties and their covariance.

Key result: If the joint posterior of $(\alpha,\beta)$ is approximately bivariate normal with means $\mu_\alpha,\mu_\beta$, variances $\sigma_\alpha^2,\sigma_\beta^2$, and covariance $\sigma_{\alpha\beta}$, then at $x=\log(d/d_{\mathrm{ref}})$, $$ \eta(d)\mid \text{data}\ \approx N\left(\mu_\alpha+x\mu_\beta,\ \sigma_\alpha^2+x^2\sigma_\beta^2+2x\sigma_{\alpha\beta}\right). $$

This follows by taking the linear combination $\eta(d)=\alpha+x\beta$: its mean is the same linear combination of means, and its variance is $\operatorname{Var}(\alpha)+x^2\operatorname{Var}(\beta)+2x\operatorname{Cov}(\alpha,\beta)$. The covariance term can either increase or decrease uncertainty at a dose. Ignoring it can give the wrong impression of how precisely the curve is known.

Worked Example: How Intercept-Slope Dependence Changes Risk at a Dose

Consider hypothetical posterior summaries with $E(\alpha)=-1.20$, $E(\beta)=0.85$, $SD(\alpha)=0.45$, $SD(\beta)=0.20$, and $\operatorname{Cov}(\alpha,\beta)=-0.055$. At $d=4d_{\mathrm{ref}}$, $x=\log(4)=1.3863$. Then

\mu_\eta=-1.20+1.3863(0.85)=-0.021645,

and the posterior variance of the linear predictor is

0.45^2+(1.3863)^2(0.20^2)+2(1.3863)(-0.055) =0.12688, \qquad SD(\eta)\approx0.3562.

The negative covariance reduces the uncertainty at this dose. If the covariance term were incorrectly omitted, the variance would be $0.2025+0.07687=0.2794$, much larger than $0.12688$. This does not mean the parameters are individually well determined: it means their joint uncertainty partly offsets when forming this particular linear combination. A different dose can have a different offset.

A posterior scatterplot, marginal intervals, and dose-specific intervals should therefore be read together. A narrow interval for a risk at one dose does not prove that both parameters are well identified. Conversely, a broad parameter cloud can still imply a relatively precise risk at a dose where intercept and slope uncertainty compensate.

Weak identification is especially plausible when few patients have completed DLT assessment, observed doses cover only a narrow range, or all observed outcomes are the same. Warning signs include posterior parameter distributions that remain close to their priors, long ridges in joint draws, strong sensitivity of remote-dose risks to small parameter changes, or posterior summaries that change materially with reasonable computational settings. Check trace plots for chains that fail to mix, effective sample sizes for excessive Monte Carlo uncertainty, and convergence diagnostics such as $\hat R$, which compares between-chain and within-chain variation. These checks assess computation, not clinical plausibility.

Dose-Specific Risk Estimates

A clinically useful diagnostic is the posterior distribution of $p(d)$ at each candidate dose. For a target toxicity probability $p_T$, the posterior overdose risk at dose $d$ is $P\{p(d)>p_T\mid\text{data}\}$. Since the expit is strictly increasing, this is equivalently the posterior probability that $\eta(d)>\operatorname{logit}(p_T)$. Under the approximate normal posterior for $\eta(d)$, it can be calculated from a standard normal tail probability. With posterior draws, estimate it by the fraction of draws for which $p(d)>p_T$.

Worked Example: Calculating Posterior Overdose Risk

Continue with the hypothetical posterior summaries above. Set $p_T=0.30$, so $\operatorname{logit}(0.30)=\log(0.30/0.70)\approx-0.8473$. At $4d_{\mathrm{ref}}$, $\eta(d)$ is approximately normal with mean $-0.021645$ and standard deviation $0.3562$. Therefore,

z=\frac{-0.8473-(-0.021645)}{0.3562}\approx-2.318, \qquad P\{p(d)>0.30\mid\text{data}\} =1-\Phi(-2.318)\approx0.9898.

Under this approximation, the posterior probability that toxicity exceeds 0.30 at $4d_{\mathrm{ref}}$ is about 0.990. This is a probability conditional on the model, prior, and observed data; it is not the probability that the model is correct. The posterior median probability at that dose is $\operatorname{expit}(-0.021645)\approx0.4946$, which is also far above the target.

Worked Example: Comparing Risks at Two Doses

At $2d_{\mathrm{ref}}$, $x=0.6931$. Using the same hypothetical summaries, the posterior mean linear predictor is

$$ -1.20+0.6931(0.85)=-0.6109. $$

Its variance is $0.45^2+(0.6931)^2(0.20^2)+2(0.6931)(-0.055)=0.1455$, giving $SD(\eta)\approx0.3814$. With the same target,

z=\frac{-0.8473-(-0.6109)}{0.3814}\approx-0.620, \qquad P\{p(2d_{\mathrm{ref}})>0.30\mid\text{data}\} =1-\Phi(-0.620)\approx0.732.

The posterior median probability there is $\operatorname{expit}(-0.6109)\approx0.352$. The two doses have different posterior risk distributions, even though both estimates come from the same parameter posterior. In practice, posterior draws are preferable to a normal approximation when the posterior is skewed or the joint distribution is not well represented by a bivariate normal.

Common Pitfalls, Alternatives, and Limits

Treating the curve at the posterior mean as the posterior mean curve. In general, $E[\operatorname{expit}\{\eta(d)\}\mid\text{data}]\neq \operatorname{expit}(E[\eta(d)\mid\text{data}])$. Summarize transformed posterior draws when a posterior mean probability is required.
Ignoring parameter covariance. Substituting marginal standard deviations while dropping the covariance term can misstate uncertainty. Use joint draws or the full variance formula.
Reporting a risk without its uncertainty. A point estimate alone can conceal substantial posterior probability above a target. Report the posterior distribution or a clearly defined risk probability alongside the estimate.
Confusing computational convergence with model adequacy. Well-mixed chains can come from a model whose prior predictive curves or posterior implications are implausible.

The main alternative to integrating over the posterior is a plug-in calculation using a point estimate of $(\alpha,\beta)$. It is simpler, but it suppresses parameter uncertainty and cannot by itself give $P\{p(d)>p_T\mid\text{data}\}$. Empirical proportions avoid the logistic curve assumption at observed doses, but can be unstable with small groups and do not directly provide a smooth estimate between doses. The BLRM borrows information across doses through its curve; that benefit comes with dependence on its functional form and prior.

Diagnostics cannot prove that the dose-toxicity relationship is correct, and sparse data may not distinguish a weakly identified fit from a genuinely broad but reasonable posterior. Prior predictive assessment, joint posterior inspection, dose-specific summaries, and computational checks answer different questions and should be considered together. In particular, extrapolation beyond the studied dose range deserves caution because slope uncertainty has increasing influence as $|\log(d/d_{\mathrm{ref}})|$ grows.

Key takeaway: Diagnose a BLRM by examining the prior’s full dose-toxicity curves, the joint posterior of intercept and slope, and posterior risk distributions at clinically relevant doses. Do not rely on a single fitted curve or a point estimate.

Check Your Understanding

Answer each question, then check the brief solutions.

Why can $\operatorname{expit}\{E[\eta(d)]\}$ differ from the posterior mean of $p(d)$?
What does a strong negative posterior covariance between $\alpha$ and $\beta$ imply about uncertainty in $\eta(d)$?
If $p_T=0.25$, write the event defining posterior overdose risk using $\eta(d)$.
Name one warning sign of weak identification and one computational diagnostic that addresses chain mixing.

Solutions. (1) The expit is nonlinear, so transforming a mean generally differs from averaging transformed draws. (2) It can reduce the variance of $\alpha+x\beta$ at some doses through the $2x\operatorname{Cov}(\alpha,\beta)$ term; the effect depends on $x$. (3) The event is $\eta(d)>\operatorname{logit}(0.25)=\log(1/3)$, approximately $-1.099$. (4) A long ridge in joint parameter draws or posterior summaries remaining close to the prior can indicate weak identification; trace plots or $\hat R$ can flag poor mixing or between-chain disagreement.