Calibrating the Protective Component's Scale

Why the Protective Scale Requires Calibration

A robust mixture prior can reduce reliance on historical information when current data conflict with it. Its protective component provides an alternative to the historical component, but the alternative is not automatically protective merely because it is included. Its scale determines which treatment effects it considers plausible and how well it predicts the data. A scale that is too narrow may fail to accommodate effects that should count as conflict; an extremely diffuse scale can assign low predictive density to ordinary observations because its probability is spread over a very wide range.

The scale is distinct from the mixture weight. The mixture weight sets the prior probability assigned to each component before observing current data. The protective scale instead determines the distribution of effects within that component. After observing data, both features matter: the scale affects the predictive evidence for each component, which in turn affects the posterior component weights.

Definition: In a two-component robust mixture prior, let \(\theta\) denote the treatment effect, \(w\) the prior probability of the historical component, and \(1-w\) the prior probability of the protective component. Write \(\theta\sim w\,\pi_H(\theta)+(1-w)\,\pi_P(\theta)\). The protective component’s scale is a parameter controlling the spread of \(\pi_P\). Its calibration means choosing that spread so that the component assigns reasonable prior probability to clinically plausible treatment effects and responds appropriately to plausible data conflict.

Scale calibration is not a search for a single number that is universally correct. It is a way to make the protective component’s assumptions explicit, relate them to the treatment-effect scale and clinical context, and examine how conclusions change when those assumptions vary.

A Normal Model for Seeing the Scale’s Role

Consider a continuous treatment effect \(\theta\), measured in a stated unit, and a current-study estimate \(Y\). For the calculations below, assume a normal sampling model \(Y\mid\theta\sim N(\theta,s^2)\), with known standard error \(s\). Let the historical and protective components be normal:

\theta\mid H\sim N(\mu_H,\tau_H^2), \qquad \theta\mid P\sim N(\mu_P,\sigma_P^2).

Here \(\sigma_P\) is the protective component’s standard deviation, the quantity being calibrated. Conditional on either component, the likelihood and prior are conjugate. The current estimate has a component-specific predictive distribution:

Y\mid H\sim N(\mu_H,\tau_H^2+s^2), \qquad Y\mid P\sim N(\mu_P,\sigma_P^2+s^2).

Thus the scale affects conflict assessment twice. First, it determines the protective prior’s spread. Second, through the predictive variance, it determines the predictive density of the observed current estimate and therefore the posterior probability assigned to each component.

Assumptions for the calculations: The estimate \(Y\) is adequately described by the stated normal likelihood, its standard error \(s\) is treated as known, the component distributions and prior mixture weight are specified before observing \(Y\), and the effect and its scales use the same units. These assumptions make the algebra transparent; real analyses must also assess model fit, endpoint definition, population comparability, and whether a normal approximation is reasonable.

Deriving the Posterior Weight and Component Means

Let \(f_H(y)\) and \(f_P(y)\) be the normal predictive densities under the historical and protective components. Bayes’ rule gives the posterior probability of the historical component:

q_H(y)=P(H\mid Y=y) =\frac{w f_H(y)}{w f_H(y)+(1-w)f_P(y)}.

The protective component receives posterior probability \(1-q_H(y)\). Within component \(j\), where \(j\) is \(H\) or \(P\), the posterior mean is a precision-weighted average of its prior mean and the current estimate:

m_j = \frac{\mu_j/\tau_j^2+y/s^2}{1/\tau_j^2+1/s^2} = \frac{s^2}{\tau_j^2+s^2}\mu_j + \frac{\tau_j^2}{\tau_j^2+s^2}y, \qquad \tau_P=\sigma_P.

Consequently, the overall posterior mean is \(E(\theta\mid y)=q_H(y)m_H+\{1-q_H(y)\}m_P\). The scale changes both \(m_P\), through the protective component’s shrinkage of the estimate, and \(q_H\), through predictive fit. Those two changes can act in different directions.

Specify a clinically meaningful effect scale.
Define the effect, its direction, and its units before choosing a numerical spread.

Translate clinical judgments into prior implications.
Ask which effect sizes should be common, uncommon, or effectively implausible under the protective component.

Check predictive implications and conflict response.
Compare reasonable scales using prior predictive distributions and representative current-study results, not only the posterior mean.

Worked Examples of Scale Calibration

Worked Example: Translating Plausible Effects Into a Scale

Suppose \(\theta\) is a treatment effect measured in points, and the protective component is centered at the neutral value, \(\mu_P=0\). Investigators judge that effects more than 1 point from zero should not be rare under this component. Compare protective standard deviations of 0.4, 1, and 2 points using the probability of an effect lying within 1 point of zero.

For a centered normal component, the probability is \(P(|\theta|\leq 1\mid P)=2\Phi(1/\sigma_P)-1\), where \(\Phi\) is the standard normal cumulative distribution function. At \(\sigma_P=0.4\), the standardized boundary is \(1/0.4=2.5\), giving \(2\Phi(2.5)-1\approx0.9876\). Only about 1.24% of the prior probability lies outside \([-1,1]\) points.

At \(\sigma_P=1\), the boundary is 1, giving \(2\Phi(1)-1\approx0.6827\); about 31.73% lies outside the interval. At \(\sigma_P=2\), the boundary is 0.5, giving \(2\Phi(0.5)-1\approx0.3829\); about 61.71% lies outside it.

The 0.4-point scale would be inconsistent with a judgment that effects beyond 1 point are not rare. The 2-point scale makes effects beyond 1 point common, which may be too diffuse if such effects are clinically implausible. The 1-point scale is a defensible candidate under the stated judgment, but the calculation does not establish it as uniquely correct. Other clinically relevant thresholds and asymmetries should also be considered.

Worked Example: How Scale Changes Conflict-Driven Reweighting

Use a deliberately simple model to isolate the effect of scale. Suppose a hypothetical current-study estimate is \(y=1.2\) points with standard error \(s=0.4\) points. Let the prior mixture weight be \(w=0.8\). Set the historical component to \(N(0,0.2^2)\) and the protective component to \(N(0,\sigma_P^2)\). The shared zero mean is a simplifying assumption for this illustration, not a general requirement.

The historical predictive variance is \(0.2^2+0.4^2=0.20\) points squared, so its density at 1.2 is

f_H(1.2) = \frac{1}{\sqrt{2\pi(0.20)}} \exp\left(-\frac{1.2^2}{2(0.20)}\right) = \frac{1}{\sqrt{2\pi(0.20)}}\exp(-3.6) \approx 0.02437.

For \(\sigma_P=0.4\), the protective predictive variance is \(0.4^2+0.4^2=0.32\). Its density is

f_P(1.2) = \frac{1}{\sqrt{2\pi(0.32)}} \exp\left(-\frac{1.2^2}{2(0.32)}\right) \approx 0.07433.

The posterior historical weight is therefore

q_H = \frac{0.8(0.02437)} {0.8(0.02437)+0.2(0.07433)} \approx 0.5674.

For \(\sigma_P=1\), the protective predictive variance is \(1.16\), and its density at 1.2 is approximately \(0.1991\). This gives \(q_H\approx0.3286\). For \(\sigma_P=2\), the predictive variance is \(4.16\), and the density is approximately \(0.1645\), giving \(q_H\approx0.3721\). The observed estimate is less compatible with the tight historical component than with each of these protective predictive distributions, so the posterior historical weight falls below its prior value of 0.8.

The posterior means show the scale’s second effect. The historical-component mean is \(m_H=\{0.2^2/(0.2^2+0.4^2)\}(1.2)=0.2400\) points. The protective means are \(0.6000\), \(1.034\), and \(1.154\) points for scales 0.4, 1, and 2, respectively. Combining each pair of means using its posterior weight gives mixture posterior means of approximately 0.3957, 0.7734, and 0.8139 points.

For comparison, the protective-only posterior means are the three protective means just calculated. Subtracting the mixture means from them gives differences of approximately 0.2043, 0.2611, and 0.3400 points. In this example, the broader protective scale both permits more movement toward the observed estimate within its component and changes the component weights. Neither effect should be attributed to scale alone without examining the full mixture calculation.

Worked Example: Why a Very Diffuse Scale Can Be Less Protective Than Expected

Keep the same model and compare the protective predictive density at \(y=1.2\) for \(\sigma_P=1\) and \(\sigma_P=2\). The predictive variances are 1.16 and 4.16 points squared. Although the second predictive distribution is wider, its density at 1.2 is lower: approximately 0.1645 rather than 0.1991. Its probability has been spread across a much wider range.

The posterior historical weights consequently rise from about 0.3286 at \(\sigma_P=1\) to about 0.3721 at \(\sigma_P=2\). The more diffuse protective component has not become more compatible with this particular observation. This is a local density comparison, not a claim that wider distributions always assign less probability to large effects. A broader scale can increase density in some regions and decrease it in others; the location of the observation relative to the component center matters.

The contrast is also visible for an estimate near zero. At \(y=0.1\) points and \(\sigma_P=1\), the protective predictive density is

f_P(0.1) = \frac{1}{\sqrt{2\pi(1.16)}} \exp\left(-\frac{0.1^2}{2(1.16)}\right) \approx 0.3688.

For the historical predictive variance of 0.20, \(f_H(0.1)\approx0.8701\). The posterior historical weight is then \(0.8(0.8701)/\{0.8(0.8701)+0.2(0.3688)\}\approx0.9042\). The data near zero reinforce the more concentrated historical component, while the estimate at 1.2 favors the protective component. This dependence on the observed value is exactly why scale should be inspected through predictive behavior rather than judged by the prior standard deviation alone.

Diagnostics, Alternatives, and Limitations

A practical calibration should make the protective prior’s implications inspectable. Plot or tabulate its prior probability over clinically meaningful effect ranges, and examine the component predictive distributions for current-study estimates that would be considered compatible, moderately discrepant, or strongly discrepant with historical evidence. Check that the likelihood’s sampling variability is included: comparing the observed estimate only with the prior distribution ignores the standard error and can misrepresent conflict.

Approach	What it changes	Useful diagnostic
Vary protective scale	Spread and predictive fit of the protective component	Prior probabilities for clinical thresholds and component predictive densities
Vary mixture weight	Prior odds assigned to the components	Compare posterior component weights across plausible prior weights
Use a single fixed prior	Removes component-level reweighting	Check whether the resulting prior and posterior remain clinically plausible

These approaches answer different questions. Changing the mixture weight does not change which treatment effects the protective component regards as plausible. Changing its scale does not change the prior odds between components, although it changes the posterior odds through the predictive densities. A single fixed prior avoids mixture reweighting but also does not provide the same data-dependent shift between components. Comparisons should therefore hold other choices fixed when assessing the specific influence of the protective scale.

Warning signs include a protective component that places almost all its probability inside a range narrower than clinically plausible effects, or one so diffuse that it assigns little density to a broad range of ordinary results. Also inspect whether small changes in scale produce large changes in posterior conclusions. Such sensitivity is not automatically a computational failure: it may reveal that the data do not determine the borrowing behavior and that the model’s prior assumptions matter materially.

The normal example has important limitations. Treatment effects may be bounded, skewed, or naturally expressed on a transformed scale, and a symmetric normal protective component may then be unsuitable. The standard error may be estimated rather than known, and the current likelihood may be non-normal. Furthermore, a scale chosen from marginal effect plausibility does not address every source of disagreement between historical and current data, such as differences in patient mix or outcome measurement. Calibration should be accompanied by checks of the model and the clinical meaning of the effect scale.

Common Mistake

Common mistake: “A wider protective component is always more protective, so increasing its standard deviation must reduce historical borrowing.” This reasoning ignores the predictive-density normalization and the observed estimate’s location. A wider component may allow its own posterior mean to move further toward the data, yet give a lower predictive density at that particular observation and thereby receive less posterior weight. Calculate or simulate the predictive and posterior implications across plausible observations instead of assuming a monotone effect.

Key takeaway: Calibrate the protective scale by linking its prior probabilities to clinical effect ranges, then check how its predictive density and posterior borrowing behave under plausible current-study results. The scale is an assumption with observable consequences, not merely a technical tuning constant.

Check Your Understanding

Answer each question, then compare with the brief solutions below.

In a robust mixture prior, what is the distinction between the protective scale and the prior mixture weight?
A centered protective normal prior has standard deviation 0.5 points. What is the probability that its effect lies within 1 point of zero? Give the expression and approximate value.
In the worked normal model, why does the posterior historical weight depend on the protective scale even though the prior mixture weight is held fixed?
For the estimate \(y=1.2\) points, the protective predictive density is about 0.1991 with scale 1 and 0.1645 with scale 2. Which scale gives the larger posterior historical weight, all else equal?
Name one diagnostic for a protective scale that is too narrow and one for a scale that is excessively diffuse.

Solutions. (1) The scale controls the spread and plausible effects within the protective component; the mixture weight sets the prior probability of selecting that component. (2) The probability is \(2\Phi(1/0.5)-1=2\Phi(2)-1\approx0.9545\). (3) Scale changes the protective predictive density, and Bayes’ rule uses that density to update the component weights. (4) Scale 2 gives the larger historical weight because its protective predictive density is lower at this observation, making the protective component less favored relative to the historical component. (5) A too-narrow scale may put nearly all prior probability inside an implausibly small clinical range; an excessively diffuse scale may have low predictive density across ordinary observations because its probability is spread too widely.