Presenting Treatment Variation Across Study Sites

Why Present Treatment Variation Across Sites?

In a multicenter study, an overall treatment estimate can conceal meaningful variation: the treatment association may be stronger at some sites and weaker at others. A model with a site-varying treatment effect allows that variation to be estimated while using information across sites. Presenting the result well requires more than listing site estimates. Readers need to know what the estimates represent, how they relate to the overall model, and why they are not separate trial results.

The focus here is a binary outcome modeled with a logistic GLMM. The earlier material on interpreting a multicenter clinical GLMM provides context for the overall model; the emphasis here is how to present its site-varying treatment terms.

The Site-Varying Treatment Model

Let \(Y_{ij}\) equal 1 if participant \(i\) at site \(j\) has the outcome and 0 otherwise. Let \(x_{ij}=1\) for treatment and \(x_{ij}=0\) for control. A model with a random site intercept and a random site treatment slope can be written as follows.

Y_{ij}\mid u_{0j},u_{1j}\sim\operatorname{Bernoulli}(p_{ij}),\qquad \operatorname{logit}(p_{ij})=\beta_0+\beta_1x_{ij}+u_{0j}+u_{1j}x_{ij}.

Here, \(\operatorname{logit}(p)=\log\{p/(1-p)\}\). The fixed coefficient \(\beta_1\) describes the mean treatment slope in the model’s site distribution. The random slope \(u_{1j}\) is site \(j\)’s deviation from that mean on the log-odds scale. The random intercept \(u_{0j}\) allows baseline outcome odds to vary by site.

Definition: In this model, the site-specific conditional treatment log odds ratio is \(\beta_1+u_{1j}\). The corresponding conditional odds ratio is \(\exp(\beta_1+u_{1j})\). The term \(u_{1j}\) alone is a deviation from the model’s mean treatment log odds ratio, not the site’s treatment effect by itself.

A standard assumption is that the site effects are independent across sites and have a bivariate normal distribution with mean zero and covariance matrix \(\Sigma\):

\begin{pmatrix}u_{0j}\\u_{1j}\end{pmatrix} \sim N\left[ \begin{pmatrix}0\\0\end{pmatrix}, \begin{pmatrix}\tau_0^2&\tau_{01}\\\tau_{01}&\tau_1^2\end{pmatrix} \right].

The variance \(\tau_1^2\) describes the modeled spread of treatment slopes across sites on the log-odds scale; \(\tau_{01}\) describes the association between site baseline deviations and treatment-slope deviations. The model also assumes that outcomes are independent conditional on the modeled random effects (and any other included predictors). In an applied analysis, the treatment allocation and data structure must support estimating site-specific treatment variation. In particular, sites need informative treatment contrasts; when treatment is not represented within sites, site-specific slopes may be confounded with differences between sites.

Turning a Random Slope into a Site Estimate

The model’s site-specific treatment contrast follows directly by comparing the two treatment values while holding the site effects fixed. At site \(j\), the control linear predictor is \(\beta_0+u_{0j}\). The treatment linear predictor is \(\beta_0+\beta_1+u_{0j}+u_{1j}\). Subtracting the control predictor from the treatment predictor gives:

(\beta_0+\beta_1+u_{0j}+u_{1j})-(\beta_0+u_{0j}) =\beta_1+u_{1j}.

Because the logit link expresses log odds, exponentiating this difference gives the site-specific conditional odds ratio. This derivation also explains why neither the random intercept nor the random slope alone is the complete site contrast: the intercept cancels, and the slope deviation must be added to the fixed treatment coefficient.

Worked Example: Calculating Site-Specific Odds Ratios

Suppose a hypothetical fitted model has a fixed treatment coefficient \(\hat{\beta}_1=-0.55\). Three sites have estimated random-slope deviations \(-0.35\), \(0.20\), and \(0.45\), respectively. Calculate and interpret each model-based site treatment odds ratio.

For Site A, the estimated log odds ratio is \(-0.55+(-0.35)=-0.90\), so the odds ratio is \(\exp(-0.90)=0.407\). Under this model, the treatment odds at Site A are estimated to be about 0.41 times the control odds, conditional on the site effects.

For Site B, the estimated log odds ratio is \(-0.55+0.20=-0.35\), and the odds ratio is \(\exp(-0.35)=0.705\). For Site C, it is \(-0.55+0.45=-0.10\), giving \(\exp(-0.10)=0.905\).

All three estimates are below 1, but their magnitudes differ. These are estimates from one model that shares information across sites, not three independent treatment-effect analyses.

Presenting the Amount of Variation

A list of site estimates does not by itself describe how much treatment effects vary across the modeled population of sites. The random-slope standard deviation, \(\tau_1\), summarizes that spread on the log-odds scale. A useful presentation reports it alongside the fixed treatment coefficient, with units (log-odds units) or a clear statement of the scale. When readers need an odds-ratio scale, the model’s distribution can also be translated transparently.

Worked Example: Translating the Random-Slope Spread

Suppose a hypothetical fitted model gives \(\hat{\beta}_1=-0.55\) and \(\hat{\tau}_1=0.25\). Under the normal random-slope assumption, approximately 95% of the modeled site-specific log odds ratios lie within 1.96 standard deviations of the mean slope:

-0.55\pm1.96(0.25) =-0.55\pm0.49 =[-1.04,-0.06].

Exponentiating the endpoints gives \([\exp(-1.04),\exp(-0.06)]=[0.353,0.942]\). This is an approximate central 95% range for the distribution of site-specific conditional odds ratios under the fitted random-effects model. It is not a confidence interval for \(\beta_1\), nor does it say that a particular site’s true odds ratio has a 95% probability of lying in that range.

A report should label this clearly as a modeled across-site distribution. The range is sensitive to the normality assumption and to uncertainty in the estimated variance; it should not be presented as if its endpoints were known precisely.

How to Present Site Estimates Without Overstating Them

Software typically estimates the random effect for a site using information from that site and the estimated distribution across sites. Such estimates are often called empirical Bayes estimates or conditional modes. They are generally partially pooled: less informative site data tend to produce estimates closer to the overall distribution’s center, while more informative data can support more distinct site estimates. The degree of pooling depends on the data and the estimated random-effect variance.

That feature is a strength of the model, but it affects the language used in a report. A site-specific model estimate is not the result of fitting an independent trial at that site. It reflects a joint analysis and the model’s assumptions about the distribution of site effects. Present estimates as model-based site contrasts, and show enough context for readers to understand their precision and the amount of variation.

Worked Example: Writing a Site-Variation Summary

Consider a hypothetical multicenter study with 12 sites. A fitted model reports \(\hat{\beta}_1=-0.55\), \(\hat{\tau}_1=0.25\), and a random-slope estimate of \(-0.30\) for Site D. Construct the site contrast and write a suitable short summary.

The Site D log odds ratio is \(-0.55+(-0.30)=-0.85\). Its odds ratio is \(\exp(-0.85)=0.427\). A concise summary could read: “In the model, the estimated treatment odds ratio at Site D was 0.43, based on a site-specific deviation of \(-0.30\) log-odds units from the mean treatment slope of \(-0.55\). Across sites, the estimated random-slope standard deviation was 0.25 log-odds units.”

This wording identifies the scale and the source of the site estimate. A complete report should also provide uncertainty using an appropriate model-based method and describe the number of participants and treatment allocation relevant to the site estimate. Do not manufacture an interval by treating a reported random-effect estimate as if it were an independently estimated fixed coefficient with an ordinary standard error.

Checks, Alternatives, and Limitations

Before presenting treatment variation, check whether the fitted model and the study design support it. A converged model is necessary but does not establish that a random-slope distribution is well estimated. Review convergence messages, the estimated variance and covariance, and whether the covariance estimate is at or near a boundary. A near-zero estimated slope variance indicates little detected variation in this model, but may also reflect limited information. Inspect site estimates for patterns that are unexpectedly extreme and check whether they arise from sparse site data or unusual treatment allocation. A small number of sites makes the assumed distribution and its variance especially consequential.

The main alternatives answer somewhat different questions. A model with a common treatment slope estimates one treatment coefficient and assumes no site-to-site variation in that slope. A model with fixed treatment-by-site interactions estimates a separate parameter for each site relative to a reference coding and does not use the same random-effects distribution to pool estimates. That approach may be useful when site contrasts themselves are the primary targets and the number of sites and data support the extra parameters. The random-slope model instead summarizes variation across sites and partially pools site estimates under its distributional assumptions.

Neither approach removes design limitations. If treatment is assigned only between sites, a site-specific treatment contrast may not be distinguishable from other site differences without additional design assumptions. If the estimated slope variance is poorly determined, presenting a finely ranked list of sites can imply more precision than the data provide. Also, the fixed treatment coefficient is a mean on the conditional log-odds scale under the model; it is not generally the marginal, population-averaged odds ratio. Keep the scale and target explicit.

Common pitfall: “The random slope for Site D is \(-0.30\), so the treatment odds ratio there is \(\exp(-0.30)=0.74\).” This treats a deviation as the full site effect. The correction is to add the deviation to the fixed treatment coefficient first: \(\hat{\beta}_1+\hat{u}_{1D}\), then exponentiate. Also describe the resulting contrast as a model-based estimate, not as an independent trial result.

Key takeaway: Present the fixed treatment slope, the random-slope spread, and selected site estimates on clearly identified scales. Explain that each site contrast combines the fixed slope and that site’s deviation, and that site estimates come from a joint model.

Check Your Understanding

Answer each question, then compare your reasoning with the brief solutions.

A model has \(\hat{\beta}_1=-0.40\) and a site random-slope estimate of \(0.10\). What is the site’s conditional log odds ratio and odds ratio?
What does a random-slope standard deviation describe, and why is it not a confidence interval for the fixed treatment coefficient?
Why should a partially pooled site estimate not be described as the result of an independent trial at that site?
What is one design concern if treatment varies only between sites?

Solutions. (1) The log odds ratio is \(-0.40+0.10=-0.30\); the odds ratio is \(\exp(-0.30)=0.741\). (2) It describes the modeled across-site spread of treatment slopes on the log-odds scale; it summarizes a random-effects distribution, not uncertainty about the fixed coefficient. (3) The estimate comes from a joint model and may be partially pooled toward the across-site distribution, so it is not an independent site analysis. (4) The treatment contrast can be confounded with other site differences, making site-specific treatment variation difficult to identify separately from site variation.