Tutorials › Pharmacometric Workflows and Reproducible Modeling › Pooling Trials With Different Assays And Protocols

Case Studies · Tutorial 394 of 399

Pooling Trials With Different Assays And Protocols

Learn to harmonize assays and protocol information, model study-specific differences, and check whether pooled pharmacokinetic estimates predict data from another trial.

Advanced 13 min read

What You'll Learn

  • Distinguish assay calibration differences from study-specific pharmacokinetic effects.
  • Represent trial-specific doses and sampling times in a pooled analysis.
  • Use paired bridge samples and equivalence margins to assess assay comparability.
  • Calculate a fixed-effect pooled estimate and assess its between-study consistency.
  • Evaluate predictions in a held-out trial without treating one successful prediction as proof of validity.
  • Recognize when study effects and assay effects cannot be separated from the available data.

Why Pooling Requires More Than Combining Rows

Pooling pharmacokinetic data can improve parameter precision and broaden the range of observed doses and sampling times. But measurements from different trials are not automatically interchangeable. Assays may differ in calibration or precision; sampling schedules may emphasize different parts of the concentration-time profile; and a trial may differ in ways that affect pharmacokinetics rather than measurement. If those distinctions are ignored, a larger combined data set can produce a more precise estimate of the wrong quantity.

The central task is to define a common pharmacokinetic target, represent how each trial collected and measured its data, and test whether trial-specific discrepancies remain after those features are accounted for. This tutorial concerns pooling across trials that are intended to inform the same target population and parameter. A difference in population or target use requires a separate transportability question.

Definition: A pooled analysis combines participant-level data from multiple trials under a shared model for the target pharmacokinetic quantities, while representing trial-specific features that can affect the observations. These features may include assay calibration and precision, actual dosing and sampling times, and residual study effects. Pooling does not mean assuming that all records were generated under identical measurement and protocol conditions.

Specify the Observation Process Before Pooling

Let \(s\) index trials, \(i\) participants, and \(j\) samples. Write \(C^{\mathrm{true}}_{sij}\) for the latent concentration predicted by the pharmacokinetic model from participant \(i\)'s dose history and the actual time of sample \(j\). A useful general observation model on the log-concentration scale is

$$ \log C^{\mathrm{obs}}_{sij} = a_s+g_s\log C^{\mathrm{true}}_{sij}+\epsilon_{sij}, \qquad \epsilon_{sij}\sim N(0,\sigma_s^2). $$

Here \(a_s\) is an assay-specific log offset, \(g_s\) is an assay-specific scale factor, and \(\sigma_s\) describes residual measurement and model error on the log scale. If the assay is calibrated to the common scale, the ideal values are \(a_s=0\) and \(g_s=1\). A trial-specific residual variance may be appropriate when assay precision differs, but it does not correct a systematic offset or scale difference.

For example, participant clearance might be represented as

$$ \log CL_{si}=\log CL+\delta_s+\eta_{si}, \qquad \eta_{si}\sim N(0,\omega_{CL}^2). $$

In this expression, \(CL\) is the common clearance target, \(\delta_s\) is a trial-specific shift in typical clearance, and \(\eta_{si}\) is participant-level variation. A reference-trial constraint such as \(\delta_1=0\), or a clearly specified centering constraint, is needed to define the common intercept. The \(\delta_s\) terms are not substitutes for assay calibration: one describes a difference in the pharmacokinetic parameter, the other a difference in how concentration is measured.

The protocol enters through the concentration model, not by pretending that every trial used the same schedule. Use recorded dose amounts, administration types, dosing times, interruptions, and sample times. A sample collected at 2.1 hours should be modeled at 2.1 hours, not rounded to a nominal 2-hour visit if that approximation could matter. Sparse schedules can still contribute information, but they may provide little information about particular parameters.

These models require that the dose and sample records are sufficiently accurate, the structural pharmacokinetic model is suitable across trials, and the residual and between-participant distributions are reasonably represented. Separating \(\delta_s\) from \(a_s\) and \(g_s\) additionally requires information that identifies those effects: for example, assay bridge samples, overlapping measurements, or design features that let the data distinguish a concentration-scale discrepancy from a true parameter shift. If assay and trial are perfectly confounded, the distinction may not be identifiable from the pooled data alone.

Worked Example: Checking Two Assays With Paired Samples

Suppose 25 hypothetical patient samples are measured by both the established assay and a candidate assay. For each sample, define \(d_i=\log(C_{\mathrm{candidate},i}/C_{\mathrm{established},i})\). Assume the paired log differences are independent across samples and approximately normally distributed. The observed mean difference is \(\bar d=0.0180\), and the sample standard deviation is \(s_d=0.0800\). Assess comparability using a prespecified allowable ratio range of 0.90 to 1.10.

A difference of zero corresponds to equal measurements on the log scale. The equivalence limits on that scale are \(\log(0.90)=-0.1053\) and \(\log(1.10)=0.09531\). The standard error of the paired mean is

$$ SE=\frac{s_d}{\sqrt{n}} =\frac{0.0800}{\sqrt{25}} =0.01600. $$

For a 90% confidence interval with \(df=24\), use \(t_{0.95,24}\approx1.711\). The interval is

$$ \bar d\pm t_{0.95,24}SE = 0.0180\pm1.711(0.01600) = 0.0180\pm0.02738 = [-0.00938,\ 0.04538]. $$

Exponentiating the endpoints gives an interval for the candidate-to-established assay ratio of approximately \([0.9907,\ 1.046]\). This entire interval lies within \([0.90,\ 1.10]\), so these hypothetical data support equivalence within the prespecified margins under the paired-sample assumptions. This is not proof that the assays are identical; it is evidence that their average difference is sufficiently small for the stated purpose and margin. The conclusion also depends on the bridge samples covering a relevant concentration range.

Why a Study Effect Is Not an Assay Correction

A systematic concentration shift can have different explanations. If all samples from one trial were measured on an assay reading 5% high, an assay offset may account for the shift. If the assay is comparable but typical clearance differs, the concentration-time profiles may reflect a study-specific pharmacokinetic effect. If the trial has a different sampling schedule, it may simply constrain some features of the model less precisely. These explanations should not be conflated.

Calibration information is especially important when assay and trial are confounded. If every participant in trial A is measured only by assay A and every participant in trial B only by assay B, a higher concentration in A could be explained by higher exposure, an assay offset, or both. Without bridge samples or other identifying information, separately estimating an assay offset and a trial-specific clearance effect can yield unstable estimates or a likelihood with a ridge of nearly equivalent explanations. The remedy is more identifying information, a defensible externally established calibration, or a more limited model—not simply adding more free parameters.

A short justification shows why the sampling schedule must also be represented correctly. Suppose the structural model predicts concentration \(f(t;\theta)\) at time \(t\), but a nominal time \(t_0\) is substituted for the recorded time \(t_0+\Delta\). For small \(\Delta\), a first-order expansion gives

$$ f(t_0+\Delta;\theta) \approx f(t_0;\theta)+ \frac{\partial f(t_0;\theta)}{\partial t}\Delta. $$

Thus, the prediction error depends on both the timing error and the local slope of the concentration-time curve. Near a steep absorption or distribution phase, even a modest timing discrepancy can matter; during a relatively flat interval, the same discrepancy may have little effect. Using actual times avoids making that approximation without justification.

Pooling Estimates and Assessing Study Consistency

A simple fixed-effect calculation illustrates the conditions required for pooling. Suppose each trial produces an estimate \(\widehat{CL}_s\) of the same clearance quantity, with known or well-estimated standard error \(SE_s\). If these estimates are unbiased, approximately normal, and independent across trials, and if a common true clearance is appropriate, the inverse-variance pooled estimate is

$$ \widehat{CL}_{\mathrm{pool}} = \frac{\sum_s w_s\widehat{CL}_s}{\sum_s w_s}, \qquad w_s=\frac{1}{SE_s^2}, \qquad SE_{\mathrm{pool}}= \sqrt{\frac{1}{\sum_s w_s}}. $$

This calculation is a transparent summary, not a replacement for an individual-level pharmacokinetic model when the trials have different dose histories, sampling schedules, or assay properties. The estimates must refer to the same parameter and scale after those differences are addressed. A useful consistency statistic is \(Q=\sum_s w_s(\widehat{CL}_s-\widehat{CL}_{\mathrm{pool}})^2\), compared approximately with a chi-square distribution with \(k-1\) degrees of freedom when there are \(k\) independent estimates and the common-effect assumptions hold. A small \(Q\) does not establish that the effects are identical, particularly when few trials provide little power to detect heterogeneity.

Worked Example: Combining Two Clearance Estimates

Consider two hypothetical trials whose assay scales have been harmonized and whose dose and sampling schedules were handled in the analyses. They estimate the same typical clearance: trial 1 gives \(4.80\ \mathrm{L/h}\) with \(SE_1=0.40\ \mathrm{L/h}\), and trial 2 gives \(5.40\ \mathrm{L/h}\) with \(SE_2=0.60\ \mathrm{L/h}\). Assume the estimates are independent and that a common clearance is defensible.

The weights are \(w_1=1/(0.40)^2=6.25\) and \(w_2=1/(0.60)^2=25/9\). Retaining the exact second weight avoids rounding error in the weighted calculation:

$$ \widehat{CL}_{\mathrm{pool}} = \frac{6.25(4.80)+(25/9)(5.40)}{6.25+25/9} = \frac{45}{325/36} = \frac{324}{65} = 4.984615\ \mathrm{L/h} \approx4.985\ \mathrm{L/h}. $$

The pooled standard error is

$$ SE_{\mathrm{pool}} = \sqrt{\frac{1}{325/36}} = \sqrt{\frac{36}{325}} = 0.3328\ \mathrm{L/h}. $$

An approximate 95% confidence interval is \(4.984615\pm1.96(0.3328)\), or about \([4.332,\ 5.637]\ \mathrm{L/h}\). Under the fixed-effect assumptions, this interval describes uncertainty in the common clearance estimated by this procedure; it is not a 95% probability statement about the fixed parameter.

The consistency statistic is

$$ Q = 6.25(4.80-4.984615)^2 + \frac{25}{9}(5.40-4.984615)^2 \approx0.6923. $$

With \(df=1\), this corresponds to a chi-square upper-tail p-value of approximately 0.405. These two estimates do not show strong evidence against a common value, but two trials cannot establish that study effects are absent. The calculation would also be inappropriate if the estimates targeted different clearance definitions, if their errors were correlated, or if assay bias had not been addressed.

Check Prediction in a Trial Not Used to Fit the Model

A pooled model can fit the combined data yet fail to predict observations from a particular trial. Prediction should therefore be assessed at the level relevant to use: concentrations, profiles, or parameter-derived quantities, with the held-out trial's assay and protocol represented. When feasible, estimate the model without one trial, predict that trial using its recorded doses and times, and compare observed data with predictive distributions that include the relevant parameter and residual uncertainty.

Worked Example: A Held-Out Concentration Prediction

Suppose a one-compartment intravenous bolus model fitted to other hypothetical trials gives a pooled \(CL=4.984615\ \mathrm{L/h}\). For illustration, take \(V=20.0\ \mathrm{L}\) as known, and consider a held-out trial with a 100 mg bolus and a sample at 8.00 hours. Assume its assay has been harmonized to the common scale. The model prediction is

$$ C(8) = \frac{100\ \mathrm{mg}}{20.0\ \mathrm{L}} \exp\left( -\frac{4.984615\ \mathrm{L/h}}{20.0\ \mathrm{L}} (8.00\ \mathrm{h}) \right) \approx0.681\ \mathrm{mg/L}. $$

Suppose the held-out measurement is \(0.700\ \mathrm{mg/L}\). If the log-scale predictive error standard deviation is \(0.250\), a simple approximate 95% predictive interval, treating the stated parameter values as fixed for this illustration, is

$$ 0.681\exp(\pm1.96(0.250)) \approx[0.417,\ 1.112]\ \mathrm{mg/L}. $$

The observed value lies inside this interval, and its log-scale standardized residual is \(\log(0.700/0.681)/0.250\approx0.11\). This single observation is compatible with the prediction, but it cannot demonstrate adequate prediction for the whole trial. A meaningful assessment examines multiple participants and times, checks whether predictive intervals achieve appropriate coverage, and looks for systematic residual trends by trial, concentration, or time. A real predictive interval should also propagate parameter uncertainty when it is material; the simplified interval here does not.

Diagnostics, Alternatives, and Limits

Useful diagnostics ask whether the model has left a pattern that its assumptions said should not remain. Inspect concentration residuals by trial, assay, concentration, nominal and actual time, dose, and predicted concentration. Examine trial-specific parameter estimates and uncertainty, assay bridge results across the concentration range, and predictive checks that reproduce each trial's sampling design. Residual trends by concentration can suggest an assay scale mismatch or structural-model inadequacy; a consistent trial-level shift can suggest calibration or study effects; poor prediction confined to a sparsely sampled phase may indicate weak information rather than a genuine parameter difference.

Several approaches are reasonable, depending on the design and estimand:

ApproachUseful whenMain caution
Naive pooling with no trial termsMeasurement and protocol processes are demonstrably aligned and study differences are negligible for the target.Unmodeled assay or study shifts can bias estimates and make uncertainty too small.
Stratified or fixed study effectsThere are specific trials of interest and enough information to estimate their contrasts.Many effects consume information; confounded effects may not be identifiable.
Hierarchical study effectsSeveral trials plausibly represent a distribution of study deviations.With few trials, the between-study variance can be poorly estimated; shrinkage does not cure assay confounding.
Separate trial analysesAssays or protocols cannot be harmonized, or a common model is not defensible.Estimates may be less precise and direct comparisons may be limited.

A common mistake is to treat a nonsignificant study-effect test as evidence that the trials are identical, then remove all trial terms and pool without qualification. A large p-value may reflect low power, especially with few trials. The correction is to report the estimated contrasts and uncertainty, assess whether they matter scientifically, and use design knowledge and predictive performance alongside the test. A second mistake is to “correct” an assay discrepancy by estimating a separate offset from the same data while also estimating a trial-specific clearance shift, even though trial and assay are completely confounded. Without an identifying source of information, the data cannot determine which explanation is correct.

Pooling also has limits. A shared structural model may not describe all trials; assay calibration may be uncertain; and one schedule may contain little information about a parameter that another schedule estimates well. A pooled estimate can be precise without being robust to these issues. Report the included trials, harmonization choices, study-effect specification, parameter uncertainty, and trial-level predictive checks so that the meaning of the pooled quantity is clear.

Key takeaway: Pool only after defining a common target and representing each trial's measurement and protocol process. Separate assay calibration from pharmacokinetic study effects when the design identifies them, and judge the result by both parameter uncertainty and prediction in trial-specific data.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions.

  1. Why can an assay offset and a trial-specific clearance shift be difficult to distinguish when each trial uses only one unique assay?
  2. A sample was collected at 3.4 hours, but the analysis substitutes the nominal 3-hour time. What feature of the concentration-time curve determines whether this matters?
  3. Two independent trial estimates of the same parameter have standard errors 0.5 and 1.0. What are their inverse-variance weights, and which estimate contributes more to a fixed-effect pooled estimate?
  4. A held-out trial's measured concentration falls inside a 95% predictive interval. What does that establish, and what does it not establish?

Solutions. (1) The observed trial difference can be attributed either to measurement scale or to pharmacokinetic parameters; if assay and trial are perfectly confounded, the data alone may not identify their separate contributions. (2) The error depends approximately on the local slope, since \(f(t_0+\Delta;\theta)-f(t_0;\theta)\approx(\partial f/\partial t)\Delta\); a steep profile is more sensitive to the time substitution. (3) The weights are \(1/(0.5)^2=4\) and \(1/(1.0)^2=1\), so the first estimate receives four times the weight. (4) The observation is compatible with the stated predictive distribution, but one observation does not establish adequate calibration, coverage, or prediction across the trial; those require evaluation across relevant participants and times.

Clinical Biostats