Why Influence Matters in Scaled Bioequivalence
In reference-scaled bioequivalence, an unusual subject can affect more than one part of the analysis. Their concentration profile may shift the estimated treatment difference, change the estimate of within-subject reference variability, or do both. Because reference variability determines the scale of the criterion, a change in that estimate can alter the apparent evidence for bioequivalence even when the treatment mean is unchanged.
An influence check asks a focused question: how much does the fitted analysis change when a particular subject’s data are omitted for diagnostic purposes? The omission is a sensitivity calculation, not a proposed primary analysis. A large change is a reason to inspect the data, design, model, and analysis record. By itself, it is not evidence that the observation is erroneous or that the subject should be excluded.
The focus here is on log-transformed pharmacokinetic endpoints in replicate designs. The earlier material on the FDA linearized reference-scaled criterion establishes the population expression \(\delta^2-\theta\sigma_{WR}^{\,2}\), where \(\delta\) is the treatment difference on the log scale, \(\sigma_{WR}^{\,2}\) is the within-subject reference variance, and \(\theta=(\ln(1.25)/0.25)^2\approx0.7967\). A larger reference variance makes this expression smaller, while a larger absolute treatment difference makes it larger. These opposing effects make influence analysis important.
Conditions for a Meaningful Influence Check
A leave-one-subject-out comparison is interpretable only relative to a defined analysis. Use the same endpoint, transformation, treatment coding, variance model, and estimation procedure in the full-data fit and each diagnostic refit. In a mixed model, refit the model after omitting the subject; do not merely delete a row while retaining variance estimates from the original fit.
The ordinary sample-variance calculation used below is a transparent illustration for a simple balanced setting. Let \(d_i\) be the difference between two log reference measurements from subject \(i\). If the two measurements have the same within-subject variance, are independent conditional on the model effects, and the differences across subjects are independent with a common mean and variance \(2\sigma_{WR}^{\,2}\), then the sample variance of the \(d_i\) values estimates \(2\sigma_{WR}^{\,2}\). Thus the simple estimate is \(s_d^2/2\). Actual reference-scaled analyses may use a specified mixed model and REML; in that setting, this calculation is a diagnostic illustration, not a replacement for the prescribed model.
This simple estimate assumes the replicate differences represent the same reference within-subject variability and are not distorted by data errors, unmodelled period or sequence effects, or an inappropriate analysis set. In a real study, inspect the design and model assumptions before interpreting a large leave-one-out change. The residual diagnostics covered earlier are useful for finding unusual residual patterns, but a residual diagnostic and an influence check answer different questions: one examines model fit, while the other measures the effect of a subject on fitted results.
Worked Example: An Extreme Reference Difference
Suppose six hypothetical subjects in a simplified replicate design have paired log-reference differences \(d_i=(-0.20,-0.10,0.00,0.10,0.20,1.00)\). Treat these values as independent, comparable replicate differences for this illustration. Calculate the simple estimate of \(\sigma_{WR}^{\,2}\) with all subjects and then omit the subject with difference \(1.00\) for a diagnostic comparison.
All six subjects. The sum of the differences is \(1.00\), so \(\bar{d}=1.00/6=0.1667\). The sum of squares about zero is \(0.04+0.01+0+0.01+0.04+1.00=1.10\). Using the identity \(\sum(d_i-\bar{d})^2=\sum d_i^2-n\bar{d}^{\,2}\), the sum of squared deviations is \(1.10-6(0.1667)^2\), or more exactly \(1.10-1.00^2/6=0.9333\). Therefore \(s_d^2=0.9333/5=0.1867\), and \(\widehat{\sigma}_{WR}^{\,2}=0.1867/2=0.09333\) log-units squared.
Omitting the subject with difference \(1.00\). The five remaining differences sum to zero, and their squared deviations sum to \(0.04+0.01+0+0.01+0.04=0.10\). Thus \(s_d^2=0.10/4=0.02500\), and the corresponding estimate is \(0.02500/2=0.01250\) log-units squared.
The diagnostic estimate is substantially smaller without that subject. The result shows sensitivity, not grounds for exclusion. Before drawing any conclusion, check the original concentration-time records, calculations, sample handling, protocol deviations, and whether the simple variance calculation is appropriate to the actual design.
Influence on the Treatment Difference
A subject can also pull the treatment contrast. In a simple complete paired comparison, let \(c_i\) be subject \(i\)'s log test-minus-reference contrast. The mean contrast is \(\bar{c}=\sum c_i/n\). This is a convenient diagnostic example; a more complex design requires the treatment estimate from the specified model, with the model refitted after each subject is omitted.
Worked Example: One Subject Shifts the Log Contrast
Consider six hypothetical subject-level log contrasts \(c_i=(-0.04,-0.02,0.00,0.02,0.04,0.20)\). Assume the paired contrasts are the appropriate summary for this simplified complete design. Calculate the mean contrast and its diagnostic value after omitting the subject with contrast \(0.20\).
With all subjects, the sum is \(0.20\), so \(\bar{c}=0.20/6=0.03333\) log units. The geometric mean test-to-reference ratio corresponding to this contrast is \(\exp(0.03333)=1.0339\), or about 103.4%.
After omitting the subject with contrast \(0.20\), the remaining contrasts sum to \(-0.04-0.02+0+0.02+0.04=0\). Their mean is therefore \(0/5=0.000\) log units, corresponding to a ratio of \(\exp(0)=1.000\), or 100.0%.
The single subject changes the estimated log contrast by \(0.03333\) log units in this illustration. This does not imply that the subject’s data are wrong: the change could reflect genuine subject variability, an unusual but valid profile, a data-processing issue, or a mismatch between the model and the design. It does show why reporting only the full-data mean can hide an important sensitivity.
In an actual scaled analysis, influence may be less obvious than one extreme paired contrast. A profile can affect model residuals, leverage, variance-component estimates, or the degrees of freedom used by the prescribed procedure. For that reason, use subject-level refits of the actual analysis rather than relying only on a visual judgment about which concentration appears most extreme.
How Influence Can Alter a Scaled Criterion
The point expression \(\delta^2-\theta\sigma_{WR}^{\,2}\) clarifies the direction of two effects. Holding the reference variance fixed, a shift of \(\delta\) away from zero increases \(\delta^2\), moving the expression upward. Holding \(\delta\) fixed, an increase in \(\sigma_{WR}^{\,2}\) decreases the expression, moving it downward. Thus an extreme profile that raises both the absolute treatment difference and estimated reference variance has competing effects. The net direction cannot be inferred from “the variance went up” or “the mean shifted” alone.
Worked Example: A Variance-Only Sensitivity Calculation
Use the six reference differences from the first example and suppose, solely to isolate the variance contribution, that the log treatment difference is fixed at \(\delta=0.1000\) in both calculations. This fixed value is a teaching device: in a genuine leave-one-subject-out analysis, the treatment estimate must also be recalculated if the omitted subject contributed to it.
With all six reference differences, \(\widehat{\sigma}_{WR}^{\,2}=0.09333\). The plug-in point expression is \(0.1000^2-0.7967(0.09333)=0.01000-0.07436=-0.06436\).
With the subject having reference difference \(1.00\) omitted, \(\widehat{\sigma}_{WR}^{\,2}=0.01250\). Holding \(\delta\) fixed as stipulated, the expression is \(0.1000^2-0.7967(0.01250)=0.01000-0.009959=0.0000413\).
The plug-in expression moves from negative to slightly positive because the reference variance estimate falls. This demonstrates how the variance component can change the direction of a point-criterion comparison. It does not establish a regulatory bioequivalence outcome. A formal FDA decision uses the applicable upper confidence bound, the point-estimate constraint, and the specified analysis. The point expression alone is not that decision procedure.
A real influence check must vary the subject’s contribution jointly. Refit the model after omitting that subject, then compare the treatment estimate, reference variance estimate, uncertainty quantities, and final upper confidence bound with the full-data results. The upper confidence bound can respond to changes in both estimates and their uncertainty; its movement cannot be deduced solely from the point-expression calculation above.
A Practical Influence-Assessment Sequence
The primary analysis remains the analysis defined for the study. A disciplined sensitivity assessment records what changes and why, without turning an exploratory deletion into a new primary analysis.
Check units, sample identity, assay and data-processing records, concentration-time ordering, and any documented protocol deviations. Correct a confirmed error through the study’s data procedures, not by an informal statistical deletion.
Record the treatment estimate, reference variance estimate, relevant uncertainty measures, and scaled-analysis result under the prespecified method.
Use identical analysis choices and omit a subject’s complete contribution, not selected measurements chosen to produce a preferred result.
Identify which subjects materially shift the treatment contrast, reference variance, upper confidence bound, or decision. Interpret the direction of each change in context.
Give the primary result and the diagnostic comparison, explain the reason for any data correction or authorized exclusion, and avoid presenting the most favorable refit as the primary result.
Alternatives, Diagnostics, and Limitations
A residual diagnostic identifies observations or patterns that fit the model poorly; a leave-one-subject-out analysis measures how dependent results are on a subject. Neither replaces the other. A large residual may have little influence in a large dataset, while a moderate residual may be influential when the sample is small or the variance estimate is unstable. The earlier material on instability of reference variance estimates in small samples is relevant: when the effective information is limited, one subject can have substantial leverage on a variance component.
Other possibilities include robust estimators or alternative variance models. These can be useful sensitivity analyses when justified, but they answer questions under different assumptions and should not silently replace the required primary method. Sensitivity to variance-model specification is a separate issue: an influence check holds the model specification fixed while varying a subject’s contribution.
Common warning signs include a large change in the reference variance estimate after one omission, a treatment estimate that changes materially, convergence problems in a refit, or a scaled result that changes category. Investigate these signs rather than treating a mechanical threshold as an exclusion rule. Leave-one-out estimates are correlated diagnostic results, and selecting the omitted subject that gives the preferred outcome creates selection bias. Moreover, a diagnostic refit with one fewer subject does not automatically have the same inferential interpretation as the prespecified full-data analysis.
The main limitation is that influence analysis diagnoses dependence on observations; it does not identify the cause of that dependence. Small sample size, genuine biological heterogeneity, data error, and model misspecification can all produce sensitivity. Conclusions therefore require statistical results alongside clinical and operational review.
Check Your Understanding
Answer each question, then compare with the solutions.
- For replicate reference differences, why is the simple variance estimate \(s_d^2/2\), and what assumptions support it?
- For the differences \((-0.20,-0.10,0.00,0.10,0.20,1.00)\), what is the sum of squares about zero and the simple reference variance estimate?
- If \(\delta=0.08\), how does the point expression change when \(\widehat{\sigma}_{WR}^{\,2}\) rises from \(0.020\) to \(0.050\), using \(\theta=0.7967\)? Does this alone establish bioequivalence?
- A leave-one-subject-out refit changes the upper confidence bound enough to change the apparent decision. What should be concluded and reported?
Solutions. (1) Under the simple model, the difference between two independent reference measurements has variance \(2\sigma_{WR}^{\,2}\). Its sample variance estimates that variance, so dividing by two estimates \(\sigma_{WR}^{\,2}\). The differences must be independent across subjects, comparable, and consistent with the model’s common variance assumptions. (2) The sum of squares about zero is \(0.04+0.01+0+0.01+0.04+1.00=1.10\). The centered sum of squares is \(1.10-1.00^2/6=0.9333\); hence \(s_d^2=0.9333/5=0.1867\), and the estimate is \(0.09333\) log-units squared. (3) At variance \(0.020\), the expression is \(0.08^2-0.7967(0.020)=0.006400-0.01593=-0.009534\). At variance \(0.050\), it is \(0.006400-0.03984=-0.03344\). Increasing the variance moves this point expression downward. Neither value alone establishes bioequivalence because the formal procedure also uses the specified uncertainty calculation and point-estimate constraint. (4) Treat the change as evidence that the result is sensitive to that subject, not as grounds for automatic exclusion. Verify the data, retain the prespecified primary analysis unless a justified correction or exclusion applies, and report the full-data result and the diagnostic refit with the model and decision changes clearly described.