Why Sensitivity Analysis Matters
A protocol deviation can affect how a bioequivalence result should be interpreted. A dosing-time error, an unplanned meal, or a missed procedure may alter exposure or make a period less comparable with the intended study conditions. Even when the deviation is documented and the data remain usable, it is reasonable to ask whether the conclusion depends on including the affected observation.
Sensitivity analysis addresses that question by repeating the analysis under one or more reasonable alternatives. It does not authorize replacing the prespecified primary analysis after seeing the results. Nor does a favorable sensitivity result erase a problematic primary result. The aim is to understand how conclusions respond to a clearly described change in assumptions or data handling.
The primary analysis and sensitivity analyses answer related but distinct questions. The primary analysis answers the question defined by the protocol’s analysis plan. A sensitivity analysis asks whether a particular concern—for example, an affected dosing period—materially changes the estimate, confidence interval, or bioequivalence decision.
Set Up the Primary Analysis Before Comparing Results
For a conventional crossover assessment of a pharmacokinetic endpoint such as AUC or Cmax, analysis is commonly performed on the natural logarithm of the endpoint. The treatment contrast is the estimated mean difference between test and reference on the log scale. Exponentiating that contrast gives the geometric mean ratio (GMR), test divided by reference. The earlier material on complete 2×2 crossover analysis establishes the design and model structure; here the emphasis is on how a deviation affects interpretation of the analysis.
Before fitting or comparing models, state what the prespecified analysis uses: the endpoint, analysis population, treatment contrast, model, confidence-interval method, and rule for handling the deviation. The precise population and procedures depend on the protocol and applicable requirements. A deviation should not automatically trigger exclusion: its relevance depends on what happened, which data it could affect, and the prespecified rules.
For the usual average bioequivalence criterion, the 90% confidence interval for the test-to-reference ratio must lie wholly within 0.80 to 1.25. If \(\hat{\delta}\) is the estimated log-scale treatment contrast, \(SE\) its standard error, and \(t_{0.95,df}\) the 95th percentile of the \(t\) distribution with the model’s degrees of freedom, the interval is:
Exponentiating both endpoints gives the confidence interval for the GMR. Equivalently, the log-scale interval must lie within \(\log(0.80)\) and \(\log(1.25)\). Passing means the entire interval is contained within the limits; it does not mean the true ratio has a 90% probability of being in this particular interval.
This calculation relies on a suitable model for the log endpoint and its uncertainty: the treatment contrast must be correctly specified, observations must be analyzed according to the design, and the model’s error assumptions must be reasonably adequate. For a standard linear-model interval, these include independent subject-level errors and an adequate normal, common-variance error approximation. The study’s design and analysis plan determine the specific model and degrees of freedom. A sensitivity analysis should not silently change those choices.
Choosing a Defensible Sensitivity Analysis
A useful sensitivity analysis starts with a question about a documented deviation, not with a desired result. For each analysis, identify the deviation, the data or assumption it could affect, the single change being made, and why that alternative is reasonable. Keep all other aspects of the analysis as close to the primary analysis as possible. If several features change at once, it becomes difficult to explain why the result changed.
For example, if a subject received an unplanned meal during one period, a justified sensitivity analysis might omit that subject or the affected period if the protocol permits and the analysis method supports it. Another might retain the observation in the primary analysis but examine the result after excluding the affected data. These are not interchangeable: removing a whole subject changes more information than removing one affected period, and a model must be appropriate for the remaining data. The rationale and exact rule should be reported.
The analysis should also preserve the study’s treatment comparison. A post hoc subgroup formed from subjects whose results look similar is not a defensible sensitivity analysis. Neither is trying many exclusions and reporting only the one that passes. The alternatives should be limited, scientifically motivated, and interpreted together with the primary analysis.
Worked Example: Primary Analysis Passes With a Documented Meal Deviation
In a hypothetical 2×2 crossover study, the protocol-specified primary analysis includes 24 subjects with evaluable endpoint data. One subject ate a snack earlier than instructed in the test period. The protocol’s prespecified handling rule retains the subject in the primary analysis. The fitted model reports a log-scale test-to-reference contrast of \(\hat{\delta}=0.0180\), \(SE=0.0250\), and \(df=21\). Assume the model and its error approximation are adequate for this illustration.
Using \(t_{0.95,21}=1.721\), the 90% interval on the log scale is:
Exponentiating the limits gives:
The point estimate is \(\exp(0.0180)=1.018\), or a GMR of 1.018. The entire 90% interval, from 0.9753 to 1.0630, is inside 0.80 to 1.25, so this primary analysis meets the stated average bioequivalence criterion. The result is interpreted as a test-to-reference ratio estimated at 1.018, with a 90% confidence interval from 0.9753 to 1.0630—not as proof that every individual’s exposure is similar.
The model output alone does not establish that the meal deviation is harmless. The record should be checked to confirm when the snack was eaten, whether the deviation could affect the endpoint, and whether the protocol’s handling rule was followed. A sensitivity analysis can then assess whether the conclusion changes under a justified alternative.
Compare Analyses on Estimates and Decisions
A sensitivity analysis should be compared with the primary analysis using more than a pass/fail label. Examine the log-scale contrast, its standard error, the 90% interval, the exponentiated GMR interval, and the reason for any change. A similar point estimate but wider interval often suggests reduced precision. A shifted point estimate suggests that the changed data or assumption affects the estimated treatment contrast. Either pattern can matter.
Worked Example: Excluding the Affected Subject
Continue the hypothetical study above. A justified sensitivity analysis excludes the subject with the documented meal deviation entirely. The refitted model reports \(\hat{\delta}=0.0110\), \(SE=0.0270\), and \(df=20\). Assume the reduced analysis still satisfies the model’s requirements and that exclusion is consistent with the analysis plan’s permitted sensitivity procedures. With \(t_{0.95,20}=1.725\):
On the ratio scale, the interval is:
This sensitivity analysis also meets the 0.80 to 1.25 criterion. The estimate changes from a GMR of 1.018 to \(\exp(0.0110)=1.011\), while the confidence interval remains inside the limits. The two analyses therefore support the same bioequivalence decision under these hypothetical assumptions. They do not prove that the deviation had no effect: they show that this specified exclusion did not change the decision in this example.
Use the documented event, including its timing, affected period, and possible relevance to the endpoint. Do not infer its impact solely from the statistical result.
Identify the prespecified population, model, and handling rule, then report the primary estimate and interval using that rule.
Explain exactly what changes and why that change addresses the deviation. Avoid outcome-driven exclusions or a collection of unmotivated analyses.
Compare estimates, uncertainty, and the bioequivalence decision. If the decision differs, investigate the source and describe the conclusion as sensitive to the analysis choice.
When the Conclusion Is Not Robust
Suppose the primary analysis meets the criterion but a reasonable sensitivity analysis does not. That disagreement means the conclusion depends on the treatment of the deviation or on an assumption changed in the sensitivity analysis. It does not mean the sensitivity analysis automatically replaces the primary analysis, and it does not establish which result is “correct.” The reason for the difference must be investigated and reported.
Worked Example: A Sensitivity Analysis Changes the Decision
Consider a separate hypothetical study with a prespecified primary analysis reporting \(\hat{\delta}=0.0180\), \(SE=0.0250\), and \(df=21\). As calculated above, its 90% ratio interval is 0.9753 to 1.0630, within the bioequivalence limits. Three subjects had protocol deviations involving study conditions. A justified sensitivity analysis excludes those subjects; the remaining data produce \(\hat{\delta}=0.100\), \(SE=0.0750\), and \(df=18\). Assume this is a reasonable alternative for the stated question and that its model assumptions are adequate. With \(t_{0.95,18}=1.734\):
The ratio interval is:
The upper limit, 1.259, exceeds 1.25, so this sensitivity analysis does not meet the criterion. The primary result remains the prespecified primary result, but the disagreement indicates that the bioequivalence conclusion is not robust to this particular, justified exclusion. The substantial shift in the estimated contrast warrants checking the subject-level records, treatment allocation, deviation timing, and model fit. The result should be reported transparently; it would be inappropriate to select whichever analysis gives the preferred conclusion.
Diagnostics, Alternatives, and Limitations
Before attributing a changed result to the deviation, verify the analysis inputs. Confirm that each subject’s treatment, period, sequence, endpoint value, and inclusion status match the data records and analysis rules. Check whether the sensitivity analysis changed more than intended, whether any observations became unusable under the model, and whether the model’s residual and variance behavior remains plausible. When the conclusion changes, inspect whether the change comes mainly from the point estimate, from increased uncertainty after removing data, or from both.
The main alternative to a sensitivity analysis is to rely on the primary analysis alone. That keeps inference aligned with the prespecified plan, but it does not show whether a documented concern could alter the conclusion. Conversely, an alternative analysis alone cannot replace the prespecified analysis without a defensible reason. Reclassifying subjects after inspecting outcomes, or searching across exclusions until one analysis passes, can produce biased conclusions and hides the uncertainty due to the analysis choice.
Sensitivity analyses have limits. Agreement across a small number of analyses does not prove that the study is free of bias, and a failure in one analysis does not by itself identify the cause. Results depend on the quality of deviation records, the scientific plausibility of the alternative, the amount of data affected, and whether the model remains appropriate. The primary analysis, the rationale for each sensitivity analysis, and all decision-relevant results should therefore be presented together.
Common Mistake
A related mistake is to treat any protocol deviation as an automatic reason for exclusion. A deviation may be immaterial to the endpoint, or the protocol may specify that the resulting data remain usable. The correct decision depends on the prespecified rules and the deviation’s documented relevance, not on a blanket rule or on which treatment comparison is more favorable.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- A primary log-scale contrast is 0.020 with \(SE=0.030\) and \(df=20\). Using \(t_{0.95,20}=1.725\), calculate the 90% ratio interval and state whether it meets the 0.80 to 1.25 criterion.
- A sensitivity analysis has nearly the same point estimate as the primary analysis but a wider confidence interval after excluding several subjects. What is one reasonable interpretation?
- Why is it not defensible to try many exclusions and report only the one that meets the bioequivalence criterion?
- The primary analysis passes, but a justified sensitivity analysis fails. What should the analyst conclude about robustness?
Solutions. (1) The log interval is \(0.020\pm1.725(0.030)=0.020\pm0.05175=[-0.03175,0.07175]\). Exponentiating gives approximately \([0.9687,1.0744]\), which lies within 0.80 to 1.25, so the criterion is met. (2) The treatment estimate appears similar, while reduced information may have increased uncertainty; inspect the standard error and model fit before attributing the change. (3) Selecting an analysis because it passes is outcome-driven and can conceal uncertainty and bias the reported conclusion. (4) The prespecified primary result remains primary, but the conclusion is sensitive to the stated alternative and should be reported with an investigation of why the analyses differ.