Why Consultation Plans Fail
A regulatory consultation can be carefully prepared and still fail to reduce uncertainty. The problem is often not the scientific model itself, but the interaction plan: the question does not identify a decision, the relevant evidence is missing, the alternative is not credible, or the discussion occurs after the development team has lost the ability to respond. A further risk arises after the interaction, when a limited comment is treated as broader agreement than the agency actually gave.
The earlier material on framing decision-focused questions and interpreting agency meeting records provides useful foundations. Here the emphasis is on spotting failures across the whole consultation plan and repairing them before the interaction. The goal is not to obtain a guaranteed endorsement. It is to make the question answerable, present enough context for a meaningful discussion, and preserve an accurate account of what the feedback does and does not establish.
A sound plan connects four elements: the decision to be made, the model-based evidence relevant to that decision, the agency's specific input being requested, and the action the sponsor could take in response. If any link is missing, even a technically sophisticated briefing package may not support a useful exchange. Agency feedback also has a defined scope: it should be read in light of the exact question, evidence, assumptions, and development context presented, and the status of the interaction and its record.
Four Common Failure Modes
Vague questions ask for general reactions without identifying the decision or the requested judgment. “Does the agency agree with our model?” is difficult to answer if “agree” could mean agreement with the model structure, data, intended use, assumptions, or proposed action. A better question names the decision and asks about a specific element that could change it.
Late engagement occurs when advice is sought only after a consequential choice has been made, or when there is no practical opportunity to address concerns before a milestone. The issue is not simply the number of weeks before a meeting. It is whether there is adequate time to prepare a focused package, discuss the question, and act on feedback before the decision becomes difficult to change.
Unsupported alternatives occur when a plan presents one favored proposal and a nominal alternative without explaining why either is scientifically plausible or what evidence would distinguish them. A useful alternative is not a straw option included to make the preferred option appear inevitable. It should be credible, connected to uncertainty in the evidence or assumptions, and linked to a different decision or next step.
Overinterpretation of comments occurs when a narrow or conditional comment is recast as broad approval. A response about one analysis, population, or assumption is not automatically endorsement of the entire modeling strategy, a future submission, or a claim not discussed. Similarly, a discussion record should be distinguished from a binding decision; the effect and status of agency feedback depend on the type of interaction and applicable framework.
A Practical Review and Repair Sequence
A useful review proceeds from the decision outward. First identify what must be decided and by when. Then identify the model claim that informs that decision, the evidence and assumptions supporting it, and the precise judgment being requested. Finally, make the possible consequences of the response explicit. This sequence is a justification for asking a narrower question: agency advice is most useful when it can be connected to a defined choice rather than an undefined request for approval.
State what development, analysis, or evidentiary choice is pending and the milestone at which it matters.
Describe the model's intended role in the decision, the population or setting to which the claim applies, and the principal assumptions.
Ask for a judgment about a specific proposal, uncertainty, or evidence gap. Avoid bundling several distinct judgments into one yes-or-no question.
Explain what evidence or assumption motivates it, how it changes the decision, and what additional work could distinguish the options.
Confirm that there is time to use the feedback, then plan to record its scope, conditions, unresolved points, and resulting actions accurately.
This sequence does not guarantee agreement. It improves the chance that any disagreement will be informative: the team can identify whether it concerns the question, evidence, assumptions, proposed use, or decision consequence, rather than leaving the source of disagreement obscure.
Worked Example: Replacing a Vague Question
A hypothetical sponsor plans to use a population pharmacokinetic model to support an exposure comparison in a new patient group. Its draft question is: “Does the agency agree with our model and proposed dose?” The package includes model diagnostics but does not say which model feature is uncertain or what answer would change the plan.
Diagnosis. The draft bundles at least two judgments: whether the model is adequate for the intended comparison and whether the proposed dose is supported. It also leaves “agree” undefined. An answer about model fit could be mistaken for endorsement of the dose, even though those are different claims.
Repair. The sponsor states that the pending decision is whether the current model-based exposure comparison is sufficient to support the proposed dose for the specified patient group, or whether additional clinical data are needed. It summarizes the relevant data, model evaluation, intended use, and assumptions that matter to that comparison. It then asks whether the presented evidence is adequate for that specific use and identifies the principal uncertainty: whether the patient-group covariate relationship is sufficiently supported by the available data.
The plan also states what follows from either response. If the evidence is considered insufficient, the sponsor will evaluate a targeted additional analysis or data collection before relying on the comparison. This repair does not assume that the agency will select one option. It makes the question, evidence, uncertainty, and consequence visible, so that a response can be interpreted within its actual scope.
Timing, Alternatives, and Evidence
A consultation is not automatically timely because it occurs before a formal submission. The relevant test is whether the advice can still inform the decision for which it is sought. If a clinical protocol is already finalized, for example, advice on an assumption that would require changing that protocol may arrive too late to be actionable. Conversely, asking before the evidence is mature enough to describe the model's intended use may produce a discussion that is too abstract to resolve the uncertainty. The plan should make the development milestone and remaining opportunities for action explicit.
A credible alternative should be presented with its rationale, not merely its name. For each option, state the supporting evidence, the assumptions on which it depends, the decision it would support, and the cost or limitation of pursuing it. If one option is preferred, explain why without suppressing material uncertainty. Acknowledging that an alternative has limitations is appropriate; presenting it as impossible without evidence is not.
Worked Example: Repairing an Unsupported Alternative
In a hypothetical consultation plan, a sponsor proposes using a model-based extrapolation for a population with limited clinical data. The only alternative listed is “conduct a new trial,” with no explanation of what question the trial would answer or why the current evidence is inadequate.
Diagnosis. The alternative is not decision-ready. It does not identify the uncertainty a trial would address, and it makes the proposed model-based approach appear preferable by default. The agency cannot readily comment on the trade-off between options.
Repair. The sponsor identifies the relevant uncertainty: whether the exposure-response relationship in the target population is sufficiently similar to that in the studied population for the proposed model-based extrapolation to support the intended decision. It describes the available evidence and the assumption of similarity. It then presents two genuine options: use the current model for the specified decision with the stated uncertainty, or collect additional data designed to inform the population difference that drives the extrapolation.
The sponsor asks whether the described evidence and assumptions are adequate for the proposed use, and what evidence gap would be most important to address if they are not. This does not claim that either option is preferred by the agency. It gives the discussion a concrete uncertainty to examine and lets the sponsor relate feedback to its next action.
The alternatives need not be equally attractive, but they should be intellectually honest and sufficiently specified to compare. If no credible alternative can yet be described, that is a signal to do more internal scientific work or to refine the consultation question, rather than to present a false choice.
Interpreting Feedback Without Overclaiming
After an interaction, distinguish what was explicitly addressed from what the team inferred. Record the question as asked, the evidence and assumptions presented, the response, any conditions or qualifications, points not resolved, and follow-up actions. If the response was silent on a topic, silence should not be rewritten as agreement. If feedback depended on a stated assumption, the conclusion should retain that condition.
This discipline also applies when agency comments are favorable. A comment that a particular analysis is informative does not by itself establish that the model is adequate for every proposed use. A comment about the evidence available at one development stage does not automatically settle the sufficiency of evidence at a later stage. The team should preserve the boundary between the agency's expressed view and its own interpretation, especially when communicating internally or summarizing the interaction in later documents.
Worked Example: Bounding an Agency Comment
In a hypothetical meeting record, the agency says that a sensitivity analysis “is helpful in characterizing the effect of the stated assumption.” The sponsor's draft summary says, “The agency agreed that the model supports the proposed dose across all intended populations.”
Diagnosis. The summary expands a comment about one sensitivity analysis and one stated assumption into agreement about the model, dose, and multiple populations. Those broader conclusions were not contained in the quoted comment.
Repair. The sponsor records that the agency considered the sensitivity analysis helpful for characterizing the effect of the stated assumption. It separately lists the model's intended uses and populations that were discussed, notes whether the dose conclusion was explicitly addressed, and labels any remaining inference as the sponsor's interpretation rather than agency agreement.
If the dose or population claim is consequential and remains unresolved, the team identifies it as an open issue and determines what evidence or clarification is needed. The repaired summary is narrower, but more useful: it accurately preserves the feedback and prevents a bounded comment from becoming an unsupported claim of broad endorsement.
Common Pitfalls and Limitations
- Asking for “agreement” without specifying the object. Replace it with the particular model claim, evidence, or proposed use on which input is sought.
- Sending a large package instead of a focused one. More material does not compensate for an unclear decision question. Organize supporting evidence around the uncertainty being discussed.
- Waiting until the preferred plan is effectively irreversible. Check whether meaningful responses can still change the plan at the relevant milestone.
- Offering a token alternative. Give each option a rationale and a decision consequence, or acknowledge that the alternative is not yet sufficiently developed.
- Recording only favorable comments. Preserve qualifications, unanswered questions, and unresolved uncertainty alongside supportive observations.
- Treating a well-prepared consultation as a guarantee. Preparation improves clarity and interpretability; it cannot ensure agreement, predict every agency concern, or replace the applicable regulatory requirements.
Consultation planning also has limits: agency feedback is based on the question and evidence available at the time, and may not resolve scientific uncertainty or determine future conclusions as new data emerge. The plan should therefore specify what is being asked now, not imply that one interaction eliminates the need to reassess the model strategy as development proceeds. The main alternatives to seeking consultation are not interchangeable substitutes: internal scientific review can improve the analysis, while a later decision may require additional evidence. Neither should be represented as agency input.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions below.
- A team asks, “Is our model acceptable?” Name two clarifications that would make the request more answerable.
- Why can a consultation held before submission still be too late to be useful?
- What information should support an alternative to the sponsor's preferred model-based approach?
- An agency comments favorably on one diagnostic but does not discuss the proposed dose. What may the sponsor accurately conclude?
- A meeting record contains a conditional comment that assumes a particular population definition. What should the sponsor preserve when summarizing the feedback?
Solutions. (1) State the pending decision and the model's specific intended use, then identify the evidence or assumption on which input is requested. (2) The relevant plan may already be difficult to change, leaving no practical opportunity to address feedback before the decision milestone. (3) Describe its scientific rationale, supporting evidence, key assumptions, decision consequence, and how it could address the uncertainty. (4) Only that the agency commented favorably on the diagnostic, within the context presented; the sponsor cannot infer agreement with the dose unless that was explicitly addressed. (5) Preserve the population definition and the conditional nature of the comment, and distinguish unresolved issues or sponsor interpretation from the agency's stated view.