Why Reviewer Responses Need a Method
A reviewer comment is not automatically a finding that the analysis is wrong, and a response is not merely a polite explanation of why the report is right. A comment may identify an actual error, ask for information that is already available, request a new analysis, challenge an interpretation, or ask for a conclusion that the data cannot support. Treating these requests as interchangeable can lead to unnecessary analyses, unsupported claims, or revisions that do not match the evidence.
A good response process makes the reasoning visible: identify what the reviewer is asking, check the relevant evidence, decide whether an analysis or report change is warranted, and record the outcome. The earlier material on creating an audit trail from data to conclusions provides the general traceability principle. Here the emphasis is on applying it to reviewer comments, including cases where the appropriate response is to clarify, qualify, or explain why a requested analysis is not justified.
Classify the Request Before Acting
Begin by paraphrasing the comment as a question or requested action. Do not assume that the wording identifies the underlying issue. For example, “Please justify the covariate model” might mean that the selection rationale is missing, that the reviewer doubts a particular effect, or that an additional sensitivity analysis is being requested. These require different responses.
A practical classification is shown below. A comment can belong to more than one category; record the primary request and any related requests separately.
| Comment type | What the reviewer is asking | First response task |
|---|---|---|
| Clarification | Explain a method, result, definition, or decision that is unclear or absent. | Check whether the information is already stated and whether it can be made clearer. |
| Evidence request | Show support for a claim or provide a relevant diagnostic, estimate, table, or reference. | Locate the evidence and verify that it addresses the claim being made. |
| Analysis request | Perform an additional analysis, sensitivity check, or alternative evaluation. | Assess scientific relevance, data availability, assumptions, and interpretability before proceeding. |
| Revision or correction | Correct an error, revise wording, qualify a conclusion, or change a figure or table. | Verify the issue, make the change if warranted, and check downstream consistency. |
| Scope or disagreement | Address an issue beyond the report’s stated scope, or respond to a premise the authors do not share. | State the scope or disagreement precisely and support it without dismissing the concern. |
Classification is a planning step, not a judgment about whether the reviewer is correct. A clear clarification request may reveal a genuine reporting omission; a request for a new analysis may turn out to be unnecessary once the relevant evidence is checked. Keep the reviewer’s requested action distinct from the scientific question that action is meant to answer.
A Traceable Response Workflow
For each comment, preserve the original wording, assign an identifier, and record a decision and its rationale. If a comment contains multiple requests, split it into numbered sub-items so that each one receives a response. Then connect the response to evidence, analysis files, and specific report locations. This structure makes omissions visible and helps another analyst reproduce what was done.
Record the specific concern and the action requested. Separate compound comments into individually answerable points.
Check the relevant data, model outputs, code or run records, and current report text. Do not accept or reject the reviewer’s factual premise without checking it.
Decide whether to correct, clarify, add evidence, perform a justified analysis, qualify the interpretation, or explain why a request is outside scope or unsupported.
For new analyses, record the question, data and model versions, methods, assumptions, outputs, and deviations from the planned approach. For text-only changes, record the revised wording and its location.
Check that reported values, interpretations, figure or table references, and section locations agree across the response letter, final report, and supporting materials.
The disposition should be explicit. “Addressed” is not enough if a reviewer cannot tell whether the team ran an analysis, changed a statement, or simply supplied an explanation. Conversely, a response need not claim a report revision when none was warranted. State what was done and where, or state why no change was made.
Worked Example: Clarifying an Existing Covariate Result
Hypothetical situation. A reviewer writes, “The report states that body weight affects clearance, but it is not clear how the effect was represented. Please clarify.” The final model report gives a covariate relationship in its methods section, but the equation’s reference weight and the interpretation of the exponent are not explained in the results text.
Classification and check. This is primarily a clarification request, not automatically a request to refit the model. The team checks the model specification and the existing output. They confirm that the fitted relationship was clearance for an individual with weight \(WT_i\) equal to typical clearance times \((WT_i/70)^{0.75}\), with weight in kilograms. The value 0.75 is the fixed allometric exponent in this hypothetical model; it is not presented as an estimated covariate effect.
Action and response. The team revises the results text to say that clearance scales with body weight relative to 70 kg using a fixed exponent of 0.75. It also clarifies that the typical clearance parameter corresponds to 70 kg, subject to the other model covariates. The response identifies the revised section and states that no refit was performed because the requested issue was an explanation of the existing specification.
Consistency check. The equation, reference value, units, and wording are checked against the model file and methods section. If the report instead described 0.75 as estimated from these data, that statement would need correction: the hypothetical specification says the exponent was fixed.
When a New Analysis Is Requested
A request for additional analysis deserves a scientific decision, not automatic compliance and not automatic refusal. Ask what uncertainty the proposed analysis would address, whether the available data can answer that question, what assumptions the analysis requires, and whether its result could change the interpretation. If the request is relevant and feasible, specify the analysis before inspecting its result where practical. This reduces the risk of choosing a method because it produces a preferred answer.
When an analysis is performed, the response should identify its purpose and enough details to make it interpretable: the data and model version used, the method, any material assumptions, the result, and the implications for the report’s conclusion. A numerical result without context is rarely a sufficient answer. If the analysis does not alter the conclusion, state that accurately; do not imply that a nonsignificant or unchanged result proves the model is correct.
If a request cannot be completed, explain the constraint specifically. “Not possible” is less informative than stating that the relevant measurements were not collected, the sample is too limited for the requested subgroup comparison, or the proposed analysis would not address the stated question. Where appropriate, identify what can be concluded from available evidence and what remains uncertain.
Worked Example: Responding to a Sensitivity-Analysis Request
Hypothetical situation. A reviewer asks whether the estimated renal-function effect on clearance depends on excluding two participants with unusually high measured renal function. The report’s primary analysis includes both participants and describes the estimated effect as uncertain.
Classification and decision. This is an analysis request. The question is whether those observations have disproportionate influence on the estimated relationship, not whether excluding data can produce a more favorable estimate. The team checks that the two observations are valid and were handled according to the prespecified data rules. Because the observations are valid and the influence question is scientifically relevant, the team conducts a clearly labelled sensitivity analysis excluding them while retaining the primary analysis unchanged.
Hypothetical results. In the primary analysis, the estimated renal-function exponent is 0.42, with a 95% confidence interval from 0.08 to 0.76. In the sensitivity analysis, it is 0.38, with a 95% confidence interval from 0.01 to 0.75. These numbers are hypothetical. The estimates are similar, while both intervals remain broad and include effects of meaningfully different magnitudes.
Response and reporting. The team reports both estimates and intervals, identifies the sensitivity analysis and the two exclusions, and states that the point estimate changed by 0.04 but the uncertainty remains substantial. It does not claim that the sensitivity analysis proves the effect is robust in every respect. The response points to the added results and explains that the primary analysis still includes all valid observations.
Why this is adequate. The reviewer’s influence concern is directly examined, the reason for the alternative analysis is stated, and the primary result is not silently replaced. The report and response distinguish a sensitivity result from the principal analysis.
Drafting a Clear and Proportionate Reply
A useful reply is direct, respectful, and specific. It ordinarily contains four elements: a concise acknowledgment of the issue, the evidence or reasoning, the action taken or reason for no action, and the location of any change. Avoid lengthy advocacy when a short factual explanation is enough. If disagreeing, address the scientific point rather than the reviewer’s presumed intent.
Use verbs that accurately describe the work. “We clarified” indicates a wording change; “we re-estimated” indicates a new fit; “we evaluated” should be followed by what was evaluated and how. Avoid saying that a request was “addressed” when the reply gives no disposition. Do not describe an analysis as prespecified if it was prompted by review.
Worked Example: Explaining Why a Requested Subgroup Estimate Is Not Reported
Hypothetical situation. A reviewer requests a separate exposure estimate for a subgroup containing six participants and asks that it be compared with the estimate for the rest of the study population. The underlying model was developed for the combined population, and no subgroup-specific model was prespecified.
Classification and assessment. This is an analysis request with a scope and interpretability concern. The team checks subgroup membership and available concentration data, then determines that six participants do not support a stable standalone model with the planned structural and variability components. Producing a point estimate is technically possible, but that fact alone would not make the estimate reliable or clinically interpretable.
Disposition. The team does not report a standalone subgroup estimate. Instead, it clarifies the subgroup’s representation in the available data and explains that a separate estimate would be highly uncertain and sensitive to individual observations. It points to the population-level evaluation already reported and states that the available data do not establish a precise subgroup-specific exposure distribution.
Response wording. A concise reply could say: “We agree that the subgroup is relevant. We did not fit a separate subgroup model because only six participants contributed data, which is insufficient to support a stable estimate with the model structure used here. We have clarified the subgroup representation and the limitation of inference in the revised limitations section.” The wording acknowledges the concern without implying that the population-level result resolves the subgroup question.
Reconciliation, Diagnostics, and Common Pitfalls
Before finalizing the response, conduct a comment-by-comment reconciliation. This is a practical diagnostic for an incomplete or internally inconsistent review cycle. Check that every comment has a disposition; every claimed change exists in the final files; every cited location is accurate; every new result is traceable to the specified analysis; and each figure, table, estimate, and interpretation agrees across documents. If a response says that an analysis was added but the report has no corresponding result, the review record is not complete.
Several approaches are useful for different purposes. A point-by-point response is best for showing that each comment was considered and is usually the clearest primary format. A narrative summary can help explain broad changes, but it should supplement rather than replace comment-level replies. An analysis appendix can hold technical detail, but the response must still summarize the result and point to the appendix. A tracked-changes file can show edits, but it does not explain why a scientific decision was made.
The central pitfall is treating reviewer agreement as the objective. For example, a team might remove valid observations, change a model, or overstate a conclusion simply to satisfy a requested interpretation. The correction is to evaluate the scientific premise, preserve the primary analysis unless there is a reason to change it, and explain any alternative analysis as such. A related mistake is the opposite: answering defensively without checking whether the report actually contains the requested information. Review is an opportunity to improve both the analysis and its communication, not a contest over who is right.
Other warning signs include vague dispositions (“considered”), unsupported claims that a result is “robust,” unexplained differences between response and report values, and reference to a report section that was renumbered during revision. These are process failures even when the underlying model is sound. A final read should therefore compare the response letter with the exact final report and archived analysis outputs, not with an earlier draft.
Check Your Understanding
Answer each question, then compare your reasoning with the brief solutions below.
- A reviewer says a figure is difficult to interpret and asks what the shaded band represents. How should this comment be classified, and what should the response establish?
- A reviewer requests a new analysis after noticing two extreme but verified observations. Why should the primary analysis not be silently replaced by an analysis excluding them?
- A response says, “We added the requested table,” but the final report contains no such table. What has failed, and what should be checked?
- A reviewer asks for a precise subgroup estimate, but the subgroup has very sparse data. Give two elements of a scientifically responsible response.
Solutions. (1) It is a clarification request. Check what the band represents, confirm that interpretation against the figure-generating output, and label or explain it in the figure or caption and response. (2) The exclusion analysis answers a different, sensitivity question and could change the result; retain the justified primary analysis, report the alternative transparently, and explain how the estimates compare. (3) The response and final report are inconsistent. Confirm whether the table was omitted, moved, or the response refers to an outdated draft; correct the report or revise the reply and verify the final location. (4) Explain the data limitation and why a separate estimate would be unstable or difficult to interpret; state what available evidence can and cannot support, and avoid presenting a precise subgroup conclusion that the data do not justify.