Why Dose Justification Needs an Evidence Matrix
A proposed dose is rarely supported by one result alone. Evidence may come from different studies, endpoints, assessment times, dose regimens, and analysis populations. A concise summary can help decision-makers see the pattern, but only if it preserves where each result came from and what it measures. Otherwise, a favorable endpoint at one time point can be mistaken for a general efficacy conclusion, or a safety percentage from one population can be compared with a percentage from another as if they were directly interchangeable.
An evidence matrix is a structured record of the findings relevant to a dose decision. Its purpose is not to calculate a new treatment effect or combine unlike results. It makes the evidence traceable: a reader can move from a proposed dose or decision criterion to the relevant study result, its uncertainty, and the limitations that affect its interpretation. The earlier material on dose-response estimands helps define the target effect; the matrix records the evidence that speaks to that target.
A row should represent one result, not a whole study. A study with two doses, several endpoints, and multiple assessment times will ordinarily contribute several rows. That granularity prevents a single label such as “positive study” from hiding which dose, endpoint, or time point supported the conclusion. The matrix is a decision aid, not a substitute for the protocol, statistical analysis plan, detailed results, or clinical interpretation.
Designing Rows That Preserve Meaning
Before entering numbers, define the decision question and the evidence fields needed to answer it. A useful minimum set is shown below. Additional fields may be needed for a particular program, but removing context to make a table shorter can make a result misleading.
| Field | What to record | Why it matters |
|---|---|---|
| Evidence source | Study identifier, phase, design, and relevant population | Shows where the result came from and how it was obtained |
| Dose and comparator | Dose level, regimen, and control or reference group | Identifies the actual contrast supporting the dose argument |
| Endpoint and time point | Measure, scale, direction of benefit, and assessment time | Prevents unlike outcomes or times from being treated as equivalent |
| Result | Estimate or event counts, with units and analysis population | Allows the reader to understand the magnitude and denominator |
| Uncertainty | Confidence interval or other prespecified uncertainty summary | Shows the range of values compatible with the analysis and assumptions |
| Interpretation and limitations | Relation to the decision criterion, missingness, precision, or other caveat | Separates the observed evidence from the judgment made about it |
State a consistent direction convention for each endpoint. For an endpoint where larger values are better, a treatment-minus-comparator contrast may be positive when favorable. For an endpoint where smaller values are better, the same subtraction may be negative when favorable. One can retain the reported contrast and explicitly label its direction, or define a favorable-oriented contrast for display. If transforming the sign, document the transformation and retain the original endpoint units. Do not make a safety risk difference look favorable merely by reversing its sign without explaining that convention.
Worked Examples
Worked Example: Orienting an Efficacy Contrast
Suppose a hypothetical randomized, placebo-controlled study evaluates a 10 mg daily dose in adults with a chronic condition. The endpoint is change from baseline in a symptom score at Week 12; lower scores are better. The hypothetical mean changes are \(-5.1\) points for 10 mg and \(-2.0\) points for placebo. The standard error of the treatment-minus-placebo contrast is 1.1 points.
The reported treatment-minus-placebo contrast is:
Because lower change is favorable on this scale, a negative contrast favors 10 mg. For a display convention in which positive values mean benefit, define the oriented benefit as placebo change minus dose change:
For an illustrative large-sample 95% interval, use \(B \pm 1.96SE\), with \(SE=1.1\) points:
The matrix should identify the endpoint as Week 12 change from baseline, record the analysis population and study design, and say that positive values of the displayed contrast favor dose. The interval is an uncertainty summary under the stated large-sample approximation; it is not a 95% probability that the true benefit lies in this particular interval. The values are hypothetical, and the interval should not be used if the analysis does not support the approximation.
Worked Example: Recording a Safety Result Without Hiding Its Denominator
In a separate hypothetical 24-week randomized study, 20 of 100 participants assigned to 20 mg daily and 8 of 100 assigned to placebo experience a specified adverse event. Define the risk difference as the event risk on dose minus the event risk on placebo. The estimated risks are \(20/100=0.20\) and \(8/100=0.08\), so:
Suppose the hypothetical study report gives a 95% score-based confidence interval for the risk difference of 3.0 to 21.0 percentage points. Record the event definition, the 24-week observation period, the dose and comparator, both event counts and denominators, the risk difference, and the interval method. The positive risk difference indicates a higher observed event risk on 20 mg in this comparison. The interval is compatible with an increase from 3.0 to 21.0 percentage points under the method’s assumptions; it does not by itself establish clinical importance or causation beyond the randomized comparison.
If exposure time, follow-up, or event ascertainment differed materially between groups, the simple risks might not adequately summarize the safety evidence. That concern belongs in the matrix’s limitation field rather than being concealed by reporting only the risk difference.
Worked Example: Using the Matrix to Find an Evidence Gap
Consider a hypothetical development program with a prespecified efficacy criterion of at least 2 points of benefit at Week 12, and a safety criterion that the upper limit of the 95% confidence interval for the excess risk of a specified adverse event should be below 10 percentage points. These criteria are illustrative, not universal. Suppose the matrix contains the following results:
| Dose | Week 12 benefit (95% CI), points | Excess adverse-event risk (95% CI), percentage points |
|---|---|---|
| 10 mg | 3.1 (0.9 to 5.3) | 3 (−2 to 8) |
| 20 mg | 4.0 (2.0 to 6.0) | 9 (1 to 17) |
For 10 mg, the point estimate exceeds the efficacy criterion, but its interval includes benefits below 2 points. The safety interval’s upper limit is 8 percentage points, below the illustrative 10-point criterion. For 20 mg, the efficacy point estimate and lower interval limit are at least 2 points, while the safety interval’s upper limit is 17 percentage points, above the safety criterion. These entries do not establish that one dose is best: they show that the interpretation depends on the decision rule and the degree of uncertainty.
A responsible matrix would also show the source studies, populations, endpoint definitions, time points, and interval methods behind these hypothetical values. If efficacy and safety came from different studies or populations, that distinction must remain visible. The next step is to assess whether the available evidence is adequate for the decision, not to treat the table as an automatic dose-selection algorithm.
Why the Matrix Structure Helps
The value of the matrix can be justified by a simple traceability argument. A dose decision is based on evidence interpreted against criteria. Each interpretation depends on a result and the context that defines that result. If the context is omitted, two numerically similar entries may refer to different populations, scales, or times and therefore answer different questions. Recording those features alongside the result preserves the chain:
Specify what efficacy, safety, or other evidence would support or constrain a proposed dose.
Match each criterion to results with the relevant population, dose contrast, endpoint, and assessment time.
Record estimates, denominators or units, and uncertainty summaries using clearly stated conventions.
State what the result does and does not support, including limitations and evidence gaps.
If a matrix lacks one of these links, the reader cannot reliably tell whether a result supports the criterion, whether it is precise enough to be informative, or whether it belongs to the target population and time frame. The structure does not remove uncertainty; it makes uncertainty and provenance harder to overlook.
Diagnostics, Alternatives, and Limitations
Review a draft matrix as carefully as a statistical output. Look for inconsistent units, reversed contrast directions, missing denominators, mismatched assessment times, duplicate rows, and entries that combine results from different analysis populations. Check that intervals correspond to the displayed estimate and contrast. A confidence interval crossing a decision threshold should be described as uncertainty around that threshold, not as a categorical pass or failure unless the prespecified rule explicitly uses that criterion. “Not reported,” “not measured,” “not estimable,” and “no observed events” are different states and should not be collapsed into a blank cell.
A matrix complements, rather than replaces, other evidence displays. A narrative summary is flexible and can explain clinical context, but it is harder to audit across many studies and may selectively emphasize salient findings. A forest plot efficiently displays estimates and intervals for a suitably comparable set of results, but it may omit study features or limitations needed for dose justification. A pooled analysis can provide a combined estimate when its assumptions and target question are appropriate; a matrix instead preserves study-level evidence and can reveal why pooling may be inappropriate. None is universally superior: the matrix is especially useful for traceability, while narrative, graphical, or quantitative synthesis may be added for other purposes.
The matrix also has limits. It cannot correct biased or poorly conducted studies, make non-comparable endpoints comparable, or convert uncertain findings into a definitive choice. It may become unwieldy if every analysis is included without a clear decision question. Use a defined scope, retain relevant results and caveats, and link entries to their source analyses. Do not rank doses solely by the number of favorable rows: studies differ in design, precision, population, and relevance, and a row count ignores those differences.
Common Pitfall
A common mistake is to place efficacy and safety numbers side by side and conclude that the dose with the largest efficacy estimate is preferred because its benefit “outweighs” its safety result. That conclusion does not follow from the table alone. The endpoints may have different scales, time frames, populations, and clinical meanings; uncertainty may also be substantial. The correction is to preserve the original measures and contexts, apply explicit decision criteria, and state where the evidence is uncertain or incomplete. A matrix organizes the basis for judgment; it does not supply the judgment automatically.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions below.
- A symptom scale is better when lower. A dose group changes by \(-4\) points and placebo by \(-1\) point. What is the treatment-minus-placebo contrast, and which group has the more favorable observed change?
- Why should a matrix record both adverse-event counts and denominators rather than only a percentage?
- A confidence interval for a dose’s efficacy estimate crosses a prespecified minimum-benefit threshold. What should the matrix say?
- When might a forest plot be useful alongside an evidence matrix, and what important information can the matrix retain that a plot may not show?
Solutions. (1) The contrast is \(-4-(-1)=-3\) points. Since lower change is favorable, the dose group has the more favorable observed change; if displaying positive benefit, placebo minus dose equals 3 points. (2) Counts and denominators show the number at risk and the basis of the percentage, helping readers assess sample size and compare groups without mistaking percentages from different denominators as equivalent. (3) Record the estimate, interval, threshold, units, and endpoint direction, and state that the uncertainty includes values on both sides of the threshold. Do not call the dose a clear pass or failure unless the prespecified decision rule requires that interpretation. (4) A forest plot can efficiently compare estimates and intervals for a set of sufficiently comparable results. The matrix can retain study design, population, dose regimen, endpoint definition, time point, analysis population, and limitations that may not fit in the plot.