Why Plan Tables and Decision Matrices?
A well-written objective explains what an analysis is meant to address. It does not, by itself, establish which participants will contribute, what analysis will be performed, what evidence will be examined, or where the result will appear. Analysis-plan tables and decision matrices make those connections visible before results are known. Their purpose is not to predetermine a favorable result; it is to make planned work traceable and reduce avoidable ambiguity.
This tutorial focuses on the design of those planning artifacts. It does not replace the detailed choices made when defining populations, model alternatives, sensitivity analyses, or uncertainty summaries. Instead, it shows how to represent such choices coherently and how to check that each planned output has a clear purpose.
Designing a Traceable Plan
A useful plan follows a chain: objective → population → analysis → evidence → output. These terms should not be treated as interchangeable. An objective states the question. A population identifies the records or participants eligible to contribute. An analysis specifies what will be done with those data. Evidence is what will be examined to assess the analysis or address the objective. The output is the planned table, figure, or written summary that communicates the evidence.
For example, “describe concentration over time” is an objective, not an analysis. “Use the prespecified pharmacokinetic analysis population to summarize observed concentrations by nominal time” describes a planned analysis more clearly. A plot of observed concentration against time, with units and a defined display scale, could be its output. The plot is evidence relevant to the objective, but it does not automatically establish that a model is adequate or that a treatment effect is causal.
The chain is useful because it exposes gaps. An objective with no output may be forgotten during reporting. An output with no objective may consume effort without a stated purpose. A figure shell that names an analysis but not its population can be populated inconsistently. If several objectives share one output, the plan should say which objective or objectives it serves rather than assuming the connection is obvious.
These artifacts rely on several planning assumptions: objectives are sufficiently stable to organize the work; population definitions are stated elsewhere or can be summarized precisely; planned methods are described at the level needed to distinguish them; and the expected evidence can be identified before results are seen. If a method or rule remains undecided, the plan should label it as unresolved and specify how and when it will be finalized. A placeholder should not disguise an open decision as a completed one.
From Objective to Table and Figure Shell
A table shell should contain the structure needed to report a result, not invented values. Include informative titles, row and column headings, units, analysis-population labels, and footnotes that define abbreviations or clarify planned summaries. Use an explicit placeholder such as “To be populated” rather than a blank cell that could be mistaken for an omitted result. A shell may specify the planned statistic or display, but should not imply that the analysis has already been conducted.
A figure shell needs comparable specificity. State the population, plotted quantities and units, planned panels or groups, and any essential display conventions. If the purpose is comparison, make the comparison clear in the title or legend. A plot that is not intended to report an analysis result—such as an exploratory diagnostic—should still be identified as such and connected to its planned use.
Worked Example: Mapping a Concentration–Time Objective
Hypothetical setting. A development team plans to characterize observed plasma concentrations after oral dosing in adults. The objective is to describe the concentration–time pattern over the planned sampling period. The analysis population has already been defined in the plan as adults with at least one usable post-dose concentration and a documented dose time.
Objective and analysis. The planned analysis is to summarize observed concentrations at each nominal sampling time in that population. The plan should identify the summary statistics and the handling of missing observations; those choices must be specified in the analysis plan rather than improvised while filling the shell.
Figure shell. A suitable shell title is “Observed plasma concentration versus nominal time after dose: planned analysis population.” The horizontal axis is nominal time after dose in hours; the vertical axis is plasma concentration in mg/L. The legend identifies the planned groupings, if any. The shell notes the intended display of individual observations and the planned summary overlay, with the summary definition stated in a footnote or referenced to the plan.
Table shell. Rows represent nominal sampling times in hours. Columns identify the number of observations and the planned concentration summaries in mg/L. Cells remain marked “To be populated.” The table is linked to the same objective and population as the figure, but serves a complementary purpose: it provides values in tabular form rather than showing the pattern across time.
Check. The output is traceable because the title names the outcome and population, the axes and table headings give units, and the planned summaries are defined rather than guessed. No conclusion about the shape or adequacy of a pharmacokinetic model is implied by this descriptive output.
Building a Decision Matrix
An analysis-plan table primarily organizes what will be done and reported. A decision matrix adds a conditional structure: if a prespecified finding occurs, what action follows, and where will that action be documented? The matrix should summarize existing planned rules, not create new thresholds after results are examined. It is especially useful when a decision depends on more than one condition or when several plausible outcomes require different next steps.
A practical matrix includes the objective or decision area, the population and analysis to which the rule applies, the criterion or evidence, the possible finding, the resulting action, and the reporting location. Where relevant, it also records the responsible reviewer or the reference to the detailed rule. If a rule is defined in another controlled section, cite that section clearly; do not duplicate it in slightly different wording that could later conflict.
Worked Example: Linking a Covariate Decision to Its Evidence
Hypothetical setting. A population pharmacokinetic analysis will evaluate whether baseline renal function is informative for apparent clearance. The plan already specifies the candidate model comparison and the retention rule. The purpose here is to record how that rule connects to evidence and reporting, not to invent a new rule.
| Decision area | Population and analysis | Evidence to review | Planned action | Reporting output |
|---|---|---|---|---|
| Renal-function covariate | Covariate-evaluable population; prespecified comparison of the base and covariate models | Results required by the prespecified retention rule, plus the planned model-behavior checks | Apply the stated rule; retain or do not retain the covariate as specified, and record any failure to evaluate it | Model-comparison table and concise rationale in the model-development summary |
The phrase “results required by the prespecified retention rule” is only adequate if the rule is easy to locate and unambiguous in the plan. If not, the matrix should reproduce the actual criteria, including how a criterion is assessed and what happens when the analysis fails or is inconclusive. The action must not be phrased as if the covariate is guaranteed to be retained.
The matrix also makes an important distinction: evidence supports a decision, but the reporting output documents both the evidence and the decision. A table containing a model comparison without a stated rationale may not show how the prespecified rule was applied.
A Short Justification for the Traceability Check
The chain from objective to output can be treated as a completeness check. Let each planned objective be associated with a population, analysis, evidence source, and output. If a link is absent, a downstream reader cannot determine whether the planned question was answered as intended. For example, an output without a named population cannot establish which records it summarizes; an analysis without an output may leave no planned reporting location for its evidence.
This is a planning argument, not a statistical theorem: a complete chain does not guarantee a valid analysis or a correct conclusion. It does provide a practical condition for traceability. Each arrow should be justified by a clear statement, table entry, or unambiguous reference. Reviewing the chain from both directions is useful: start with each objective and locate its outputs, then start with each output and identify the objective it serves.
Diagnostics, Alternatives, and Common Failures
A plan can be audited before analysis begins. Check whether population names match the definitions used elsewhere, whether units and time references are explicit, and whether each planned output has an objective. For a decision matrix, verify that criteria are prespecified, all plausible outcomes have an action or escalation path, and the corresponding reporting location is named. A reviewer should be able to distinguish “not assessed,” “assessment failed,” and “criterion not met”; these are different outcomes and should not collapse into an empty cell.
Consider an analysis that cannot be completed because the required data are unavailable or the model fails to run. The matrix should direct the team to document the failure and any planned alternative, not silently treat missing evidence as a negative result. If a decision is inconclusive, the plan should indicate whether additional review, a sensitivity analysis, or a documented limitation is appropriate under the prespecified approach.
A decision matrix is more compact than prose when decisions branch across several outcomes. Prose can be better for explaining scientific rationale, assumptions, and exceptions. The strongest plans often use both: a concise matrix for navigation and a nearby narrative or cross-reference for definitions and justification. A table shell is appropriate when the main need is consistent tabulation; a figure shell is appropriate when the planned evidence is best communicated visually. None of these formats replaces the analysis specification itself.
The artifacts also have limits. They cannot ensure that the selected population or method is scientifically appropriate, that data are adequate, or that an analysis succeeds. Overly detailed shells may become brittle if the plan changes, while vague shells invite inconsistent reporting. When a justified change is made, update the controlled plan and record the change and its rationale; do not leave obsolete shells that appear to describe the analysis actually performed.
A Repeatable Review Workflow
Use the approved objective wording and assign each objective a stable identifier if the project uses identifiers.
Name the applicable population and summarize the planned analysis precisely enough to distinguish it from related analyses.
Specify what will be reviewed and the table, figure, or narrative location that will report it.
Reference or restate the prespecified criteria, identify resulting actions, and include failure or inconclusive pathways where relevant.
Trace every objective to its outputs and every planned output back to an objective; resolve orphaned items and ambiguous links.
Worked Example: Auditing a Small Plan
Worked Example: Finding a Missing Link
Hypothetical plan. An objective is to assess the robustness of the primary model to a prespecified alternative handling of below-quantification observations. The plan names the primary analysis population and the alternative analysis, and lists a sensitivity-analysis table. However, the table shell has no population label, and the plan does not say where differences between the primary and sensitivity results will be summarized.
Audit. The objective, population, and analysis are identifiable, but the reporting chain is incomplete. The table’s omitted population label makes its scope uncertain. Listing the table does not specify how the evidence will be interpreted or where the comparison will be explained.
Correction. Add the applicable population to the shell title or heading, including any difference from the primary population. Add the planned comparison summary to the reporting plan and name its location. The corrected chain now makes clear which analysis generated the evidence and where the evidence and its implications will be reported. The audit does not decide whether the alternative handling is preferable; that depends on the prespecified analysis plan and the observed evidence.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions below.
- A table shell lists concentration summaries but gives no population. What traceability problem does this create?
- Why should a decision matrix include an action for an analysis that fails or is inconclusive?
- A figure shell has a clear title but omits units from both axes. What should be corrected before the shell is approved?
- A result does not meet a prespecified decision criterion. Is that equivalent to having no evidence? Explain.
- A planned output has no linked objective. Give two reasonable actions for resolving this.
Solutions. (1) A reader cannot determine which participants or records the summaries describe, so the output cannot be reliably linked to the planned population; add the population label or an unambiguous reference. (2) Otherwise, a failed or inconclusive analysis may be mistaken for a negative finding or omitted without explanation; specify the planned documentation or escalation path. (3) Add units to the axis labels and check that the quantities and time reference are also explicit, so the display can be interpreted correctly. (4) No. Evidence may have been generated but may not satisfy the decision criterion; distinguish that outcome from missing evidence or analysis failure. (5) Confirm whether the output supports an unstated objective and add the valid link, or remove it from the planned outputs if it serves no required purpose.