Why Documentation Is Part of the Analysis
A competing-risk result is interpretable only when readers can tell which participants, events, and model produced it. Two analyses can use the same dataset and report the same estimand yet differ because they coded an event differently, excluded different participants, handled missing covariates differently, or made different modeling choices. Reporting those decisions is not administrative detail: it allows readers to understand what was estimated and gives another analyst a route to check the work.
Documentation supports reproducibility, but it does not guarantee that another analyst will obtain identical numbers. Differences in data access, software implementations, numerical tolerances, and computing environments can matter. The practical goal is to record enough information to identify the analysis, verify its inputs and decisions, and investigate discrepancies.
The methods section should contain the choices a reader needs to interpret the analysis. A supplement can hold implementation detail that would interrupt the main account, such as a complete event-code map, variable-level missingness, or software code. The supplement should be clearly signposted from the methods; moving information there does not make it optional.
A Traceable Record of Decisions
A useful record links each decision to the analysis it affected and to the reported result. For example, a reader should be able to trace a cumulative incidence estimate back to the analysis population, event definitions, follow-up rules, and software implementation used to calculate it. Report decisions specifically, not with phrases such as “standard methods were used.”
Name the software and version, relevant packages and versions, and the analysis-code location or availability. State the data extract or cut-off used when it is relevant and permissible to disclose.
State eligibility and exclusions, analysis population, time origin and follow-up rules, and how each event type and censoring status was represented.
Describe missing-data procedures and model choices at a level that allows the analysis to be reconstructed, including covariates and relevant implementation details.
Check participant and event counts against tables and figures, report diagnostics relevant to the chosen method, and describe departures from the analysis plan with their rationale and impact.
Event coding should be explicit: name each cause, identify the value assigned to it, and distinguish censoring from event codes. State how same-time records, invalid or inconsistent records, and the end of follow-up were handled if they arose or could affect the analysis. The methods should also identify the analysis population in operational terms, not simply call it “the study cohort.”
For a model, document enough to distinguish it from plausible alternatives. This commonly includes the outcome and event of interest, model type, covariates and their coding, any stratification or interactions, handling of ties if relevant, and how uncertainty was calculated. Do not imply that a model specification alone establishes its assumptions; report the checks performed and any concerning findings. A reader should be able to distinguish, for example, a cause-specific hazard model from a subdistribution hazard model without having to infer it from a software function name.
Worked Example: Making Event Coding and the Analysis Population Auditable
Suppose a hypothetical trial enrolls 240 people. The prespecified analysis requires a baseline covariate and at least one day of follow-up. Seven people lack the baseline covariate and five have no follow-up time, leaving 228 in the analysis population. During follow-up, 31 experience the event of interest, 18 experience a competing event, and 179 are censored before either event. Write a concise methods description and check the counts.
A suitable description is: “The analysis included participants with the required baseline covariate and at least one day of follow-up. Of 240 enrolled participants, 7 were excluded for missing the baseline covariate and 5 for having no follow-up time; 228 participants were analyzed. Event status was coded 1 for the event of interest, 2 for the competing event, and 0 for censoring. Follow-up ended at the first recorded event or the participant’s last event-free contact.”
The event totals should reconcile to the analysis population:
The exclusions also reconcile: \(240 - 7 - 5 = 228\). These arithmetic checks do not prove that the coding or exclusions were clinically correct. They do reveal an internal inconsistency if the stated counts fail to add up, and they give readers a clear accounting to compare with the cohort flow diagram and results table. The code map belongs in the methods or a clearly referenced supplement; a statement that “events were coded numerically” is insufficient.
Missing Data and Model Details
Missing-data reporting should identify which variables had missing values, how much was missing, and what procedure was used in each analysis. If a complete-case analysis was used, state the criteria for a complete case and report the resulting analysis count. If imputation was used, describe the imputation approach, variables included, number of imputed datasets or imputations, and how estimates and uncertainty were combined. Report relevant software details and any prespecified sensitivity analyses. Do not claim that a method removes bias without stating the assumptions on which that claim depends.
Model documentation should be specific enough that a knowledgeable reader could recreate the design matrix and analysis population. State transformations, reference categories for categorical predictors, and units or scaling for continuous predictors. If the analysis plan prespecified a model, report it and note departures. If diagnostics indicated convergence or estimation problems, state what occurred and what was done; do not silently omit an unstable model or present a revised model as if it had always been planned.
Worked Example: Reporting Missing Covariates
In a hypothetical analysis of 180 eligible participants, 34 have at least one missing baseline covariate and 146 have complete covariate records. The prespecified primary analysis uses multiple imputation with 20 imputed datasets. A complete-case analysis is also reported as a sensitivity analysis. Draft a transparent report and verify the counts.
The counts reconcile because \(180 - 34 = 146\). A suitable report is: “Among 180 eligible participants, 34 (18.9%) had at least one missing baseline covariate. The primary analysis used 20 multiply imputed datasets. The imputation model included the outcome event indicator, follow-up information, all analysis covariates, and auxiliary variables listed in the supplement. Estimates and standard errors were combined across imputations using Rubin’s rules. A complete-case analysis of 146 participants was conducted as a sensitivity analysis.”
The percentage is \(34/180 \times 100 = 18.89\%\), reported as 18.9%. The description should be revised if the imputation model did not include one of the listed variable groups, or if the actual combining procedure differed. The report should also give missingness by variable, preferably in the supplement, because 34 participants with any missing value does not show which covariates were incomplete. The sensitivity analysis is informative as a comparison; agreement between analyses does not, by itself, prove that missingness was harmless.
Plan Deviations and Where Details Belong
A deviation is a meaningful departure from the analysis plan, not merely a change in wording. Examples include changing an eligibility rule, adding or removing a covariate, changing the missing-data approach, or replacing a planned model after a documented estimation problem. Record what changed, when the decision was made relative to examining outcome results, why it changed, and which analyses or conclusions it affected. If no deviation occurred, a brief statement can say so.
Avoid describing a post hoc decision as prespecified. If an unplanned analysis is presented, label it exploratory or otherwise describe its status accurately. Explain whether the main planned analysis remains available, and report enough information to assess the consequences of the change. This makes the distinction between planned evidence and later investigation visible without implying that every deviation invalidates the analysis.
The main methods and supplement serve different purposes. The main text should let a reader understand the analysis without searching through technical appendices; the supplement should provide the detailed record needed to check or reproduce it. A compact cross-reference can make this division explicit.
| Report in the main methods | Place in a supplement when detail is extensive |
|---|---|
| Software and version; analysis population; event and censoring definitions; missing-data approach; model type and core specification; whether deviations occurred | Full code map; variable-level missingness; detailed covariate coding; code or syntax; extended diagnostics; deviation log and sensitivity-analysis details |
Checks, Failure Modes, and Limits
Before submission, compare the methods, flow diagram, tables, figures, and analysis code. The number analyzed should agree across them, event counts should match the stated code definitions, and reported model covariates should match the actual fitted model. Confirm that software and package versions refer to the environment that generated the results, not simply the environment currently installed. If code or data cannot be shared because of privacy or governance restrictions, state the restriction and describe an appropriate route for access or independent checking where possible.
Common warning signs include an event-code map with no censoring value, inconsistent denominators, unexplained changes in sample size between models, a methods description that omits the missing-data procedure, or a supplement that is cited but does not contain the promised detail. These are reporting failures that can obstruct review; some may also reveal errors in the analysis itself. Documentation cannot repair an invalid model, guarantee that missingness assumptions hold, or make restricted data publicly available. It makes choices and limitations visible so readers can judge what the analysis supports.
Compared with a brief narrative alone, a structured supplement offers greater detail and auditability but can be harder to navigate. Compared with publishing code alone, a methods narrative explains why key choices were made and what the analysis means. The strongest reporting uses both: concise contextual description in the main text, with code and detailed specifications in a clearly organized supplement where appropriate.
Worked Example: Documenting a Deviation After a Model Fails to Converge
In a hypothetical study, the analysis plan specified an adjusted competing-risk regression with five covariates. The planned fit does not converge because one covariate category has very few observations. Before interpreting the result, the analyst checks the data and code, then fits a prespecified-in-the-report sensitivity model omitting that covariate. What should be documented?
The report should identify the planned model and state that it did not converge, describe the diagnostic checks and the sparse category, and explain the revised model and its status as a deviation or sensitivity analysis. It should give the revised covariate set and affected analysis count, if that count changed, and report the results without presenting the revised fit as the original planned analysis. If the five-covariate model could not produce an estimate, that failure should not be hidden by reporting only the alternative.
A concise statement could read: “The planned adjusted model did not converge. Review of the data and model specification identified a sparse category for [covariate]. We therefore report the planned model as not estimable and present a sensitivity model omitting that covariate; this change was made after the convergence problem was identified. The number analyzed was unchanged.” This wording is appropriate only if each detail is true. If the analysis count changed, the final sentence must give the revised count and reason. Reporting the failure and the response lets readers distinguish numerical troubleshooting from a successful prespecified fit.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- A report says “events were coded 1 and 2.” What essential coding information is missing?
- A study has 300 eligible participants, 24 with missing covariates, and uses complete cases. How many participants enter that analysis, and what should the report state?
- A model was changed after a convergence problem. Name three details that should be reported about the change.
- Why is “software was used for all analyses” inadequate documentation?
- Give one type of detail that can reasonably go in a supplement, and explain what the main methods should still say.
Solutions. (1) The report must identify which event type each value represents and give the censoring code and its meaning. (2) The analysis includes \(300 - 24 = 276\) participants; report the eligibility and complete-case rule, missingness, and resulting analysis count. (3) State the planned model and failure, the diagnostic checks and reason for the change, and the revised specification and its status; also report any change in analysis count or affected results. (4) It omits the software name and version, and potentially relevant package versions, so readers cannot identify the implementation. (5) A full event-code map or detailed code can go in a supplement, but the main methods should still summarize the event definitions and clearly point to that detail.