Reporting an End-to-End Integrative Case Study

What an End-to-End Report Must Make Auditable

A multi-omics result is only as interpretable as the decisions that produced it. A reported molecular signature, classifier, or subtype depends on who entered the cohort, which assays passed quality control, how measurements were transformed, how the data layers were joined, and how validation was conducted. If those choices are not visible, readers cannot tell whether the result is robust, reproducible, or relevant to the stated population.

A useful report is not a list of software commands. It links each decision to its rationale and to the consequence for the analysis. The central question is whether another analyst could reconstruct the cohort and analysis from the description and specified materials, while understanding what the evidence does—and does not—support.

Definition: A reproducible end-to-end integrative report documents the cohort, assays, processing, integration, validation, uncertainty, and limits of interpretation in enough detail to audit the analytic path and, given lawful access to the data and specified computational materials, reproduce the reported results.

The report should make the analysis unit explicit (for example, one participant rather than one specimen), distinguish planned decisions from exploratory ones, and give denominators whenever samples or features are removed. It should also identify the target population and outcome. A complete-case analysis of people with all assays, for example, describes a selected subset if assay availability is related to clinical characteristics.

Document the Cohort and the Assay Path

Begin with eligibility criteria, recruitment or data sources, dates where relevant, and the number of participants at each stage. Give reasons for exclusions and distinguish participant exclusions from assay-level failures. For each omics layer, report the starting and retained sample counts, the quality-control rules, and the handling of missing measurements. State whether the analysis used matched specimens collected at the same visit or measurements from different times.

For each assay, report enough to identify the measurement and processing path: assay type, input material, quality-control criteria, normalization or transformation, feature filtering, and the versioned tools or reference resources used. Explain how feature identifiers were harmonized across layers and how duplicates or ambiguous mappings were treated. A phrase such as “standard preprocessing was applied” is not reproducible because it does not specify the actual operations or their order.

Then describe the integration decisions in a way that distinguishes design from result. State which layers entered the analysis, how features were scaled or selected, how samples were matched, and what happened when a participant lacked one layer. If a transformation, feature filter, or tuning decision was estimated from data, say which partition supplied that information. In prediction analyses, all steps that learn from outcomes or estimate preprocessing parameters must be confined to training data; the held-out set must not influence feature selection, scaling, threshold choice, or tuning.

Worked Example: Reporting Cohort Flow

Consider a hypothetical study assessing a two-layer expression-and-protein classifier. Of 112 records screened, 8 were excluded before enrollment: 4 did not meet eligibility criteria, 2 withdrew, and 2 lacked consent. Thus, \(112-8=104\) participants entered the analysis cohort. Quality control retained RNA measurements for 96 and protein measurements for 92. Four participants failed quality control for both assays, so the number with both layers available was:

n_{\text{paired}}=104-\left[(104-96)+(104-92)-4\right] =104-(8+12-4)=88.

The paired-sample retention proportion is \(88/104=0.8462\), or 84.6%. This number describes availability of both assays among the 104 enrolled participants; it is not a retention rate from the initial screening records. The report should give both denominators and the separate assay counts.

Suppose the 88 paired participants were split by participant into 42 development cases and 46 held-out test cases. A concise flow description would state the 112 screened, the 8 pre-enrollment exclusions and reasons, the assay-specific pass counts, the 88 paired cases, and the 42/46 split. It would also say whether the split was prespecified and whether the test set remained untouched during model development. This makes sample attrition visible instead of allowing the final analysis count to appear without context.

Validation and Uncertainty Belong Together

For each reported performance measure, identify the evaluation set, the number of participants contributing to it, the outcome definition, and any decision threshold. Report the confusion-matrix counts alongside derived quantities when possible. Sensitivity, specificity, and accuracy can look precise as percentages while resting on very few cases; the denominators reveal that limitation.

For a binomial proportion, the Wilson score interval is one commonly used interval. If \(x\) of \(n\) independent cases have the property of interest, \(\hat{p}=x/n\). For a nominal 95% interval, use \(z=1.96\) in:

\frac{\hat{p}+\frac{z^2}{2n}}{1+\frac{z^2}{n}} \ \pm\ \frac{z}{1+\frac{z^2}{n}} \sqrt{\frac{\hat{p}(1-\hat{p})}{n}+\frac{z^2}{4n^2}}.

Its usual interpretation is that the procedure has approximately 95% coverage over repeated samples under the binomial model; actual coverage can differ from 95% and depends on \(n\) and \(p\), in part because the binomial distribution is discrete. It is not a statement that the fixed parameter has a 95% probability of lying in this particular computed interval.

Justification: Under the binomial model, \(X\sim\mathrm{Binomial}(n,p)\) and \(\hat{p}=X/n\). Inverting the score-test acceptance condition \(\left|\hat{p}-p\right|\le z\sqrt{p(1-p)/n}\) gives \(n(\hat{p}-p)^2\le z^2p(1-p)\). Solving this quadratic inequality for \(p\) yields the Wilson lower and upper limits in the formula above. The inversion explains why the limits depend on the unknown \(p\) before algebraic rearrangement; it does not make the interval’s coverage exactly 95% for every \(n\) and \(p\).

Worked Example: Reporting Held-Out Specificity and Its Uncertainty

In a hypothetical held-out test set, 26 participants do not have the target condition. The classifier correctly labels 19 of them as negative and incorrectly labels 7 as positive. Specificity is the proportion of participants without the condition who are classified as negative:

\widehat{\mathrm{specificity}}=\frac{TN}{TN+FP} =\frac{19}{19+7}=\frac{19}{26}=0.7308.

To calculate the nominal 95% Wilson interval, use \(\hat{p}=19/26=0.7308\), \(n=26\), and \(z=1.96\), so \(z^2=3.8416\). The adjusted center is:

\frac{0.7308+3.8416/(2\cdot26)}{1+3.8416/26} \approx 0.7011.

The half-width is:

\frac{1.96}{1+3.8416/26} \sqrt{\frac{0.7308(1-0.7308)}{26}+\frac{3.8416}{4(26)^2}} \approx 0.1619.

The resulting interval is approximately \(0.5392\) to \(0.8629\). A suitable report would say: “Among 26 held-out participants without the condition, specificity was 73.1% (nominal 95% Wilson interval, 53.9% to 86.3%).” The interval is wide, reflecting the limited number of negative cases. The binomial interpretation assumes independent test participants and a relevant sampling process; it does not account for distribution shift between the test set and the intended deployment population.

Report Estimates Without Overstating Biology

A report should distinguish prediction from association. A classifier’s ability to discriminate outcomes does not establish that its features cause those outcomes, nor does it establish a biological mechanism. Likewise, an adjusted model coefficient describes a conditional association under the model’s assumptions; adjustment does not automatically remove confounding or turn an observational result into a causal effect.

For an estimated coefficient, report the model, the meaning and units of the predictor, the comparison or scale, the uncertainty measure, and the relevant adjustment variables. Explain whether the analysis was prespecified or exploratory and how multiple testing or selection was handled. A biological interpretation should be proportional to the evidence: a pathway-level pattern may motivate a mechanistic hypothesis, but it is not itself experimental confirmation of that mechanism.

Worked Example: Translating an Adjusted Association Into Careful Report Language

Suppose a hypothetical logistic regression relates a prespecified integrated score to disease status, adjusting for age and sex. The score is standardized, so a one-unit increase corresponds to one standard deviation. The fitted coefficient is \(\hat{\beta}=0.62\) with \(SE=0.24\). Under the approximate Wald method, the 95% interval on the log-odds scale is:

0.62\pm1.96(0.24)=0.62\pm0.4704 =[0.1496,\ 1.0904].

The Wald statistic is \(z=0.62/0.24=2.583\), giving a two-sided \(p\)-value of approximately 0.0098 under the model. Exponentiating the coefficient and interval gives the odds-ratio scale:

\exp(0.62)=1.859,\qquad [\exp(0.1496),\exp(1.0904)]\approx[1.161,\ 2.975].

A careful statement is: “In this hypothetical adjusted model, a one-standard-deviation increase in the integrated score was associated with 1.86 times the odds of disease (approximate 95% Wald interval, 1.16 to 2.98; \(p\approx0.0098\)), conditional on age and sex.” The interval and test rely on the model’s large-sample approximation and appropriate standard errors. The result is an adjusted association, not evidence that increasing the score causes disease. The report should also state how the score was constructed and whether its construction was independent of the outcome data used for this estimate.

Reproducibility Materials, Diagnostics, and Alternatives

The report should identify the code and software environment needed to reconstruct the analysis, including package and reference-database versions, parameter settings, random seeds where relevant, and the order of operations. Provide a data dictionary and a clear description of derived variables. When data cannot be shared, explain the access restrictions and provide permissible code, metadata, or synthetic examples so that the workflow can still be inspected. Record important deviations from the planned analysis rather than silently replacing the original plan.

Diagnostics should address the specific steps reported. Check participant counts and identifiers across layers, sample and feature exclusions, missingness patterns, batch or population structure, and whether the final analysis set differs materially from the target cohort. For a predictive analysis, inspect the partitioning and confirm that all learned transformations and selection steps were confined to development data. For a fitted model, report relevant checks of model fit and uncertainty assumptions. Inconsistent counts, unexplained feature losses, suspiciously strong performance, or a major gap between internal and external validation are reasons to investigate and qualify conclusions—not details to omit.

Different validation designs answer different questions. A held-out test set offers a direct evaluation if it was not used for model development, but may be imprecise in a small cohort. Resampling within a development cohort can use data more efficiently for internal validation, but does not establish transportability to a new population. Independent-cohort validation offers stronger evidence of generalizability to that cohort, while still requiring a description of differences in recruitment, assays, and case mix. These approaches are not interchangeable; the report should explain what its chosen design supports.

A concise main-text report with a detailed supplement can be effective, but a supplement cannot repair missing denominators or ambiguous claims in the main conclusion. The main text should communicate the key cohort flow, analysis design, central estimates, uncertainty, and limitations; supplementary material can provide full processing parameters, diagnostic plots, and extended results. No amount of computational detail compensates for limitations such as small sample size, nonrepresentative recruitment, incomplete assay overlap, or lack of external validation.

Common pitfall: “The model achieved 90% accuracy, so the integrated signature is a validated biological marker.” This reasoning confuses a performance estimate with biological validation and omits the test-set denominator, uncertainty, and validation design. Report the number and composition of test cases, the metric with uncertainty, how the test set was protected from model development, and the limits on biological interpretation. A strong result can support further validation; it does not by itself establish mechanism or clinical utility.

A Practical Reporting Sequence

Define the question and cohort.
State the target population, analysis unit, eligibility rules, outcome, and intended use of the analysis.

Account for samples and assays.
Show participant flow, assay-specific quality-control counts, matched-sample availability, and reasons for exclusions or missing layers.

Specify processing and integration.
Describe transformations, feature handling, layer alignment, and which data partitions informed each learned decision.

Present validation and uncertainty.
Name the validation design, provide denominators and relevant estimates, and interpret intervals and tests under their assumptions.

Bound the conclusion.
Separate predictive performance, association, and mechanism; state limitations and identify materials needed to reproduce the work.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions.

A report says 80 of 100 screened participants were analyzed, but gives no exclusion reasons or assay counts. Name two important details that should be added.
A specificity estimate is 19/26. What is the estimated specificity, and why should its interval not be described as having exactly 95% coverage?
A transcriptomic feature filter was chosen using all samples before a train/test split. Why is this a reporting and validation problem, even if the outcome labels were not used in the filter?
A one-standard-deviation increase in an adjusted score has an estimated odds ratio of 1.86, with interval 1.16 to 2.98. Give a defensible interpretation and one conclusion that is not justified.

Solutions. (1) Give the reasons for the 20 exclusions and distinguish participant exclusions from assay failures; report assay-specific counts and the number with all required layers. (2) \(19/26=0.7308\), or 73.1%. Wilson intervals have approximately nominal coverage under the binomial model, but actual coverage can vary with \(n\) and \(p\); discreteness prevents claiming exact 95% coverage in general. (3) A filter estimated from all samples allows information about the held-out feature distribution to influence model development. The report should disclose the procedure; for an unbiased held-out evaluation, any data-learned filtering should be performed using development data only and then applied to the test data. (4) The estimate supports an adjusted association: conditional on the model’s adjustment variables, the odds are estimated to be 1.86 times as high per one-standard-deviation increase, with the stated interval. It does not establish that the score causes disease or that it is a clinically useful marker.