Why Reproducible Reporting Matters
A differential-abundance result is interpretable only when a reader can tell what “abundance” means in that analysis. A reported increase could refer to a larger fraction of captured cells, a change in a log-ratio, or a modeled difference in expected counts. Those are distinct quantities. A reproducible report therefore describes not only the fitted model, but also the target contrast, the cell labels, the denominator, the biological replication unit, and the uncertainty calculation.
The reporting goal is not to claim that a particular model is universally best. It is to make the analysis auditable: another analyst should be able to reconstruct which samples and cells were analyzed, understand the model’s assumptions, and interpret the reported effect on its stated scale. The introductory material on differential-abundance interpretation provides context for why relative abundance is not the same as absolute cell number; here the emphasis is on documenting the analysis precisely.
Specify the Target Before Reporting the Model
Begin with the biological and statistical target in words. For example: “the adjusted difference between conditions in the expected fraction of quality-controlled captured cells assigned to cell type A, among donors represented by the study.” This statement makes the cell type, denominator, conditions, and target population visible. If the analysis instead targets a log-ratio of cell type A to a specified reference set, name that reference set and state that the effect is on the log-ratio scale.
A report should define cell labels operationally: which annotation procedure or marker rules were used, at what resolution labels were retained, and how ambiguous or unassigned cells were handled. If label uncertainty or the choice of cell-type resolution could change the result, report the relevant sensitivity analysis rather than presenting the labels as error-free. Earlier material on sensitivity to annotation uncertainty and resolution gives the relevant concerns.
State the denominator for each abundance outcome. Examples include all retained cells in a sample, all cells within a specified compartment, or a named reference composition. Describe the quality-control rules that determine which cells enter the numerator and denominator. A denominator that differs across samples because of undocumented filtering changes the meaning of the outcome.
Also report the biological replication unit. In a multi-donor study, cells from one donor are not independent biological replicates. The report should state how cells were summarized or modeled within each donor and how donor-to-donor variability entered the inference. If samples or donor-condition profiles were missing, state which were absent, how they were handled, and how the missingness affects the set of donors represented in the contrast.
Worked Example: Making a Denominator Explicit
Suppose a hypothetical study retains two samples per condition, each from a different donor. The report uses all quality-controlled cells in each sample as the denominator. In one control sample, 18 of 200 cells are assigned to cell type A; in one treated sample, 27 of 225 cells are assigned to cell type A. Calculate and describe the sample proportions.
For the control sample, the observed proportion is \(18/200=0.0900\), or 9.00%. For the treated sample, it is \(27/225=0.120\), or 12.0%. The observed difference for these two samples is \(0.120-0.0900=0.0300\), equivalent to 3.00 percentage points.
This arithmetic describes the two samples, not a donor-population effect estimate with inferential uncertainty. A reproducible report would say that the denominator is all quality-controlled captured cells in each sample and would describe the donor-level model used to estimate the condition contrast. It would not treat the 425 cells as 425 independent biological replicates.
Document the Donor-Level Model
State the outcome and its sampling model before interpreting a coefficient. For a beta-binomial analysis of cell-type counts, let \(Y_i\) be the number of cells assigned to the target type for donor-sample \(i\), and let \(N_i\) be the total number of cells in the stated denominator. One possible model is:
Here \(T_i\) encodes condition, \(\mathbf{z}_i\) contains the stated covariates, and \(\rho\) represents extra-binomial variation among donor-samples. Under this parameterization, the conditional mean is \(N_i p_i\), and the variance is \(N_i p_i(1-p_i)\{1+(N_i-1)\rho\}\). The model assumes that the specified beta-binomial distribution is an adequate representation of donor-sample counts conditional on the covariates, that included donor-samples are independent unless dependence is explicitly modeled, and that the cell counts and denominators are correctly defined.
For any model, report its family or likelihood, link function, outcome, denominator or offset, condition coding, covariates, and treatment of repeated observations or donor dependence. Identify whether covariates were selected in advance and give their units or coding. Explain how the coefficient corresponds to the target estimand. A condition coefficient in a logit model is a conditional log-odds contrast; it is not a percentage-point difference in proportions.
If the reported target is marginal rather than conditional, describe how it was calculated. For example, predictions under each condition may be averaged over a stated covariate distribution and then contrasted. State the population over which predictions were averaged and whether the contrast is a difference, ratio, or log-ratio. Do not call a model coefficient an adjusted marginal effect unless it was actually calculated on that marginal scale.
Worked Example: Reporting a Conditional Log-Odds Effect
Consider a hypothetical donor-level beta-binomial model with condition coded \(T_i=1\) for treatment and \(T_i=0\) for control, adjusted for a prespecified batch covariate. Suppose the fitted condition coefficient is \(\hat{\beta}_1=0.60\), with \(SE=0.25\). For illustration, assume the model’s large-sample normal approximation is adequate for this coefficient.
A 95% Wald interval on the log-odds scale is \(\hat{\beta}_1 \pm 1.96SE\):
The Wald statistic is \(z=0.60/0.25=2.40\), giving a two-sided normal-approximation p-value of about 0.0164. Exponentiating the estimate and interval gives an adjusted odds ratio of \(\exp(0.60)=1.82\), with interval \((\exp(0.110),\exp(1.090))=(1.12,\ 2.97)\).
The report can state that, conditional on the batch covariate and under the specified model, treatment is associated with 1.82 times the odds of a cell being assigned to the target type rather than the remaining denominator cells, compared with control. The 95% interval describes uncertainty in that conditional odds ratio. It does not say that the target cell type’s proportion increased by 82%, nor that there is a 95% probability that the fixed parameter lies in this particular interval.
Justify Uncertainty and Report Diagnostics
The usual Wald interval in the example follows from an approximate sampling statement. If an estimator \(\hat{\beta}\) is approximately normal with mean \(\beta\) and estimated standard error \(SE\), then the standardized quantity \((\hat{\beta}-\beta)/SE\) is approximately standard normal. The central 95% of a standard normal distribution lies between \(-1.96\) and \(1.96\). Rearranging that event gives the interval \(\hat{\beta}\pm1.96SE\). This justification depends on the approximation being reasonable; it does not make the interval reliable when the model or sampling assumptions fail.
State how uncertainty was obtained: for example, a model-based standard error, a donor-level bootstrap, or a posterior interval. Identify the interval level and whether it is on the model scale or transformed scale. For a transformed effect, give the transformation and ensure interval endpoints are transformed consistently. Report estimates and intervals in enough precision to reproduce the stated interpretation.
Diagnostics should address whether the fitted model describes the observed donor-level data and whether any one donor dominates the result. Report the checks performed and the important findings, not only that “assumptions were checked.” Depending on the model, relevant checks can include residual patterns, overdispersion, influential donors, fitted-versus-observed counts, and posterior predictive comparisons. Earlier material on residual and overdispersion diagnostics and posterior predictive checks covers these tools. State any model changes prompted by diagnostics and whether the primary conclusion was sensitive to reasonable alternatives.
A clear report also identifies failure signals and their consequences. Examples include separation or unstable estimates, very sparse cell types, poorly estimated dispersion, a covariate pattern that leaves condition effects non-identifiable, or a result that changes substantially when one donor is omitted. These are reasons to qualify the inference and report the limitation; they are not evidence that a nominal p-value alone resolves the problem.
Multiplicity and Reproducibility Details
When many cell types, contrasts, or both are tested, specify the family of hypotheses to which multiplicity adjustment applies. State the procedure, the significance threshold if one was prespecified, and whether the reported values are raw p-values, adjusted p-values, or both. For example, the Benjamini–Hochberg procedure controls the false discovery rate under independence and also under certain forms of positive dependence; it should not be described as controlling the probability of any false positive. If its dependence conditions are not suitable, a more conservative procedure may be considered, and the choice should be named.
The Benjamini–Hochberg adjusted values are formed by ordering the \(m\) raw p-values, scaling the value at rank \(r\) by \(m/r\), and applying a reverse cumulative minimum so the adjusted values do not decrease with rank. The full list of tested hypotheses matters: changing the family changes the adjustment. Include unselected tests in the declared family, and report enough results for readers to see which hypotheses were tested.
Worked Example: Reporting Benjamini–Hochberg Adjustment
Suppose a hypothetical analysis tests five cell types for one prespecified condition contrast. The sorted raw p-values are 0.004, 0.012, 0.030, 0.040, and 0.200. For \(m=5\), the unmonotonized scaled values are \(5p_{(r)}/r\): 0.020, 0.030, 0.050, 0.050, and 0.200.
The reverse cumulative minima are unchanged in this example, so the corresponding adjusted p-values in sorted order are 0.020, 0.030, 0.050, 0.050, and 0.200. At a prespecified false-discovery-rate threshold of 0.05, the first four tests meet the adjusted-p-value threshold. This does not mean each selected result has a 5% probability of being false; the procedure targets the expected proportion of false discoveries among rejections under its stated conditions.
A reproducible report would identify the five-cell-type family, the single contrast, the Benjamini–Hochberg adjustment, and the threshold, and would provide effect estimates and uncertainty alongside adjusted p-values. Reporting only the four selected cell types would obscure the testing family.
For computational reproducibility, document the software and relevant package versions, the analysis code or a durable code location, random seeds when stochastic computation is used, and the input data version or processing steps needed to recreate the analysis. Report the rules for exclusions, failed fits, and unassigned labels. These details complement—not replace—the statistical specification.
Common Reporting Mistakes and Limits
- Calling a relative abundance an absolute count. A fraction among captured cells does not establish a change in the number of cells in tissue. State the observed sampling basis and its limitations.
- Leaving the denominator implicit. “Cell type A increased” is incomplete if readers cannot tell whether the fraction is among all cells or a restricted reference set.
- Using cells as independent replicates. This can overstate information about a condition effect when cells share donors. Identify the donor-level unit and account for its variation.
- Interpreting a coefficient on the wrong scale. A logit coefficient is a log-odds contrast; it is not directly a percentage-point change. Name the scale and any transformation.
- Reporting adjusted significance without the testing family. State what was included in the multiplicity procedure and which adjustment was used.
- Presenting diagnostics as a checkbox. Name the checks and summarize relevant failures or sensitivity, including donor influence and model fit.
No reporting checklist can remove limitations caused by capture bias, annotation errors, sparse donors, or an unrepresentative donor population. The report should distinguish what the fitted analysis estimates from broader biological claims it cannot establish. In particular, observed relative abundance is conditional on the cells captured and retained under the stated protocol.
Check Your Understanding
Answer each question, then compare with the concise solutions.
- A report gives a logit-model condition coefficient of 0.40. What scale is this effect on, and what is its odds-ratio interpretation?
- Why is a denominator needed to interpret a reported cell-type proportion?
- In a study with 12 donors and thousands of cells per donor, what must be clarified about the replication unit?
- Five cell types were tested, but a report describes adjustment “among significant results.” What key information is missing?
Solutions. (1) The coefficient is a conditional log-odds contrast; exponentiating gives \(\exp(0.40)=1.49\), so the modeled odds are 1.49 times as large for the coded contrast, conditional on included covariates. (2) A proportion is a numerator divided by a specified total or reference set; changing that set changes the quantity being estimated. (3) The report must state that donors, rather than individual cells, provide biological replication and explain how within-donor dependence is handled. (4) The report must name the complete hypothesis family and the adjustment procedure; selecting significant results first does not define a transparent multiplicity correction.