Managing Data Quality for Bayesian Analyses

Why Data Quality Matters for Bayesian Evidence

A Bayesian analysis combines observed data with a probability model and a prior distribution. Its posterior is conditional on the data being the data the analysis claims they are: correctly linked to participants, measured using the intended definitions, and sufficiently complete and accurate for the stated model. If those conditions fail, calculations can be internally correct while the resulting evidence is misleading.

Data quality deserves particular attention when information accumulates during a trial or when external information contributes to an analysis. A duplicate record can make a small interim dataset appear larger; a misclassified endpoint can change the apparent event rate; and a difference in endpoint definitions can make an external success count non-comparable with the current trial. Bayesian computation does not automatically repair these problems. The posterior describes uncertainty under the data and model supplied to it, not uncertainty about whether the records were correct unless that uncertainty is explicitly represented.

The introductory material on assessing relevance and quality of external data addresses whether an external source is suitable to inform a trial. Here the emphasis is operational: what to verify in the records and data flow before using accumulating or external information in a Bayesian calculation.

Checks That Protect the Analysis

The checks should follow the information path from source to analysis. Their exact implementation depends on the trial, data source, endpoint, and planned use. A clean-looking analysis dataset is not sufficient evidence of quality if important transformations or source discrepancies have not been examined.

• Identity and uniqueness. Confirm that each participant has one valid analysis identity, that records from different participants have not been merged, and that repeated visits or exports have not been mistaken for additional participants. Repeated measurements may be legitimate, but their unit of analysis must match the model.
• Completeness and follow-up. Check that required fields and assessment windows are accounted for, and describe which participants have not yet had an opportunity for endpoint assessment. A field that is blank is not automatically a failure, a success, or evidence of no event.
• Endpoint definition and coding. Verify the endpoint rule, time window, units, event coding, and adjudication status against the analysis specification. Check whether the same definition was applied across sites, calendar periods, and data sources.
• Accuracy and source reconciliation. Compare analysis-critical values with appropriate source records, such as randomization or endpoint records. Investigate impossible values, inconsistent dates, conflicting status fields, and unexplained changes to derived variables.
• Consistency and protocol version. Check that eligibility, treatment assignment, follow-up, and endpoint processing follow the applicable protocol and analysis definitions. When amendments affect data collection or definitions, identify which participants or observations were affected.
• Timeliness and maturity. Establish the data cut-off and determine whether expected visits, adjudications, or outcome updates are still pending. An interim snapshot can be accurate as of its cut-off but incomplete for the intended endpoint.
• Provenance and transformations. Be able to identify the source and processing applied to each analysis variable, including external data. Changes in units, coding, inclusion rules, or deduplication should be understandable and consistently applied.

These checks do not replace the statistical assumptions. For example, validating that a binary endpoint is coded correctly does not establish that outcomes are independent conditional on the model parameters. Conversely, a well-fitting model does not establish that the source records are correct. Data-quality evidence and model diagnostics answer related but distinct questions.

How a Data Discrepancy Changes a Posterior

A simple beta-binomial model makes the sensitivity to outcome errors visible. Suppose \(Y\) is the number of participants with an event among \(n\) participants, and conditional on an event probability \(p\), outcomes are independent Bernoulli observations with the same probability. With a Beta\((a,b)\) prior for \(p\), where \(a>0\) and \(b>0\), the posterior is Beta\((a+Y,b+n-Y)\). This result assumes a fixed binary endpoint, correctly counted participants, and the specified independent Bernoulli model.

The posterior mean is \((a+Y)/(a+b+n)\). If one participant among the fixed \(n\) is changed from an event to a non-event, the posterior mean decreases by \(1/(a+b+n)\). This is a mathematical consequence of the model, not a claim that every uncertain classification should be recoded. If the participant's outcome is genuinely unresolved, assigning it a value merely to obtain a posterior would conceal the uncertainty.

Quality Checks When Data Accumulate

Accumulating data can influence interim summaries, predictive calculations, or prespecified decisions. At each planned data cut, quality checks should establish which participants and observations are included, whether each has reached the relevant assessment point, and whether the same data rules have been applied as at earlier cuts. The checks should be proportionate to the decision: an error affecting treatment assignment or a primary endpoint may be more consequential than an error in a nonessential descriptive field.

A particularly important distinction is between additional information and repeated representation of the same information. If the analysis treats duplicated participant records as independent observations, it can increase the apparent sample size and alter the posterior as if genuinely new participants had been observed. The following example uses a simple model to show the arithmetic; it does not imply that a beta-binomial model is appropriate for every accumulating trial.

Useful diagnostics include counts by participant, site, visit, and data cut; duplicate-key and impossible-date checks; reconciliation of randomized participants with analysis records; and comparisons of critical counts across successive snapshots. A sudden change may reflect genuine new information, a data-processing change, late reporting, or an error. The change should be explained before it is interpreted as a clinical pattern.

External Information: Quality, Compatibility, and Weight

External data require checks of both record quality and comparability. A source may have complete records yet differ from the current trial in outcome definitions, assessment schedules, eligibility, treatment context, or calendar period. Such differences are not all data-entry errors, but they can make direct use of the external counts inappropriate. The relevance of a source and the justification for borrowing from it are separate from whether its records are accurate.

A simple pooled calculation illustrates why duplicate external records are consequential. Pooling assumes that all observations share a common event probability, that outcomes are independent, and that the endpoint and participant definitions are comparable. These are strong assumptions. The calculation below is only a demonstration of the impact of a data error, not a recommendation to pool sources without justification.

Interpreting Unresolved Quality Concerns

A Bayesian estimate is conditional on the observations included, their coding, the analysis model, and the prior. Consequently, unresolved quality concerns limit the scope of the interpretation. One may report a posterior calculated from a clearly defined dataset, but should state the relevant limitations and avoid implying that the posterior accounts for unmodeled errors. If the uncertainty about data correctness is material, it may be appropriate to delay a decision, correct the data, or show analyses under defensible alternative data definitions. The choice depends on the nature and impact of the concern.

A sensitivity analysis can show whether a conclusion changes under plausible corrections or exclusions, but it cannot transform a poor-quality source into a reliable one. Nor does agreement across several analyses prove data integrity if they share the same defective records. For unresolved records, alternatives should be based on documented possibilities, not chosen because they produce a preferred result. Missing outcomes and their assumptions require a dedicated treatment; simply coding them as failures or successes is not a neutral quality fix.

Quality concerns also differ in severity. A typographical issue in a variable unused by the analysis may have little consequence. Uncertain participant identity, treatment assignment, primary endpoint status, or data maturity can directly affect the evidence and decision. The assessment should therefore connect each issue to the affected records, model inputs, posterior summaries, and decision rules, and explain what remains uncertain after any correction.

Common Pitfalls and Practical Safeguards

• “The posterior accounts for all uncertainty.” Not unless the relevant uncertainty is represented in the probability model. Ordinary posterior calculations condition on the supplied data and assumptions.
• “A larger dataset is always more informative.” Duplicate or mislinked records can inflate apparent information without adding independent participants.
• “The data passed a plausibility check, so they are accurate.” Plausible values can still be miscoded, drawn from the wrong source, or based on a different endpoint definition. Checks should target the analysis-critical variables and their provenance.
• “A sensitivity analysis resolves a quality failure.” It can reveal how conclusions depend on alternatives, but cannot determine which records are correct without evidence.
• “External records can be pooled because both datasets report the same label.” The endpoint rule, follow-up window, participant definition, and source quality must also be examined, and the statistical assumptions for borrowing must be justified.

A practical sequence is to define the analysis-critical data, specify checks and acceptable limits before reviewing results where feasible, investigate discrepancies, document the resolution and its effect on the analysis, and state remaining limitations when interpreting the posterior. This keeps data correction separate from outcome-driven selection.