PROJECT 2 EXAMPLE: Feedback X Prevalence Using Dermatology Stimuli: Complete Statistical Analysis

1. Trial at a Glance

This page covers the registry results for PROJECT 2 EXAMPLE: Feedback X Prevalence Using Dermatology Stimuli (NCT05244122), a na na study. The primary outcome is Change in D' Between Pairs of Blocks..

2. Clinical Question

3. Trial Design

This completed study is registered as a single-group behavioral experiment, not as a conventional comparison of randomized treatment arms. Allocation is listed as not applicable, and masking is listed as none. The registry names feedback and prevalence as the behavioral interventions. Its summary describes blocks with low or high prevalence, with or without feedback, allowing the study to examine whether the history of preceding blocks relates to decisions in subsequent blocks.

Participants used the online DiagnosUs medical-imaging labeling app to classify images of skin spots as melanoma or benign nevi. The summary describes participants with varying degrees of expertise and says that over 300,000 individual judgements were collected. The registry lists enrollment of 1121. It does not report how participants’ expertise was defined or how the total number of judgements was distributed among participants and conditions.

The arm description states that observers completed blocks of 80 trials. Low-prevalence blocks contained 20% cancer cases, or 16 images; high-prevalence blocks contained 50%, or 40 images. The primary-outcome timeframe says participation could last from two blocks in one day to 24 blocks over 6 days. The registry does not report the complete schedule, the assignment or ordering procedure, or whether participants completed the same number of blocks.

The record lists a start date of 2021-06-22, a primary completion date of 2021-06-27, and status as completed. The phase is listed as not applicable. Brigham and Women's Hospital is named as the sponsor. These design details describe the registry entry; they do not establish that any particular condition caused a change in performance.

4. Participant Flow and Accounting

The registry reports how participants moved through each stage of the study. All counts below are exactly as posted; missing counts are not inferred from other stages.

Overall Study

Reasons for not completing this period:

Analysis population for the primary outcome, as defined in the registry: Blocks having D' \< 0.5 were excluded.

5. Baseline Characteristics

The table reproduces the baseline characteristics posted in the registry, which describe the participants analysed at baseline and help show whether the randomized groups were comparable.

6. Primary Endpoint

The registry lists the primary outcome as Change in D' Between Pairs of Blocks., measured over Participants could be in the study for as little as two blocks in one day up to 24 blocks collected over 6 days..

The primary outcome is titled “Change in D′ Between Pairs of Blocks” and is measured in z-score units. D′, or d-prime, is defined as the z-transform of the true-positive rate minus the z-transform of the false-positive rate. A true positive is a melanoma image classified as melanoma; a false positive is a benign-nevus image classified as melanoma. The registry says a correction of 0.5 error is added to avoid calculation problems when the rates are at their endpoints, z=0 or z=1.

In general, D′ is used as a measure of discrimination: it represents how well responses distinguish target images from non-target images, rather than simply how often a participant chooses one response. The registry states that D′ of zero indicates no ability to discriminate and D′ greater than zero indicates some ability to discriminate. It defines the change of interest as D′ for Block 2 when it follows Block 1, compared with D′ for Block 2 averaged across all conditions.

The record says blocks with D′ below 0.5 were excluded. It reports a 95% confidence interval as the dispersion measure, but gives no numerical mean or interval for the outcome. The endpoint therefore establishes what was intended to be measured, not the size or direction of any observed change.

7. Why No Re-analysis Is Shown

8. Statistical Interpretation

9. Statistical Methods Explained

The primary measure is D′, calculated from the z-transformed true-positive and false-positive rates. General explanation: a true-positive rate describes the share of target items correctly identified, while a false-positive rate describes the share of non-target items incorrectly called targets. A sensitivity measure that incorporates both rates can distinguish discrimination performance from a general tendency to choose the target response. D′ is therefore relevant when researchers want to assess whether people separate melanoma images from benign images, rather than merely count how often they label an image as melanoma.

The registry specifies an adjustment of 0.5 error to avoid calculation problems when a rate produces z=0 or z=1. General explanation: z-transformation maps rates to a scale used in signal-detection measures, and boundary rates can yield values that cannot be handled as ordinary finite z-scores. A correction can make the calculation possible. The registry does not report further details about how this correction was applied to particular observations.

The outcome is described as a change between pairs of blocks. Its stated contrast compares D′ for Block 2 after Block 1 with D′ for Block 2 averaged across conditions. This wording identifies the scientific comparison, but it does not fully specify the statistical estimator. For example, the registry does not say whether the contrast is calculated at the participant level, at the block level, or using another aggregation procedure. It also does not report a model formula or covariates.

General explanation: when the same participant contributes multiple blocks, the resulting observations may be correlated. Treating related observations as independent can make uncertainty estimates inaccurate. Analyses of repeated measurements commonly account for participant-level clustering or repeated observations, but which method is appropriate depends on the design and analysis plan. The registry record does not report whether dependence was modeled or how it was handled.

The registry names a 95% confidence interval as the dispersion measure. General explanation: a confidence interval expresses the statistical uncertainty around an estimated effect under the model and sampling assumptions used. It is not a guarantee that the true effect lies within a particular interval, and it does not replace the effect estimate itself. Because neither the mean nor the interval limits are reported, the precision of the primary result cannot be evaluated.

The record says blocks with D′ below 0.5 were excluded. This is an analysis-population rule for the outcome as described, but the registry does not state the rationale, timing, or number of exclusions. General explanation: exclusions based on observed outcome values can affect the population to which an estimate applies and may influence an estimate if excluded observations differ systematically from retained observations. Assessing that possibility would require information about exclusions and the analysis population, which is not reported in the registry record.

The registry lists two outcome measures but zero analyses. It does not report a statistical testing procedure, significance threshold for the outcome analysis, model assumptions, or any adjustment for multiple comparisons. The absence of a reported analysis is not evidence that no analysis was performed outside the registry; it means that the registry record does not document one. No inferential claim should be reconstructed from the outcome definition alone.

General explanation: over-dispersion occurs when observed variability exceeds what a statistical model assumes. If present, it can make standard errors too small and confidence intervals too narrow unless the analysis accounts for it. The record gives no outcome-level data or model information with which to assess over-dispersion, and it does not state whether this issue was examined.

Any sample-size figures described as planning illustrations are not observed power. The stated planning approach uses a two-sided significance level of 0.05, equal allocation between two arms, a normal approximation, and fractions of an observed effect. Those assumptions are illustrations and do not describe this single-group study’s allocation. The registry record does not report the observed effect used for such planning, a trial-specific sample-size justification, or a power calculation for the study as conducted.

10. Multiplicity, Missing Data and Analysis Populations

The registry lists two outcome measures, but does not report a multiplicity strategy. It does not say whether one outcome was designated as primary for hypothesis testing, whether several comparisons were planned, or whether any adjustment for multiple testing was used. The title of the primary outcome identifies the measure of interest, but the registry record does not report an inferential analysis or testing hierarchy. Without those details, the effect of multiple outcomes or comparisons on interpretation cannot be assessed.

Missing-data methods are not reported in the registry record. It does not describe missing responses, incomplete block sequences, participant withdrawals, or how any such observations were handled. The stated participation timeframe ranges from two blocks in one day to 24 blocks over 6 days, but that range alone does not establish how incomplete participation was treated analytically.

The record specifies that blocks with D′ below 0.5 were excluded. It does not report how many blocks met that criterion, whether any other exclusions occurred, or the number of participants or blocks included in an analysis. This threshold-based exclusion should not be treated as a missing-data procedure: the registry presents it as an outcome-related inclusion rule, and does not describe its relationship to missing observations.

The analysis populations are not reported in detail. There is no registry account of an intention-to-treat population, a per-protocol population, or another named analysis set. Because allocation is listed as not applicable and the study is single-group, terminology developed for randomized treatment comparisons may not map directly to this design. The record does not provide enough information to establish which participants, blocks, or responses contributed to any statistical result.

11. What the Result Does, and Does Not, Mean

The registry record establishes that the study examined visual decisions about skin images under blocks with different prevalence and feedback conditions. It also defines a primary outcome based on D′ in Block 2 in relation to the preceding block and an average across conditions. These details describe the research question and planned measurement. They do not establish the size, direction, or statistical certainty of a result.

No numerical primary-outcome estimate, confidence interval limits, or analysis result is reported in the registry record. Accordingly, the posted information cannot show whether D′ changed, how large a change was observed, or how compatible the data are with different possible effect sizes. The presence of a defined outcome and a completed study status is not a substitute for a reported result.

The study summary’s statement that over 300,000 individual judgements were collected describes the scale of response collection. It does not alone establish how many independent observations support a particular estimate, how much participants varied, or how precise the estimate would be. The registry does not provide the analysis method or enough detail about the distribution of responses to answer those questions.

General explanation: behavioral findings from image-labeling tasks concern performance under the studied task and conditions. They do not, by themselves, establish how a clinician will perform in a different setting or whether a decision process improves patient outcomes. The registry describes an online experiment using skin images and a labeling app; it does not report a clinical-care outcome.

The record’s exclusion of blocks with D′ below 0.5 also defines a restriction on the outcome population. Any interpretation of an estimate would need to account for that rule and for the unreported number of excluded blocks. No claim about broader populations, alternative tasks, or clinical practice can be drawn from the registry description alone.

12. Why This Trial Matters Statistically

The study addresses a statistical feature of sequential decision tasks: measurements made in one block may depend on the context established by earlier blocks. Examining feedback and target prevalence within a block-based task can make the order and history of observations important to interpretation. This is relevant to biostatisticians and clinical researchers because an apparent condition effect can be difficult to interpret if the design’s sequence and repeated observations are not clearly described.

The study also illustrates the difference between the number of responses and the number of participants. The registry reports enrollment of 1121 and says over 300,000 individual judgements were collected. Repeated responses can provide detailed information about behavior, but they are not automatically independent observations. Statistical analysis must match the data structure, and the registry record does not report how repeated responses were modeled.

D′ is a useful example of an outcome that combines correct detection and false alarms into a discrimination measure. Its definition makes clear why raw classification frequency alone may not answer the scientific question. The record also states an exclusion rule for low D′ blocks, underscoring that outcome construction and analysis-population rules affect what an estimate represents.

Finally, the record demonstrates the boundary between a well-described research question and an interpretable statistical result. It identifies conditions, outcome construction, and a confidence-interval type, but reports no numerical primary result or analysis. Readers therefore need to separate what the study set out to measure from what the registry record allows them to conclude.

13. Limitations

• The posted summary statistics are the only data these results rest on; patient-level data, full protocols and statistical analysis plans are not available.
• Recalculated results are simple, unadjusted calculations from posted summaries. They do not replace the sponsor's prespecified analysis.

The most consequential limitation for interpreting results from the registry record is that numerical outcome findings are not reported. There is no mean change, confidence interval limit, or inferential result to assess. The record also lists zero analyses, so it does not document the statistical procedure used to produce an estimate.

The design details are incomplete. Allocation is listed as not applicable, masking as none, and the study as single-group, but the sequence-assignment method and the ordering of feedback and prevalence conditions are not reported. Without those details, the potential influence of order, learning, or participant characteristics cannot be evaluated. The registry does not specify whether participants completed comparable sequences or how many contributed to each condition.

The primary outcome excludes blocks with D′ below 0.5, but the rationale and number of excluded blocks are not reported. The registry also does not describe missing responses, incomplete participation, or analysis-population definitions. These omissions limit assessment of how the analyzed observations relate to the collected judgements.

Although over 300,000 individual judgements are described, the registry does not report their distribution across participants, blocks, or conditions, or how dependence among repeated observations was handled. It also does not give model assumptions, covariate adjustments, or multiplicity procedures. These are not necessarily flaws in the study; they are details that cannot be assessed from the registry record.

The online image-labeling task and the stated range of participant expertise define the context of the research, but the registry does not provide further participant characteristics or evidence about performance in clinical care. The record describes a behavioral experiment, not a reported evaluation of patient outcomes.

14. Related Tutorials

15. Related Statistical Calculators

16. Sources

• ClinicalTrials.gov: PROJECT 2 EXAMPLE: Feedback X Prevalence Using Dermatology Stimuli — NCT05244122.
• Registry record last updated 2022-10-26; status last verified 2022-10.
• Results first posted 2022-10-26.
• Data retrieved from ClinicalTrials.gov on 2026-11-09.
• Group sizes, means, event counts and sponsor-reported p-values are reproduced as posted. Effect estimates, confidence intervals and p-values not attributed to the sponsor are independently calculated from the posted summary statistics.

17. Record Summary

NCT05244122 is a completed, single-group behavioral study sponsored by Brigham and Women's Hospital. The registry lists enrollment of 1121, allocation as not applicable, and masking as none. The study examined feedback and low- or high-prevalence blocks in an online task in which participants classified skin images as melanoma or benign nevi.

The primary outcome is change in D′ between pairs of blocks, with a 95% confidence interval as the stated dispersion measure. The registry defines D′ using true-positive and false-positive rates and says blocks with D′ below 0.5 were excluded. Participation could range from two blocks in one day to 24 blocks over 6 days; the arm description states that blocks contained 80 trials.

The record reports no numerical primary-outcome estimate, confidence interval limits, or analysis results. It lists two outcome measures and zero analyses. The study status is completed, but the registry description alone does not quantify the observed change or its precision.