AI-based Medical Device Validation for Early Melanoma Detection: Complete Statistical Analysis

1. Trial at a Glance

This page covers the registry results for AI-based Medical Device Validation for Early Melanoma Detection (NCT06221397), a registered phase not stated study. The primary outcome is Area Under the ROC Curve (AUC) for Melanoma Detection.

2. Clinical Question

This is an observational validation study, not a trial in which the registry reports that participants were assigned to different care strategies. The study group consisted of adults whose lesions were already suspected of malignancy. The registry describes photographing lesions and using clinical information and biopsy results, while participants continued to receive their usual medical care.

The central statistical question is about discrimination: whether the system tends to assign higher predicted probabilities to lesions ultimately classified as melanoma than to lesions classified as non-melanoma. The primary outcome is an AUC, rather than a direct estimate of how often the system is correct at a particular operating threshold. Other listed outcomes include accuracy across ICD categories, sensitivity, specificity, predictive values, and AUC for malignancy detection. Their values are not reported in the registry record.

The summary also raises a comparison with dermatologists’ judgments. However, the primary outcome definition focuses on the device’s ability to distinguish melanoma from non-melanoma cases. A numerical comparison between device and clinician performance, including the method used to compare them, is not reported in the registry record.

3. Trial Design

The registry record identifies this as an observational study of a computer-aided diagnosis system using artificial intelligence algorithms for early, noninvasive, in vivo cutaneous melanoma detection. Its status is completed. The recorded study start is 2020-09-17, and the primary completion date is 2023-11-13. The sponsor is AI Labs Group S.L.

The study group is described as adult patients with skin lesions suspected of being malignant who attended the Dermatology Departments of Hospital Universitario Cruces and Hospital Universitario Basurto. The registry reports enrollment of 105 patients and notes that 114 lesions were registered because a participant could have more than one lesion. Those are different units of observation: patient counts describe people, while lesion counts describe the lesions evaluated. The record does not provide further detail about how analyses account for lesions from the same person.

Participants’ lesions were photographed using a special camera attached to a smartphone. The study summary says researchers used participants’ clinical data and biopsy results, and compared the AI results with final diagnoses made by doctors or through biopsy. It also describes a comparison with expert dermatologists’ clinical judgment. The registry record does not report a randomized allocation or a separate control arm; the listed study group is patients with suspected cutaneous malignancy. Accordingly, performance comparisons should not be interpreted as randomized treatment effects.

The primary outcome was measured at the time of the single clinical visit, identified as baseline. It is the AUC for melanoma detection. The registry lists 11 outcome measures, including secondary measures of classification accuracy, AUC for malignancy detection, sensitivity, specificity, and predictive values. Although the study status is completed, outcome estimates and analysis results are not reported in the registry record. Details of recruitment procedures, the protocol’s statistical plan, and any prespecified sample-size justification are not reported in the registry record.

4. Participant Flow and Accounting

The registry reports how participants moved through each stage of the study. All counts below are exactly as posted; missing counts are not inferred from other stages.

Before assignment: After enrollment, the most significant event was a pilot data analysis of the first 40 participants. Researchers found that the initial cohort, which only included high-suspicion cases requiring biopsy, was not representative of daily clinical practice. Consequently, the intended sample was extended to 200 to include common benign lesions like nevi. But finally, 105 patients were enrolled. Two participants were excluded from the final analysis due to non-conclusive clinical diagnoses.

Overall Study

Reasons for not completing this period:

Analysis population for the primary outcome, as defined in the registry: This difference is due to the fact that a single participant could have more than one lesion; as a result, there were 105 patients but 114 lesions registered.

5. Baseline Characteristics

The table reproduces the baseline characteristics posted in the registry, which describe the participants analysed at baseline and help show whether the randomized groups were comparable.

6. Primary Endpoint

The registry lists the primary outcome as Area Under the ROC Curve (AUC) for Melanoma Detection, measured over At the time of the single clinical visit (Baseline)..

The primary endpoint is the area under the receiver operating characteristic curve for melanoma detection. The registry defines it as a measure of the device’s ability to distinguish melanoma from non-melanoma cases using predicted probabilities. The measurement timeframe is the baseline clinical visit, and the registry specifies a 95% confidence interval as the dispersion measure. The AUC estimate and confidence interval are not reported in the registry record.

In general, an ROC curve describes how sensitivity and specificity vary as a classifier’s decision threshold changes. The AUC compresses that discrimination pattern into a single summary. It is useful when the question is whether predicted scores tend to rank cases with the target condition above cases without it, without choosing one particular threshold in advance. It does not, by itself, identify a clinically preferred threshold or describe the consequences of false-positive and false-negative classifications.

The primary endpoint is distinct from accuracy, sensitivity, specificity, and predictive values, which are also listed as secondary outcome areas. Those measures answer different questions and may depend on a selected threshold or on the frequency of the condition in the evaluated group. The registry record does not report the threshold used for any classification-based secondary measure, nor the corresponding numerical results.

The unit structure matters to interpretation. The registry reports 105 patients and 114 lesions, explaining that participants could contribute more than one lesion. An AUC calculated at the lesion level may have uncertainty affected by within-person clustering. The registry record does not report whether the confidence interval accounts for that dependence or how the analysis handled multiple lesions from the same participant.

7. Why No Re-analysis Is Shown

8. Statistical Interpretation

9. Statistical Methods Explained

The methods can be understood by separating the performance question, the uncertainty around that performance, the observation unit, and the proposed comparisons. The registry record defines the primary outcome but does not report numerical analyses. The explanations below describe the general meaning of the stated statistical measures, rather than asserting that a particular calculation or analysis was performed.

What does an AUC describe?

In general, a classifier assigns scores or probabilities to observations. An ROC curve traces sensitivity against the false-positive rate as the decision threshold changes. The area under that curve summarizes how well the scores separate cases from non-cases across thresholds. It is a discrimination measure: it concerns ranking, not whether predicted probabilities are calibrated to observed risk. A favorable discrimination summary would not, on its own, establish that the system is suitable for a particular clinical use or that its classifications improve outcomes.

The registry’s primary endpoint is specifically AUC for melanoma detection, using predicted probabilities to distinguish melanoma from non-melanoma lesions. The outcome is measured at the baseline visit. Because the registry record does not report an AUC estimate, readers cannot assess the observed discrimination from this record. The listed 95% confidence interval is a requested or specified dispersion measure, but its bounds are not reported.

Why report a confidence interval?

In general, a confidence interval conveys sampling uncertainty around an estimate under the statistical model and analysis procedure used. It can help readers distinguish a relatively precise estimate from one with substantial uncertainty. Its interpretation depends on appropriate assumptions, including how observations were sampled and whether their dependence was handled. Here, the registry record names a 95% confidence interval for the primary endpoint but does not give its limits or explain the calculation method. The uncertainty therefore cannot be independently assessed.

Why do patient and lesion counts both matter?

The registry reports 105 patients and 114 lesions, and explains that a participant could contribute more than one lesion. This means the data may include multiple lesion observations from a person. In general, observations from the same person can share characteristics and may not be statistically independent. If an analysis assumes independence when observations are clustered, uncertainty estimates can be misleading. Appropriate approaches can account for clustering, but the registry record does not report whether such an approach was used. It also does not state whether the primary analysis unit was the lesion, the patient, or another defined unit.

The difference between patient-level and lesion-level interpretation is substantive as well as statistical. A lesion-level result describes classification of lesions in the study data; it is not automatically a patient-level measure. The record’s stated primary endpoint refers to melanoma and non-melanoma cases, while its population description distinguishes patients from registered lesions. The precise mapping between those units and the analysis is not reported.

How should sensitivity, specificity, and predictive values be read?

In general, sensitivity describes the fraction of cases with the target condition that a classifier identifies at a chosen threshold. Specificity describes the fraction of cases without the condition that it classifies as negative. Positive and negative predictive values describe the chance that a positive or negative classification corresponds to the target status in the evaluated population. These measures provide information about threshold-specific classification, unlike an AUC that summarizes discrimination over thresholds. Predictive values can also vary with the prevalence of the condition in the population being assessed. The registry lists these outcome areas, but does not report their estimates, thresholds, or the underlying classification counts.

What do planning assumptions mean?

The registry’s planning assumptions describe illustrations based on the observed effect, not the study’s own design. They state a two-sided significance level of 0.05, equal allocation between two arms, and a normal approximation, with the effect to detect defined as a stated fraction of the observed effect. These assumptions describe a hypothetical planning exercise. They should not be mistaken for evidence that the observational study assigned participants equally to two arms, or for observed power. The registry record does not report a randomized two-arm comparison, and the numerical observed effect and planning outputs are not reported in the registry record.

In general, a sample-size calculation estimates the number of observations that might be needed under specified assumptions to detect a chosen effect with a chosen probability. Such a calculation is conditional on those assumptions and does not measure the probability that a completed study’s finding is true. The assumptions alone cannot establish the study’s original design rationale, because the sample-size justification and statistical plan are not reported in the registry record.

What can be concluded from the analysis entries?

The registry record reports no analysis records. It does not describe adjustment for covariates, subgroup analyses, handling of missing observations, or a method for comparing AI performance with clinician judgment. Those details are necessary for evaluating how the stated objectives were translated into statistical procedures. Their absence does not establish that no analyses occurred; it means those procedures are not reported in the registry record.

10. Multiplicity, Missing Data and Analysis Populations

The registry lists 11 outcome measures, with a primary outcome of AUC for melanoma detection and secondary outcomes spanning classification accuracy, malignancy detection, sensitivity, specificity, and predictive values. Multiple outcomes create a multiplicity question: when several statistical questions are assessed, the overall chance of a chance-positive finding can differ from the chance associated with any one prespecified test. In general, researchers may address this through prespecification, adjustment, a hierarchy of tests, or careful interpretation. The registry record does not report a multiplicity strategy or the results of statistical tests.

The record describes the primary outcome timeframe as the baseline clinical visit, but does not report the primary endpoint result or an analysis population for that result. It does not describe how missing images, incomplete clinical data, unavailable biopsy results, or otherwise unevaluable lesions were handled. Missing-data methods and the number or characteristics of missing observations are not reported in the registry record.

The record also does not define an intention-to-diagnose or other analysis population, state whether every eligible lesion was included, or describe exclusions from analysis. It is therefore not possible to determine from the registry record whether the analysis set matched the enrolled group or how any exclusions may have affected estimates.

The distinction between patients and lesions adds a further analysis-population issue. The record reports 105 patients and 114 lesions, with the explanation that some participants had more than one lesion. It does not report whether analyses were conducted per lesion or per patient, whether dependence within participants was addressed, or whether different outcomes used different analysis sets. These omissions limit assessment of both the estimates and their uncertainty.

11. Safety Results

The registry reports serious adverse events as affected participants divided by participants at risk.

Deaths

Other (non-serious) adverse events

Participants with at least one non-serious event at or above the registry's 0% reporting threshold.

The adverse-event record lists the group as patients with suspected cutaneous malignancy. It reports 105 participants at risk for serious adverse events and zero affected by a serious adverse event. For other adverse events, it reports 105 at risk and zero affected. The record also reports zero deaths, and no serious adverse-event terms are listed.

These are the figures recorded for this study group; they do not establish that there was no risk. In general, an event count of zero means no events of that type were recorded in the reported data, not that an event is impossible or that the intervention has been shown safe in every setting. Interpretation also depends on the period and method of safety surveillance, the definitions used, and how completely events were captured. The timeframe and detailed surveillance procedures are not reported in the registry record.

The study is observational and its summary says participants continued to receive medical care as usual. The adverse-event figures should not be read as a randomized safety comparison between AI-supported and non-AI-supported care. The registry record does not report a separate control group or a causal analysis linking the device to adverse events. The counts describe recorded events, not a comparative estimate of risk.

12. What the Result Does, and Does Not, Mean

The registry record establishes that a completed observational study evaluated an AI-based CAD system for skin-lesion analysis and defined AUC for melanoma detection as its primary outcome. It also describes the intended comparison with clinical judgment and final diagnoses, the lesion-imaging process, and several secondary performance measures. Those details explain what the study was designed to examine, but they do not reveal the numerical result of the primary analysis.

The most important boundary is that the primary AUC estimate and its 95% confidence interval are not reported in the registry record. Without them, the system’s measured ability to discriminate melanoma from non-melanoma cases cannot be characterized. Nor can the precision of that estimate be evaluated. The record likewise does not report the numerical results for the secondary measures, such as sensitivity, specificity, predictive values, or classification accuracy.

The study summary’s proposed comparison with dermatologists should also be read as an objective, not a demonstrated comparative finding. The registry record does not give paired performance estimates, a contrast between clinician and AI results, or a statistical method for such a comparison. It therefore does not establish that either approach performed better.

In general, diagnostic performance measures describe agreement or discrimination against a reference classification in the evaluated data. They do not by themselves show that using a tool changes patient outcomes, improves care, or has favorable consequences in a different population. The registry describes an observational evaluation and says that medical care was not changed by the study. It does not report a clinical-impact analysis.

The record reports 105 patients and 114 lesions, noting that some people contributed more than one lesion. Results at lesion level should not automatically be treated as results at patient level, and uncertainty may depend on whether the analysis accounts for clustering. That method is not reported. Thus, even a numerical lesion-level result would require careful interpretation of its unit and analysis assumptions.

The adverse-event record reports zero serious adverse events, zero other adverse events, and zero deaths among the listed group. Those counts describe what the registry records; they do not establish an absence of risk or a comparative safety benefit. Overall, the registry supports a description of the study question and design, but not a conclusion about diagnostic performance, comparative superiority, or clinical impact.

13. Why This Trial Matters Statistically

This study raises a statistically important question for diagnostic validation: whether an algorithm can distinguish melanoma from non-melanoma lesions using image-based predictions. AUC is a relevant summary for that discrimination question because it considers performance across decision thresholds. The registry’s choice of primary outcome therefore focuses on a recognizable feature of diagnostic classifiers, rather than restricting evaluation to a single reported classification threshold.

The study also illustrates why performance metrics should not be treated as interchangeable. AUC summarizes discrimination, whereas sensitivity and specificity describe classification at a threshold, and predictive values relate classifications to condition status in the evaluated group. Accuracy across ICD categories addresses a different classification task. The registry’s listed outcome domains reflect several dimensions of performance, but their results are not reported in the registry record.

A further statistical issue is that the study includes multiple lesions from some participants. The registry reports 105 patients and 114 lesions. When observations are clustered within people, uncertainty may depend on how that dependence is modeled. Reporting the analysis unit and the approach to clustering would help readers interpret both an estimate and its confidence interval. Those methods are not reported in the registry record.

The record also highlights the distinction between validation and clinical impact. Comparing predicted classifications with final diagnoses can assess diagnostic performance, while showing that a tool improves care requires different evidence. The study summary states that participants’ care continued as usual, and the registry describes an observational design. No clinical-impact result is reported.

Finally, a completed registry entry is most informative when it includes estimates and uncertainty alongside outcome definitions. Here the primary endpoint is specified, but its AUC and confidence interval are not reported. That gap prevents quantitative assessment and limits comparison with other diagnostic approaches. The statistical importance of the study question is clear, but the registry record does not provide the outcome data needed to assess the answer.

14. Limitations

• The posted summary statistics are the only data these results rest on; patient-level data, full protocols and statistical analysis plans are not available.
• Recalculated results are simple, unadjusted calculations from posted summaries. They do not replace the sponsor's prespecified analysis.

The central limitation for interpretation is the absence of reported primary and secondary outcome estimates. Although the primary endpoint and its 95% confidence interval are specified, neither the estimate nor interval bounds are reported in the registry record. The values for the listed secondary measures are also not reported. Consequently, statistical precision and diagnostic performance cannot be assessed from this record.

The registry describes a comparison with expert dermatologists’ judgments and final diagnoses, but does not give the clinician results, a direct comparative estimate, or a method for comparing paired assessments. The reference standard details are limited to the summary’s mention of doctors or biopsy. How final diagnoses were established, and whether adjudication procedures were standardized, are not reported in the registry record.

The population is described as adults presenting at two named dermatology departments with lesions suspected of malignancy. The record does not report additional participant characteristics or the distribution of melanoma and non-melanoma diagnoses. Without those details, it is not possible to assess the composition of the evaluated sample or how performance might vary across patient or lesion subgroups.

The record reports 105 patients and 114 lesions but does not report the analysis unit, methods for accounting for multiple lesions per participant, or the confidence-interval procedure. It also does not describe missing-data handling, exclusions, the analysis population, a prespecified threshold for threshold-based measures, or a multiplicity approach. Those omissions restrict appraisal of the analysis and its uncertainty.

The observational design does not provide a randomized comparison of care strategies. The registry states that care was not changed for participants, and does not report a clinical-impact analysis. The completed status and the adverse-event counts do not resolve these limitations or substitute for unreported performance results.

15. Related Tutorials

16. Related Statistical Calculators

17. Sources

• ClinicalTrials.gov: AI-based Medical Device Validation for Early Melanoma Detection — NCT06221397.
• Registry record last updated 2026-03-16; status last verified 2026-02.
• Results first posted 2026-03-16.
• Data retrieved from ClinicalTrials.gov on 2026-11-09.
• Group sizes, means, event counts and sponsor-reported p-values are reproduced as posted. Effect estimates, confidence intervals and p-values not attributed to the sponsor are independently calculated from the posted summary statistics.

18. Record Summary

NCT06221397 is a completed observational study of an AI-based CAD system for skin-lesion analysis, sponsored by AI Labs Group S.L. The registry records a study start of 2020-09-17, primary completion on 2023-11-13, and enrollment of 105. The study group comprised adults with lesions suspected of malignancy at Hospital Universitario Cruces and Hospital Universitario Basurto. The registry notes 114 lesions because some participants could contribute more than one.

The primary endpoint is AUC for melanoma detection at the baseline clinical visit, with a 95% confidence interval specified. The study summary describes comparison with dermatologists’ judgments and final diagnoses made by doctors or through biopsy. The listed secondary outcomes include accuracy, AUC for malignancy detection, sensitivity, specificity, and predictive values.

The primary estimate, confidence interval, secondary outcome results, and analysis methods are not reported in the registry record. The adverse-event record lists zero serious events, zero other events, and zero deaths. These figures describe recorded counts and do not constitute a comparative safety analysis.