Validation of Criteria for Identification of Epileptiform Discharges in EEG Recordings of Patients With Epilepsy: Complete Statistical Analysis

1. Trial at a Glance

This page covers the registry results for Validation of Criteria for Identification of Epileptiform Discharges in EEG Recordings of Patients With Epilepsy (NCT03533374), a registered phase not stated study. The primary outcome is Inter-rater Agreement of the International Federation of Clinical Neurophysiology (IFCN) Criteria (Cut-off=4) in Sensor Space and of Detection of Epileptiform Discharges (EDs) in Source Space.

2. Clinical Question

The clinical context is the visual identification of sharp EEG discharges that may be classified as epileptiform. The record says the IFCN set contains six operational criteria and that at least four need to be present for a discharge to be classified as epileptiform. It describes the criteria as largely based on expert opinion and says their sensitivity, specificity, optimal combination, and inter-rater agreement were uncertain. These statements explain the motivation for evaluating the criteria; they are not findings from the study.

The central question is therefore about consistency between raters, rather than whether one treatment is better than another. Agreement is relevant because visual assessment requires interpretation of EEG features. The registry record does not identify the number of raters, their selection, or the reference standard used for any accuracy assessment.

3. Trial Design

The registry describes a completed diagnostic-validation study, not a randomized treatment trial. It lists patients with epileptic seizures and patients with non-epileptic seizures. Both groups are described as receiving EEG and visual evaluation. The study began on 2012-01-01, and its primary completion date is listed as 2018-01-01. Enrollment is listed as 100; the record does not say whether this is planned or actual enrollment, nor does it give group-specific counts.

EEG was recorded using a Nicolet-One system with a standard 25-electrode array. Recordings with sharp transients were inspected by physicians described as experts. The primary outcome concerns agreement in two settings: applying the IFCN criteria with a cut-off of 4 in sensor space, and detecting epileptiform discharges in source space. The registry lists a one-year timeframe for this outcome. It does not clarify the starting point or operational meaning of that timeframe.

The study objectives also include assessing the combination of criteria with the best sensitivity and specificity and evaluating accuracy in source space. Those objectives describe intended questions, not reported results. The record does not provide a detailed sequence of study procedures, explain how recordings or discharges were selected for rating, or state whether raters were blinded to participant classification or to one another’s assessments.

Allocation is not stated, and the two groups are not described as randomized or as a like-for-like control comparison. They should be understood as diagnostic groups within an evaluation of EEG criteria, not as treatment arms. The registry record does not report a protocol, a sample-size justification, or a detailed statistical analysis plan. It lists zero analysis records, which limits the detail available about how the stated objectives were addressed.

4. Participant Flow and Accounting

The registry reports how participants moved through each stage of the study. All counts below are exactly as posted; missing counts are not inferred from other stages.

Overall Study

5. Baseline Characteristics

The table reproduces the baseline characteristics posted in the registry, which describe the participants analysed at baseline and help show whether the randomized groups were comparable.

6. Primary Endpoint

The registry lists the primary outcome as Inter-rater Agreement of the International Federation of Clinical Neurophysiology (IFCN) Criteria (Cut-off=4) in Sensor Space and of Detection of Epileptiform Discharges (EDs) in Source Space, measured over 1 year.

The primary outcome is inter-rater agreement for the IFCN criteria, using a cut-off of 4 in sensor space, and for detection of epileptiform discharges in source space. Agreement is quantified with Gwet’s AC1, with a 95% confidence interval. The registry record lists the unit as an agreement coefficient and the timeframe as one year.

Agreement measures how consistently raters classify or assess the same material. It is distinct from diagnostic accuracy: raters can agree with one another without necessarily matching an external reference standard. Conversely, an accuracy assessment requires a reference classification and a defined comparison between the assessment and that reference. Although diagnostic accuracy is among the study objectives, the registry record does not identify a reference standard or report sensitivity or specificity estimates.

AC1 is used to quantify agreement beyond chance. The record explains that Gwet’s coefficient was chosen because, compared with Cohen’s Kappa, it is less affected by prevalence and marginal probability, thereby avoiding the issue known as the paradoxes of kappa. This choice concerns the behavior of the agreement statistic; it does not by itself establish whether agreement was high or whether classifications were accurate.

No observed AC1 estimate or confidence interval is reported in the registry record. As a result, the endpoint definition can be described, but the degree and precision of agreement cannot be interpreted from this record.

7. Why No Re-analysis Is Shown

8. Statistical Interpretation

9. Statistical Methods Explained

What does an agreement coefficient measure?

Agreement coefficients summarize how often or how consistently assessors assign the same classifications, while accounting for agreement that might occur by chance according to the coefficient’s model. The primary outcome uses Gwet’s AC1 for beyond-chance agreement. It is applied to the IFCN criteria with a cut-off of 4 in sensor space and to detection of epileptiform discharges in source space. The coefficient is not itself a measure of sensitivity or specificity, and it does not establish that a classification is correct.

The registry record explains the choice of AC1 by noting that, compared with Cohen’s Kappa, it is less affected by prevalence and marginal probability. These features matter because agreement statistics can behave differently when ratings or categories are distributed unevenly. The record names this concern as the paradoxes of kappa. In general, a statistic’s robustness to a particular distributional issue does not eliminate the need to understand the ratings, categories, and analysis population used to calculate it.

Why report a confidence interval?

An agreement estimate calculated from observed ratings is subject to sampling uncertainty. A confidence interval expresses that uncertainty around the estimate under the method’s assumptions. Its width indicates how precisely the agreement coefficient has been estimated: a narrower interval generally conveys more precision, and a wider one less. The registry specifies a 95% confidence interval but does not report its limits or the AC1 estimate. No assessment of precision or strength of agreement is therefore possible from the record.

Confidence intervals are also useful because a single point estimate can obscure uncertainty. When an interval spans values that would be interpreted differently, the point estimate alone may give an incomplete picture. That general principle cannot be applied to this study’s result without its estimate and interval. The registry’s stated interpretation categories are poor for values below 0, slight from 0·01-0·20, fair from 0·21-0·40, moderate from 0·41-0·60, substantial from 0·61-0·80, and almost perfect from 0·81-1·00. These are the record’s interpretive categories; no category can be assigned without the observed coefficient.

What is the role of the IFCN cut-off?

The study focuses on six IFCN operational criteria for identifying epileptiform discharges. The record says that at least four criteria need to be present to classify a discharge as epileptiform. The primary outcome specifies the cut-off of 4 for the sensor-space agreement analysis. This threshold defines the classification rule being evaluated; it is not an outcome result. The record does not provide the individual criteria ratings, the distribution of classifications, or the results of analyses using other cut-offs.

In general, a classification threshold turns a set of observed features into a decision category. Different thresholds can change how often observations are classified in each category and may affect measures of agreement and accuracy. The registry describes an objective of identifying the combination of criteria with the best accuracy, but does not report how combinations were compared or which combination performed best. It also does not state whether the threshold was prespecified or selected during analysis.

How are agreement and diagnostic accuracy different?

Agreement describes consistency among raters or assessments. Diagnostic accuracy describes how classifications correspond to a reference standard. Sensitivity and specificity are accuracy measures that require a defined reference classification and a rule for comparing assessments against it. The study objectives mention sensitivity, specificity, and accuracy, but the registry record does not report estimates or specify the reference standard in the material describing the primary outcome. It is therefore important not to interpret a proposed accuracy objective as a result, or to treat agreement as evidence of accuracy.

The study also evaluates assessment in sensor space and source space. In general, sensor space refers to signals represented in recording channels, while source space refers to reconstructed signals associated with the regions generating electrical currents. The registry describes this distinction and says advances in signal analysis make source-space reconstruction possible. It does not report technical processing details, the reconstruction procedure, or comparative results between the two spaces.

What can the study design tell readers?

The record describes EEG and expert visual evaluation for patients with epileptic seizures and patients with non-epileptic seizures. Both groups had recordings made using a Nicolet-One system and a standard 25-electrode array, and recordings with sharp transients were inspected by physicians. In general, design details such as how recordings were sampled, whether assessors were blinded, and how multiple assessments were linked are relevant to interpreting agreement. Those details are not reported in the registry record.

The registry lists a one-year timeframe for the primary outcome, but does not explain its operational meaning. The record lists enrollment of 100 without group-specific counts, and does not report the number of raters or rating units. These omissions prevent a fuller assessment of how the agreement estimate was constructed and how representative the analyzed material may be.

The registry describes planning illustrations based on a stated fraction of an observed effect, using a two-sided significance level of 0.05, equal allocation between the two arms, and a normal approximation. For time-to-event planning, it describes total events; for other planning, it describes participants per arm. These are planning illustrations, not observed power, and the record does not provide the calculated planning values in the information available for this description. They should not be confused with the study’s enrollment or with evidence about the precision of its reported outcome.

10. Multiplicity, Missing Data and Analysis Populations

The registry record does not report how multiplicity was handled. The stated objectives include agreement for the IFCN criteria in sensor space, detection of epileptiform discharges in source space, assessment of the combination of criteria with the best accuracy, and evaluation of sensitivity and specificity. It does not describe which analyses were designated primary beyond the named agreement outcome, how additional outcomes were prioritized, or whether any adjustment was planned for multiple comparisons or criterion combinations.

Multiplicity matters in general when several hypotheses, outcomes, or candidate rules are evaluated. Conducting many analyses can increase the chance that an apparently notable finding arises by chance. An adjustment or a prespecified hierarchy can address some aspects of this issue, but the registry record does not state whether either approach was used. The objective of finding the best-performing combination also raises questions about selection and validation that cannot be resolved from the available record.

Missing-data procedures are not reported in the registry record. It does not state whether any EEG recordings, discharges, or ratings were missing, excluded, or unusable, or how incomplete ratings would be treated in an agreement analysis. The number of participants or observations included in each analysis is not reported. The listed enrollment alone does not describe the analysis population.

The record does not report analysis-population definitions, such as criteria for inclusion in the primary agreement analysis, nor does it provide sensitivity analyses for missing or excluded assessments. It lists zero analysis records. These omissions mean that readers cannot assess whether the analyzed material differed from the enrolled population, or whether analytic choices could have influenced the results. No statistical analysis plan or protocol is reported in the registry record.

11. What the Result Does, and Does Not, Mean

The registry record supports a clear description of the study’s purpose: evaluating agreement when experts identify epileptiform discharges using IFCN criteria in sensor space and assess discharges in source space. It also identifies the EEG equipment and electrode array used, the two seizure-classification groups, and Gwet’s AC1 with a 95% confidence interval as the primary agreement measure.

It does not report the observed agreement coefficient or its confidence interval. Therefore, it does not establish whether raters agreed poorly or strongly, nor how precisely agreement was estimated. It also does not report the sensitivity, specificity, or other accuracy results named among the study objectives. The registry record cannot support a conclusion about the performance of the IFCN criteria, the optimal combination of criteria, or the comparative accuracy of sensor-space and source-space assessment.

Even if an agreement estimate were available, agreement would not by itself show that raters’ classifications were correct. Accuracy requires comparison with a reference classification. Likewise, an objective to find the best-performing criteria combination is not evidence that one combination was identified or validated. These distinctions are important for interpreting diagnostic-validation research in general.

The completed status and enrollment listing do not fill these reporting gaps. The record does not state group-specific numbers, the number of raters, or the number of ratings contributing to analyses. It also does not describe missing observations or analysis-population rules. Readers should therefore treat the record as a description of a study and its intended measurements, rather than as a report of the study’s statistical findings.

No treatment comparison is at issue in this record. The groups are not randomized treatment arms, and the study is not a like-for-like control comparison. The statistics described concern how EEG findings were assessed, not the effect of a treatment or a recommendation for patient care.

12. Why This Trial Matters Statistically

This study addresses a statistical issue that is central to visual diagnostic assessment: whether different raters apply operational criteria consistently. A criterion set can be clearly described yet still require evaluation of how reliably it is used. Measuring agreement helps distinguish a rule that can be applied reproducibly from one whose interpretation may vary across assessors.

The study also distinguishes sensor-space review from source-space assessment. The registry describes source space as an approach made possible by reconstructing electrical currents in regions of the brain that generate them, rather than viewing signals only in recording channels. Evaluating agreement in both settings is statistically relevant because a change in representation may alter the information presented for assessment and the consistency of interpretation. The record states this as a study objective but does not report comparative results.

Using Gwet’s AC1 reflects attention to how agreement statistics can be affected by the prevalence of categories and the marginal probabilities of ratings. The registry says AC1 is less affected by these features than Cohen’s Kappa and was chosen to avoid the paradoxes of kappa. This is a methodological rationale, not a guarantee of validity or a result. The coefficient still needs to be interpreted alongside its estimate, confidence interval, data structure, and rating process.

The record also raises a broader question about translating criteria into diagnostic performance. Agreement, sensitivity, and specificity are different statistical properties. Assessing one does not automatically answer the others. A complete interpretation would require reported estimates and enough information about the reference classification and analysis procedures.

Because numerical outcomes are not reported in the registry record, the study’s statistical importance lies in the questions and measurement framework it identifies, not in a result that can be assessed from this record. Its stated objectives highlight reliability, accuracy, and the potential distinction between two EEG representations, while the missing estimates leave those questions unresolved for readers of the registry record.

13. Limitations

• The posted summary statistics are the only data these results rest on; patient-level data, full protocols and statistical analysis plans are not available.
• Recalculated results are simple, unadjusted calculations from posted summaries. They do not replace the sponsor's prespecified analysis.

The most direct limitation for interpretation is that the registry record does not report the primary AC1 estimate or its 95% confidence interval. It also does not report numerical findings for sensitivity, specificity, or the accuracy objectives. Without those results, the study’s observed agreement and diagnostic performance cannot be evaluated.

Important details about the rating process are not reported in the registry record. These include the number and characteristics of raters, whether they assessed the same recordings independently, whether they were blinded to participant classification or other ratings, and how disagreements or incomplete ratings were handled. The record does not specify the number of recordings or discharges used in each analysis, or the group-specific participant counts.

The record describes both sensor-space and source-space assessment but does not provide technical details of the source reconstruction or explain how the two assessments were compared. It identifies accuracy and selection of a best-performing criteria combination as objectives, but does not identify a reference standard, analysis procedures, or a validation approach for any selected combination.

The registry lists enrollment of 100 and a one-year timeframe for the primary outcome, but does not clarify the status of enrollment or the operational definition of that timeframe. It does not report a sample-size justification, protocol, statistical analysis plan, missing-data handling, analysis-population definitions, or multiplicity procedures. It lists zero analysis records. These omissions limit assessment of the methods and prevent conclusions about the robustness or generalizability of any findings.

The study is not a randomized treatment comparison, and the seizure-classification groups are not described as like-for-like controls. Comparisons between those groups should not be interpreted as treatment effects. These limitations concern what can be established from the registry record and do not imply that unreported procedures were or were not performed.

14. Related Tutorials

15. Related Statistical Calculators

16. Sources

• ClinicalTrials.gov: Validation of Criteria for Identification of Epileptiform Discharges in EEG Recordings of Patients With Epilepsy — NCT03533374.
• Registry record last updated 2020-07-22; status last verified 2019-07.
• Results first posted 2020-07-22.
• Data retrieved from ClinicalTrials.gov on 2026-11-09.
• Group sizes, means, event counts and sponsor-reported p-values are reproduced as posted. Effect estimates, confidence intervals and p-values not attributed to the sponsor are independently calculated from the posted summary statistics.

17. Record Summary

The registry record describes a completed diagnostic-validation study of visual EEG assessment for patients with epileptic seizures and patients with non-epileptic seizures. It lists enrollment of 100, a start date of 2012-01-01, and a primary completion date of 2018-01-01. Both groups are described as having EEG recorded with a Nicolet-One system and a standard 25-electrode array, with experts inspecting recordings containing sharp transients.

The primary outcome is inter-rater agreement for the IFCN criteria with a cut-off of 4 in sensor space and detection of epileptiform discharges in source space, measured using Gwet’s AC1 with a 95% confidence interval. The registry also describes objectives concerning the criteria combination with the best accuracy and assessment in source space.

The registry record does not report the primary estimate, its confidence interval, or diagnostic accuracy results. It lists three outcome measures and zero analysis records. Participant characteristics, group sizes, rater details, analysis populations, and a statistical analysis plan are not reported in the registry record.