Electrically Evoked Compound Action Potentials Human Observation Medtronic Algorithm Comparison Study: Complete Statistical Analysis

1. Trial at a Glance

This page covers the registry results for Electrically Evoked Compound Action Potentials Human Observation Medtronic Algorithm Comparison Study (NCT04765735), a randomized na study. The primary outcome is The Proportion of Subjects With a Reduction in Overstimulation Sensation With SCS Using a Closed-loop Algorithm Compared With SCS in Open-loop.

2. Clinical Question

The question is about a within-subject comparison of two testing periods, not simply a comparison of separate groups receiving different treatments. Each subject’s average intensity score during closed-loop testing is compared with that subject’s average during open-loop testing. A subject is counted as having a reduction when the closed-loop average is less than the open-loop average.

The registry describes a randomized, single-blind, crossover study, with testing conducted in clinic over one day. Randomization determined which testing condition came first. The primary analysis was pre-specified to report the total proportion with success together, regardless of randomization sequence. The record does not provide sequence-specific outcome estimates or the number of participants who met the endpoint.

This is a question about a sensory experience measured during protocol-prescribed activities. It is not an endpoint for overall pain relief, long-term function, or continued benefit outside the clinic. Those outcomes are not reported as primary measures in the registry record. The endpoint definition concerns the relative average scores during the two test periods; it does not, by itself, indicate the size of any change or how many subjects experienced one.

3. Trial Design

The registry describes ECHO-MAC as a prospective, multi-center, randomized, single-blind, crossover, non-significant risk IDE research study. It was designed to evaluate in-clinic performance of a closed-loop algorithm in spinal cord stimulation therapy with an ENS device. The listed condition is chronic pain, and the listed intervention is open-loop and closed-loop testing using an external spinal cord stimulator.

The crossover structure means that both testing conditions were experienced in sequence by participants, rather than assigning one group to open-loop testing only and another group to closed-loop testing only. The registry lists two sequences: open-loop testing followed by closed-loop testing, and closed-loop testing followed by open-loop testing. Randomization assigned the testing sequence. The registry does not report the number assigned to each sequence or sequence-specific endpoint results.

The record identifies the study as single-blind but does not state which parties were blinded or how blinding was maintained. It also does not report details about the implementation of randomization, allocation concealment, or any procedures intended to address period or carryover effects. Those details should not be inferred from the crossover label alone.

Enrollment is listed as 52. The study start date is 2021-02-23, the primary completion date is 2021-06-29, and the status is completed. The primary endpoint timeframe is in-clinic testing over one day. These dates describe the registry record’s study milestones; they do not indicate the duration of any participant’s benefit or follow-up beyond the stated endpoint timeframe.

The registry identifies one primary outcome measure and one analysis. It describes the analysis population as all randomized subjects, regardless of sequence, while the endpoint description defines the proportion among subjects who have in-clinic testing. The record does not reconcile those descriptions with a reported denominator or explain how missing endpoint observations were handled.

4. Participant Flow and Accounting

The registry reports how participants moved through each stage of the study. All counts below are exactly as posted; missing counts are not inferred from other stages.

Before assignment: Subjects were randomized at the In-Clinic Testing Visit following their Enrollment/Baseline Visit (consent visit). Ten Subjects exited the study prior to randomization resulting in 42 subjects that were randomized.

Overall Study

Analysis population for the primary outcome, as defined in the registry: Intent-to-treat analysis set included all subjects who were randomized. Analysis was pre-specified in the study statistical analysis plan to report out on the total proportion of subjects with success together, regardless of the randomization sequence.

5. Baseline Characteristics

The table reproduces the baseline characteristics posted in the registry, which describe the participants analysed at baseline and help show whether the randomized groups were comparable.

6. Primary Endpoint

The registry lists the primary outcome as The Proportion of Subjects With a Reduction in Overstimulation Sensation With SCS Using a Closed-loop Algorithm Compared With SCS in Open-loop, measured over In-clinic testing (1 day).

The primary endpoint is the proportion of subjects with a reduction in overstimulation sensation during closed-loop testing compared with open-loop testing. During protocol-prescribed activities, subjects rated the intensity of each overstimulation sensation on a 5-point Likert scale. An average intensity score was calculated for each subject during each testing period.

The endpoint classifies a subject as having a reduction when that subject’s average score during the closed-loop period is less than the average during the open-loop period. The measure is binary at the participant level: a subject either meets this stated criterion or does not. The primary result would ordinarily report the count or proportion meeting the criterion among the defined analysis population, but the registry record does not report that result.

A participant-level proportion is useful for expressing how often the specified direction of within-person change occurred. It preserves the paired nature of the crossover comparison by assessing each subject against their own score in the other condition. It also gives a readily interpretable summary of the endpoint’s success rule. However, this classification does not show the magnitude of score change for each subject, the distribution of scores, or whether a lower score is important to participants. Those details are not reported in the registry record.

The endpoint is specific to the in-clinic sensory rating procedure. It should not be read as a measure of overall pain, long-term symptom control, or treatment effectiveness beyond the stated testing conditions. The record does not report an observed endpoint proportion, uncertainty interval, or test result.

7. Why No Re-analysis Is Shown

8. Statistical Interpretation

9. Statistical Methods Explained

What does the endpoint count?

The endpoint turns two average intensity scores for each subject into a participant-level classification. The subject is counted as having a reduction if the closed-loop average is less than the open-loop average. Otherwise, the subject does not meet the stated success criterion. The primary measure is the proportion of subjects classified as having a reduction, rather than the average difference in score across all subjects.

In general, converting a continuous or ordinal rating into a yes-or-no outcome can make a result easier to communicate, but it discards information. The classification does not retain how far apart a subject’s two averages were. It also treats any lower closed-loop average as meeting the rule, without distinguishing the size of the reduction. The registry record does not provide the underlying scores or the resulting count of subjects who met the rule.

Why does the crossover design matter?

In a crossover study, each participant experiences the conditions in sequence. The primary endpoint compares each subject’s average under closed-loop testing with that same subject’s average under open-loop testing. This paired structure can account for stable differences between people because the comparison is within subject. General statistical methods for paired data rely on the pairing being represented appropriately in the analysis.

The sequence was randomized: one sequence began with open-loop testing, and the other began with closed-loop testing. Randomizing order can help reduce systematic differences in which condition is experienced first. It does not, by itself, establish that period effects or carryover effects are absent. A period effect is a change related to when a measurement occurs; carryover refers to effects of an earlier condition that persist into a later period. The record does not report whether these effects were assessed or how they were handled.

What does the analysis population mean?

The primary analysis description identifies an intent-to-treat population that includes all subjects who were randomized. The endpoint description, however, defines the primary proportion among subjects who have in-clinic testing. These statements identify a broad analysis principle and a testing-based endpoint population, respectively. The registry record does not give the endpoint denominator or explain how randomized subjects without evaluable in-clinic data would be treated.

In general, intent-to-treat analysis aims to retain participants in the groups or sequences to which they were randomized. In crossover studies, the exact implementation also depends on how the analysis represents sequence, period, and within-subject observations. The record states that the primary analysis combines success across randomization sequences, but it does not provide a full statistical model or further analytic details.

How should uncertainty be understood?

A proportion estimated from a study is subject to sampling uncertainty: another study of a similar population might yield a different estimate. A confidence interval is a general way to express the precision of an estimate across a range of values compatible with the data and method. The registry record does not report a primary endpoint estimate or confidence interval, so the precision of the endpoint result cannot be assessed from the record.

For a binary participant-level endpoint, inference depends on the numerator and denominator and on the analysis method. In a crossover setting, the paired observations and sequence structure may also matter. General methods should reflect the design rather than treating observations from the same subject as independent. No detailed model, variance method, or inferential procedure is reported in the registry record.

What can planning calculations say?

General sample-size planning asks how many participants would be needed to detect a specified effect under stated assumptions. The planning illustration described for this study uses a two-sided significance level of 0.05, equal allocation between two arms, and a normal approximation; it is based on a stated fraction of the observed effect. These are planning assumptions, not a description of the trial’s own design or evidence of its achieved ability to detect an effect.

The registry record does not report the observed primary endpoint effect needed to interpret planning calculations based on that effect. It also does not report a sample-size justification from the study protocol. Planning outputs cannot be treated as observed power, and they do not show whether the endpoint result was statistically significant. Those conclusions require reported outcome estimates and inferential results, which are not reported in the registry record.

Why might over-dispersion matter in general?

Over-dispersion occurs when outcome variability is greater than a statistical model assumes. In general, a model that underestimates variability can produce overly narrow uncertainty intervals or misleading test results. This is a general consideration for statistical modeling, not a reported feature of this trial. The endpoint is presented as a participant-level binary proportion, and the registry record does not report a model or a finding of over-dispersion.

10. Exact Estimates for Each Arm

No two-arm comparison is shown for this trial, but each arm's result can still be summarized with exact interval estimates computed from the posted counts. The Clopper-Pearson interval is exact and conservative; the Wilson interval is a common closer-to-nominal alternative; the Bayesian credible interval uses a uniform Beta(1, 1) prior. For small groups the three can differ noticeably.

R code and output

arm    <- c("Intent-to-treat (ITT)")
n      <- c(42)
events <- c(41)
for (i in seq_along(n)) {
  ci <- binom.test(events[i], n[i])$conf.int          # Clopper-Pearson exact interval
  cr <- qbeta(c(0.025, 0.975), 1 + events[i], 1 + n[i] - events[i])   # Bayesian, uniform Beta(1,1) prior
  wi <- (function(x, n) { p <- x / n; z <- qnorm(0.975); d <- 1 + z^2 / n
    c((p + z^2 / (2 * n) - z * sqrt(p * (1 - p) / n + z^2 / (4 * n^2))) / d, (p + z^2 / (2 * n) + z * sqrt(p * (1 - p) / n + z^2 / (4 * n^2))) / d) })(events[i], n[i])
  cat(sprintf("ARM|%s|%d|%d|%.1f|%.1f|%.1f|%.1f|%.1f|%.1f|%.1f\n", arm[i], n[i], events[i], 100 * events[i] / n[i],
              100 * ci[1], 100 * ci[2], 100 * wi[1], 100 * wi[2], 100 * cr[1], 100 * cr[2]))
}

ARM  Intent-to-treat (ITT)  42  41  97.6  87.4  99.9  87.7  99.6  87.7  99.4

11. Multiplicity, Missing Data and Analysis Populations

The registry record lists one primary outcome measure and one analysis. It does not report a multiplicity adjustment, a hierarchy of primary and secondary hypotheses, or a broader testing strategy. With multiple outcomes or repeated formal tests, the chance of a false-positive finding can increase unless the analysis plan addresses multiplicity. This is a general statistical concern; the record does not provide enough detail to establish whether any additional testing was conducted.

The primary analysis is described as reporting the total proportion of subjects with success together, regardless of randomized sequence. Sequence-specific endpoint results are not reported. The record also does not describe interim analyses or stopping rules, so neither their presence nor their absence should be inferred.

For missing data, the endpoint description refers to the proportion among subjects who have in-clinic testing, whereas the analysis population is described as all randomized subjects. The registry record does not report how many randomized participants had evaluable endpoint data, the number with missing ratings, or how missing observations were handled. It does not specify whether any imputation, sensitivity analysis, or other missing-data method was used.

In general, missing outcome data can affect both the analysis denominator and the interpretation of a proportion. The direction and size of any effect depend on why data are missing and on the values that would otherwise have been observed; these cannot be determined from the record. The safety information lists 42 at risk, but that is not identified as the denominator for the primary endpoint and cannot resolve the endpoint’s missing-data details.

The intent-to-treat description establishes that all randomized subjects were included in the analysis population as specified, but it does not reveal how unavailable endpoint measurements were incorporated. The record does not report a participant flow accounting or enough analytic detail to assess the consequences of missingness.

12. Safety Results

The registry reports serious adverse events as affected participants divided by participants at risk.

Deaths

Other (non-serious) adverse events

Participants with at least one non-serious event at or above the registry's 0% reporting threshold.

The registry’s safety information lists an ITT group with 42 at risk for serious adverse events and 0 affected. It also lists 42 at risk for other adverse events and 0 affected. The record lists 0 deaths, and no serious adverse-event terms are listed.

These are registry counts, not estimates of risk beyond the recorded group and categories. The record does not define the observation period for these safety counts or describe event collection and classification procedures. It also does not explain why the at-risk count differs from the enrollment figure of 52, or how the safety group relates to the participants with primary endpoint data. The two at-risk entries should not be added together; the record does not establish that they represent separate participant groups.

A zero count means no affected participants were recorded in the relevant category in this registry entry. In general, zero recorded events do not establish that an event is impossible, nor do they by themselves quantify how precisely a safety risk is known. The record does not provide additional detail about exposure, duration of observation, or event-level circumstances. These figures should therefore be read narrowly as the reported safety counts, without extending them to other settings or follow-up periods.

13. What the Result Does, and Does Not, Mean

The primary endpoint has a clear within-subject rule: a lower average overstimulation-sensation score during closed-loop testing than during open-loop testing qualifies as a reduction. The record describes the rating process and the intended summary as a proportion. It does not report how many subjects met the rule, the resulting proportion, or an uncertainty estimate. The actual primary outcome therefore cannot be characterized from the registry record.

Even if a proportion were reported, its meaning would remain specific to this endpoint and setting. It would describe the frequency of a lower average rating during the closed-loop period under the study’s in-clinic testing procedure. It would not, on its own, describe the size of score changes, an individual participant’s experience outside the study, or longer-term outcomes.

The record does not report results for pain relief, function, or sustained benefit as primary outcomes. It also does not provide evidence from which to draw a conclusion about overall treatment effectiveness. The study summary frames the work as an evaluation of in-clinic algorithm performance, and the primary measure addresses overstimulation sensation during protocol-prescribed activities.

The randomized crossover design supports a comparison across two conditions experienced in sequence, but the record does not report sequence-specific results or methods addressing period and carryover effects. The primary analysis description says success was to be reported together regardless of randomization sequence. Without the outcome data and analytic details, the extent to which the observed result might depend on testing order cannot be assessed.

The safety entry reports zero affected for serious adverse events and zero affected for other adverse events among 42 at risk in each category, and it records zero deaths. Those counts are limited to the registry’s stated safety information; they do not establish the absence of risk beyond the recorded observations. Overall, the registry record defines the question and reports limited safety counts, but does not report the primary efficacy outcome needed to assess the observed endpoint.

14. Why This Trial Matters Statistically

This study raises a useful statistical issue: how to summarize paired, within-person measurements when testing conditions occur in sequence. The endpoint compares each participant’s average intensity rating during closed-loop testing with that participant’s average during open-loop testing. That is different from comparing independent groups, and the randomized order is part of the design that shapes interpretation.

The success rule also illustrates a tradeoff in endpoint construction. A binary classification can provide a direct summary of how many participants meet a stated criterion, while losing information about the amount of change in the underlying rating. Readers need the definition of that criterion, the analysis denominator, and the observed numerator and proportion to understand the result. The registry record gives the definition but does not report the endpoint count or proportion.

The distinction between an intent-to-treat population and participants with evaluable in-clinic testing is another important reporting issue. The record describes the primary analysis population as all randomized subjects and also defines the endpoint among subjects who have in-clinic testing. Without a reported denominator or missing-data method, the relationship between those descriptions remains unclear.

Finally, the record illustrates why a completed study’s outcome definition and safety counts are not substitutes for a reported primary result. The absence of a reported endpoint estimate prevents assessment of the direction, magnitude, and precision of the primary comparison. The crossover design and endpoint rule can be described, but they cannot supply the missing outcome evidence.

15. Limitations

• The posted summary statistics are the only data these results rest on; patient-level data, full protocols and statistical analysis plans are not available.
• Recalculated results are simple, unadjusted calculations from posted summaries. They do not replace the sponsor's prespecified analysis.

The principal limitation for interpreting this study from the registry record is that the primary endpoint result is not reported. There is no observed count or proportion of participants meeting the reduction criterion, and no effect estimate, confidence interval, or p-value. Consequently, the direction and precision of the primary comparison cannot be assessed from the registry entry.

The record does not give the primary endpoint denominator or explain the relationship between the intent-to-treat analysis population of all randomized subjects and the endpoint population described as subjects with in-clinic testing. It does not report missing rating counts, reasons for missingness, or a method for handling missing endpoint data. These gaps limit assessment of who contributed to the outcome summary.

Although randomization assigned the order of conditions, the record does not report sequence-specific results or describe analysis of period or carryover effects. It also does not specify who was blinded, how blinding was maintained, or how randomization was implemented. These design details may matter when interpreting a within-subject comparison collected over sequential periods, but they are not reported in the registry record.

The stated endpoint is narrowly tied to average intensity ratings during protocol-prescribed activities in clinic over one day. The registry record does not report the underlying ratings or describe outcomes beyond this endpoint. It therefore does not permit assessment of the magnitude of individual changes or extension of the finding to longer-term outcomes.

The safety entry lists 42 at risk in each adverse-event category, while enrollment is listed as 52. The record does not explain this difference or identify the safety observation period. The listed zero counts should not be interpreted as covering participants or periods beyond those recorded.

16. Related Tutorials

17. Related Statistical Calculators

18. Sources

• ClinicalTrials.gov: Electrically Evoked Compound Action Potentials Human Observation Medtronic Algorithm Comparison Study — NCT04765735.
• Registry record last updated 2024-02-26; status last verified 2024-02.
• Results first posted 2024-02-26.
• Data retrieved from ClinicalTrials.gov on 2026-11-09.
• Group sizes, means, event counts and sponsor-reported p-values are reproduced as posted. Effect estimates, confidence intervals and p-values not attributed to the sponsor are independently calculated from the posted summary statistics.

19. Record Summary

The registry record describes ECHO-MAC, a completed, prospective, multi-center, randomized, single-blind crossover study of open-loop and closed-loop spinal cord stimulation testing for chronic pain. Enrollment is listed as 52. Study dates are 2021-02-23 for start and 2021-06-29 for primary completion.

The primary endpoint is the proportion of subjects whose average overstimulation-sensation intensity score was lower during closed-loop than open-loop testing. Ratings were made during protocol-prescribed activities on a 5-point Likert scale, with testing conducted in clinic over one day. The analysis description specifies all randomized subjects and says success was to be reported together across randomization sequences.

The registry record does not report the primary endpoint count or proportion, an effect estimate, or inferential statistics. Its safety entry lists 42 at risk and 0 affected for serious adverse events, 42 at risk and 0 affected for other adverse events, and 0 deaths. The record does not explain the relationship between the safety counts and enrollment or the endpoint analysis population.