← Clinical Trials
Depression Phase 4 Continuous Endpoint NCT00305565

Study Comparing Outcomes for Patients With Treatment Resistant Depression Who Receive VNS Therapy at Different Doses: Complete Statistical Analysis

NCT00305565 compared low-, medium-, and high-dose Vagus Nerve Stimulation Therapy in a randomized study of patients with treatment-resistant depression. The primary posted result is the mean change in IDS-C score from baseline to study week 22.

Trial start: 2006-01  ·  Enrollment: 331  ·  Sponsor: Cyberonics, Inc.
Scope of this record

This page provides an independent statistical analysis and educational interpretation of publicly reported results. ClinicalTrials.gov provides the official trial registry record. Numerical trial results on this page are restricted to the results posted in the ClinicalTrials.gov record; calculations added by Clinical Biostats are labelled as such.

1. Trial at a Glance

This page covers the registry results for Study Comparing Outcomes for Patients With Treatment Resistant Depression Who Receive VNS Therapy at Different Doses (NCT00305565), a randomized phase 4 study. The primary outcome is Inventory of Depressive Symptomatology-Clinician Administered (IDS-C) Mean Change From Baseline to Week 22 of the Acute Phase (ITT Population)..

331
Enrollment
Phase 4
3
Arms
In the primary outcome
-0.33
Mean difference
Medium Dose
-3.09–2.43
95% CI
Two-sided
FeatureRegistry value
ClinicalTrials.gov identifierNCT00305565
PhasePhase 4
ConditionDepression
AllocationRandomized
Design modelParallel
MaskingTriple
Enrollment331
Arms in the primary outcome3
Lead sponsorCyberonics, Inc.
Results postedYes

2. Clinical Question

Population

The registry describes patients with treatment-resistant depression. The primary outcome is identified as being measured in the intention-to-treat population, but the registry record does not provide further eligibility details in the posted description.

Intervention

Vagus Nerve Stimulation (VNS) Therapy administered at different amounts of electrical charge. The three groups are named Low Dose, Medium Dose, and High Dose; their specific dose settings are not reported in the registry record.

Comparator

The primary comparisons are Medium Dose versus Low Dose and High Dose versus Low Dose. Each group received VNS Therapy, so the comparison concerns different dose levels rather than VNS Therapy versus no device intervention.

Primary question

How did mean change in the Inventory of Depressive Symptomatology-Clinician Administered (IDS-C) score from baseline to study week 22 differ between the dose groups? The IDS-C ranges from 0 to 84, and higher scores indicate worse outcomes.

The registry describes this as a postmarket medical device study intended to compare the safety and effectiveness of VNS Therapy administered at different electrical charges for treatment-resistant depression. The primary endpoint is a continuous score change over the acute phase. The study also lists responder and remitter outcomes, including outcomes measured at study week 50 during the long-term phase. Those secondary measures address different ways of expressing symptom change, but the central statistical question remains whether the group mean change on the IDS-C differed by assigned dose at week 22.

The posted results show mean changes of -9.96 for Low Dose, -10.29 for Medium Dose, and -10.3 for High Dose. The sponsor’s reported comparisons and simple unadjusted recalculations both describe contrasts close to zero relative to their uncertainty. These estimates describe the groups’ recorded outcomes; they do not by themselves establish that one dose level is better, worse, or equivalent to another.

3. Trial Design

NCT00305565 is registered as a completed Phase 4, randomized, triple-masked, parallel-group study, with enrollment of 331. The sponsor is Cyberonics, Inc. The registry lists Depression as the condition and identifies the intervention as VNS Therapy, a device. The study began in 2006-01 and had primary completion in 2010-02.

Participants were allocated among Low Dose, Medium Dose, and High Dose groups. All three groups are described as receiving VNS Therapy. The registry’s brief summary states that the objective was to compare the safety and effectiveness of the therapy administered at different amounts of electrical charge. Specific charge settings and additional arm procedures are not reported in the registry record. The word “triple” describes the recorded masking designation, but the identities of the masked parties are not reported in the registry record.

The primary outcome was mean change in IDS-C score from baseline to study week 22 in the intention-to-treat population. The registry also lists secondary outcomes measuring IDS-C and QIDS-C response or remission at different time points. The posted secondary outcome descriptions include an acute-phase assessment at week 22 and long-term assessments at week 50. The registry lists 38 outcome measures and 35 posted secondary measures.

Enrollment and the analysis counts for particular outcomes are not identical: the primary outcome groups have reported counts of 102.0, 101.0, and 107.0, while secondary analyses use different counts. Reasons for differences in outcome-specific counts and details of the analysis populations are not reported in the registry record. The trial’s original sample-size assumptions, allocation ratio, full protocol, and planned statistical analysis are also not reported in the registry record.

4. Participant Flow and Accounting

The registry reports how participants moved through each stage of the study. All counts below are exactly as posted; missing counts are not inferred from other stages.

Before assignment: A total of 331 subjects at 29 sites were actually enrolled and implanted with the Vagus Nerve Stimulation (VNS) Therapy System, and were included in the safety dataset. Twenty-one subjects were excluded from the intent-to-treat (ITT) dataset, leaving 310 in the ITT population.

Overall Study (Safety Population)

StageLow DoseMedium DoseHigh Dose
STARTED111107113
COMPLETED9795106
NOT COMPLETED14127

Overall Study (ITT Population)

StageLow DoseMedium DoseHigh Dose
STARTED102101107
COMPLETED8990102
NOT COMPLETED13115

5. Baseline Characteristics

The table reproduces the baseline characteristics posted in the registry, which describe the participants analysed at baseline and help show whether the randomized groups were comparable.

CharacteristicUnitLow DoseMedium DoseHigh DoseTotal
Participants analysed102101107310
Age, Continuousyears49.1 (SD 10.5)47.2 (SD 11.0)47.4 (SD 10.8)47.9 (SD 10.8)
Sex: Female, Male: FemaleParticipants686973210
Sex: Female, Male: MaleParticipants343234100
Race/Ethnicity, Customized: Caucasianparticipants9796104297
Race/Ethnicity, Customized: Asianparticipants3104
Race/Ethnicity, Customized: African-Americanparticipants2338
Race/Ethnicity, Customized: Hispanicparticipants0101
Region of Enrollment: United Statesparticipants102101107310
Unipolar or Bipolar Depression: Unipolar Depression (Single Episode)participants6101531
Unipolar or Bipolar Depression: Unipolar Depression (Recurrent)participants767166213
Unipolar or Bipolar Depression: Bipolar Depressionparticipants20202666
Lifetime Episodes of Depression: 0-2 Episodesparticipants16172861
Lifetime Episodes of Depression: 3-5 Episodesparticipants23212266
Lifetime Episodes of Depression: 6-10 Episodesparticipants13161847
Lifetime Episodes of Depression: >10 Episodesparticipants504639135
Duration of IllnessYears29.8 (SD 12.1)26.3 (SD 10.9)27.0 (SD 12.1)27.7 (SD 11.8)
Length of Current Major Depressive EpisodeMonths106.7 (SD 122.8)106.1 (SD 107.3)111.3 (SD 146.3)108.1 (SD 126.5)

6. Primary Endpoint

The registry lists the primary outcome as Inventory of Depressive Symptomatology-Clinician Administered (IDS-C) Mean Change From Baseline to Week 22 of the Acute Phase (ITT Population)., measured over From Baseline to Study Week 22.

ElementRegistry entry
Result typeLeast Squares Mean
Measure of spreadStandard Error
Unitunits on a scale
Endpoint classificationContinuous Endpoint

The primary endpoint is the mean change from baseline to study week 22 on the Inventory of Depressive Symptomatology-Clinician Administered (IDS-C), for the intention-to-treat population. The IDS-C is a 30-item measure designed to assess the severity of depressive symptoms. Its scale ranges from 0 to 84, with higher values considered worse outcomes. The registry reports the outcome as a least-squares mean, with a standard error as its measure of dispersion.

A continuous score preserves information about the amount of change on the scale rather than reducing each participant’s outcome to a category such as responder or non-responder. In general, this can make a mean change useful for comparing group-level shifts in a measured symptom score. It does not show how every participant changed, nor does a group mean describe the range of individual experiences.

Because higher IDS-C scores are considered worse, a negative change indicates a lower score at follow-up than at baseline. The posted group means are -9.96 for Low Dose, -10.29 for Medium Dose, and -10.3 for High Dose. These are group-level changes, not direct comparisons between groups. The relevant dose comparisons ask whether the mean change differed between each active dose group and Low Dose. The registry record does not report a threshold for a clinically important difference on the IDS-C, so the clinical importance of a particular numerical contrast cannot be determined from the endpoint description alone.

7. Results Posted in the Registry

Mean difference: Medium Dose

-0.33

95% CI: -3.09 to 2.43

Recalculated from the posted summary statistics

Primary outcome by arm, with 95% confidence intervals
Primary outcome by arm with 95% confidence intervals-12.7-11.7-10.6-9.6-8.6-7.6Low Dose (n=102)Medium Dose (n=101)High Dose (n=107)units on a scale (95% CI)
Treatment effect versus the control arm
Forest plot of mean difference vs control (95% CI)Medium DoseHigh Dose-3.53-2.25-0.970.311.592.87mean difference vs control (95% CI)

8. Secondary Efficacy Endpoints

The registry also posts results for secondary outcomes. For each outcome below, the endpoint type, the test applied to the posted summary statistics, the result and its interpretation are given in turn. Secondary endpoints are usually not adjusted for multiple testing, so they support rather than confirm the primary finding.

Secondary outcomeTypeEffect measureEstimate (95% CI)Planning size, 80% power
Inventory of Depressive Symptomatology-Clinician Administered (IDS-C) Percent Responders From Baseline to Week 22 of the Acute Phase (ITT Population).Binary EndpointRisk difference+1.0 pts (-9.5 to 11.6)20,850 per arm
Inventory of Depressive Symptomatology-Clinician Administered (IDS-C) Percent Remitters From Baseline to Week 22 of the Acute Phase (ITT Population).Binary EndpointRisk difference+3.1 pts (-4.4 to 10.6)1,170 per arm
Inventory of Depressive Symptomatology-Clinician Administered (IDS-C) Percent Responders From Baseline to Week 50 of the Long-term Phase (ITT Population).Binary EndpointRisk difference+11.9 pts (-1.7 to 25.6)243 per arm

Inventory of Depressive Symptomatology-Clinician Administered (IDS-C) Percent Responders From Baseline to Week 22 of the Acute Phase (ITT Population).

Treatment effect versus the control arm
Forest plot of risk difference (percentage points) vs control (95% CI)Medium DoseHigh Dose-12-5.78-0.055.691117risk difference (percentage points) vs control (95% CI)

Inventory of Depressive Symptomatology-Clinician Administered (IDS-C) Percent Remitters From Baseline to Week 22 of the Acute Phase (ITT Population).

Treatment effect versus the control arm
Forest plot of risk difference (percentage points) vs control (95% CI)Medium DoseHigh Dose-5.80-1.772.276.311014risk difference (percentage points) vs control (95% CI)

Inventory of Depressive Symptomatology-Clinician Administered (IDS-C) Percent Responders From Baseline to Week 50 of the Long-term Phase (ITT Population).

Treatment effect versus the control arm
Forest plot of risk difference (percentage points) vs control (95% CI)Medium DoseHigh Dose-15-6.432.33112029risk difference (percentage points) vs control (95% CI)

The posted secondary outcomes cover responder and remitter percentages on the IDS-C at week 22 and week 50, sustained response at week 50, and responder or remitter outcomes on the QIDS-C at week 22 and week 50. The secondary measures therefore include categorical summaries of symptom change as well as assessments at different time points. The registry lists 35 posted secondary measures, while the number of posted results represented in the recorded summaries is more limited.

Across the secondary endpoints with posted arm summaries, patterns are not uniform: some percentages are higher in Medium or High Dose than in Low Dose, while other contrasts are close to one another. The recalculated unadjusted intervals for the listed contrasts are broad and include zero. These results do not overturn the primary pattern of similar estimated mean changes with uncertain differences; nor do they establish a consistent dose-related pattern across outcomes and time points.

Secondary endpoints can help describe other aspects of the measured outcomes, but their inferential weight depends on how they were planned, analyzed, and interpreted together. When many outcomes or comparisons are examined, the chance of seeing an apparently notable result by chance can increase. An unadjusted p-value does not account for that broader set of tests. The registry record does not report a multiplicity adjustment, a hierarchy of outcome testing, or a prespecified strategy for selecting which secondary findings would be confirmatory.

The secondary outcomes should therefore be read as supportive descriptions rather than independent proof of a dose difference. Differences in analysis counts across time points and outcomes also matter when interpreting percentages. Reasons for those differences and the missing-data procedures are not reported in the registry record. The registry’s endpoint descriptions identify the outcome and timeframe, but do not provide enough detail to assess every operational definition used to classify response, remission, or sustained response.

Secondary outcomes registered for this trial, for reference (only those above were recalculated):

9. Statistical Interpretation

Clinical Biostats interpretation

For the primary endpoint, mean IDS-C change at week 22 was -9.96 in the Low Dose group, -10.29 in the Medium Dose group, and -10.3 in the High Dose group. The negative group means indicate lower IDS-C scores at follow-up than at baseline; the scale description identifies higher scores as worse outcomes. The between-group comparisons are much smaller in magnitude than the group changes from baseline and are estimated with uncertainty.

The recalculated unadjusted mean difference for Medium Dose versus Low Dose is -0.33, with a 95% confidence interval from -3.088 to 2.428 and p = 0.815. For High Dose versus Low Dose, the recalculated difference is -0.34, with a 95% confidence interval from -3.057 to 2.377 and p = 0.806. Both intervals include zero, so the posted estimates are compatible with no difference as well as with differences in either direction within the intervals. These unadjusted calculations are not a replacement for the sponsor’s analysis.

The sponsor’s mixed-model analysis reports a mean difference (net) of 0.33 for Low Dose versus Medium Dose, with a 95% confidence interval from -2.39 to 3.05 and p = 0.813. For Low Dose versus High Dose, it reports 0.34, with a 95% confidence interval from -2.34 to 3.02 and p = 0.803. The direction of the displayed differences is reversed relative to the recalculated contrasts because the group order is reversed: the sponsor reports Low Dose versus the other group, while the recalculation expresses the other group versus Low Dose. The estimates are similar in magnitude, and the sponsor’s own analysis takes precedence.

The p-values do not provide evidence of a clear difference in these comparisons under the reported analyses. They are not probabilities that a dose is effective or ineffective, and they do not establish that the dose groups are equivalent. Demonstrating equivalence would require an explicit equivalence margin and an analysis designed for that purpose; these are not reported in the registry record. The intervals are important because they show the range of effects reasonably compatible with each estimate under the analysis assumptions, not merely whether a p-value crosses a threshold.

The secondary outcome descriptions include responder and remitter percentages at week 22 and week 50. These categorical outcomes present a related but distinct view of symptom change. Their estimates vary across endpoints and comparisons, and unadjusted analyses carry less weight when many outcomes are considered. The registry record does not report a multiplicity strategy that would allow those results to be interpreted as a single confirmatory family of tests.

Overall, the posted results describe similar primary mean changes across the three dose groups, with uncertain between-group contrasts. They do not establish the absence of a potentially meaningful difference, and the registry does not report a clinical-importance threshold for interpreting the IDS-C contrasts. The statistical findings should be read as estimates from this study, with uncertainty, rather than as a definitive ranking of dose levels.

10. Statistical Methods Explained

The primary outcome is continuous, so the registry reports group least-squares means for change in IDS-C score and standard errors. A least-squares mean is an estimated group mean from a statistical model. In general, such a model can account for specified features of the analysis; the exact covariates and model details for the sponsor’s primary analysis are not reported in the registry record. The standard error describes the estimated uncertainty of a mean, rather than the spread of individual participants’ scores.

What does a group mean change represent?

Each group’s mean change summarizes the average score change among the participants included in that outcome analysis. The Low Dose mean change is -9.96, the Medium Dose mean change is -10.29, and the High Dose mean change is -10.3. Since higher IDS-C values indicate worse outcomes, negative values correspond to lower scores after baseline. The group mean does not show the distribution of individual changes, and the registry record does not provide those individual data.

What is a mean difference?

A mean difference compares the average change in one group with the average change in another. The recalculated Medium Dose versus Low Dose difference is -0.33, and the recalculated High Dose versus Low Dose difference is -0.34. These simple calculations are based on posted summary statistics and are unadjusted. The recalculation uses an approximate pooled standard deviation of 10.02, inferred from the reported standard errors and group sizes. As a general statistical point, a standardized effect expresses a difference in relation to a measure of variability; the posted standardized estimates are -0.033 for Medium Dose versus Low Dose and -0.034 for High Dose versus Low Dose. The sponsor’s model-based results are the primary reference for the study’s reported analysis.

How should a confidence interval be read?

A confidence interval gives a range of values compatible with the estimate under the statistical model and its assumptions. The sponsor reports 95% confidence intervals for both primary comparisons. The intervals extend on either side of zero, so they include no difference as well as possible differences in either direction. A wide interval indicates that the estimate is not precise enough to distinguish among a broad range of plausible effects. A confidence interval is not a guarantee that the true value lies within its particular bounds, nor is it a range of individual participants’ outcomes.

What does the p-value add?

A p-value summarizes how unusual data at least as inconsistent with a specified null hypothesis would be if that hypothesis and the analysis assumptions held. The sponsor reports p = 0.813 for Low Dose versus Medium Dose and p = 0.803 for Low Dose versus High Dose. The recalculated unadjusted p-values are 0.815 and 0.806, respectively. These are not probabilities that the null hypothesis is true, and they do not measure the size or clinical importance of a difference. A p-value should be read alongside the estimate and its confidence interval.

The sponsor used a mixed models analysis and reported mean differences (net). The model’s detailed specification, including handling of repeated measurements, covariates, missing observations, and any adjustments, is not reported in the registry record. The registry identifies the primary population as intention-to-treat. In general, intention-to-treat analysis aims to compare participants according to their randomized group, but the registry does not describe how this population was defined or how missing endpoint data were handled.

The recalculated comparisons are simple unadjusted analyses from posted summary statistics; they are useful as a consistency check, not as a substitute for the sponsor’s model. Their estimates have the opposite sign convention because the order of the groups is reversed. The reported estimates are close in magnitude, and the sponsor’s own analysis takes precedence. In general, model-based and summary-statistic estimates can differ because they may use different assumptions, data handling, or adjustment; the exact reasons for any small numerical differences cannot be determined from the registry record.

For binary secondary endpoints, the registry reports percentages and the recalculated comparisons include risk differences. A risk difference expresses the difference between group proportions. In general, confidence intervals for a risk difference indicate the uncertainty around that contrast, and an interval including zero is compatible with no difference. The registry also gives unadjusted p-values for these recalculated comparisons. They should not be interpreted independently of the number of outcomes and comparisons assessed.

General statistical context: variability matters because a mean difference is harder to estimate precisely when participant outcomes vary widely. Over-dispersion is a concern for count or binary data when observed variability is greater than a model assumes; accounting for it can affect uncertainty estimates. The registry does not report whether over-dispersion was assessed or relevant to the sponsor’s analyses. No particular distributional diagnosis or correction can therefore be inferred.

11. Confidence Intervals and Precision

The sponsor’s primary confidence intervals are 95% intervals. For Low Dose versus Medium Dose, the reported mean difference (net) is 0.33, with an interval from -2.39 to 3.05. For Low Dose versus High Dose, the reported mean difference (net) is 0.34, with an interval from -2.34 to 3.02. Both intervals cross zero, indicating that the estimated contrasts are not precise enough to rule out no difference under the reported model. They also permit differences in either direction within their bounds.

The recalculated contrasts use the reverse group order. For Medium Dose versus Low Dose, the estimate is -0.33 and the interval is -3.088 to 2.428. For High Dose versus Low Dose, the estimate is -0.34 and the interval is -3.057 to 2.377. These recalculated intervals are based on unadjusted summary-statistic methods. The sponsor’s model-based analyses remain the reference for the study’s reported results.

Precision is about how tightly the data locate an effect estimate, not whether a result is important. An interval that includes zero does not establish equivalence, and it does not show that all clinically relevant differences have been excluded. The registry record does not identify a threshold for a clinically important IDS-C difference or an equivalence margin, so the range cannot be judged against either benchmark.

The standard errors for the group means are 0.99 for Low Dose, 1.0 for Medium Dose, and 0.97 for High Dose. These quantify uncertainty in the estimated means, not participant-to-participant variability. In general, greater variability and fewer observations tend to reduce precision, while additional information can improve it. A complete assessment would also depend on the analysis model, outcome data availability, and assumptions; the registry record does not provide enough detail to assess all of these elements.

12. Planning a Similar Trial: Sample Size

This is a planning illustration, not a re-evaluation of the original trial. It asks: if a new study wanted to detect a difference like the one observed, or a smaller one, how large would it need to be? It is deliberately not “observed power” of the finished trial, which is not a meaningful quantity.

Effect to detectParticipants per arm, 80% powerParticipants per arm, 90% power
100% of the observed effect14,48519,391
75% of the observed effect25,75134,472
50% of the observed effect57,93877,562
Sample size needed versus the size of the effect to detect
Sample size needed versus size of effect to detect10,00020,00050,000100,000200,00040%60%80%100%80% power90% powersize of effect to detect, as % of the effect observed (log vertical scale: participants per arm)
Treat these figures cautiously. The observed difference is imprecise: its confidence interval includes no difference at all. A sample size built on a noisy estimate can be far too small or too large, so the smaller-effect rows are the more cautious guide.
Method and assumptions. Two-sided α = 0.05, 1:1 allocation, two-sample comparison of means with the pooled standard deviation. Standard deviation was approximated from model-based standard errors, so treat it as rough. Calculated for the comparison: Medium Dose.

The planning figures are participants per arm for the two-group comparisons, not total participants across the trial. The planning illustration uses a two-sided significance level of 0.05, equal allocation between the two arms, and a normal approximation. It considers effects equal to 100%, 75%, or 50% of the observed effect. The original trial’s own design assumptions and sample-size justification are not reported in the registry record.

The 80% power column means the number of participants per arm that the planning calculation estimates would be needed to have an 80% chance of detecting the specified effect under those assumptions. The 90% power column uses a higher target chance of detecting that same specified effect, and therefore generally requires more participants per arm. These are planning illustrations, not a description of the enrollment target or analysis plan actually used in the trial.

Smaller effects are harder to distinguish from random variation, so planning to detect a smaller fraction of an observed effect requires more participants. The estimates depend on the observed effect used as their starting point and on the stated assumptions; they should not be treated as reliable enrollment targets without a design-specific justification. They are not observed power: observed power is not a useful way to reinterpret a completed study’s result, and these figures instead illustrate hypothetical future planning under stated conditions.

The primary outcome is continuous, while some secondary outcomes are percentages. The planning calculations are therefore illustrative and are tied to their respective outcome types and observed estimates. The registry record does not report the original rationale for enrolling 331 participants, nor does it report whether a different planning method was used for the completed study.

13. How the Sample Size Is Derived

Power is the probability that a study of a given size rejects the null hypothesis when a specific true effect exists. It depends on four things: the effect to detect, the variability of the outcome, the significance level (here a two-sided α of 0.05, so z1−α/2 = 1.960) and the desired power (z1−β = 0.842 for 80% and 1.282 for 90%).

Formula

n = 2 (z1−α/2 + z1−β)2 σ2 / Δ2

n is the number of participants in each arm, Δ is the difference in means to detect and σ is the common standard deviation.

Worked calculation for this trial

  1. Observed difference Δ = -0.330; pooled standard deviation σ = 10.024.
  2. n = 2 × (1.96 + 0.842)2 × 10.0242 / -0.3302 = 14,484.4, rounded up to 14,485 participants per arm for 80% power.

The smaller-effect rows in the table above repeat the same calculation with the effect multiplied by 0.75 and 0.5 (for ratios, applied to the logarithm of the ratio).

R code

alpha <- 0.05
z_a   <- qnorm(1 - alpha / 2)          # two-sided
delta <- -0.329999999999998                      # observed difference in means
sigma <- 10.0240978800318                     # pooled standard deviation
n_per_arm <- function(power, frac = 1) {
  ceiling(round(2 * (z_a + qnorm(power))^2 * sigma^2 / (delta * frac)^2, 6))
}
for (f in c(1, 0.75, 0.5))
  cat(sprintf("%3.0f%% of observed effect: n per arm = %s (80%% power), %s (90%% power)\n", 100 * f,
              format(n_per_arm(0.80, f), big.mark = ","), format(n_per_arm(0.90, f), big.mark = ",")))

# Check against R's built-in function (t-based, so very slightly larger)
cat("power.t.test:", round(power.t.test(delta = delta, sd = sigma, power = 0.80)$n, 1), "\n")

# Simulation: exact two-sample t-test using simulated means and variances
set.seed(305565)
n    <- n_per_arm(0.80)
reps <- 20000
m_diff <- rnorm(reps, delta, sigma * sqrt(2 / n))             # difference in sample means
s2     <- sigma^2 * rchisq(reps, 2 * (n - 1)) / (2 * (n - 1))   # pooled sample variance
tstat  <- m_diff / sqrt(s2 * 2 / n)
cat(sprintf("Simulated power at n = %s per arm: %.3f (%d replicates)\n", format(n, big.mark = ","), mean(abs(tstat) > qt(0.975, 2 * (n - 1))), reps))

Output from running the code

100% of observed effect: n per arm = 14,485 (80% power), 19,391 (90% power)
 75% of observed effect: n per arm = 25,751 (80% power), 34,472 (90% power)
 50% of observed effect: n per arm = 57,938 (80% power), 77,562 (90% power)
power.t.test: 14485.4 
Simulated power at n = 14,485 per arm: 0.799 (20000 replicates)

The simulation draws many sets of group means and variances at the planned sample size and counts how often the two-sample t-test rejects the null. It should land close to the target power; at very small required sample sizes it can fall below it, because the normal approximation is rough there.

14. Analyses Reported by the Sponsor

MethodEstimate95% CIP-value
Mixed Models AnalysisMean Difference (Net) 0.34-2.34 to 3.020.803
Mixed Models AnalysisMean Difference (Net) 0.33-2.39 to 3.050.813

The sponsor reports mixed-model comparisons of Low Dose with Medium Dose and High Dose. For Low Dose versus Medium Dose, the reported mean difference (net) is 0.33, with a 95% confidence interval from -2.39 to 3.05 and p = 0.813. For Low Dose versus High Dose, it is 0.34, with a 95% confidence interval from -2.34 to 3.02 and p = 0.803.

The simple recalculations express the comparisons in the opposite order: Medium Dose versus Low Dose is -0.33, with a 95% confidence interval from -3.088 to 2.428 and p = 0.815; High Dose versus Low Dose is -0.34, with an interval from -3.057 to 2.377 and p = 0.806. Reversing the order of groups reverses the sign of a difference. The confidence limits also appear in reverse order and with small numerical differences because the recalculations and sponsor analysis are not identical methods.

The estimates are close in magnitude, and both sets of results indicate considerable uncertainty around the between-group contrasts. The recalculated values are unadjusted checks based on summary statistics, whereas the sponsor reports a mixed-model analysis. The sponsor’s own analysis takes precedence for interpreting the trial’s reported result. Details that would explain the exact model specification or any small differences are not reported in the registry record.

15. Multiplicity, Missing Data and Analysis Populations

The registry identifies one primary endpoint and lists a range of secondary outcomes, including repeated outcome types at multiple time points and measures using both IDS-C and QIDS-C. It reports two sponsor analyses for the primary endpoint, comparing Medium Dose and High Dose separately with Low Dose. The record also lists 38 outcome measures and 35 posted secondary measures. Assessing several comparisons raises a multiplicity question: the more tests considered, the greater the possibility that at least one result may appear notable by chance even when no corresponding effect exists.

A multiplicity plan might specify which outcomes are confirmatory, how tests are ordered, or whether the significance threshold is adjusted. Such a plan is not reported in the registry record. The secondary risk-difference p-values are unadjusted recalculations, so they should be interpreted cautiously and not treated as if they were corrected for the full collection of secondary tests. The registry does not explain how the two primary comparisons were handled as a family of tests.

The primary outcome is labeled as intention-to-treat, but the registry record does not provide the detailed analysis-population definition or explain how participants with missing week-22 outcomes were handled. Outcome-specific analysis counts vary, and reasons for those differences are not reported. No missing-data method, sensitivity analysis, or imputation approach is described in the registry record. As a general matter, missing outcomes can affect both the precision and the validity of a group comparison if their occurrence is related to participant outcomes or treatment group.

The registry record also does not report interim analyses, stopping rules, the original sample-size justification, or a full statistical analysis plan. This absence does not establish that these procedures were not used; it means their details cannot be assessed from the registry record. The record’s completed status and posted results do not fill in those methodological details.

16. Safety Results

The registry reports serious adverse events as affected participants divided by participants at risk.

ArmParticipants with serious adverse eventsParticipants at riskPercentage
Low Dose3111127.9%
Medium Dose1910717.8%
High Dose1611314.2%
Participants with at least one serious adverse event
Serious adverse events by armLow Dose27.9%Medium Dose17.8%High Dose14.2%% of participants with at least one serious adverse event

Other (non-serious) adverse events

Participants with at least one non-serious event at or above the registry's 1% reporting threshold.

ArmParticipants affected of at risk
Low Dose103 of 111
Medium Dose104 of 107
High Dose111 of 113

Most frequent serious adverse event terms

Event termOrgan systemLow DoseMedium DoseHigh Dose
Body as a WholeGeneral disorders19 of 1119 of 10713 of 113
Nervous SystemNervous system disorders9 of 11110 of 1074 of 113
Digestive SystemGastrointestinal disorders3 of 1111 of 1071 of 113
Respiratory SystemRespiratory, thoracic and mediastinal disorders2 of 1111 of 1071 of 113
Cardiovascular SystemCardiac disorders1 of 1111 of 1071 of 113
Urogenital SystemRenal and urinary disorders2 of 1111 of 1070 of 113
Endocrine SystemEndocrine disorders0 of 1111 of 1070 of 113
Skin and Appendages SystemSkin and subcutaneous tissue disorders0 of 1111 of 1070 of 113

Most frequent non-serious adverse event terms

Event termOrgan systemLow DoseMedium DoseHigh Dose
Respiratory SystemRespiratory, thoracic and mediastinal disorders87 of 11197 of 10799 of 113
Body as a WholeGeneral disorders80 of 11178 of 10792 of 113
Nervous SystemNervous system disorders73 of 11162 of 10785 of 113
Digestive SystemGastrointestinal disorders37 of 11145 of 10747 of 113
Cardiovascular SystemCardiac disorders12 of 11113 of 10715 of 113
Special Senses SystemEar and labyrinth disorders11 of 1116 of 10715 of 113
Metabolic SystemMetabolism and nutrition disorders7 of 1119 of 10714 of 113
Musculoskeletal SystemMusculoskeletal and connective tissue disorders6 of 11112 of 10711 of 113
Urogenital SystemRenal and urinary disorders8 of 11113 of 1078 of 113
Skin and Appendages SystemSkin and subcutaneous tissue disorders9 of 1117 of 1076 of 113

Serious events with the highest arm-level rate: body as a whole (17.1%), nervous system (9.3%), digestive system (2.7%), respiratory system (1.8%), urogenital system (1.8%).

Do not overextend the safety comparison. Serious-adverse-event counts are safety observations, not efficacy outcomes, and should not be combined with the primary result into a single benefit-risk number.

The adverse-event summary gives group counts at risk and counts affected by serious and other adverse events. For serious adverse events, the listed at-risk and affected counts are 111 and 31 for Low Dose, 107 and 19 for Medium Dose, and 113 and 16 for High Dose. For other adverse events, the corresponding counts are 111 and 103, 107 and 104, and 113 and 111. These are counts recorded for the groups, not comparisons adjusted for differences in exposure or follow-up.

The listed serious-event terms include Body as a Whole (17.1%), Nervous System (9.3%), Digestive System (2.7%), Respiratory System (1.8%), Urogenital System (1.8%), and Cardiovascular System (0.9%). These categories describe the registry’s reported terms; the record does not clarify whether they are mutually exclusive or how events were attributed to particular participants. Death counts are not reported in the registry record.

These figures describe recorded events and should not be interpreted as a complete causal comparison between dose levels. The registry summary does not provide event timing, exposure-adjusted rates, detailed event narratives, or a statistical analysis of safety differences. In general, a count affected is not the same as the number of event episodes if participants can experience more than one event; the registry’s presentation does not explain this distinction.

17. What the Result Does, and Does Not, Mean

The primary results show negative mean IDS-C changes in all three dose groups from baseline to week 22. Because the scale description says higher values are worse, those group means correspond to lower scores at follow-up. The primary between-group estimates are close to zero relative to their confidence intervals, and the sponsor’s reported comparisons do not show a clearly distinguished mean change between Low Dose and either other group under the reported model.

This finding does not establish that the dose levels are equivalent. A result that does not clearly distinguish groups can arise when true differences are small, when estimates are imprecise, or both. Equivalence requires a prespecified margin defining how much difference would count as acceptably small, together with an analysis designed to assess that question. Neither an equivalence margin nor an equivalence analysis is reported in the registry record.

The group mean changes also do not establish how individual participants fared. A mean can conceal variation, including participants with larger changes, smaller changes, or no change. The registry lists categorical responder and remitter outcomes, but their interpretation is limited by unadjusted comparisons, multiple outcomes, and incomplete reporting of analysis details. The record does not report a clinical-importance threshold for the IDS-C mean difference, so statistical estimates cannot be translated into a threshold-based judgment of clinical importance from this information alone.

The study is described as a comparison of different VNS Therapy dose levels, not a comparison with no device intervention. It therefore addresses differences among the three dose groups in the enrolled study population. Its results do not answer every question about VNS Therapy, other populations, or other treatment options. No medical advice follows from these registry findings.

Safety counts should also be read as descriptive registry summaries. They indicate that adverse events were recorded in the groups, but the listed counts alone do not establish that an event was caused by a device or that one dose level has a different safety profile. More detailed methods and event information would be needed to interpret such comparisons.

18. Why This Trial Matters Statistically

This trial is statistically instructive because the three group mean changes from baseline are visibly more substantial than the estimated differences between dose groups. A within-group change and a between-group contrast answer different questions: the first describes change over time within each group, while the second compares the groups’ changes. The latter is central to determining whether the randomized dose assignments produced distinguishable outcomes.

The uncertainty intervals reinforce why effect estimates matter more than a simple significant-or-not label. The primary sponsor intervals include zero and extend in both directions. That pattern does not prove no difference; it shows that the estimates remain compatible with a range of possible contrasts. Reporting the estimate, interval, and analysis method together gives a more informative account than relying on a p-value alone.

The study also illustrates the difference between an analysis reported by the sponsor and a simple recalculation from summary data. The recalculated estimates provide a transparent check, but use an unadjusted method and a reversed comparison order. The sponsor’s mixed-model analysis is the study’s reported analysis and takes precedence. Readers should not mistake a recalculated result for a reconstruction of the full model.

Finally, the many secondary outcomes highlight the importance of a prespecified multiplicity strategy and transparent reporting of missing data and analysis populations. Without those details, secondary results are best treated as descriptive or exploratory. The registry record gives useful design and outcome information, but leaves uncertainty about several choices needed for a complete statistical appraisal.

19. Limitations

Several limitations specific to interpreting this trial arise from the detail available in the registry record. The Low, Medium, and High Dose labels do not include the electrical charge settings, so the dose contrast cannot be characterized in those terms from the record. The three groups all received VNS Therapy; there is no no-device comparator in the listed arms.

The primary endpoint is identified as an intention-to-treat outcome, but the registry does not define the analysis population, describe missing-data handling, or explain why the outcome analysis counts differ from overall enrollment. The sponsor’s mixed-model specification is also not reported in enough detail to assess its covariates, repeated-measures structure, or assumptions. The record does not provide a protocol or full statistical analysis plan.

The primary confidence intervals cross zero, but the registry does not state a clinically important difference threshold or an equivalence margin for IDS-C change. Consequently, the estimates cannot be judged against a trial-defined threshold for clinical importance or equivalence. The primary findings concern a week-22 measure, while several secondary measures extend to week 50; the available descriptions do not establish that these time points answer the same question.

There are numerous secondary outcomes, but multiplicity handling and the confirmatory status of individual secondary tests are not reported. The reported secondary comparisons are unadjusted recalculations. This makes isolated secondary results less persuasive as evidence of a dose effect. Safety information is also summarized without event timing, detailed narratives, or an analysis comparing risks while accounting for exposure. Deaths are not reported in the registry record.

These are limitations of what can be established from the registry record, not evidence that a particular method or procedure was absent from the full study.

20. Related Tutorials

21. Related Statistical Calculators

22. Sources

Continue through the Clinical Biostats statistical pathway

Use this trial's endpoint as a starting point for deeper study of effect measures, confidence intervals, and sample-size planning.

23. Record Summary

NCT00305565 is a completed Phase 4 randomized, triple-masked, parallel-group postmarket medical device study sponsored by Cyberonics, Inc. It enrolled 331 patients with treatment-resistant depression and compared Low Dose, Medium Dose, and High Dose VNS Therapy. The study began in 2006-01 and reached primary completion in 2010-02.

The primary endpoint was mean IDS-C change from baseline to study week 22 in the intention-to-treat population. The sponsor reported mixed-model comparisons of Low Dose with Medium Dose and High Dose. The confidence intervals include zero, and the registry record does not report an equivalence margin or a clinical-importance threshold. Secondary measures include IDS-C and QIDS-C response or remission outcomes at week 22 or week 50. Multiplicity procedures, missing-data methods, detailed model specifications, and the original sample-size justification are not reported in the registry record.

Clinical Biostats