← Clinical Trials
Sleep Deprivation NA Continuous Endpoint NCT03600194

Pilot Study Comparing the Effectiveness of Two Different Methods of Acoustic Stimulation: Complete Statistical Analysis

This analysis examines a randomized, single-blind crossover pilot of two acoustic-stimulation systems, comparing active and sham conditions on slow-wave activity and two cognitive measures. The registry reports a mixed-model analysis for the primary outcome, while unadjusted comparisons from the posted group summaries give a different inferential picture.

Trial start: 2018-06-01  ·  Enrollment: 38  ·  Sponsor: Philips Respironics
Scope of this record

This page provides an independent statistical analysis and educational interpretation of publicly reported results. ClinicalTrials.gov provides the official trial registry record. Numerical trial results on this page are restricted to the results posted in the ClinicalTrials.gov record; calculations added by Clinical Biostats are labelled as such.

1. Trial at a Glance

This page covers the registry results for Pilot Study Comparing the Effectiveness of Two Different Methods of Acoustic Stimulation (NCT03600194), a randomized na study. The primary outcome is The Effects of the Auditory Stimulation on the Cumulative or Average Slow-wave Activity (SWA) Delivered by the PowerSleep & Northwestern Systems in Adults With Mildly Sleep Restricted Schedules as Determined by Device and PSG Scoring Outputs..

38
Enrollment
NA
4
Arms
In the primary outcome
253.00
Mean difference
PowerSleep Stim
-674.58–1,181
95% CI
Two-sided
FeatureRegistry value
ClinicalTrials.gov identifierNCT03600194
PhaseNA
ConditionSleep Deprivation, Insufficient Sleep Syndrome, Healthy
AllocationRandomized
Design modelCrossover
MaskingSingle
Enrollment38
Arms in the primary outcome4
Lead sponsorPhilips Respironics
Results postedYes

2. Clinical Question

Population

Adults with mildly sleep-restricted schedules, described in the registry summary as generally healthy people aged 21–50 who reported shortened sleep related to lifestyle. Eligibility also involved reported sleep duration and timing characteristics. The registry record lists sleep deprivation, insufficient sleep syndrome, and healthy participants among the conditions.

Intervention

Active acoustic stimulation during deep sleep, using either the PowerSleep prototype or the Northwestern system. The registry describes soft tones delivered by the devices, with the Northwestern setup using headphones.

Comparator

Sham use of the corresponding device setup, with no audio tones played. The primary outcome comparisons include PowerSleep stimulation versus PowerSleep sham, and Northwestern stimulation versus PowerSleep sham in the posted summary-based calculations; the registry’s primary description also says each prototype was compared with its own sham condition.

Primary question

Does active acoustic stimulation change cumulative or average slow-wave activity (SWA) across nonREM sleep compared with sham, and does the relative change differ between the two prototypes? The registry’s stated hypothesis anticipated an increase of at least 5% under active use, but the posted group summaries and the sponsor’s reported analysis need to be read separately.

This was a pilot study, not a definitive comparison. The registry describes adults with sleep-restricted schedules and a randomized, single-blind, sham-controlled crossover design. Participants were to use the systems in a laboratory setting; the brief summary describes two nights of in-lab use with active versus sham conditions, while the primary-outcome description frames the comparison as one night under active use versus one night under sham. The record does not resolve that difference in phrasing.

The clinical question is about a sleep measurement, not a patient-centered health outcome. SWA is recorded as a continuous quantity in microvolts^2, and the posted primary measure is a mean. The registry also lists brief psychomotor vigilance and paired associated learning as secondary measures. Those outcomes may help describe performance alongside SWA, but they do not replace the primary sleep-activity question.

3. Trial Design

The registry identifies this as a randomized, single-blind, sham-controlled pilot crossover trial. A crossover design means that a participant can receive more than one study condition over time, allowing outcomes under different conditions to be compared within the same person. In general, this can account for stable differences between participants, but valid interpretation depends on the sequence and timing of conditions and on appropriate analysis. The registry record does not report the treatment sequence, washout arrangements, or the details needed to assess possible carryover effects.

The study involved two device systems, PowerSleep and Northwestern, each with active and sham configurations. Active conditions used soft audio tones during deep sleep; sham configurations were described as having no tones. The primary outcome description says the active and sham conditions for each prototype were compared, and that the prototypes’ relative ability to increase SWA would also be compared. The registry’s posted group summaries list 11 observations for each of the four condition groups. In a crossover study, those group counts should not automatically be read as distinct participants in four separate parallel arms.

The registry lists enrollment of 38 and status as completed. Its brief summary also describes a plan to consent up to 60 individuals to complete a total of 10 participants, while the posted outcome groups each have 11 observations. The record does not explain how those enrollment and completion statements relate to the outcome counts. The expected study duration for each participant was up to 5 weeks; the primary outcome timeframe is also listed as 5 weeks.

The study started on 2018-06-01 and had primary completion on 2019-07-25. Phase is listed as not applicable. The sponsor is Philips Respironics. The registry does not report the randomization sequence, allocation concealment, detailed blinding procedures, prespecified analysis population, or a full crossover analysis plan. Those omissions limit how much can be inferred about implementation and analysis from the record alone.

4. Participant Flow and Accounting

The registry reports how participants moved through each stage of the study. All counts below are exactly as posted; missing counts are not inferred from other stages.

Before assignment: Because of all of the different randomization variations, participants could have received the powersleep or the northwestern device first and then could have been randomized to stim or sham first for both set of devices, data has been presented by device rather than overnight device configurations.

Screening

StageAll Participants ConsentedPowerSleep First, Northwestern SecondNorthwestern First, PowerSleep Second
STARTED3800
COMPLETED1100
NOT COMPLETED2700

In Lab Phase

StageAll Participants ConsentedPowerSleep First, Northwestern SecondNorthwestern First, PowerSleep Second
STARTED056
COMPLETED056
NOT COMPLETED000

5. Baseline Characteristics

The table reproduces the baseline characteristics posted in the registry, which describe the participants analysed at baseline and help show whether the randomized groups were comparable.

CharacteristicUnitScreening Population
Participants analysed38
Age, Continuousyears29.4 (SD 6.8)
Sex: Female, Male: FemaleParticipants24
Sex: Female, Male: MaleParticipants14
Ethnicity (NIH/OMB): Hispanic or LatinoParticipants6
Ethnicity (NIH/OMB): Not Hispanic or LatinoParticipants32
Ethnicity (NIH/OMB): Unknown or Not ReportedParticipants0
Race (NIH/OMB): American Indian or Alaska NativeParticipants0
Race (NIH/OMB): AsianParticipants7
Race (NIH/OMB): Native Hawaiian or Other Pacific IslanderParticipants0
Race (NIH/OMB): Black or African AmericanParticipants10
Race (NIH/OMB): WhiteParticipants17
Race (NIH/OMB): More than one raceParticipants3
Race (NIH/OMB): Unknown or Not ReportedParticipants1
Region of Enrollment: United Statesparticipants38
Body Mass Index (BMI)kg/m^226.2 (SD 5.6)

6. Primary Endpoint

The registry lists the primary outcome as The Effects of the Auditory Stimulation on the Cumulative or Average Slow-wave Activity (SWA) Delivered by the PowerSleep & Northwestern Systems in Adults With Mildly Sleep Restricted Schedules as Determined by Device and PSG Scoring Outputs., measured over 5 weeks.

ElementRegistry entry
Result typeMean
Measure of spreadStandard Deviation
Unitmicrovolts^2
Endpoint classificationContinuous Endpoint

The primary endpoint is cumulative or average slow-wave activity (SWA) across the whole night of nonREM sleep, measured in microvolts^2. The registry describes the outcome as a mean, with standard deviation as the measure of spread, and says the results are determined using device and polysomnography scoring outputs. The timeframe is 5 weeks.

SWA is used in this study as a continuous physiological measure of slow-wave activity during sleep. The study’s stated hypothesis is that active acoustic stimulation would increase SWA relative to sham, and the registry says the relative ability of the two prototypes would also be compared. This makes the endpoint directly aligned with the proposed mechanism being studied: the systems deliver tones during deep sleep, and the measured outcome is the amount of slow-wave activity recorded over nonREM sleep.

A mean describes the average observed value within a condition; the standard deviation describes how dispersed individual observations are around that mean. These summaries give a basic view of the outcome, but they do not by themselves account for the crossover structure or show how the same participants’ measurements relate across conditions. The registry does not report the scoring algorithm, the exact definition of the cumulative versus average measure used for the posted result, or further measurement-quality details. Those elements are important for judging reproducibility and clinical interpretation, but are not reported in the registry record.

7. Results Posted in the Registry

Mean difference: PowerSleep Stim

253.00

95% CI: -674.58 to 1,181

Recalculated from the posted summary statistics

Primary outcome by arm, with 95% confidence intervals
Primary outcome by arm with 95% confidence intervals1,0641,5021,9392,3762,8143,251PowerSleep Stim (n=11)PowerSleep Sham (n=11)Northwestern Stim (n=11)Northwestern Sham (n=11)microvolts^2 (95% CI)
Treatment effect versus the control arm
Forest plot of mean difference vs control (95% CI)PowerSleep StimNorthwestern StimNorthwestern Sham-1,039-508235541,0861,617mean difference vs control (95% CI)

8. Secondary Efficacy Endpoints

The registry also posts results for secondary outcomes. For each outcome below, the endpoint type, the test applied to the posted summary statistics, the result and its interpretation are given in turn. Secondary endpoints are usually not adjusted for multiple testing, so they support rather than confirm the primary finding.

Secondary outcomeTypeEffect measureEstimate (95% CI)Planning size, 80% power
The Effects of Active and Sham Conditions of PowerSleep and Northwestern University Systems on Psychomotor Vigilance Task - BriefContinuous EndpointMean difference65.10 (-56.61 to 186.81)79 per arm
The Effects of Active and Sham Conditions of PowerSleep and Northwestern University Systems on Paired Associated LearningContinuous EndpointMean difference-1.50 (-8.29 to 5.29)462 per arm

The Effects of Active and Sham Conditions of PowerSleep and Northwestern University Systems on Psychomotor Vigilance Task - Brief

Treatment effect versus the control arm
Forest plot of mean difference vs control (95% CI)PowerSleep StimNorthwestern Stim-76-203793150206mean difference vs control (95% CI)

The Effects of Active and Sham Conditions of PowerSleep and Northwestern University Systems on Paired Associated Learning

Treatment effect versus the control arm
Forest plot of mean difference vs control (95% CI)PowerSleep StimNorthwestern Stim-9.83-5.38-0.943.507.9412mean difference vs control (95% CI)

The secondary outcomes are a brief psychomotor vigilance task measured in milliseconds and paired associated learning measured in words. Across these endpoints, the group summaries do not show a uniform pattern in which active conditions consistently move in the same direction relative to sham. The displayed summary-based contrasts also have broad confidence intervals, and the registry does not report sponsor analyses for either secondary endpoint.

These measures provide additional context about performance, but they are not substitutes for the primary SWA endpoint. The primary question concerns slow-wave activity during sleep; the secondary outcomes concern task performance. Their results therefore address related but distinct questions. The registry does not report a joint model linking changes in SWA to changes in either cognitive outcome, so no relationship between the endpoints can be inferred.

The secondary comparisons are unadjusted. In general, when several outcomes and contrasts are examined, the chance of observing an apparently notable result by chance can increase. Adjustments or a prespecified hierarchy can help control that issue, but the registry record does not report whether either was used for these outcomes. The secondary findings should consequently carry less weight than a prespecified, fully reported primary analysis, particularly in a pilot with limited observations and incomplete analytical detail.

The broad pattern is best understood as exploratory rather than as confirmation of a consistent cognitive effect. The registry record does not report a multiplicity strategy, clinically important thresholds for these tasks, or modeled estimates with uncertainty. The secondary data can motivate further study, but they do not resolve the primary analysis discrepancy or establish that an SWA change corresponds to a performance change.

Secondary outcomes registered for this trial, for reference (only those above were recalculated):

9. Statistical Interpretation

Clinical Biostats interpretation

For the posted group summaries, mean SWA was 2084.0 microvolts^2 under PowerSleep stimulation and 1831.0 microvolts^2 under PowerSleep sham. The unadjusted summary-based mean difference for PowerSleep stimulation versus PowerSleep sham is 253.0, with a 95% confidence interval from -674.6 to 1181.0 and an unadjusted p-value of 0.593. The interval includes zero, so these summary-based data are compatible with a range of effects in either direction as well as with little difference.

Mean SWA was 2317.0 microvolts^2 under Northwestern stimulation. Compared with PowerSleep sham, the unadjusted summary-based mean difference is 486.1, with a 95% confidence interval from -461.3 to 1433.0 and an unadjusted p-value of 0.315. This interval also includes zero and is wide. The Northwestern sham mean was 1723.0 microvolts^2; its unadjusted difference versus PowerSleep sham is -107.8, with a 95% confidence interval from -855.7 to 640.1 and an unadjusted p-value of 0.778. That interval likewise includes zero.

These are recalculated, unadjusted comparisons from the posted summary statistics. They are not the sponsor’s reported mixed-model analysis. The registry reports that analysis as a superiority test across the four groups, with p = .019, but gives no model estimate, confidence interval, or further result detail. Because the sponsor’s analysis differs from the summary-based calculations, the sponsor’s own analysis takes precedence as the registry-reported primary analysis. The p-value alone does not identify which conditions differ, the size of a difference, or its precision.

The posted summaries therefore do not support treating the recalculations as a replacement for the crossover model. Nor should the sponsor’s overall p-value be interpreted as evidence that every active condition differed from its sham comparator. The record does not report condition-specific estimates from the mixed model, so the magnitude and direction of the sponsor’s modeled contrasts cannot be described. This distinction is central: an overall test across conditions and individual pairwise contrasts address different questions.

The study is explicitly described as a pilot, with 11 observations per condition in the posted outcome summaries. In a small study, estimates can be imprecise, and a nonsignificant unadjusted p-value does not establish equivalence or rule out potentially meaningful effects. Conversely, a reported p-value of .019 for an overall model does not establish the size, reliability, or clinical importance of any particular contrast. The registry record does not state a threshold for a meaningful SWA change beyond the hypothesis that the increase would be at least 5%; it does not provide a modeled estimate with which to compare that hypothesis.

10. Statistical Methods Explained

The primary outcome is continuous, so its posted summaries are means and standard deviations. A mean gives the central average of the measurements in a condition. A standard deviation describes how much individual values vary around that average. For SWA, the reported means range across the four conditions, and the standard deviations are substantial relative to those means. That spread matters because the uncertainty around a mean difference depends not only on the difference between averages but also on how variable the observations are and how many observations contribute.

What does a mean difference represent?

A mean difference expresses the average outcome in one condition relative to another, in the original outcome units. The summary-based comparisons use PowerSleep sham, with a mean of 1831.0 microvolts^2, as the reference. A positive estimate means the active or other listed condition has a higher posted mean than that reference; a negative estimate means a lower mean. These recalculations are unadjusted comparisons based on the posted group summaries. They do not provide the sponsor’s model-adjusted result.

Because the trial is a crossover, the observations across conditions may come from the same participants. An analysis designed for crossover data can account for within-person relationships, which may differ from treating each condition as an unrelated group. The registry reports a mixed models analysis for the primary outcome, but does not report its model specification, covariance structure, treatment sequence terms, period effects, or condition-specific estimates. The available group summaries do not provide the within-person correlations needed to recreate that analysis. Thus, the simple recalculations are useful descriptive contrasts, but should not be treated as a reconstruction of the sponsor’s mixed model.

How should the confidence intervals be read?

A confidence interval describes the uncertainty around an estimated effect under the method used to construct it. A wide interval indicates limited precision: many effect sizes remain compatible with the data and method. If an interval includes zero for a mean difference, the estimates are compatible with both directions of difference and with no difference. That does not prove that the conditions are equivalent. Equivalence requires a prespecified margin and an analysis designed to test whether the plausible differences fall within it; such a margin and analysis are not reported in the registry record.

The recalculated intervals are unadjusted. They are not the confidence intervals from the sponsor’s mixed model, which are not reported. Their interpretation is therefore limited to the summary-based comparison method. They also do not account for the set of multiple comparisons or outcomes unless a method specifically doing so is reported; the registry record does not report such an adjustment.

Why does over-dispersion matter?

In general, over-dispersion means that observed variability is greater than a statistical model expects. This is more commonly discussed for count outcomes than for a continuous measure such as SWA, but the broader principle applies: if variability is large relative to the estimated effect, uncertainty grows and estimates become less precise. The posted standard deviations show that there is considerable spread within conditions. A model should represent the outcome’s variability appropriately. The registry does not report a specific over-dispersion assessment or a model diagnostic, so none can be inferred.

What does the sponsor’s test establish?

The registry lists a mixed models analysis and a superiority test across PowerSleep stimulation, PowerSleep sham, Northwestern stimulation, and Northwestern sham, with p = .019. In general, a model can use the structure of repeated measurements and account for specified sources of variation. An overall test can assess whether the modeled condition means are consistent with a common value, but it does not, by itself, identify which pairs differ or give the size and direction of each pairwise contrast. The registry does not give the model’s test statistic, degrees of freedom, parameter estimates, confidence intervals, or pairwise results. The reported p-value should therefore be read as a limited summary of the sponsor’s overall test, not as a complete account of the comparative effects.

The discrepancy flag indicates that the recalculated summary-based analysis does not match the sponsor’s reported result. In that circumstance, the sponsor’s own analysis takes precedence for describing the registry-reported analysis. The recalculations remain a separate, transparent reading of posted means and standard deviations, but they cannot resolve why the results differ. Missing details about the mixed model and crossover data structure prevent a full reconciliation.

11. Confidence Intervals and Precision

The recalculated confidence intervals are broad. For PowerSleep stimulation versus PowerSleep sham, the estimated mean difference is 253.0 microvolts^2, with a 95% confidence interval from -674.6 to 1181.0. For Northwestern stimulation versus PowerSleep sham, the estimate is 486.1, with a 95% confidence interval from -461.3 to 1433.0. Each interval includes zero and spans values on both sides of it. The Northwestern sham comparison with PowerSleep sham is also imprecise: its estimate is -107.8, and the 95% confidence interval ranges from -855.7 to 640.1.

In general, a confidence interval gives a range of effect estimates compatible with the observed data under the calculation method. It is not a guarantee that the true effect lies within the interval, and a wide interval is not evidence of no effect. Here, the breadth of the recalculated intervals means the posted summaries do not pin down the size or direction of the contrasts with much precision. The relatively small number of observations per condition and the variability in the outcome are relevant to that uncertainty.

These intervals come from unadjusted recalculations, not from the sponsor’s mixed model. The registry does not report confidence intervals for the sponsor’s primary analysis. It therefore does not permit a comparison of the modeled estimate’s precision with the summary-based estimates. The sponsor’s reported p-value of .019 does not convey the missing effect estimate or its uncertainty. Interpretation should keep the overall model test separate from the imprecise pairwise recalculations.

12. Planning a Similar Trial: Sample Size

This is a planning illustration, not a re-evaluation of the original trial. It asks: if a new study wanted to detect a difference like the one observed, or a smaller one, how large would it need to be? It is deliberately not “observed power” of the finished trial, which is not a meaningful quantity.

Effect to detectParticipants per arm, 80% powerParticipants per arm, 90% power
100% of the observed effect303405
75% of the observed effect538720
50% of the observed effect1,2091,618
Sample size needed versus the size of the effect to detect
Sample size needed versus size of effect to detect2005001,0002,0005,00040%60%80%100%80% power90% powersize of effect to detect, as % of the effect observed (log vertical scale: participants per arm)
Treat these figures cautiously. The observed difference is imprecise: its confidence interval includes no difference at all. A sample size built on a noisy estimate can be far too small or too large, so the smaller-effect rows are the more cautious guide.
Method and assumptions. Two-sided α = 0.05, 1:1 allocation, two-sample comparison of means with the pooled standard deviation. Calculated for the comparison: PowerSleep Stim.

The planning figures are participants per arm, not total participants or total events. The figures are illustrations based on the observed effect in each recalculated comparison, not the original trial’s design assumptions. The illustration uses a two-sided significance level of 0.05, equal allocation between two arms, and a normal approximation. It considers detecting the full observed effect, 75% of the observed effect, and 50% of the observed effect. The original trial’s sample-size assumptions and justification are not reported in the registry record.

The 80% power column gives the estimated participants per arm for a design with an 80% chance of detecting the specified effect under those planning assumptions, if that effect were the true effect and the test assumptions held. The 90% power column gives the corresponding per-arm planning figure for a 90% chance. These are design illustrations, not results about the chance that this completed pilot detected an effect.

The figures vary by comparison because the observed estimates and variability differ. For example, planning for 100% of the observed Northwestern stimulation effect versus PowerSleep sham gives 86 participants per arm for 80% power and 115 for 90% power. Planning for 75% of that observed effect gives 152 per arm for 80% power and 204 for 90% power; for 50%, the figures are 342 and 458. Smaller effects require more participants because they are harder to distinguish from random variation at the same significance level and target power.

These are not observed power calculations. They should not be read as evidence that the trial was adequately or inadequately powered for its original question. The assumptions behind the trial’s own planning, including any crossover-specific assumptions, are not reported.

13. How the Sample Size Is Derived

Power is the probability that a study of a given size rejects the null hypothesis when a specific true effect exists. It depends on four things: the effect to detect, the variability of the outcome, the significance level (here a two-sided α of 0.05, so z1−α/2 = 1.960) and the desired power (z1−β = 0.842 for 80% and 1.282 for 90%).

Formula

n = 2 (z1−α/2 + z1−β)2 σ2 / Δ2

n is the number of participants in each arm, Δ is the difference in means to detect and σ is the common standard deviation.

Worked calculation for this trial

  1. Observed difference Δ = 253.000; pooled standard deviation σ = 1109.901.
  2. n = 2 × (1.96 + 0.842)2 × 1109.9012 / 253.0002 = 302.1, rounded up to 303 participants per arm for 80% power.

The smaller-effect rows in the table above repeat the same calculation with the effect multiplied by 0.75 and 0.5 (for ratios, applied to the logarithm of the ratio).

R code

alpha <- 0.05
z_a   <- qnorm(1 - alpha / 2)          # two-sided
delta <- 253                      # observed difference in means
sigma <- 1109.90142355076                     # pooled standard deviation
n_per_arm <- function(power, frac = 1) {
  ceiling(round(2 * (z_a + qnorm(power))^2 * sigma^2 / (delta * frac)^2, 6))
}
for (f in c(1, 0.75, 0.5))
  cat(sprintf("%3.0f%% of observed effect: n per arm = %s (80%% power), %s (90%% power)\n", 100 * f,
              format(n_per_arm(0.80, f), big.mark = ","), format(n_per_arm(0.90, f), big.mark = ",")))

# Check against R's built-in function (t-based, so very slightly larger)
cat("power.t.test:", round(power.t.test(delta = delta, sd = sigma, power = 0.80)$n, 1), "\n")

# Simulation: exact two-sample t-test using simulated means and variances
set.seed(3600194)
n    <- n_per_arm(0.80)
reps <- 20000
m_diff <- rnorm(reps, delta, sigma * sqrt(2 / n))             # difference in sample means
s2     <- sigma^2 * rchisq(reps, 2 * (n - 1)) / (2 * (n - 1))   # pooled sample variance
tstat  <- m_diff / sqrt(s2 * 2 / n)
cat(sprintf("Simulated power at n = %s per arm: %.3f (%d replicates)\n", format(n, big.mark = ","), mean(abs(tstat) > qt(0.975, 2 * (n - 1))), reps))

Output from running the code

100% of observed effect: n per arm = 303 (80% power), 405 (90% power)
 75% of observed effect: n per arm = 538 (80% power), 720 (90% power)
 50% of observed effect: n per arm = 1,209 (80% power), 1,618 (90% power)
power.t.test: 303.1 
Simulated power at n = 303 per arm: 0.800 (20000 replicates)

The simulation draws many sets of group means and variances at the planned sample size and counts how often the two-sample t-test rejects the null. It should land close to the target power; at very small required sample sizes it can fall below it, because the normal approximation is rough there.

14. Analyses Reported by the Sponsor

MethodEstimate95% CIP-value
Mixed Models Analysisnot reported0.019
Where the two analyses differ, rely on the sponsor's. The simple re-calculation above reaches a different conclusion from the sponsor's analysis, usually because the sponsor adjusts for covariates, stratification or repeated measures, or (for event rates) uses a negative binomial model that allows for over-dispersion and so gives wider intervals.

The registry reports a sponsor analysis using mixed models, described as a superiority analysis across all four condition groups. Its reported p-value is .019. The record does not provide an effect estimate, confidence interval, test statistic, model specification, or pairwise contrasts for that analysis. As a result, the overall test cannot be translated into a condition-specific difference or a measure of its precision from the registry information alone.

The unadjusted recalculations use the posted means and standard deviations to compare each listed condition with PowerSleep sham. Their confidence intervals include zero, and their unadjusted p-values are 0.593 for PowerSleep stimulation, 0.315 for Northwestern stimulation, and 0.778 for Northwestern sham. These calculations are not the sponsor’s mixed model and do not incorporate the reported model result. Since the discrepancy flag is true, the sponsor’s own analysis takes precedence when describing the primary analysis reported by the registry.

The difference between the two analyses cannot be explained from the record. A crossover model may use repeated measurements and within-participant information that are unavailable in the group summaries, but the record does not provide enough model details or participant-level data to establish whether that explains the discrepancy. The sponsor’s p-value should not be treated as a pairwise finding, while the recalculated contrasts should not be treated as a replacement for the sponsor’s primary model.

15. Multiplicity, Missing Data and Analysis Populations

Multiplicity arises when multiple hypotheses are tested. This trial includes four condition groups, more than one possible pairwise comparison, a primary outcome, and two secondary outcomes. In general, testing many contrasts without an adjustment can increase the chance of at least one false-positive finding. A prespecified testing hierarchy, family-wise error adjustment, or another multiplicity strategy can clarify how to interpret multiple results. The registry record does not report whether multiplicity was considered, how comparisons were prioritized, or whether the reported p-value was adjusted.

The sponsor’s reported p = .019 is attached to a mixed-model superiority analysis across the four groups, but the record does not state whether it is an omnibus test or how its result relates to planned pairwise contrasts beyond identifying it as a superiority analysis. The record does not provide the exact contrast structure or an adjusted analysis plan. The summary-based recalculations are explicitly unadjusted and should not be taken as a multiplicity-controlled set of conclusions.

Missing-data handling is also not reported in the registry record. It does not state whether all randomized participants contributed to each endpoint, how incomplete sleep recordings or task assessments were handled, or whether any imputation was used. The posted group summaries list 11 observations for each condition, but they do not explain the analysis population or whether any observations were excluded. In a crossover study, missing data in one condition can affect whether a participant contributes to within-person comparisons.

The registry record does not report the analysis population definition, such as an intention-to-treat or per-protocol population, nor does it describe deviations from assigned conditions. Details of the mixed-model specification are also absent, including how repeated measurements and any period or sequence effects were represented. These omissions prevent readers from assessing the robustness of the primary result to alternative analysis choices.

These are reporting limitations, not evidence that particular methods were or were not used. The registry does not report an interim-analysis plan, stopping rules, or a formal missing-data sensitivity analysis. Without those details, the inferential results remain difficult to audit and reproduce.

16. Safety Results

The registry reports serious adverse events as affected participants divided by participants at risk.

ArmParticipants with serious adverse eventsParticipants at riskPercentage
Screening Population0380.0%
PowerSleep Stim0110.0%
PowerSleep Sham0110.0%
Northwestern Stim0110.0%
Northwestern Sham0110.0%

Deaths

ArmDeaths (participants affected of at risk)
Screening Population0 of 38
PowerSleep Stim0 of 11
PowerSleep Sham0 of 11
Northwestern Stim0 of 11
Northwestern Sham0 of 11

Other (non-serious) adverse events

Participants with at least one non-serious event at or above the registry's 0% reporting threshold.

ArmParticipants affected of at risk
Screening Population1 of 38
PowerSleep Stim0 of 11
PowerSleep Sham1 of 11
Northwestern Stim0 of 11
Northwestern Sham1 of 11

Most frequent non-serious adverse event terms

Event termOrgan systemScreening PopulationPowerSleep StimPowerSleep ShamNorthwestern StimNorthwestern Sham
InfluenzaImmune system disorders1 of 380 of 110 of 110 of 110 of 11
Skin IrritationSkin and subcutaneous tissue disorders0 of 380 of 110 of 110 of 111 of 11
AllergiesImmune system disorders0 of 380 of 111 of 110 of 110 of 11
Do not overextend the safety comparison. Serious-adverse-event counts are safety observations, not efficacy outcomes, and should not be combined with the primary result into a single benefit-risk number.

The adverse-event record lists no serious adverse events in the screening population or in any of the four condition groups. It also lists zero deaths in each group. For other adverse events, the screening population has one event among 38 at risk; PowerSleep sham has one among 11 at risk; Northwestern sham has one among 11 at risk. The PowerSleep stimulation and Northwestern stimulation groups each list zero other adverse events among 11 at risk.

These counts should be read as recorded events, not as a complete assessment of comparative safety. The registry does not list the terms for serious events, and no serious events are shown. It does not provide details about the single other-event terms, severity, timing, relationship to a device, or how events were collected. The screening population count and condition-group counts should not be added together; the crossover design means the same people may contribute under more than one condition, and the record does not explain the relationship between those populations.

With a small pilot and limited event detail, the counts describe what was recorded in the registry but do not establish that events could not occur. No medical advice or broader safety conclusion follows from these figures alone.

17. What the Result Does, and Does Not, Mean

The primary result has two distinct components in the registry: an overall mixed-model superiority test reported with p = .019, and unadjusted recalculations from the posted group summaries. The sponsor’s model is the registry-reported primary analysis and takes precedence over the recalculations because the discrepancy flag is true. However, the registry gives no modeled estimate or confidence interval, so the size, direction, and precision of the sponsor’s result cannot be described. Its p-value alone does not identify which condition contrasts account for the overall result.

The recalculated comparisons offer a separate view of the posted means. Their estimated differences are uncertain, with confidence intervals spanning zero. That does not establish that active stimulation and sham are equivalent, nor does it prove that there is no effect. The estimates are based on small condition groups and do not reproduce the crossover mixed model. The registry’s hypothesis of an increase of at least 5% is stated, but the sponsor’s model results do not provide the condition-specific estimates needed to assess that hypothesis directly.

The trial is a pilot study, and its purpose includes assessing feasibility as well as efficacy. The registry record does not report a detailed feasibility result. It also does not provide enough information about the crossover sequence, missing data, model structure, or participant-level outcomes to establish how robust the analyses are. The posted cognitive outcomes add exploratory information but do not demonstrate a consistent relationship between sleep activity and performance.

Accordingly, the registry supports a cautious description of the statistical record, not a clinical conclusion. The overall test is reported as statistically notable, while the summary-based pairwise estimates are imprecise and do not match that analysis. The available record cannot resolve the discrepancy or establish the magnitude or clinical importance of a particular device-condition effect.

18. Why This Trial Matters Statistically

This study illustrates why analysis design matters as much as the arithmetic of comparing averages. In a crossover trial, the same participants can experience multiple conditions, so within-person information may be important. A mixed model may account for features of repeated measurements, whereas simple comparisons of condition means do not necessarily use that information. When the registry reports only group summaries alongside an overall model p-value, the two approaches can yield different inferential messages that cannot be reconciled without the model details.

It also shows the importance of reporting estimates and uncertainty alongside p-values. The sponsor’s analysis gives p = .019 but no effect estimate or confidence interval. The unadjusted recalculations provide estimates and intervals, but they do not reproduce the sponsor’s analysis. A p-value without an effect size cannot describe the magnitude of a difference, and an estimate without a suitable model cannot fully answer a crossover question.

Finally, the planning illustrations demonstrate how sample-size requirements depend on the effect size one aims to detect and the target power. They are not a substitute for the original design justification, which is not reported. In a pilot, estimates can be unstable, and an observed effect is not necessarily a reliable planning target. The distinctions between pilot evidence, confirmatory analysis, and planning calculations are essential when interpreting small clinical studies.

19. Limitations

Several limitations are specific to this registry record. First, the study is identified as a crossover trial, but the record does not report condition sequence, period effects, washout arrangements, or other details needed to understand potential carryover. It also does not provide the mixed-model specification. The sponsor’s reported p-value therefore cannot be tied to a clearly described set of modeled contrasts.

Second, the sponsor’s overall result and the unadjusted summary-based recalculations differ. The sponsor analysis takes precedence, but the record lacks estimates and intervals from that analysis, so the discrepancy cannot be investigated. The simple recalculations use condition-level means and standard deviations and do not capture within-person relationships that may be relevant in a crossover design.

Third, the registry’s participant accounting is not fully clear: enrollment is listed as 38, the summary describes consenting up to 60 to complete 10 participants, and the posted outcome groups each list 11 observations. The record does not explain these figures or identify the analysis population. Missing-data procedures and exclusions are not reported.

Fourth, the primary description refers to one night under active use versus one night under sham, while the brief summary describes two nights of in-lab use with active versus sham conditions. The record does not clarify the relationship between those descriptions. Additional limitations include absent primary-model estimates, no reported multiplicity strategy, limited adverse-event detail, and no sponsor-reported analyses for the secondary outcomes. These gaps constrain interpretation and reproducibility; they do not establish that unreported procedures were not conducted.

20. Related Tutorials

21. Related Statistical Calculators

22. Sources

Continue through the Clinical Biostats statistical pathway

Use this trial's endpoint as a starting point for deeper study of effect measures, confidence intervals, and sample-size planning.

23. Record Summary

NCT03600194 is a completed, randomized, single-blind crossover pilot sponsored by Philips Respironics. It compared active and sham acoustic stimulation using PowerSleep and Northwestern systems in adults described as having mildly sleep-restricted schedules. The registry lists 38 participants enrolled and a primary completion date of 2019-07-25.

The primary outcome was mean cumulative or average SWA during nonREM sleep, measured in microvolts^2. The registry reports a mixed-model superiority analysis with p = .019, but does not report its effect estimate or confidence interval. Unadjusted recalculations from posted summaries have broad confidence intervals; because the reported analyses differ, the sponsor’s analysis takes precedence. Two secondary outcomes and adverse-event counts are also posted, but analytical detail is limited.

Important details about the crossover analysis, multiplicity, missing data, and analysis populations are not reported in the registry record. The findings should be read with those limitations in mind.

Clinical Biostats