This page provides an independent statistical analysis and educational interpretation of publicly reported results. ClinicalTrials.gov provides the official trial registry record. Numerical trial results on this page are restricted to the results posted in the ClinicalTrials.gov record; calculations added by Clinical Biostats are labelled as such.
1. Trial at a Glance
This page covers the registry results for Pilot Study Comparing the Effectiveness of Two Different Methods of Acoustic Stimulation (NCT03600194), a randomized na study. The primary outcome is The Effects of the Auditory Stimulation on the Cumulative or Average Slow-wave Activity (SWA) Delivered by the PowerSleep & Northwestern Systems in Adults With Mildly Sleep Restricted Schedules as Determined by Device and PSG Scoring Outputs..
| Feature | Registry value |
|---|---|
| ClinicalTrials.gov identifier | NCT03600194 |
| Phase | NA |
| Condition | Sleep Deprivation, Insufficient Sleep Syndrome, Healthy |
| Allocation | Randomized |
| Design model | Crossover |
| Masking | Single |
| Enrollment | 38 |
| Arms in the primary outcome | 4 |
| Lead sponsor | Philips Respironics |
| Results posted | Yes |
2. Clinical Question
Population
Adults with mildly sleep-restricted schedules, described in the registry summary as generally healthy people aged 21–50 who reported shortened sleep related to lifestyle. Eligibility also involved reported sleep duration and timing characteristics. The registry record lists sleep deprivation, insufficient sleep syndrome, and healthy participants among the conditions.
Intervention
Active acoustic stimulation during deep sleep, using either the PowerSleep prototype or the Northwestern system. The registry describes soft tones delivered by the devices, with the Northwestern setup using headphones.
Comparator
Sham use of the corresponding device setup, with no audio tones played. The primary outcome comparisons include PowerSleep stimulation versus PowerSleep sham, and Northwestern stimulation versus PowerSleep sham in the posted summary-based calculations; the registry’s primary description also says each prototype was compared with its own sham condition.
Primary question
Does active acoustic stimulation change cumulative or average slow-wave activity (SWA) across nonREM sleep compared with sham, and does the relative change differ between the two prototypes? The registry’s stated hypothesis anticipated an increase of at least 5% under active use, but the posted group summaries and the sponsor’s reported analysis need to be read separately.
This was a pilot study, not a definitive comparison. The registry describes adults with sleep-restricted schedules and a randomized, single-blind, sham-controlled crossover design. Participants were to use the systems in a laboratory setting; the brief summary describes two nights of in-lab use with active versus sham conditions, while the primary-outcome description frames the comparison as one night under active use versus one night under sham. The record does not resolve that difference in phrasing.
The clinical question is about a sleep measurement, not a patient-centered health outcome. SWA is recorded as a continuous quantity in microvolts^2, and the posted primary measure is a mean. The registry also lists brief psychomotor vigilance and paired associated learning as secondary measures. Those outcomes may help describe performance alongside SWA, but they do not replace the primary sleep-activity question.
3. Trial Design
The registry identifies this as a randomized, single-blind, sham-controlled pilot crossover trial. A crossover design means that a participant can receive more than one study condition over time, allowing outcomes under different conditions to be compared within the same person. In general, this can account for stable differences between participants, but valid interpretation depends on the sequence and timing of conditions and on appropriate analysis. The registry record does not report the treatment sequence, washout arrangements, or the details needed to assess possible carryover effects.
The study involved two device systems, PowerSleep and Northwestern, each with active and sham configurations. Active conditions used soft audio tones during deep sleep; sham configurations were described as having no tones. The primary outcome description says the active and sham conditions for each prototype were compared, and that the prototypes’ relative ability to increase SWA would also be compared. The registry’s posted group summaries list 11 observations for each of the four condition groups. In a crossover study, those group counts should not automatically be read as distinct participants in four separate parallel arms.
The registry lists enrollment of 38 and status as completed. Its brief summary also describes a plan to consent up to 60 individuals to complete a total of 10 participants, while the posted outcome groups each have 11 observations. The record does not explain how those enrollment and completion statements relate to the outcome counts. The expected study duration for each participant was up to 5 weeks; the primary outcome timeframe is also listed as 5 weeks.
The study started on 2018-06-01 and had primary completion on 2019-07-25. Phase is listed as not applicable. The sponsor is Philips Respironics. The registry does not report the randomization sequence, allocation concealment, detailed blinding procedures, prespecified analysis population, or a full crossover analysis plan. Those omissions limit how much can be inferred about implementation and analysis from the record alone.
4. Participant Flow and Accounting
The registry reports how participants moved through each stage of the study. All counts below are exactly as posted; missing counts are not inferred from other stages.
Before assignment: Because of all of the different randomization variations, participants could have received the powersleep or the northwestern device first and then could have been randomized to stim or sham first for both set of devices, data has been presented by device rather than overnight device configurations.
Screening
| Stage | All Participants Consented | PowerSleep First, Northwestern Second | Northwestern First, PowerSleep Second |
|---|---|---|---|
| STARTED | 38 | 0 | 0 |
| COMPLETED | 11 | 0 | 0 |
| NOT COMPLETED | 27 | 0 | 0 |
In Lab Phase
| Stage | All Participants Consented | PowerSleep First, Northwestern Second | Northwestern First, PowerSleep Second |
|---|---|---|---|
| STARTED | 0 | 5 | 6 |
| COMPLETED | 0 | 5 | 6 |
| NOT COMPLETED | 0 | 0 | 0 |
5. Baseline Characteristics
The table reproduces the baseline characteristics posted in the registry, which describe the participants analysed at baseline and help show whether the randomized groups were comparable.
| Characteristic | Unit | Screening Population |
|---|---|---|
| Participants analysed | 38 | |
| Age, Continuous | years | 29.4 (SD 6.8) |
| Sex: Female, Male: Female | Participants | 24 |
| Sex: Female, Male: Male | Participants | 14 |
| Ethnicity (NIH/OMB): Hispanic or Latino | Participants | 6 |
| Ethnicity (NIH/OMB): Not Hispanic or Latino | Participants | 32 |
| Ethnicity (NIH/OMB): Unknown or Not Reported | Participants | 0 |
| Race (NIH/OMB): American Indian or Alaska Native | Participants | 0 |
| Race (NIH/OMB): Asian | Participants | 7 |
| Race (NIH/OMB): Native Hawaiian or Other Pacific Islander | Participants | 0 |
| Race (NIH/OMB): Black or African American | Participants | 10 |
| Race (NIH/OMB): White | Participants | 17 |
| Race (NIH/OMB): More than one race | Participants | 3 |
| Race (NIH/OMB): Unknown or Not Reported | Participants | 1 |
| Region of Enrollment: United States | participants | 38 |
| Body Mass Index (BMI) | kg/m^2 | 26.2 (SD 5.6) |
6. Primary Endpoint
The registry lists the primary outcome as The Effects of the Auditory Stimulation on the Cumulative or Average Slow-wave Activity (SWA) Delivered by the PowerSleep & Northwestern Systems in Adults With Mildly Sleep Restricted Schedules as Determined by Device and PSG Scoring Outputs., measured over 5 weeks.
| Element | Registry entry |
|---|---|
| Result type | Mean |
| Measure of spread | Standard Deviation |
| Unit | microvolts^2 |
| Endpoint classification | Continuous Endpoint |
The primary endpoint is cumulative or average slow-wave activity (SWA) across the whole night of nonREM sleep, measured in microvolts^2. The registry describes the outcome as a mean, with standard deviation as the measure of spread, and says the results are determined using device and polysomnography scoring outputs. The timeframe is 5 weeks.
SWA is used in this study as a continuous physiological measure of slow-wave activity during sleep. The study’s stated hypothesis is that active acoustic stimulation would increase SWA relative to sham, and the registry says the relative ability of the two prototypes would also be compared. This makes the endpoint directly aligned with the proposed mechanism being studied: the systems deliver tones during deep sleep, and the measured outcome is the amount of slow-wave activity recorded over nonREM sleep.
A mean describes the average observed value within a condition; the standard deviation describes how dispersed individual observations are around that mean. These summaries give a basic view of the outcome, but they do not by themselves account for the crossover structure or show how the same participants’ measurements relate across conditions. The registry does not report the scoring algorithm, the exact definition of the cumulative versus average measure used for the posted result, or further measurement-quality details. Those elements are important for judging reproducibility and clinical interpretation, but are not reported in the registry record.
7. Results Posted in the Registry
- PowerSleep Stim: mean 2,084 versus 1,831 on control; difference 253.00 (95% CI -674.58 to 1,181), standardized effect d = 0.23; z-test p = 0.593.
- Northwestern Stim: mean 2,317 versus 1,831 on control; difference 486.10 (95% CI -461.28 to 1,433), standardized effect d = 0.43; z-test p = 0.315.
- Northwestern Sham: mean 1,723 versus 1,831 on control; difference -107.80 (95% CI -855.70 to 640.10), standardized effect d = -0.12; z-test p = 0.778.
Mean difference: PowerSleep Stim
95% CI: -674.58 to 1,181
Recalculated from the posted summary statistics
8. Secondary Efficacy Endpoints
The registry also posts results for secondary outcomes. For each outcome below, the endpoint type, the test applied to the posted summary statistics, the result and its interpretation are given in turn. Secondary endpoints are usually not adjusted for multiple testing, so they support rather than confirm the primary finding.
| Secondary outcome | Type | Effect measure | Estimate (95% CI) | Planning size, 80% power |
|---|---|---|---|---|
| The Effects of Active and Sham Conditions of PowerSleep and Northwestern University Systems on Psychomotor Vigilance Task - Brief | Continuous Endpoint | Mean difference | 65.10 (-56.61 to 186.81) | 79 per arm |
| The Effects of Active and Sham Conditions of PowerSleep and Northwestern University Systems on Paired Associated Learning | Continuous Endpoint | Mean difference | -1.50 (-8.29 to 5.29) | 462 per arm |
The Effects of Active and Sham Conditions of PowerSleep and Northwestern University Systems on Psychomotor Vigilance Task - Brief
- Endpoint type: Continuous: a mean value per participant, compared as a difference in means.
- Test: Unadjusted two-sample z-test for the difference in means, using the posted standard deviations.
- Result:
- PowerSleep Stim: mean 310.70 versus 245.60 on control; difference 65.10 (95% CI -56.61 to 186.81), standardized effect d = 0.45; z-test p = 0.294.
- Northwestern Stim: mean 247.40 versus 245.60 on control; difference 1.80 (95% CI -31.99 to 35.59), standardized effect d = 0.04; z-test p = 0.917.
- Interpretation:
- The 95% confidence interval includes no difference, so these data do not show a statistically significant difference between PowerSleep Stim and the control. The interval gives the range of effects that remain compatible with the data.
- The 95% confidence interval includes no difference, so these data do not show a statistically significant difference between Northwestern Stim and the control. The interval gives the range of effects that remain compatible with the data.
The Effects of Active and Sham Conditions of PowerSleep and Northwestern University Systems on Paired Associated Learning
- Endpoint type: Continuous: a mean value per participant, compared as a difference in means.
- Test: Unadjusted two-sample z-test for the difference in means, using the posted standard deviations.
- Result:
- PowerSleep Stim: mean 8.70 versus 10.20 on control; difference -1.50 (95% CI -8.29 to 5.29), standardized effect d = -0.18; z-test p = 0.665.
- Northwestern Stim: mean 12.40 versus 10.20 on control; difference 2.20 (95% CI -6.45 to 10.85), standardized effect d = 0.21; z-test p = 0.618.
- Interpretation:
- The 95% confidence interval includes no difference, so these data do not show a statistically significant difference between PowerSleep Stim and the control. The interval gives the range of effects that remain compatible with the data.
- The 95% confidence interval includes no difference, so these data do not show a statistically significant difference between Northwestern Stim and the control. The interval gives the range of effects that remain compatible with the data.
The secondary outcomes are a brief psychomotor vigilance task measured in milliseconds and paired associated learning measured in words. Across these endpoints, the group summaries do not show a uniform pattern in which active conditions consistently move in the same direction relative to sham. The displayed summary-based contrasts also have broad confidence intervals, and the registry does not report sponsor analyses for either secondary endpoint.
These measures provide additional context about performance, but they are not substitutes for the primary SWA endpoint. The primary question concerns slow-wave activity during sleep; the secondary outcomes concern task performance. Their results therefore address related but distinct questions. The registry does not report a joint model linking changes in SWA to changes in either cognitive outcome, so no relationship between the endpoints can be inferred.
The secondary comparisons are unadjusted. In general, when several outcomes and contrasts are examined, the chance of observing an apparently notable result by chance can increase. Adjustments or a prespecified hierarchy can help control that issue, but the registry record does not report whether either was used for these outcomes. The secondary findings should consequently carry less weight than a prespecified, fully reported primary analysis, particularly in a pilot with limited observations and incomplete analytical detail.
The broad pattern is best understood as exploratory rather than as confirmation of a consistent cognitive effect. The registry record does not report a multiplicity strategy, clinically important thresholds for these tasks, or modeled estimates with uncertainty. The secondary data can motivate further study, but they do not resolve the primary analysis discrepancy or establish that an SWA change corresponds to a performance change.
Secondary outcomes registered for this trial, for reference (only those above were recalculated):
- The Effects of Active and Sham Conditions of PowerSleep and Northwestern University Systems on Psychomotor Vigilance Task - Brief
- The Effects of Active and Sham Conditions of PowerSleep and Northwestern University Systems on Paired Associated Learning
9. Statistical Interpretation
For the posted group summaries, mean SWA was 2084.0 microvolts^2 under PowerSleep stimulation and 1831.0 microvolts^2 under PowerSleep sham. The unadjusted summary-based mean difference for PowerSleep stimulation versus PowerSleep sham is 253.0, with a 95% confidence interval from -674.6 to 1181.0 and an unadjusted p-value of 0.593. The interval includes zero, so these summary-based data are compatible with a range of effects in either direction as well as with little difference.
Mean SWA was 2317.0 microvolts^2 under Northwestern stimulation. Compared with PowerSleep sham, the unadjusted summary-based mean difference is 486.1, with a 95% confidence interval from -461.3 to 1433.0 and an unadjusted p-value of 0.315. This interval also includes zero and is wide. The Northwestern sham mean was 1723.0 microvolts^2; its unadjusted difference versus PowerSleep sham is -107.8, with a 95% confidence interval from -855.7 to 640.1 and an unadjusted p-value of 0.778. That interval likewise includes zero.
These are recalculated, unadjusted comparisons from the posted summary statistics. They are not the sponsor’s reported mixed-model analysis. The registry reports that analysis as a superiority test across the four groups, with p = .019, but gives no model estimate, confidence interval, or further result detail. Because the sponsor’s analysis differs from the summary-based calculations, the sponsor’s own analysis takes precedence as the registry-reported primary analysis. The p-value alone does not identify which conditions differ, the size of a difference, or its precision.
The posted summaries therefore do not support treating the recalculations as a replacement for the crossover model. Nor should the sponsor’s overall p-value be interpreted as evidence that every active condition differed from its sham comparator. The record does not report condition-specific estimates from the mixed model, so the magnitude and direction of the sponsor’s modeled contrasts cannot be described. This distinction is central: an overall test across conditions and individual pairwise contrasts address different questions.
The study is explicitly described as a pilot, with 11 observations per condition in the posted outcome summaries. In a small study, estimates can be imprecise, and a nonsignificant unadjusted p-value does not establish equivalence or rule out potentially meaningful effects. Conversely, a reported p-value of .019 for an overall model does not establish the size, reliability, or clinical importance of any particular contrast. The registry record does not state a threshold for a meaningful SWA change beyond the hypothesis that the increase would be at least 5%; it does not provide a modeled estimate with which to compare that hypothesis.
10. Statistical Methods Explained
The primary outcome is continuous, so its posted summaries are means and standard deviations. A mean gives the central average of the measurements in a condition. A standard deviation describes how much individual values vary around that average. For SWA, the reported means range across the four conditions, and the standard deviations are substantial relative to those means. That spread matters because the uncertainty around a mean difference depends not only on the difference between averages but also on how variable the observations are and how many observations contribute.
What does a mean difference represent?
A mean difference expresses the average outcome in one condition relative to another, in the original outcome units. The summary-based comparisons use PowerSleep sham, with a mean of 1831.0 microvolts^2, as the reference. A positive estimate means the active or other listed condition has a higher posted mean than that reference; a negative estimate means a lower mean. These recalculations are unadjusted comparisons based on the posted group summaries. They do not provide the sponsor’s model-adjusted result.
Because the trial is a crossover, the observations across conditions may come from the same participants. An analysis designed for crossover data can account for within-person relationships, which may differ from treating each condition as an unrelated group. The registry reports a mixed models analysis for the primary outcome, but does not report its model specification, covariance structure, treatment sequence terms, period effects, or condition-specific estimates. The available group summaries do not provide the within-person correlations needed to recreate that analysis. Thus, the simple recalculations are useful descriptive contrasts, but should not be treated as a reconstruction of the sponsor’s mixed model.
How should the confidence intervals be read?
A confidence interval describes the uncertainty around an estimated effect under the method used to construct it. A wide interval indicates limited precision: many effect sizes remain compatible with the data and method. If an interval includes zero for a mean difference, the estimates are compatible with both directions of difference and with no difference. That does not prove that the conditions are equivalent. Equivalence requires a prespecified margin and an analysis designed to test whether the plausible differences fall within it; such a margin and analysis are not reported in the registry record.
The recalculated intervals are unadjusted. They are not the confidence intervals from the sponsor’s mixed model, which are not reported. Their interpretation is therefore limited to the summary-based comparison method. They also do not account for the set of multiple comparisons or outcomes unless a method specifically doing so is reported; the registry record does not report such an adjustment.
Why does over-dispersion matter?
In general, over-dispersion means that observed variability is greater than a statistical model expects. This is more commonly discussed for count outcomes than for a continuous measure such as SWA, but the broader principle applies: if variability is large relative to the estimated effect, uncertainty grows and estimates become less precise. The posted standard deviations show that there is considerable spread within conditions. A model should represent the outcome’s variability appropriately. The registry does not report a specific over-dispersion assessment or a model diagnostic, so none can be inferred.
What does the sponsor’s test establish?
The registry lists a mixed models analysis and a superiority test across PowerSleep stimulation, PowerSleep sham, Northwestern stimulation, and Northwestern sham, with p = .019. In general, a model can use the structure of repeated measurements and account for specified sources of variation. An overall test can assess whether the modeled condition means are consistent with a common value, but it does not, by itself, identify which pairs differ or give the size and direction of each pairwise contrast. The registry does not give the model’s test statistic, degrees of freedom, parameter estimates, confidence intervals, or pairwise results. The reported p-value should therefore be read as a limited summary of the sponsor’s overall test, not as a complete account of the comparative effects.
The discrepancy flag indicates that the recalculated summary-based analysis does not match the sponsor’s reported result. In that circumstance, the sponsor’s own analysis takes precedence for describing the registry-reported analysis. The recalculations remain a separate, transparent reading of posted means and standard deviations, but they cannot resolve why the results differ. Missing details about the mixed model and crossover data structure prevent a full reconciliation.
11. Confidence Intervals and Precision
The recalculated confidence intervals are broad. For PowerSleep stimulation versus PowerSleep sham, the estimated mean difference is 253.0 microvolts^2, with a 95% confidence interval from -674.6 to 1181.0. For Northwestern stimulation versus PowerSleep sham, the estimate is 486.1, with a 95% confidence interval from -461.3 to 1433.0. Each interval includes zero and spans values on both sides of it. The Northwestern sham comparison with PowerSleep sham is also imprecise: its estimate is -107.8, and the 95% confidence interval ranges from -855.7 to 640.1.
In general, a confidence interval gives a range of effect estimates compatible with the observed data under the calculation method. It is not a guarantee that the true effect lies within the interval, and a wide interval is not evidence of no effect. Here, the breadth of the recalculated intervals means the posted summaries do not pin down the size or direction of the contrasts with much precision. The relatively small number of observations per condition and the variability in the outcome are relevant to that uncertainty.
These intervals come from unadjusted recalculations, not from the sponsor’s mixed model. The registry does not report confidence intervals for the sponsor’s primary analysis. It therefore does not permit a comparison of the modeled estimate’s precision with the summary-based estimates. The sponsor’s reported p-value of .019 does not convey the missing effect estimate or its uncertainty. Interpretation should keep the overall model test separate from the imprecise pairwise recalculations.
12. Planning a Similar Trial: Sample Size
This is a planning illustration, not a re-evaluation of the original trial. It asks: if a new study wanted to detect a difference like the one observed, or a smaller one, how large would it need to be? It is deliberately not “observed power” of the finished trial, which is not a meaningful quantity.
| Effect to detect | Participants per arm, 80% power | Participants per arm, 90% power |
|---|---|---|
| 100% of the observed effect | 303 | 405 |
| 75% of the observed effect | 538 | 720 |
| 50% of the observed effect | 1,209 | 1,618 |
The planning figures are participants per arm, not total participants or total events. The figures are illustrations based on the observed effect in each recalculated comparison, not the original trial’s design assumptions. The illustration uses a two-sided significance level of 0.05, equal allocation between two arms, and a normal approximation. It considers detecting the full observed effect, 75% of the observed effect, and 50% of the observed effect. The original trial’s sample-size assumptions and justification are not reported in the registry record.
The 80% power column gives the estimated participants per arm for a design with an 80% chance of detecting the specified effect under those planning assumptions, if that effect were the true effect and the test assumptions held. The 90% power column gives the corresponding per-arm planning figure for a 90% chance. These are design illustrations, not results about the chance that this completed pilot detected an effect.
The figures vary by comparison because the observed estimates and variability differ. For example, planning for 100% of the observed Northwestern stimulation effect versus PowerSleep sham gives 86 participants per arm for 80% power and 115 for 90% power. Planning for 75% of that observed effect gives 152 per arm for 80% power and 204 for 90% power; for 50%, the figures are 342 and 458. Smaller effects require more participants because they are harder to distinguish from random variation at the same significance level and target power.
These are not observed power calculations. They should not be read as evidence that the trial was adequately or inadequately powered for its original question. The assumptions behind the trial’s own planning, including any crossover-specific assumptions, are not reported.
13. How the Sample Size Is Derived
Power is the probability that a study of a given size rejects the null hypothesis when a specific true effect exists. It depends on four things: the effect to detect, the variability of the outcome, the significance level (here a two-sided α of 0.05, so z1−α/2 = 1.960) and the desired power (z1−β = 0.842 for 80% and 1.282 for 90%).
Formula
n is the number of participants in each arm, Δ is the difference in means to detect and σ is the common standard deviation.
Worked calculation for this trial
- Observed difference Δ = 253.000; pooled standard deviation σ = 1109.901.
- n = 2 × (1.96 + 0.842)2 × 1109.9012 / 253.0002 = 302.1, rounded up to 303 participants per arm for 80% power.
The smaller-effect rows in the table above repeat the same calculation with the effect multiplied by 0.75 and 0.5 (for ratios, applied to the logarithm of the ratio).
R code
alpha <- 0.05
z_a <- qnorm(1 - alpha / 2) # two-sided
delta <- 253 # observed difference in means
sigma <- 1109.90142355076 # pooled standard deviation
n_per_arm <- function(power, frac = 1) {
ceiling(round(2 * (z_a + qnorm(power))^2 * sigma^2 / (delta * frac)^2, 6))
}
for (f in c(1, 0.75, 0.5))
cat(sprintf("%3.0f%% of observed effect: n per arm = %s (80%% power), %s (90%% power)\n", 100 * f,
format(n_per_arm(0.80, f), big.mark = ","), format(n_per_arm(0.90, f), big.mark = ",")))
# Check against R's built-in function (t-based, so very slightly larger)
cat("power.t.test:", round(power.t.test(delta = delta, sd = sigma, power = 0.80)$n, 1), "\n")
# Simulation: exact two-sample t-test using simulated means and variances
set.seed(3600194)
n <- n_per_arm(0.80)
reps <- 20000
m_diff <- rnorm(reps, delta, sigma * sqrt(2 / n)) # difference in sample means
s2 <- sigma^2 * rchisq(reps, 2 * (n - 1)) / (2 * (n - 1)) # pooled sample variance
tstat <- m_diff / sqrt(s2 * 2 / n)
cat(sprintf("Simulated power at n = %s per arm: %.3f (%d replicates)\n", format(n, big.mark = ","), mean(abs(tstat) > qt(0.975, 2 * (n - 1))), reps))
Output from running the code
100% of observed effect: n per arm = 303 (80% power), 405 (90% power)
75% of observed effect: n per arm = 538 (80% power), 720 (90% power)
50% of observed effect: n per arm = 1,209 (80% power), 1,618 (90% power)
power.t.test: 303.1
Simulated power at n = 303 per arm: 0.800 (20000 replicates)
The simulation draws many sets of group means and variances at the planned sample size and counts how often the two-sample t-test rejects the null. It should land close to the target power; at very small required sample sizes it can fall below it, because the normal approximation is rough there.
14. Analyses Reported by the Sponsor
| Method | Estimate | 95% CI | P-value |
|---|---|---|---|
| Mixed Models Analysis | not reported | 0.019 |
The registry reports a sponsor analysis using mixed models, described as a superiority analysis across all four condition groups. Its reported p-value is .019. The record does not provide an effect estimate, confidence interval, test statistic, model specification, or pairwise contrasts for that analysis. As a result, the overall test cannot be translated into a condition-specific difference or a measure of its precision from the registry information alone.
The unadjusted recalculations use the posted means and standard deviations to compare each listed condition with PowerSleep sham. Their confidence intervals include zero, and their unadjusted p-values are 0.593 for PowerSleep stimulation, 0.315 for Northwestern stimulation, and 0.778 for Northwestern sham. These calculations are not the sponsor’s mixed model and do not incorporate the reported model result. Since the discrepancy flag is true, the sponsor’s own analysis takes precedence when describing the primary analysis reported by the registry.
The difference between the two analyses cannot be explained from the record. A crossover model may use repeated measurements and within-participant information that are unavailable in the group summaries, but the record does not provide enough model details or participant-level data to establish whether that explains the discrepancy. The sponsor’s p-value should not be treated as a pairwise finding, while the recalculated contrasts should not be treated as a replacement for the sponsor’s primary model.
15. Multiplicity, Missing Data and Analysis Populations
Multiplicity arises when multiple hypotheses are tested. This trial includes four condition groups, more than one possible pairwise comparison, a primary outcome, and two secondary outcomes. In general, testing many contrasts without an adjustment can increase the chance of at least one false-positive finding. A prespecified testing hierarchy, family-wise error adjustment, or another multiplicity strategy can clarify how to interpret multiple results. The registry record does not report whether multiplicity was considered, how comparisons were prioritized, or whether the reported p-value was adjusted.
The sponsor’s reported p = .019 is attached to a mixed-model superiority analysis across the four groups, but the record does not state whether it is an omnibus test or how its result relates to planned pairwise contrasts beyond identifying it as a superiority analysis. The record does not provide the exact contrast structure or an adjusted analysis plan. The summary-based recalculations are explicitly unadjusted and should not be taken as a multiplicity-controlled set of conclusions.
Missing-data handling is also not reported in the registry record. It does not state whether all randomized participants contributed to each endpoint, how incomplete sleep recordings or task assessments were handled, or whether any imputation was used. The posted group summaries list 11 observations for each condition, but they do not explain the analysis population or whether any observations were excluded. In a crossover study, missing data in one condition can affect whether a participant contributes to within-person comparisons.
The registry record does not report the analysis population definition, such as an intention-to-treat or per-protocol population, nor does it describe deviations from assigned conditions. Details of the mixed-model specification are also absent, including how repeated measurements and any period or sequence effects were represented. These omissions prevent readers from assessing the robustness of the primary result to alternative analysis choices.
These are reporting limitations, not evidence that particular methods were or were not used. The registry does not report an interim-analysis plan, stopping rules, or a formal missing-data sensitivity analysis. Without those details, the inferential results remain difficult to audit and reproduce.
16. Safety Results
The registry reports serious adverse events as affected participants divided by participants at risk.
| Arm | Participants with serious adverse events | Participants at risk | Percentage |
|---|---|---|---|
| Screening Population | 0 | 38 | 0.0% |
| PowerSleep Stim | 0 | 11 | 0.0% |
| PowerSleep Sham | 0 | 11 | 0.0% |
| Northwestern Stim | 0 | 11 | 0.0% |
| Northwestern Sham | 0 | 11 | 0.0% |
Deaths
| Arm | Deaths (participants affected of at risk) |
|---|---|
| Screening Population | 0 of 38 |
| PowerSleep Stim | 0 of 11 |
| PowerSleep Sham | 0 of 11 |
| Northwestern Stim | 0 of 11 |
| Northwestern Sham | 0 of 11 |
Other (non-serious) adverse events
Participants with at least one non-serious event at or above the registry's 0% reporting threshold.
| Arm | Participants affected of at risk |
|---|---|
| Screening Population | 1 of 38 |
| PowerSleep Stim | 0 of 11 |
| PowerSleep Sham | 1 of 11 |
| Northwestern Stim | 0 of 11 |
| Northwestern Sham | 1 of 11 |
Most frequent non-serious adverse event terms
| Event term | Organ system | Screening Population | PowerSleep Stim | PowerSleep Sham | Northwestern Stim | Northwestern Sham |
|---|---|---|---|---|---|---|
| Influenza | Immune system disorders | 1 of 38 | 0 of 11 | 0 of 11 | 0 of 11 | 0 of 11 |
| Skin Irritation | Skin and subcutaneous tissue disorders | 0 of 38 | 0 of 11 | 0 of 11 | 0 of 11 | 1 of 11 |
| Allergies | Immune system disorders | 0 of 38 | 0 of 11 | 1 of 11 | 0 of 11 | 0 of 11 |
The adverse-event record lists no serious adverse events in the screening population or in any of the four condition groups. It also lists zero deaths in each group. For other adverse events, the screening population has one event among 38 at risk; PowerSleep sham has one among 11 at risk; Northwestern sham has one among 11 at risk. The PowerSleep stimulation and Northwestern stimulation groups each list zero other adverse events among 11 at risk.
These counts should be read as recorded events, not as a complete assessment of comparative safety. The registry does not list the terms for serious events, and no serious events are shown. It does not provide details about the single other-event terms, severity, timing, relationship to a device, or how events were collected. The screening population count and condition-group counts should not be added together; the crossover design means the same people may contribute under more than one condition, and the record does not explain the relationship between those populations.
With a small pilot and limited event detail, the counts describe what was recorded in the registry but do not establish that events could not occur. No medical advice or broader safety conclusion follows from these figures alone.
17. What the Result Does, and Does Not, Mean
The primary result has two distinct components in the registry: an overall mixed-model superiority test reported with p = .019, and unadjusted recalculations from the posted group summaries. The sponsor’s model is the registry-reported primary analysis and takes precedence over the recalculations because the discrepancy flag is true. However, the registry gives no modeled estimate or confidence interval, so the size, direction, and precision of the sponsor’s result cannot be described. Its p-value alone does not identify which condition contrasts account for the overall result.
The recalculated comparisons offer a separate view of the posted means. Their estimated differences are uncertain, with confidence intervals spanning zero. That does not establish that active stimulation and sham are equivalent, nor does it prove that there is no effect. The estimates are based on small condition groups and do not reproduce the crossover mixed model. The registry’s hypothesis of an increase of at least 5% is stated, but the sponsor’s model results do not provide the condition-specific estimates needed to assess that hypothesis directly.
The trial is a pilot study, and its purpose includes assessing feasibility as well as efficacy. The registry record does not report a detailed feasibility result. It also does not provide enough information about the crossover sequence, missing data, model structure, or participant-level outcomes to establish how robust the analyses are. The posted cognitive outcomes add exploratory information but do not demonstrate a consistent relationship between sleep activity and performance.
Accordingly, the registry supports a cautious description of the statistical record, not a clinical conclusion. The overall test is reported as statistically notable, while the summary-based pairwise estimates are imprecise and do not match that analysis. The available record cannot resolve the discrepancy or establish the magnitude or clinical importance of a particular device-condition effect.
18. Why This Trial Matters Statistically
This study illustrates why analysis design matters as much as the arithmetic of comparing averages. In a crossover trial, the same participants can experience multiple conditions, so within-person information may be important. A mixed model may account for features of repeated measurements, whereas simple comparisons of condition means do not necessarily use that information. When the registry reports only group summaries alongside an overall model p-value, the two approaches can yield different inferential messages that cannot be reconciled without the model details.
It also shows the importance of reporting estimates and uncertainty alongside p-values. The sponsor’s analysis gives p = .019 but no effect estimate or confidence interval. The unadjusted recalculations provide estimates and intervals, but they do not reproduce the sponsor’s analysis. A p-value without an effect size cannot describe the magnitude of a difference, and an estimate without a suitable model cannot fully answer a crossover question.
Finally, the planning illustrations demonstrate how sample-size requirements depend on the effect size one aims to detect and the target power. They are not a substitute for the original design justification, which is not reported. In a pilot, estimates can be unstable, and an observed effect is not necessarily a reliable planning target. The distinctions between pilot evidence, confirmatory analysis, and planning calculations are essential when interpreting small clinical studies.
19. Limitations
- The posted summary statistics are the only data these results rest on; patient-level data, full protocols and statistical analysis plans are not available.
- Recalculated results are simple, unadjusted calculations from posted summaries. They do not replace the sponsor's prespecified analysis.
Several limitations are specific to this registry record. First, the study is identified as a crossover trial, but the record does not report condition sequence, period effects, washout arrangements, or other details needed to understand potential carryover. It also does not provide the mixed-model specification. The sponsor’s reported p-value therefore cannot be tied to a clearly described set of modeled contrasts.
Second, the sponsor’s overall result and the unadjusted summary-based recalculations differ. The sponsor analysis takes precedence, but the record lacks estimates and intervals from that analysis, so the discrepancy cannot be investigated. The simple recalculations use condition-level means and standard deviations and do not capture within-person relationships that may be relevant in a crossover design.
Third, the registry’s participant accounting is not fully clear: enrollment is listed as 38, the summary describes consenting up to 60 to complete 10 participants, and the posted outcome groups each list 11 observations. The record does not explain these figures or identify the analysis population. Missing-data procedures and exclusions are not reported.
Fourth, the primary description refers to one night under active use versus one night under sham, while the brief summary describes two nights of in-lab use with active versus sham conditions. The record does not clarify the relationship between those descriptions. Additional limitations include absent primary-model estimates, no reported multiplicity strategy, limited adverse-event detail, and no sponsor-reported analyses for the secondary outcomes. These gaps constrain interpretation and reproducibility; they do not establish that unreported procedures were not conducted.
20. Related Tutorials
21. Related Statistical Calculators
22. Sources
- ClinicalTrials.gov: Pilot Study Comparing the Effectiveness of Two Different Methods of Acoustic Stimulation — NCT03600194.
- Registry record last updated 2021-06-14; status last verified 2021-05.
- Results first posted 2021-06-14.
- Data retrieved from ClinicalTrials.gov on 2026-11-09.
- Group sizes, means, event counts and sponsor-reported p-values are reproduced as posted. Effect estimates, confidence intervals and p-values not attributed to the sponsor are independently calculated from the posted summary statistics.
Continue through the Clinical Biostats statistical pathway
Use this trial's endpoint as a starting point for deeper study of effect measures, confidence intervals, and sample-size planning.
23. Record Summary
NCT03600194 is a completed, randomized, single-blind crossover pilot sponsored by Philips Respironics. It compared active and sham acoustic stimulation using PowerSleep and Northwestern systems in adults described as having mildly sleep-restricted schedules. The registry lists 38 participants enrolled and a primary completion date of 2019-07-25.
The primary outcome was mean cumulative or average SWA during nonREM sleep, measured in microvolts^2. The registry reports a mixed-model superiority analysis with p = .019, but does not report its effect estimate or confidence interval. Unadjusted recalculations from posted summaries have broad confidence intervals; because the reported analyses differ, the sponsor’s analysis takes precedence. Two secondary outcomes and adverse-event counts are also posted, but analytical detail is limited.
Important details about the crossover analysis, multiplicity, missing data, and analysis populations are not reported in the registry record. The findings should be read with those limitations in mind.