Virtual Hope Box - Effectiveness of a Smartphone App for Coping With Suicidal Ideation: Complete Statistical Analysis

1. Trial at a Glance

This page covers the registry results for Virtual Hope Box - Effectiveness of a Smartphone App for Coping With Suicidal Ideation (NCT01982773), a randomized na study. The primary outcome is Beck Scale for Suicidal Ideation.

2. Clinical Question

3. Trial Design

This was a randomized, parallel-group clinical trial comparing the Virtual Hope Box smartphone app with Enhanced Treatment as Usual. The registry lists the study as completed, with an overall enrollment of 118. The study began in 2014-03, and primary completion was recorded as 2016-04. The sponsor was the National Center for Telehealth and Technology.

The intervention group used the Virtual Hope Box app on personal smartphones. The comparator group received printed materials intended to guide coping with suicidal thoughts, including coping strategies and emergency contact information. Both interventions are classified as behavioral in the registry. The trial was open-label, so masking was listed as none. The registry does not report further information about the randomization procedure, concealment of allocation, or the practical steps used to deliver either intervention.

The primary outcome listed is the Beck Scale for Suicidal Ideation, with the timeframe recorded as baseline. The arm-level outcome summaries report 58 participants for the app group and 59 for Enhanced Treatment as Usual. Those summaries describe baseline scores rather than follow-up scores. Although the registry summary describes hypotheses about pre-post change and differences between conditions, the posted primary outcome information does not include a follow-up assessment or a change-from-baseline comparison.

The trial is listed as phase not applicable. The registry record does not report the trial’s original sample-size justification, planned analysis model, missing-data approach, or interim-analysis plan. These omissions limit what can be said about how the reported summary statistics relate to the trial’s original analysis plan.

4. Participant Flow and Accounting

The registry reports how participants moved through each stage of the study. All counts below are exactly as posted; missing counts are not inferred from other stages.

Overall Study

Reasons for not completing this period:

5. Baseline Characteristics

The table reproduces the baseline characteristics posted in the registry, which describe the participants analysed at baseline and help show whether the randomized groups were comparable.

6. Primary Endpoint

The registry lists the primary outcome as Beck Scale for Suicidal Ideation, measured over Baseline.

The primary measure is the Beck Scale for Suicidal Ideation, a self-report measure of suicidal ideation during the past week. The registry describes 21 items covering desire for death, preparation for suicide, and actual suicide desire. Its score range is 0 to 48, with higher scores indicating greater risk for suicidal ideation.

This measure is relevant to the trial’s stated topic because it quantifies reported suicidal thoughts, attitudes, and intentions rather than relying only on a general description of distress or coping. A numerical scale can support a comparison of group summaries, but interpretation depends on when it was measured and how change was analyzed. In this record, the primary outcome timeframe is baseline. The displayed arm summaries are therefore starting-point descriptions, not evidence of improvement or worsening over time.

The registry lists the mean and standard deviation for each group. A mean describes the group’s average score, while the standard deviation describes the spread of individual scores around that average. These summaries do not show individual responses, the distribution’s shape, or how participants’ scores changed after baseline. The registry record does not report a threshold for a clinically important difference on this measure, so the size of a group difference cannot be classified using such a threshold from the available information.

7. Results Posted in the Registry

• Virtual Hope Box Smartphone App: mean 3.31 versus 3.58 on control; difference -0.27 (95% CI -1.25 to 0.71), standardized effect d = -0.10; z-test p = 0.591.

8. Statistical Interpretation

9. Statistical Methods Explained

What does a mean difference represent?

A mean difference expresses the gap between two group averages in the units of the measured outcome. Here the outcome is measured in Beck Scale for Suicidal Ideation units. The calculation is oriented as the Virtual Hope Box mean minus the Enhanced Treatment as Usual mean. A negative estimate therefore indicates a lower average baseline score in the app group, while a positive estimate would indicate a higher average. The reported estimate is -0.27 units.

A difference between means is a summary, not a description of every participant. People within each group may have scores above and below their group mean. The standard deviations, 2.6 in the app group and 2.83 in the comparator group, describe that within-group variability. They are not standard errors and do not express uncertainty in the same way as a confidence interval. A standard deviation characterizes dispersion among observed scores; a confidence interval characterizes uncertainty around an estimated quantity.

How should the p-value be understood?

The unadjusted p-value is 0.591. In general, a p-value describes how compatible data are with a specified null hypothesis under the test’s assumptions. It is not the probability that the null hypothesis is true, and it does not measure the probability that a treatment works. A relatively large p-value does not prove equality or rule out meaningful differences; it indicates that the observed comparison is not unusual enough under the null hypothesis to provide strong evidence against it by that measure.

This p-value is an unadjusted result from the simple comparison of the reported group summaries. The registry record does not list an analysis plan or a sponsor-reported analysis for this outcome. The p-value should therefore not be mistaken for an adjusted estimate or a result from an analysis model described in the record. The registry does not report whether covariates, clustering, repeated measurements, or other design features were included in an original analysis.

What information does the confidence interval add?

A confidence interval presents a range of estimates compatible with the observed data under the method used. For this mean difference, the interval extends from -1.254 to a positive value and includes zero. In general, including zero means the interval does not distinguish the estimate from no group difference at the confidence level used to construct it. Because the upper bound is recorded with more than three decimal places, it is not reproduced; the important interpretation is that the interval crosses zero.

Intervals also communicate precision. A narrow interval would constrain plausible values more tightly; a wide interval would leave more uncertainty about the size and direction of the difference. The interval allows values on both sides of zero, so it does not rule out either a lower or a higher app-group mean. It should not be interpreted as a range containing a fixed proportion of individual participants’ scores.

Why distinguish a baseline comparison from a treatment effect?

A randomized trial can compare groups at follow-up, compare changes from baseline, or use a model that accounts for baseline values. Those approaches answer questions about outcomes after assignment. A single baseline comparison is different: it describes scores before the intervention could have produced a change. In this record, the primary outcome timeframe is baseline, and no follow-up score or change estimate is posted. Thus the reported mean difference cannot be read as the effect of the app.

Random allocation is valuable because it is intended to make assigned groups comparable on average, including for characteristics not measured. However, randomization does not guarantee matching baseline means in a particular trial. Baseline summaries can help describe the groups, but a baseline significance test is not a substitute for a post-assignment outcome analysis. The registry record does not supply enough detail to assess balance across other characteristics or to determine whether baseline adjustment was planned.

What are the limits of these summary statistics?

Means and standard deviations compress the observations into a few quantities. They do not reveal skewness, unusual values, clustering of scores, or missingness patterns. The registry record does not report individual-level observations, distributions, analysis populations, or a missing-data method. The observed score range is described as 0 to 48, but the arm summaries alone cannot show how scores were distributed across that range.

For a continuous outcome, an unadjusted mean comparison is often interpretable when the mean is a suitable summary and the analysis assumptions are reasonable. General statistical considerations include independence of observations and an appropriate estimate of uncertainty. The registry does not report diagnostic checks or details sufficient to evaluate those assumptions. No claim should be made that a particular model, distributional check, or correction was used.

The trial summary also mentions hoped-for changes in coping and perceived reasons for living, as well as patient and clinician views of the app. The posted primary outcome results do not answer those questions. The registry lists four outcome measures overall, but no secondary outcomes are posted and no analyses are listed. That means the baseline mean comparison should be treated as a narrow statistical description, rather than a complete account of the trial’s stated aims.

10. Confidence Intervals and Precision

The unadjusted mean difference is -0.27 Beck Scale for Suicidal Ideation units, comparing the app group with Enhanced Treatment as Usual. The reported confidence interval runs from -1.254 to a positive value and crosses zero. It therefore includes no difference as well as possible differences in either direction. The unadjusted p-value is 0.591.

A confidence interval is useful because it shows uncertainty around the point estimate, rather than presenting one number as if it were exact. In general, the interval’s endpoints represent estimates that remain compatible with the data under the method used to calculate the interval. The interval is not a range for individual scores, and it does not mean that there is a particular probability that the true value lies inside the realized interval.

The interval’s span across zero means the result does not precisely establish the direction of a baseline group difference. It does not prove that the means are equal. The estimate is also not a treatment-effect interval: the recorded timeframe is baseline, and the registry does not report a follow-up contrast or change-from-baseline estimate for this outcome.

Precision should be considered alongside the scale and the observed variability. The group standard deviations are 2.6 and 2.83, compared with a mean difference of -0.27. The record does not report a threshold for clinical importance, so the statistical estimate cannot be translated into a classification of clinical importance from the registry information alone. Nor can the interval be used to infer the range of individual responses.

The upper confidence limit is recorded as 0.715 in the registry-derived comparison, a value with more than three decimal places. It is not reproduced numerically; its positive sign and the fact that the interval crosses zero are sufficient for the interpretation. More precise claims about the magnitude of the upper limit are not necessary to understand the uncertainty conveyed by this result.

11. Planning a Similar Trial: Sample Size

This is a planning illustration, not a re-evaluation of the original trial. It asks: if a new study wanted to detect a difference like the one observed, or a smaller one, how large would it need to be? It is deliberately not “observed power” of the finished trial, which is not a meaningful quantity.

The planning figures are participants per arm, not total event counts. They are illustrations based on the observed mean difference, rather than a report of the trial’s original design calculation. The assumptions listed for the illustration are a two-sided significance level of 0.05, equal allocation between the two arms, and a normal approximation. The effect to detect is set in turn to 100%, 75%, or 50% of the observed effect.

For planning around 100% of the observed effect, the figures are 1,592 participants per arm for 80% power and 2,131 participants per arm for 90% power. For 75% of the observed effect, they are 2,829 participants per arm for 80% power and 3,788 participants per arm for 90% power. For 50% of the observed effect, they are 6,366 participants per arm for 80% power and 8,522 participants per arm for 90% power.

In plain language, the 80% power column gives the number of participants per group that the stated planning calculation estimates would be needed to have an 80% chance of detecting the chosen effect, if its assumptions hold. The 90% column gives the corresponding planning target for a 90% chance. These are design targets under specified assumptions, not descriptions of what happened in the trial.

The pattern illustrates a general statistical principle: smaller effects are harder to distinguish from random variation, so planning to detect a smaller effect requires more participants. The displayed values are not observed power. The registry record does not report the original sample-size assumptions or justification, so these illustrations should not be treated as the trial’s own planned sample size or as evidence about the trial’s ability to detect its intended effect.

12. How the Sample Size Is Derived

Power is the probability that a study of a given size rejects the null hypothesis when a specific true effect exists. It depends on four things: the effect to detect, the variability of the outcome, the significance level (here a two-sided α of 0.05, so z_1−α/2 = 1.960) and the desired power (z_1−β = 0.842 for 80% and 1.282 for 90%).

Formula

n is the number of participants in each arm, Δ is the difference in means to detect and σ is the common standard deviation.

Worked calculation for this trial

1. Observed difference Δ = -0.270; pooled standard deviation σ = 2.718.
2. n = 2 × (1.96 + 0.842)² × 2.718² / -0.270² = 1,591.3, rounded up to 1,592 participants per arm for 80% power.

The smaller-effect rows in the table above repeat the same calculation with the effect multiplied by 0.75 and 0.5 (for ratios, applied to the logarithm of the ratio).

R code

alpha <- 0.05
z_a   <- qnorm(1 - alpha / 2)          # two-sided
delta <- -0.27                      # observed difference in means
sigma <- 2.71843337236725                     # pooled standard deviation
n_per_arm <- function(power, frac = 1) {
  ceiling(round(2 * (z_a + qnorm(power))^2 * sigma^2 / (delta * frac)^2, 6))
}
for (f in c(1, 0.75, 0.5))
  cat(sprintf("%3.0f%% of observed effect: n per arm = %s (80%% power), %s (90%% power)\n", 100 * f,
              format(n_per_arm(0.80, f), big.mark = ","), format(n_per_arm(0.90, f), big.mark = ",")))

# Check against R's built-in function (t-based, so very slightly larger)
cat("power.t.test:", round(power.t.test(delta = delta, sd = sigma, power = 0.80)$n, 1), "\n")

# Simulation: exact two-sample t-test using simulated means and variances
set.seed(1982773)
n    <- n_per_arm(0.80)
reps <- 20000
m_diff <- rnorm(reps, delta, sigma * sqrt(2 / n))             # difference in sample means
s2     <- sigma^2 * rchisq(reps, 2 * (n - 1)) / (2 * (n - 1))   # pooled sample variance
tstat  <- m_diff / sqrt(s2 * 2 / n)
cat(sprintf("Simulated power at n = %s per arm: %.3f (%d replicates)\n", format(n, big.mark = ","), mean(abs(tstat) > qt(0.975, 2 * (n - 1))), reps))

Output from running the code

100% of observed effect: n per arm = 1,592 (80% power), 2,131 (90% power)
 75% of observed effect: n per arm = 2,829 (80% power), 3,788 (90% power)
 50% of observed effect: n per arm = 6,366 (80% power), 8,522 (90% power)
power.t.test: 1592.2 
Simulated power at n = 1,592 per arm: 0.801 (20000 replicates)

The simulation draws many sets of group means and variances at the planned sample size and counts how often the two-sample t-test rejects the null. It should land close to the target power; at very small required sample sizes it can fall below it, because the normal approximation is rough there.

13. Multiplicity, Missing Data and Analysis Populations

The registry lists four outcome measures overall, but no secondary outcomes are posted and no analyses are listed. The primary outcome is identified as the Beck Scale for Suicidal Ideation at baseline. The trial summary also describes hypotheses involving coping and perceived reasons for living, and asks whether patients and clinicians would find the app beneficial and readily integrated into practice. The registry record does not provide posted results for those questions.

When a study assesses multiple outcomes, timepoints, or comparisons, the chance of finding an apparently noteworthy result by chance can increase. Statistical methods for handling multiple testing may include adjustments or a prespecified hierarchy. Whether any such method was planned or used is not reported in the registry record. The unadjusted p-value should therefore be understood as unadjusted, and not as evidence that multiplicity was accounted for.

The record does not report how missing outcome data were handled, how many participants had missing baseline data, whether an intention-to-treat population was used, or how analysis populations were defined. It also does not report imputation, sensitivity analyses, or reasons for missingness. The differences between outcome and safety denominators should not be used to infer who was missing from a particular analysis; the record does not explain those denominators.

These omissions matter because analysis choices can affect the interpretation of a trial. Without the missing-data plan, analysis-population definitions, and a complete set of outcome analyses, the baseline comparison cannot be placed within a full confirmatory analysis framework. The registry record also does not report the original statistical analysis plan, interim analyses, or a sample-size justification.

14. Safety Results

The registry reports serious adverse events as affected participants divided by participants at risk.

Other (non-serious) adverse events

Participants with at least one non-serious event at or above the registry's 0% reporting threshold.

Most frequent non-serious adverse event terms

The registry lists serious adverse events for 58 participants at risk in the Virtual Hope Box group and 60 at risk in Enhanced Treatment as Usual. The number of participants affected by serious adverse events is recorded as 0 in each group. For other adverse events, the record lists 58 and 60 participants at risk, with 1 participant affected in each group.

These figures describe the registry’s recorded affected-participant counts and at-risk denominators. They do not provide event types, severity details for other adverse events, timing, or a fuller account of how events were collected. The record lists no serious-event terms. Deaths are not reported in the registry record.

A count of 0 affected participants in the serious-event field should not be interpreted as proof that no safety concerns could occur, nor does it describe risk outside the reported study groups and recording period. The registry record does not give enough detail to assess completeness of safety ascertainment or compare event rates with other populations. The adverse-event figures are descriptive registry information, not a standalone conclusion about safety.

15. What the Result Does, and Does Not, Mean

The posted primary outcome information describes baseline Beck Scale for Suicidal Ideation scores in the randomized groups. The app group mean was 3.31, and the Enhanced Treatment as Usual group mean was 3.58. The unadjusted mean difference was -0.27, with a confidence interval that crosses zero and an unadjusted p-value of 0.591. These data do not establish a clear baseline difference between the group means.

That is a limited conclusion. Baseline scores precede the intervention’s potential effects, so the comparison does not show whether the app changed suicidal ideation scores. It also does not show whether participants’ coping or perceived reasons for living changed, whether differences emerged later, or whether patients and clinicians found the app useful. The registry summary states those as study hypotheses or aims, but the relevant outcome analyses are not posted in the record.

The result does not show that the app works, and it does not show that the app does not work. It is not a treatment-effect estimate. The confidence interval includes zero and values in both directions, while the p-value is unadjusted. Neither a non-small p-value nor a baseline comparison establishes equivalence, absence of benefit, or harm.

The record also does not report a clinical-importance threshold for the scale, so the mean difference cannot be translated into a conclusion about clinical importance. The standard deviations describe variation in observed scores, but the registry record does not give individual-level results or distributions that would clarify how scores varied among participants.

A fuller assessment of the trial’s stated questions would require reported follow-up outcomes, analysis methods, missing-data information, and relevant results for the other aims. Those details are not reported in the registry record. The appropriate reading is therefore specific: the available statistics describe baseline scores and their uncertainty, not intervention efficacy or the overall clinical value of the app.

16. Why This Trial Matters Statistically

This trial is statistically interesting because its stated hypotheses concern change over time and differences between randomized groups, while the posted primary outcome timeframe is baseline. That distinction is central to clinical-trial interpretation. A baseline comparison can describe the groups before intervention effects are assessed, but it cannot answer whether outcomes improved after assignment or whether improvement differed between groups.

The randomized, parallel design identifies a comparative framework: one group received a smartphone app and the other received printed coping materials. Yet the value of that framework for estimating an intervention effect depends on outcome data and analyses after assignment. The registry record does not post such an analysis for the primary measure. Recognizing that boundary prevents a baseline statistic from being mistaken for a treatment result.

The comparison also illustrates the separate roles of a point estimate, confidence interval, and p-value. The estimate gives a direction and scale for the observed mean difference; the interval communicates uncertainty; and the p-value summarizes compatibility with a null hypothesis under the test’s assumptions. None alone establishes clinical importance, and the interval crossing zero does not prove equality.

Finally, the planning illustrations show how assumed effect size connects to required sample size under stated assumptions. Smaller target effects require more participants in those calculations. But these illustrations are distinct from the trial’s own design rationale, which is not reported, and they are not observed power. Keeping those distinctions clear is useful when reading registry results that include descriptive outcome statistics but do not provide a complete analysis record.

17. Limitations

• The posted summary statistics are the only data these results rest on; patient-level data, full protocols and statistical analysis plans are not available.
• Recalculated results are simple, unadjusted calculations from posted summaries. They do not replace the sponsor's prespecified analysis.

The principal limitation for interpreting the primary outcome is that the registry records its timeframe as baseline. The available arm summaries are not follow-up results, and no change-from-baseline comparison is reported. This prevents an assessment of whether the outcome changed during the trial or whether changes differed between assigned groups.

The registry record does not report the statistical analysis plan, the original sample-size justification, or the handling of missing data. It does not define analysis populations, report imputation or sensitivity analyses, or explain whether adjustment for baseline scores or other characteristics was planned. No analyses are listed, and no secondary outcomes are posted. The described hypotheses about coping, reasons for living, and integration into practice therefore cannot be evaluated from the posted primary outcome statistics.

The study was open-label, with masking listed as none. The registry does not explain how this feature was addressed in outcome collection or analysis. The primary scale is self-reported, and the record does not provide information about completion procedures, adherence to the app, or the amount of app use. No such details should be inferred from the intervention description alone.

The mean and standard deviation do not reveal individual score distributions or missingness patterns. The record also does not report a clinical-importance threshold for the scale. Safety information is limited to affected-participant counts and at-risk denominators; event details and deaths are not reported in the registry record. Together, these limitations mean the posted information supports a narrow description of baseline scores, rather than a comprehensive appraisal of outcomes, intervention effects, or safety.

18. Related Tutorials

19. Related Statistical Calculators

20. Sources

• ClinicalTrials.gov: Virtual Hope Box - Effectiveness of a Smartphone App for Coping With Suicidal Ideation — NCT01982773.
• Registry record last updated 2018-01-30; status last verified 2017-07.
• Results first posted 2018-01-30.
• Data retrieved from ClinicalTrials.gov on 2026-11-09.
• Group sizes, means, event counts and sponsor-reported p-values are reproduced as posted. Effect estimates, confidence intervals and p-values not attributed to the sponsor are independently calculated from the posted summary statistics.

21. Record Summary

NCT01982773 is a completed, randomized, parallel-group, open-label trial of the Virtual Hope Box smartphone app compared with Enhanced Treatment as Usual. The registry lists an enrollment of 118, a start date of 2014-03, primary completion in 2016-04, and the National Center for Telehealth and Technology as sponsor.

The listed primary outcome is the Beck Scale for Suicidal Ideation at baseline. The app group mean was 3.31 with a standard deviation of 2.6 among 58 participants; the comparator mean was 3.58 with a standard deviation of 2.83 among 59 participants. The simple unadjusted mean difference was -0.27, with an unadjusted p-value of 0.591 and a confidence interval crossing zero.

The registry record does not post follow-up or change results for this primary outcome. It lists no secondary outcomes posted and no analyses listed. Thus the available primary outcome statistics describe baseline scores, not an intervention effect.