Feasibility of Depression Care Management by E-mail: Complete Statistical Analysis

1. Trial at a Glance

This page covers the registry results for Feasibility of Depression Care Management by E-mail (NCT00755235), a randomized na study. The primary outcome is 20-Item Symptom Checklist Depression Scale.

2. Clinical Question

3. Trial Design

This was a randomized, double-masked, parallel-group trial marked as completed in the registry record. The listed phase is NA. Kaiser Permanente is named as the sponsor. The study title identifies the work as a feasibility trial, and the brief summary frames its aims as assessing whether electronic depression care management could be feasible, acceptable, and potentially effective. The registry record does not report a protocol or a detailed account of how feasibility and acceptability were assessed.

The record lists 208 participants enrolled, with a start date of 2009-04 and a primary completion date of 2010-04. The intervention was depression care management by secure messaging. The comparator was usual care without additional education or care-management services. The record does not describe the randomization procedure, allocation concealment, or the exact masking arrangements. The label double-masked is recorded, but which groups of people were masked and how masking was maintained are not reported in the registry record.

The primary outcome was the 20-Item Symptom Checklist Depression Scale, measured at baseline and after 6 months of treatment. The posted outcome summary includes 104 participants in the intervention arm and 93 in the usual-care arm. The primary-outcome description says the analysis included participants participating in outcome assessment. How participants were selected for that assessment, and why the posted arm counts do not account for the full enrollment, are not reported in the registry record.

Treatment satisfaction is listed as a secondary outcome, and the record lists two outcome measures overall. The numerical secondary-outcome findings are not reported in the registry record. The record lists zero analyses. The study’s original sample-size justification, planned analysis methods, and any interim-analysis procedures are not reported in the registry record.

4. Participant Flow and Accounting

The registry reports how participants moved through each stage of the study. All counts below are exactly as posted; missing counts are not inferred from other stages.

Overall Study

Analysis population for the primary outcome, as defined in the registry: Analysis included all participants participating in outcome assessment

5. Baseline Characteristics

The table reproduces the baseline characteristics posted in the registry, which describe the participants analysed at baseline and help show whether the randomized groups were comparable.

6. Primary Endpoint

The registry lists the primary outcome as 20-Item Symptom Checklist Depression Scale, measured over Measured at baseline and after 6 months of treatment.

The primary endpoint is the 20-Item Symptom Checklist Depression Scale, a measure of depression severity adapted from the longer SCL-90. It is a continuous measure, reported as a mean with standard deviation. The registry record describes its scale as ranging from 0 to 4 and says scores exceeding 1.5 indicate moderate depression. The outcome was measured at baseline and after 6 months of treatment.

A depression-severity scale offers a standardized way to summarize symptom scores across participants. Reporting the mean describes the group’s average score, while the standard deviation describes the spread of individual scores around that average. Those summaries help readers understand both the typical score and how much participants’ scores vary within each arm.

The endpoint is relevant to the trial’s stated focus on depression care management. However, the registry record does not explain why this particular scale was selected as the primary outcome, report a threshold for a clinically important difference, or describe whether the analysis compared follow-up scores, changes from baseline, or an adjusted measure. The posted arm means and comparison should therefore be interpreted as reported, without assuming an unreported analysis of change from baseline.

7. Results Posted in the Registry

• Secure Messaging Care Management: mean 0.95 versus 1.17 on control; difference -0.22 (95% CI -0.43 to -0.01), standardized effect d = -0.29; z-test p = 0.044.

8. Statistical Interpretation

9. Statistical Methods Explained

The primary outcome is continuous, so its arm summaries are means and standard deviations. A mean is the arithmetic average of the measured scores in a group. A standard deviation describes how dispersed those scores are around the group mean. A smaller standard deviation indicates scores are more concentrated near the mean; a larger one indicates more spread. The posted summaries are means of 0.95 and 1.17, with standard deviations of 0.71 and 0.81, for secure messaging care management and usual care, respectively.

What does a mean difference describe?

A mean difference compares the average outcome in one arm with the average in the other. The registry comparison expresses the difference as the intervention-arm mean minus the usual-care mean. Its estimate is -0.22. A negative value means that the intervention-arm average is lower on the scale in this comparison. Because the scale is described as a depression-severity measure, the sign identifies the direction of the score contrast; it does not by itself establish the importance of the difference or resolve other questions about trial conduct and analysis.

The posted comparison gives a 95% confidence interval from -0.434 to -0.006. In general, a confidence interval is a range generated by a statistical method to reflect uncertainty in an estimate. Its interpretation depends on the assumptions and analysis used. It is not a range containing a fixed proportion of individual participants’ scores, and it does not measure variation in the outcome itself. The standard deviations describe score variation; the confidence interval describes uncertainty in the estimated comparison.

How should the p-value be read?

The unadjusted p-value is 0.044. In general, a p-value indicates how unusual the observed data, or data more extreme, would be under a specified null model and the assumptions of the test. It does not give the probability that the treatment effect is zero, the probability that the finding will recur, or a measure of clinical importance. The registry record does not report the full testing procedure or whether a particular significance threshold was prespecified. The stated value is unadjusted, so it should not be read as accounting for a multiplicity procedure.

The registry record does not report the statistical test used for the sponsor’s primary analysis, any covariate adjustment, or how baseline measurements entered the analysis. The presence of measurements at baseline and after 6 months does not establish that the posted comparison is a difference in change. The description of participants included in outcome assessment also does not establish whether all randomized participants were analyzed according to their original assignment. Those details are not reported in the registry record.

Why do assumptions matter?

In general, methods for comparing means rely on assumptions about the observations and the way the data were collected. For example, a method may require independent observations or may rely on a model for the distribution of scores. If participants’ outcomes are related, or if the distribution behaves differently from the model’s assumptions, the uncertainty estimate and test result may be affected. These are general considerations; the registry record does not identify the particular model or diagnostic checks used for this outcome.

In general, over-dispersion refers to variability that is greater than a statistical model expects. It can make model-based uncertainty estimates too small if the model does not account for the extra variation. The posted outcome is continuous and summarized with means and standard deviations, and the registry record does not report an over-dispersion assessment. This general issue should not be mistaken for a documented problem in this trial.

The record lists 208 participants enrolled, but the posted arm summaries report 104 and 93 participants. The outcome description says the analysis included participants participating in outcome assessment. The registry record does not explain the difference in counts, provide a participant flow account, or report whether missing outcomes were imputed, weighted, or otherwise addressed. Without those details, it is not possible to assess from the registry record whether missing outcome data could have affected the comparison.

The record lists two outcome measures, including treatment satisfaction as a secondary outcome, and zero analyses. It does not report a statistical analysis plan, the hierarchy of outcomes, any interim analyses, or a multiple-testing adjustment. The absence of those details means that the posted p-value cannot be evaluated against an unreported prespecified analysis plan. It also prevents a fuller assessment of whether choices made during analysis could have influenced which result was emphasized.

The comparison and interval describe a group-level estimate, not the outcome of every participant. In general, an estimated mean difference can be useful for summarizing an average contrast while still concealing variation among individuals. The arm standard deviations offer some information about score dispersion, but the registry record does not provide individual-level data or subgroup analyses. No claim about individual responses follows from the group means alone.

10. Confidence Intervals and Precision

The posted mean difference is -0.22, with a 95% confidence interval from -0.434 to -0.006. This interval describes uncertainty in the estimated difference under the method used to construct it. The upper endpoint is close to zero, so the interval includes values representing a small negative difference as well as values farther from zero in the negative direction. The lower endpoint indicates that larger negative differences are also compatible with the interval. The range should not be interpreted as a prediction of individual scores.

In general, confidence intervals are more informative when read alongside the point estimate and the outcome scale. The estimate is expressed in scale units, and the interval gives a range of estimates consistent with the analysis at the stated confidence level. The registry record does not report a threshold for a clinically important difference, so the interval cannot be used by itself to determine whether the range represents an important change in depression severity.

The standard deviations, 0.71 in the secure messaging arm and 0.81 in usual care, describe variation in participants’ scores. They are different quantities from the confidence interval around the mean difference. The reported arm sizes are 104 and 93. The registry record does not report a more detailed account of the outcome-assessment population, the handling of missing data, or the method used to calculate the interval. These limitations constrain how precisely the interval’s assumptions can be evaluated.

The unadjusted p-value of 0.044 is consistent with a comparison near the conventional 0.05 reference, but it does not replace the interval as a description of the range of estimated differences. Neither the p-value nor the confidence interval establishes clinical importance or guarantees that a result would be similar in another study.

11. Planning a Similar Trial: Sample Size

This is a planning illustration, not a re-evaluation of the original trial. It asks: if a new study wanted to detect a difference like the one observed, or a smaller one, how large would it need to be? It is deliberately not “observed power” of the finished trial, which is not a meaningful quantity.

The planning figures are participants per arm, not total event counts. The calculation is a planning illustration based on the observed effect, not the trial’s original design justification. The registry record’s stated assumptions are a two-sided significance level of 0.05, equal allocation between the two arms, and a normal approximation. Each planning row uses the stated fraction of the observed effect as the effect to detect.

The 80% power column gives the number of participants per arm in a planning scenario designed to have an 80% chance of detecting the assumed effect under the stated assumptions, if that effect is real and the assumptions hold. The 90% power column gives the corresponding number per arm for a 90% chance. These are design calculations, not probabilities that the trial’s finding is true and not measures of the power actually achieved in this completed trial.

For planning at 100% of the observed effect, the figures are 187 participants per arm for 80% power and 251 participants per arm for 90% power. At 75% of the observed effect, they are 333 per arm and 445 per arm. At 50% of the observed effect, they are 748 per arm and 1,001 per arm. The pattern reflects a general statistical principle: smaller effects are harder to distinguish from random variation and therefore require more participants to detect with a given power target.

These figures are planning illustrations, not observed power. They use an effect estimated from this trial, which is itself uncertain, and they do not show what the trial’s actual power was. The original sample-size assumptions and justification are not reported in the registry record.

12. How the Sample Size Is Derived

Power is the probability that a study of a given size rejects the null hypothesis when a specific true effect exists. It depends on four things: the effect to detect, the variability of the outcome, the significance level (here a two-sided α of 0.05, so z_1−α/2 = 1.960) and the desired power (z_1−β = 0.842 for 80% and 1.282 for 90%).

Formula

n is the number of participants in each arm, Δ is the difference in means to detect and σ is the common standard deviation.

Worked calculation for this trial

1. Observed difference Δ = -0.220; pooled standard deviation σ = 0.759.
2. n = 2 × (1.96 + 0.842)² × 0.759² / -0.220² = 186.8, rounded up to 187 participants per arm for 80% power.

The smaller-effect rows in the table above repeat the same calculation with the effect multiplied by 0.75 and 0.5 (for ratios, applied to the logarithm of the ratio).

R code

alpha <- 0.05
z_a   <- qnorm(1 - alpha / 2)          # two-sided
delta <- -0.22                      # observed difference in means
sigma <- 0.758823313105772                     # pooled standard deviation
n_per_arm <- function(power, frac = 1) {
  ceiling(round(2 * (z_a + qnorm(power))^2 * sigma^2 / (delta * frac)^2, 6))
}
for (f in c(1, 0.75, 0.5))
  cat(sprintf("%3.0f%% of observed effect: n per arm = %s (80%% power), %s (90%% power)\n", 100 * f,
              format(n_per_arm(0.80, f), big.mark = ","), format(n_per_arm(0.90, f), big.mark = ",")))

# Check against R's built-in function (t-based, so very slightly larger)
cat("power.t.test:", round(power.t.test(delta = delta, sd = sigma, power = 0.80)$n, 1), "\n")

# Simulation: exact two-sample t-test using simulated means and variances
set.seed(755235)
n    <- n_per_arm(0.80)
reps <- 20000
m_diff <- rnorm(reps, delta, sigma * sqrt(2 / n))             # difference in sample means
s2     <- sigma^2 * rchisq(reps, 2 * (n - 1)) / (2 * (n - 1))   # pooled sample variance
tstat  <- m_diff / sqrt(s2 * 2 / n)
cat(sprintf("Simulated power at n = %s per arm: %.3f (%d replicates)\n", format(n, big.mark = ","), mean(abs(tstat) > qt(0.975, 2 * (n - 1))), reps))

Output from running the code

100% of observed effect: n per arm = 187 (80% power), 251 (90% power)
 75% of observed effect: n per arm = 333 (80% power), 445 (90% power)
 50% of observed effect: n per arm = 748 (80% power), 1,001 (90% power)
power.t.test: 187.7 
Simulated power at n = 187 per arm: 0.796 (20000 replicates)

The simulation draws many sets of group means and variances at the planned sample size and counts how often the two-sample t-test rejects the null. It should land close to the target power; at very small required sample sizes it can fall below it, because the normal approximation is rough there.

13. Multiplicity, Missing Data and Analysis Populations

The registry record lists the primary depression-scale outcome and treatment satisfaction as a secondary outcome, with two outcome measures overall. It lists zero analyses. Numerical results for treatment satisfaction are not reported in the registry record. The record does not describe a prespecified testing hierarchy, a multiplicity adjustment, or how results across outcomes were to be interpreted together.

In general, multiplicity arises when several hypotheses are tested. Conducting multiple tests can increase the chance that at least one result appears statistically notable by chance. Adjustments or a prespecified hierarchy can address that concern, but the registry record does not report whether either approach was planned or used. The primary comparison is identified as unadjusted; the record does not report a multiplicity-adjusted result.

The primary-outcome description says the analysis included participants participating in outcome assessment. Although the record lists 208 enrolled participants, the arm summaries include 104 in secure messaging care management and 93 in usual care. Reasons for the difference, participant flow after randomization, and the number with missing primary outcomes are not reported in the registry record. It is therefore not possible to determine from the posted information whether missing outcomes differed between arms or could have affected the result.

The registry record does not report how missing values were handled, whether the analysis followed an intention-to-treat principle, or whether sensitivity analyses were performed. It also does not report the analysis population for the secondary outcome. These omissions limit assessment of how completely the analysis represents the randomized groups and how robust the findings are to alternative assumptions about missing data.

Interim analyses and stopping rules are not reported in the registry record. The record also does not provide a statistical analysis plan, so the timing of analysis decisions and the prespecification of model choices cannot be assessed. These details are relevant to interpreting a p-value and to judging how much confidence to place in a single posted result.

14. Safety Results

The registry reports serious adverse events as affected participants divided by participants at risk.

Other (non-serious) adverse events

Participants with at least one non-serious event at or above the registry's 1% reporting threshold.

Most frequent serious adverse event terms

Serious events with the highest arm-level rate: suicide attempt (0.0%).

The posted serious-adverse-event figures report zero affected participants among 104 at risk in the secure messaging care-management arm and zero among 93 at risk in the usual-care arm. The listed serious-event term, suicide attempt, is reported as 0.0%. The figures therefore describe what is recorded in the registry for the stated groups and event category.

Zero recorded events do not establish that an event could never occur or that the arms have identical safety profiles. In general, the ability to characterize uncommon events depends on the amount and completeness of safety follow-up. The registry record does not report detailed safety definitions, the collection procedures, or follow-up duration for adverse events, so the scope of these counts cannot be assessed further.

The record also lists zero participants affected by other adverse events in each arm. Death counts are null and therefore are not reported in the registry record. These entries should not be interpreted as a statement about outcomes beyond the categories and information recorded.

15. What the Result Does, and Does Not, Mean

The posted results show a mean depression-scale score of 0.95 in the secure messaging care-management arm and 1.17 in usual care. The reported mean difference is -0.22, with a 95% confidence interval from -0.434 to -0.006 and an unadjusted p-value of 0.044. These statistics describe the difference in the posted arm summaries and the uncertainty associated with that estimate under the analysis used.

The result does not establish the clinical importance of the difference. The registry record does not report a threshold for a meaningful difference on this scale. It also does not explain whether the comparison accounts for baseline scores or how missing outcomes were handled. Thus, the posted comparison should not be described as a change from baseline unless the analysis method confirms that interpretation.

The study is identified as a pilot trial focused on the feasibility and acceptability of depression care management by electronic secure messaging, as well as its potential effectiveness. The primary-outcome comparison alone does not answer all of those aims. Numerical findings for treatment satisfaction and quantitative feasibility results are not reported in the registry record.

The posted group summaries do not establish that each participant experienced a similar outcome, nor do they show how results vary across subgroups. The standard deviations indicate that individual scores varied within each arm, but the registry record does not provide individual-level results. The study’s enrollment, outcome-assessment counts, and incomplete analysis details also limit conclusions about how representative the analyzed participants were of everyone enrolled.

These findings are a record of one completed pilot trial. They do not, on their own, establish whether the intervention would produce the same results in other settings or populations. The registry record does not provide enough information about the usual-care services, messaging procedures, or analysis plan to resolve those broader questions. Interpretation should remain tied to the measured outcome, the participants represented in the posted analysis, and the uncertainty in the estimated difference.

16. Why This Trial Matters Statistically

This trial illustrates several important statistical considerations in a pilot study. The registry record gives an estimated mean difference, a confidence interval, and an unadjusted p-value, but does not report the full statistical analysis plan. Readers therefore need to distinguish the numerical result from details that would explain exactly how it was derived.

The primary outcome was measured at baseline and after 6 months of treatment, but the registry record does not say whether the posted comparison uses follow-up scores, change scores, or an adjustment for baseline. These approaches can answer related but distinct statistical questions. Without that information, the difference should be described as the posted mean comparison rather than assigned a more specific interpretation.

The record also highlights the importance of outcome completeness. Enrollment is listed as 208, while the posted arm results include 104 and 93 participants. The primary-outcome description refers to participants who took part in outcome assessment, but reasons for the difference in counts and methods for handling missing data are not reported. Transparent participant flow and missing-data methods help readers understand which participants contribute to a result and how exclusions might affect its interpretation.

Finally, the study’s stated feasibility and acceptability aims are not fully represented by the numerical primary-outcome comparison. Treatment satisfaction is listed as a secondary outcome, but its results are not reported in the registry record. A pilot trial can raise useful questions about implementation and measurement, yet a single estimated effect also has uncertainty. The accompanying planning illustrations show how required sample sizes change when smaller effects are targeted; they are not evidence about the trial’s observed power or its original design rationale.

17. Limitations

• The posted summary statistics are the only data these results rest on; patient-level data, full protocols and statistical analysis plans are not available.
• Recalculated results are simple, unadjusted calculations from posted summaries. They do not replace the sponsor's prespecified analysis.

Several trial-specific limitations affect interpretation of the posted result. The study is described as a pilot, and the registry record does not report the original sample-size justification or the assumptions used to plan enrollment. The planning figures based on the observed effect are not a substitute for that original rationale.

The record lists 208 enrolled participants, while the outcome summaries include 104 in secure messaging care management and 93 in usual care. It says the primary analysis included participants participating in outcome assessment, but does not explain why other enrolled participants are absent from the posted outcome counts. Participant flow, missing-outcome numbers, and missing-data methods are not reported in the registry record.

Although the outcome was measured at baseline and after 6 months of treatment, the registry record does not specify how baseline scores were used in the comparison. It does not report the statistical test, model, covariate adjustments, sensitivity analyses, or whether the analysis followed the original randomized allocation. The interval and p-value therefore cannot be independently assessed against a detailed analytic description in the record.

Feasibility and acceptability are among the study’s stated aims, but corresponding quantitative results are not reported in the registry record. Treatment satisfaction is listed as a secondary outcome without numerical results. The record also does not describe the details of the secure-messaging intervention or the services encompassed by usual care, limiting understanding of what was compared.

Finally, the record does not report a clinically important difference threshold for the depression scale, a multiplicity plan, or interim-analysis procedures. These gaps do not negate the posted statistics, but they constrain judgments about clinical relevance, missingness, analysis choices, and the breadth of conclusions that can be drawn.

18. Related Tutorials

19. Related Statistical Calculators

20. Sources

• ClinicalTrials.gov: Feasibility of Depression Care Management by E-mail — NCT00755235.
• Registry record last updated 2017-11-13; status last verified 2017-10.
• Results first posted 2012-06-01.
• Data retrieved from ClinicalTrials.gov on 2026-11-09.
• Group sizes, means, event counts and sponsor-reported p-values are reproduced as posted. Effect estimates, confidence intervals and p-values not attributed to the sponsor are independently calculated from the posted summary statistics.

21. Record Summary

The registry record identifies NCT00755235 as a completed randomized, double-masked, parallel pilot trial sponsored by Kaiser Permanente. Enrollment was 208, with a start date of 2009-04 and primary completion in 2010-04. The comparison was depression care management by secure messaging versus usual care without added education or care-management services.

The primary outcome was the 20-Item Symptom Checklist Depression Scale, measured at baseline and after 6 months of treatment. Posted means were 0.95 for secure messaging care management and 1.17 for usual care. The mean difference was -0.22, with a 95% confidence interval from -0.434 to -0.006 and an unadjusted p-value of 0.044. The record also lists treatment satisfaction as a secondary outcome, but numerical results are not reported in the registry record.

The original analysis plan, sample-size justification, missing-data procedures, and details of how baseline scores entered the comparison are not reported in the registry record. Serious adverse events and other adverse events are recorded as affecting zero participants in each arm; death counts are not reported.