COVID-19 Watch + COVID-19 Pulse: Complete Statistical Analysis

1. Trial at a Glance

This page covers the registry results for COVID-19 Watch + COVID-19 Pulse (NCT04581863), a randomized na study. The primary outcome is Difference in Days Alive and Out of Hospital.

2. Clinical Question

3. Trial Design

This was a randomized, parallel-group trial with no masking reported. The comparison was between COVID Watch alone and COVID Watch plus a pulse oximeter. The University of Pennsylvania sponsored the trial. The registry lists enrollment of 2097 and a completed status. The recorded start date is 2020-11-30, and the primary completion date is 2021-02-05.

The shared COVID Watch program used text-based assessments twice daily for 14 days. It was described as escalating reports of worsening symptoms to a nurse through telemedicine when symptoms were not severe enough to recommend an immediate emergency department visit. The added-device group was prompted to text an oxygen-saturation measurement twice daily after walking in place for 1 minute. The registry record does not report further details about allocation procedures, such as how the random sequence was generated or concealed, so those features cannot be characterized from this record.

The primary outcome was days alive and out of hospital, measured over 30 days. The registry description says the outcome period begins at the date of a positive COVID-19 test for people tested in outpatient or emergency department settings, or at hospital discharge for people enrolled in COVID Watch as inpatients. The primary outcome description specifies patients confirmed to be COVID positive. The posted group summaries are means with standard deviations, and the reported comparison is a mean difference.

The registry record does not report the original statistical analysis plan, the trial's sample-size justification, or detailed eligibility and recruitment procedures. The posted summaries support a descriptive comparison of group means and an unadjusted comparison based on those summaries; they do not establish what additional analyses the investigators may have performed outside the posted record.

4. Participant Flow and Accounting

The registry reports how participants moved through each stage of the study. All counts below are exactly as posted; missing counts are not inferred from other stages.

Overall Study

Reasons for not completing this period:

5. Baseline Characteristics

The table reproduces the baseline characteristics posted in the registry, which describe the participants analysed at baseline and help show whether the randomized groups were comparable.

6. Primary Endpoint

The registry lists the primary outcome as Difference in Days Alive and Out of Hospital, measured over 30 days.

Days alive and out of hospital measures time spent alive without being in hospital during the outcome period. The registry specifies a 30-day timeframe and describes the measure as tracking safety outside the hospital after a positive COVID-19 test, or after hospital discharge for participants enrolled as inpatients. It also states that initial analyses include in-hospital deaths. The registry description says deaths occurring outside the hospital in 2020 would be added to a follow-up analysis when National Death Index information became available.

This endpoint combines survival and hospital use into a time-based measure. A group mean summarizes the average number of days meeting the endpoint definition; the standard deviation describes variation among participants around that average. A mean difference compares the groups on the same scale, in days. A value closer to 30 days represents more time alive and out of hospital within the stated 30-day period, but the mean alone does not show how many participants experienced particular events or how outcomes were distributed across individuals.

Using a time-based outcome can capture more than a single yes-or-no status at one moment. Interpretation still depends on the endpoint definition and the completeness of outcome ascertainment. The registry record does not report further details about how missing endpoint observations were handled or whether any additional endpoint definitions were used in the analysis.

7. Results Posted in the Registry

• COVID Watch + Pulse Oximeter: mean 29.40 versus 29.50 on control; difference -0.10 (95% CI -0.32 to 0.12), standardized effect d = -0.04; z-test p = 0.377.

8. Statistical Interpretation

9. Statistical Methods Explained

The primary outcome is continuous: it is measured in days, and the registry identifies the reported parameter as a mean with standard deviation. The main comparison is a difference between group means. The posted estimate is -0.1 days for COVID Watch plus a pulse oximeter relative to COVID Watch alone. The sign depends on that stated ordering: a negative value means the added-device group's mean is lower. The group means and this difference describe the same endpoint from different perspectives.

What does the mean describe?

A mean is an average across the analyzed observations. It is useful for summarizing a continuous outcome, but it does not reveal whether outcomes are concentrated near the mean or spread across a broad range, nor whether a few unusual values influence the average. The standard deviations are 2.3 days in the COVID Watch group and 2.8 days in the added-device group. A standard deviation describes variation among observations; it is not a confidence interval and does not by itself quantify the precision of the estimated group difference.

Because the groups are randomized and parallel, the intended comparison is between outcomes in separate groups. The posted mean difference is an unadjusted summary comparison. The registry record does not report an adjusted estimate, covariates used in adjustment, or detailed model specifications. The reported comparison therefore should not be described as adjusted for baseline characteristics.

How should the difference be read?

The raw mean difference remains in days, the same unit as the endpoint. A difference of -0.1 days is the estimated average contrast, not a statement that every participant in one group had fewer days alive and out of hospital. Individual outcomes can vary around each group's mean. The posted standard deviations provide context for that variation, while the confidence interval addresses uncertainty in the estimated mean contrast.

The registry also posts a standardized mean difference of -0.039, calculated using a pooled standard deviation of 2.562. Standardization divides the contrast by a measure of spread so the result is expressed in standard-deviation units rather than days. It can help describe the relative size of an effect in statistical terms, but it does not provide a threshold for clinical importance. The registry record does not report such a threshold for this endpoint.

What do the p-value and interval add?

The unadjusted p-value is 0.377. In general, a p-value is calculated under a specified null model and describes how often data at least as extreme as those observed would arise under that model. It is not a measure of effect size, the probability that a treatment is effective, or the probability that chance alone caused the results. Interpretation also depends on the test, assumptions, analysis population, and whether the comparison was selected among multiple analyses. The full statistical plan and multiplicity approach are not reported in the registry record.

A confidence interval describes a range of estimates compatible with the data and the interval procedure's assumptions. The reported interval for the mean difference includes zero, so it encompasses no difference as well as differences in either direction. Its breadth conveys uncertainty: a narrow interval would indicate more precise estimation than a broad one. The interval's confidence level is not stated in the registry record, so it should not be assigned a level based on assumption.

The interval is attached to the mean difference, not to either group's standard deviation, and it is not a range containing a fixed proportion of individual outcomes. General statistical interpretation of an interval depends on its construction and the sampling and modeling assumptions behind it. The registry record does not report enough methodological detail to assess all those assumptions.

What is the role of the planning illustration?

The planning figures are based on fractions of the observed effect and use a two-sided significance level of 0.05, equal allocation between two arms, and a normal approximation. They are planning illustrations, not results from the trial's original design calculation. The original sample-size assumptions and justification are not reported in the registry record. Smaller target effects require more participants under the stated planning assumptions because they are harder to distinguish from random variation. The figures should therefore be read as conditional illustrations, not as evidence that the trial was planned to detect those exact effects.

The posted summaries do not establish whether outcome distributions met a particular model assumption, whether robust or alternative analyses were conducted, or whether any subgroup comparisons were performed. Such details are not reported in the registry record. These limitations counsel against reading the unadjusted summary comparison as a complete description of every analysis undertaken.

10. Confidence Intervals and Precision

The estimated mean difference is -0.1 days, and the reported confidence interval extends from a negative value to a positive value, crossing zero. The interval therefore includes the possibility of no mean difference under its construction, while also allowing for differences in either direction. It is not appropriate to interpret the interval as proof that the groups are identical. The interval's confidence level is not reported in the registry record.

Precision concerns how tightly the data estimate the contrast, rather than whether the estimate is important. The group means are close, but means alone do not convey uncertainty. The standard deviations, 2.3 and 2.8 days, describe participant-level spread, whereas the confidence interval describes uncertainty around the estimated group contrast. These quantities answer different questions.

The interval's crossing of zero and the unadjusted p-value of 0.377 indicate that the posted results do not sharply distinguish the estimated contrast from zero using that comparison. They do not establish equivalence or rule out every potentially meaningful difference. To evaluate equivalence, the analysis would need a prespecified margin defining an acceptable range, and the registry record does not report one.

Precision is also conditional on outcome ascertainment and analysis choices. The registry describes how the 30-day period is anchored for outpatient or emergency department testing and for inpatient enrollment, but does not report missing-data methods or detailed analysis populations. These unreported features limit assessment of how completely the interval reflects uncertainty in the underlying trial population.

11. Planning a Similar Trial: Sample Size

This is a planning illustration, not a re-evaluation of the original trial. It asks: if a new study wanted to detect a difference like the one observed, or a smaller one, how large would it need to be? It is deliberately not “observed power” of the finished trial, which is not a meaningful quantity.

The sample-size display is a planning illustration based on the observed effect, not a record of the original trial design calculation. Its entries are participants per arm, not total participants or total events. The stated assumptions are a two-sided significance level of 0.05, equal allocation between the two arms, and a normal approximation. The effect to detect is set to the stated fraction of the observed effect. The registry record does not report the trial's original sample-size assumptions or justification.

The 80% power column gives the planned participants per arm under those assumptions for a design with an 80% chance of detecting the specified effect if that effect were the true one and the planning model were correct. The 90% power column gives the corresponding per-arm planning number for a design with a 90% chance under the same conditional framework. These are design probabilities, not guarantees about a particular study's results. This is not observed power.

The rows for 100% of the observed effect, 75% of the observed effect, and 50% of the observed effect represent progressively smaller target effects. Smaller effects require more participants to detect under otherwise unchanged assumptions. For the 100% effect illustration, the figures are 10,307 participants per arm for 80% power and 13,798 participants per arm for 90% power. For 75% of the observed effect, they are 18,323 and 24,529 participants per arm, respectively. For 50% of the observed effect, they are 41,227 and 55,191 participants per arm, respectively.

The curve should be read as a visual continuation of these conditional planning calculations, not as observed trial performance. These illustrations use the observed effect as their starting point, so they are sensitive to the uncertainty in that estimate. They should not be interpreted as a recommendation for a future study without a separate design rationale and assumptions appropriate to that study.

12. How the Sample Size Is Derived

Power is the probability that a study of a given size rejects the null hypothesis when a specific true effect exists. It depends on four things: the effect to detect, the variability of the outcome, the significance level (here a two-sided α of 0.05, so z_1−α/2 = 1.960) and the desired power (z_1−β = 0.842 for 80% and 1.282 for 90%).

Formula

n is the number of participants in each arm, Δ is the difference in means to detect and σ is the common standard deviation.

Worked calculation for this trial

1. Observed difference Δ = -0.100; pooled standard deviation σ = 2.562.
2. n = 2 × (1.96 + 0.842)² × 2.562² / -0.100² = 10,306.6, rounded up to 10,307 participants per arm for 80% power.

The smaller-effect rows in the table above repeat the same calculation with the effect multiplied by 0.75 and 0.5 (for ratios, applied to the logarithm of the ratio).

R code

alpha <- 0.05
z_a   <- qnorm(1 - alpha / 2)          # two-sided
delta <- -0.100000000000001                      # observed difference in means
sigma <- 2.56234713938723                     # pooled standard deviation
n_per_arm <- function(power, frac = 1) {
  ceiling(round(2 * (z_a + qnorm(power))^2 * sigma^2 / (delta * frac)^2, 6))
}
for (f in c(1, 0.75, 0.5))
  cat(sprintf("%3.0f%% of observed effect: n per arm = %s (80%% power), %s (90%% power)\n", 100 * f,
              format(n_per_arm(0.80, f), big.mark = ","), format(n_per_arm(0.90, f), big.mark = ",")))

# Check against R's built-in function (t-based, so very slightly larger)
cat("power.t.test:", round(power.t.test(delta = delta, sd = sigma, power = 0.80)$n, 1), "\n")

# Simulation: exact two-sample t-test using simulated means and variances
set.seed(4581863)
n    <- n_per_arm(0.80)
reps <- 20000
m_diff <- rnorm(reps, delta, sigma * sqrt(2 / n))             # difference in sample means
s2     <- sigma^2 * rchisq(reps, 2 * (n - 1)) / (2 * (n - 1))   # pooled sample variance
tstat  <- m_diff / sqrt(s2 * 2 / n)
cat(sprintf("Simulated power at n = %s per arm: %.3f (%d replicates)\n", format(n, big.mark = ","), mean(abs(tstat) > qt(0.975, 2 * (n - 1))), reps))

Output from running the code

100% of observed effect: n per arm = 10,307 (80% power), 13,798 (90% power)
 75% of observed effect: n per arm = 18,323 (80% power), 24,529 (90% power)
 50% of observed effect: n per arm = 41,227 (80% power), 55,191 (90% power)
power.t.test: 10307.5 
Simulated power at n = 10,307 per arm: 0.801 (20000 replicates)

The simulation draws many sets of group means and variances at the planned sample size and counts how often the two-sample t-test rejects the null. It should land close to the target power; at very small required sample sizes it can fall below it, because the normal approximation is rough there.

13. Multiplicity, Missing Data and Analysis Populations

The registry record lists 16 outcome measures and zero posted analyses. It reports the primary outcome comparison discussed in the results, but does not post secondary outcome results in the record. The record does not describe how the investigators handled multiplicity across outcome measures, possible subgroup analyses, or other comparisons. It is therefore not possible to determine from the registry record whether a multiplicity adjustment was planned or applied.

Multiplicity matters because examining many outcomes or comparisons can increase the chance of at least one apparently unusual result. Whether adjustment is appropriate depends on the goals and structure of the analyses. Without a reported analysis plan, the posted unadjusted p-value should not be treated as evidence that all potential comparisons were controlled under a single error-rate procedure.

Missing-data handling is not reported in the registry record. The outcome description specifies a 30-day assessment and explains how the period begins for outpatient testing, emergency department testing, and inpatient enrollment, but it does not say how participants with unavailable endpoint information were treated. It also does not report whether imputation, complete-case analysis, or sensitivity analyses were used. No such method should be inferred from the group summaries alone.

The analysis population is not fully described in the posted outcome entry. The registry specifies that the primary outcome is for patients confirmed to be COVID positive, but does not report the number included in the outcome analysis, the handling of participants without confirmation, or whether the comparison followed an intention-to-treat approach. The randomized enrollment figures should not be assumed to equal the analysis population for this endpoint.

The registry description says initial analyses include in-hospital deaths and that deaths occurring outside the hospital in 2020 would be added in a follow-up analysis when National Death Index information became available. The record does not provide enough detail to determine how that follow-up affected the posted result or whether all relevant deaths were ascertained. These are material limits on judging completeness and analysis choices.

14. Safety Results

The registry reports serious adverse events as affected participants divided by participants at risk.

Deaths

Other (non-serious) adverse events

Participants with at least one non-serious event at or above the registry's 5% reporting threshold.

The adverse-event record lists 1024 participants at risk in the COVID Watch group and 1025 at risk in the COVID Watch plus pulse oximeter group. It records zero serious adverse events and zero other adverse events in each group. Deaths are listed separately: 3 for COVID Watch and 5 for COVID Watch plus a pulse oximeter. These are counts as posted; they should not be combined with the adverse-event counts or treated as rates.

The record does not report the safety observation period, the definitions used for serious or other adverse events, or details about how events were collected. A zero count in a registry entry means no events are recorded in that category in the posted data; without ascertainment details, it does not establish that no events occurred outside the recorded information. The separately listed deaths are not characterized further in the adverse-event entry, so their causes and relation to either intervention are not reported in the registry record.

15. What the Result Does, and Does Not, Mean

The posted primary outcome results show means of 29.5 days alive and out of hospital for COVID Watch and 29.4 days for COVID Watch plus a pulse oximeter over the 30-day period. The estimated mean difference is -0.1 days for the added-device group relative to COVID Watch alone. The reported confidence interval crosses zero, and the unadjusted p-value is 0.377. Together, these statistics describe a small observed difference with uncertainty that includes values on either side of zero.

This result does not prove that the approaches are equivalent. Equivalence requires a prespecified acceptable difference and an analysis designed to assess it; neither is reported in the registry record. It also does not show that every participant had a similar outcome, because a group mean can conceal variation in individual experiences. The reported standard deviations indicate spread around the group means, but do not give the full outcome distributions.

The comparison addresses the addition of a pulse oximeter to a text-based remote-monitoring program, not a comparison of remote monitoring with no monitoring. Both groups received COVID Watch. The results therefore cannot isolate the effect of text messaging itself. Nor do the posted primary outcome summaries establish effects on outcomes that are not reported, including unposted secondary outcomes.

Interpretation is bounded by the information in the registry record. It does not report full analysis populations, missing-data methods, the confidence level, multiplicity procedures, or the complete statistical analysis plan. These omissions make it difficult to assess some aspects of precision and robustness. The results are best read as the posted descriptive and unadjusted comparison for the stated endpoint, rather than as a complete account of every possible clinical or statistical question.

16. Why This Trial Matters Statistically

This trial addresses a statistical challenge in remote monitoring: whether adding a measurement device to an existing text-based service changes a patient-centered outcome. Because both groups received COVID Watch, the randomized contrast focuses on the incremental addition of pulse oximetry within that program. This design distinction is important when interpreting what the comparison can answer.

The primary endpoint combines survival and time outside the hospital over a defined period. Such a time-based measure can summarize more than a single status at follow-up, while still requiring careful attention to its definition, timing, and ascertainment. The registry identifies different starting points for outpatient or emergency department testing and inpatient enrollment, illustrating why endpoint timing needs to be clear when combining participants enrolled in different care settings.

The posted means are close, and the mean difference is small relative to the standard deviations. The confidence interval includes zero, emphasizing the difference between an observed estimate and a precise conclusion. A non-zero estimate can arise even when uncertainty is substantial; conversely, an interval crossing zero does not demonstrate that effects in either direction are impossible. Statistical interpretation should consider the estimate, its uncertainty, the endpoint's meaning, and the analysis plan together.

The record also illustrates the limits of registry results as a standalone statistical account. The primary summaries are available, but the detailed analysis plan, missing-data procedures, multiplicity handling, and analysis populations are not reported. Those details are necessary to evaluate how the posted estimate was obtained and how broadly it can be interpreted. In this context, clear separation of the randomized comparison, the endpoint definition, and the uncertainty around the mean difference is especially important.

17. Limitations

• The posted summary statistics are the only data these results rest on; patient-level data, full protocols and statistical analysis plans are not available.
• Recalculated results are simple, unadjusted calculations from posted summaries. They do not replace the sponsor's prespecified analysis.

The registry record gives a concise endpoint description but does not report detailed eligibility criteria, recruitment procedures, or the number of participants included in the primary outcome analysis. Although the outcome is specified for patients confirmed to be COVID positive, the record does not provide the analysis population or explain how confirmation status affected inclusion. Enrollment should therefore not be assumed to be the endpoint analysis denominator.

The outcome's timing can differ by enrollment setting: the registry describes starting at the positive test for outpatient and emergency department testing, or at discharge for those enrolled as inpatients. The record does not report how this distinction was handled in analysis or whether it affected comparability. The description also notes a planned follow-up incorporating deaths outside the hospital, but does not provide details on the resulting ascertainment or its effect on the posted outcome.

Masking is listed as none. The registry record does not report further procedures for reducing bias in outcome assessment or data collection. It also does not provide detailed information about adherence to pulse-oximeter prompts, missing readings, use of the device, or delivery of the COVID Watch service. These details cannot be inferred from the group means.

Finally, the registry does not report the full statistical analysis plan, missing-data methods, multiplicity procedures, or a clinical threshold for a meaningful difference in days alive and out of hospital. The confidence interval level is also not stated. These omissions limit assessment of analytic robustness and the practical importance of the estimated contrast.

18. Related Tutorials

19. Related Statistical Calculators

20. Sources

• ClinicalTrials.gov: COVID-19 Watch + COVID-19 Pulse — NCT04581863.
• Registry record last updated 2023-11-24; status last verified 2023-11.
• Results first posted 2023-11-24.
• Data retrieved from ClinicalTrials.gov on 2026-11-09.
• Group sizes, means, event counts and sponsor-reported p-values are reproduced as posted. Effect estimates, confidence intervals and p-values not attributed to the sponsor are independently calculated from the posted summary statistics.

21. Record Summary

NCT04581863, titled COVID-19 Watch + COVID-19 Pulse, was a randomized, parallel-group trial with no masking reported. The sponsor was the University of Pennsylvania. The registry lists the trial as completed, with enrollment of 2097, a start date of 2020-11-30, and a primary completion date of 2021-02-05.

The comparison was COVID Watch alone versus COVID Watch plus a pulse oximeter. The primary endpoint was mean days alive and out of hospital over 30 days. Posted means were 29.5 days and 29.4 days, respectively; the estimated mean difference was -0.1 days, with an unadjusted p-value of 0.377. The reported confidence interval includes zero. The registry record does not report the complete analysis plan, analysis populations, missing-data methods, or multiplicity procedures.