This page provides an independent statistical analysis and educational interpretation of publicly reported results. ClinicalTrials.gov provides the official trial registry record. Numerical trial results on this page are restricted to the results posted in the ClinicalTrials.gov record; calculations added by Clinical Biostats are labelled as such.
1. Trial at a Glance
This page covers the registry results for EHR Embedded Comparative Effectiveness Studies--WWC (NCT04514679), a randomized na study. The primary outcome is Compliance Rate.
| Feature | Registry value |
|---|---|
| ClinicalTrials.gov identifier | NCT04514679 |
| Phase | NA |
| Condition | Obesity |
| Allocation | Randomized |
| Design model | Parallel |
| Masking | None (open label) |
| Enrollment | 60 |
| Arms in the primary outcome | 2 |
| Lead sponsor | Dartmouth-Hitchcock Medical Center |
| Results posted | Yes |
2. Clinical Question
Population
Patients receiving obesity treatment at Dartmouth-Hitchcock Medical Center were assigned to one of two visit sequences. The primary outcome population was participants with health record data; the registry record says this excluded 3 patients. The registry record does not report further eligibility details or a demographic description of the participants.
Intervention
In the Dietician / Health Coach sequence, patients first saw a dietician and then a health coach. The primary question was whether participants completed the two visits in the assigned order. This was one of two behavioral intervention sequences listed for the obesity demonstration project.
Comparator
In the Health Coach / Dietician sequence, patients first saw a health coach and then a dietician. The comparison concerns the order of the two visits, rather than a comparison with no visits or no intervention.
Primary question
What percentage of participants in each assigned sequence completed both visits in the correct order, from baseline through approximately 2 months? The registry record reports compliance of 87.0% for Dietician / Health Coach and 76.0% for Health Coach / Dietician. These are posted results for this comparison, not evidence about weight change or other health outcomes.
The project’s official title describes work on creating valid instrumental variables for learning health care systems. Its brief summary says the project aimed to develop informatics systems and statistical methods to support randomization processes in electronic health record systems for comparative effectiveness research. The registry record describes two demonstration projects, including weight-loss interventions and common practices for chronic low back pain. The results concern the obesity visit-sequence comparison. The registry record does not report a separate clinical efficacy question or a formal hypothesis for these arms.
3. Trial Design
The registry record identifies this study as randomized, with no masking and a parallel model. The two parallel arms assigned different orders for two obesity-treatment visits: Dietician / Health Coach and Health Coach / Dietician. The record lists phase as not applicable. Dartmouth-Hitchcock Medical Center is the sponsor. The study status is completed, with a start date of 2020-10-23 and a primary completion date of 2021-09-27.
Enrollment is listed as 60. The primary outcome population is described as those with health record data, which eliminated 3 patients. The posted primary outcome results include 28 participants in the Dietician / Health Coach arm and 29 in the Health Coach / Dietician arm. The registry record does not explain the difference between the enrollment figure, the health-record-data exclusion, and the outcome denominators, or provide a participant-flow account. It also does not report the original target sample size or the assumptions used to choose it.
The primary endpoint was compliance with the assigned visit order, measured as a percentage of participants from baseline through approximately 2 months. The record also lists change in body mass index at 30 days since baseline as a secondary outcome. Although three outcome measures are indicated, only one secondary outcome is posted. The record lists no reported analyses. It does not describe random sequence generation, allocation concealment, how the electronic record supported assignment, or whether any departures from assigned visit order occurred for reasons other than those captured by the compliance endpoint.
The project’s broader description includes a second demonstration concerning chronic low back pain, but the posted arms and outcome results addressed are the two obesity-treatment sequences. The registry record does not report additional design details for the separate demonstration in the results summarized for these arms.
4. Participant Flow and Accounting
The registry reports how participants moved through each stage of the study. All counts below are exactly as posted; missing counts are not inferred from other stages.
Overall Study
| Stage | Dietician / Health Coach | Health Coach / Dietician |
|---|---|---|
| STARTED | 30 | 30 |
| COMPLETED | 30 | 30 |
| NOT COMPLETED | 0 | 0 |
Analysis population for the primary outcome, as defined in the registry: Those who had health record data, which eliminated 3 patients.
5. Baseline Characteristics
The table reproduces the baseline characteristics posted in the registry, which describe the participants analysed at baseline and help show whether the randomized groups were comparable.
Population: Age was unavailable due to missing data in the patient record.
| Characteristic | Unit | Dietician / Health Coach | Health Coach / Dietician | Total |
|---|---|---|---|---|
| Participants analysed | 28 | 29 | 57 | |
| Age, Continuous | years | 45.0 (SD 14.4) | 45.2 (SD 13.5) | 45.1 (SD 13.8) |
| Sex: Female, Male: Female | Participants | 23 | 27 | 50 |
| Sex: Female, Male: Male | Participants | 5 | 2 | 7 |
| Ethnicity (NIH/OMB): Hispanic or Latino | Participants | 0 | 0 | 0 |
| Ethnicity (NIH/OMB): Not Hispanic or Latino | Participants | 28 | 28 | 56 |
| Ethnicity (NIH/OMB): Unknown or Not Reported | Participants | 0 | 1 | 1 |
| Race (NIH/OMB): American Indian or Alaska Native | Participants | 0 | 0 | 0 |
| Race (NIH/OMB): Asian | Participants | 0 | 0 | 0 |
| Race (NIH/OMB): Native Hawaiian or Other Pacific Islander | Participants | 0 | 0 | 0 |
| Race (NIH/OMB): Black or African American | Participants | 0 | 0 | 0 |
| Race (NIH/OMB): White | Participants | 28 | 29 | 57 |
| Race (NIH/OMB): More than one race | Participants | 0 | 0 | 0 |
| Race (NIH/OMB): Unknown or Not Reported | Participants | 0 | 0 | 0 |
| Region of Enrollment: United States | participants | 28 | 29 | 57 |
6. Primary Endpoint
The registry lists the primary outcome as Compliance Rate, measured over Baseline through approximately 2 months.
| Element | Registry entry |
|---|---|
| Result type | Number |
| Measure of spread | 95% Confidence Interval |
| Unit | percentage of participants |
| Endpoint classification | Binary Endpoint |
The primary endpoint is the percentage of participants in each arm who completed both visits in the correct assigned order. The measurement window is baseline through approximately 2 months. This endpoint directly reflects whether the intended sequence was carried out, rather than whether participants lost weight or experienced another health change. It is therefore a measure of implementation and adherence to the assigned sequence.
The posted percentages are 87.0% for Dietician / Health Coach and 76.0% for Health Coach / Dietician. The corresponding confidence intervals are 66.0% to 97.0% and 54.0% to 90.0%. The registry record identifies the primary outcome population as those who had health record data, eliminating 3 patients. It does not report how compliance was determined from the records, whether the endpoint definition allowed any timing variation, or how incomplete records were handled beyond describing the population.
Using order compliance as the primary endpoint is relevant to a project developing randomization and informatics processes in routine electronic health record settings: the outcome indicates whether the sequence assigned by the study was completed as intended. It does not, by itself, establish that either visit sequence produces better weight outcomes. Body mass index change is listed separately as a secondary endpoint, and the registry record does not report a primary clinical weight-loss endpoint.
7. Results Posted in the Registry
- Health Coach / Dietician: 75.9% (22/29) versus 85.7% (24/28) on control; risk difference -9.9 percentage points (95% CI -30.1 to 10.4), risk ratio 0.89 (0.69–1.14); unadjusted two-proportion z-test p = 0.346.
Risk difference: Health Coach / Dietician
95% CI: -30.1 to 10.4 percentage points
Recalculated from the posted summary statistics
8. Secondary Efficacy Endpoints
The registry also posts results for secondary outcomes. For each outcome below, the endpoint type, the test applied to the posted summary statistics, the result and its interpretation are given in turn. Secondary endpoints are usually not adjusted for multiple testing, so they support rather than confirm the primary finding.
| Secondary outcome | Type | Effect measure | Estimate (95% CI) | Planning size, 80% power |
|---|---|---|---|---|
| Change in Body Mass Index (BMI) at 30 Days | Continuous Endpoint | Mean difference | -0.09 (-0.72 to 0.54) | 2,810 per arm |
Change in Body Mass Index (BMI) at 30 Days
- Endpoint type: Continuous: a mean value per participant, compared as a difference in means.
- Test: Unadjusted two-sample z-test for the difference in means, using the posted standard errors.
- Result:
- Health Coach / Dietician: mean -0.16 versus -0.07 on control; difference -0.09 (95% CI -0.72 to 0.54), standardized effect d = -0.07; z-test p = 0.779.
- Interpretation:
- The 95% confidence interval includes no difference, so these data do not show a statistically significant difference between Health Coach / Dietician and the control. The interval gives the range of effects that remain compatible with the data.
The registry record lists change in body mass index at 30 days since baseline as a secondary outcome, while the primary outcome measures completion of the assigned visit sequence through approximately 2 months. The outcomes therefore address different questions: one concerns implementation of the assigned order, and the other concerns short-term change in a health measure. The secondary outcome adds context about the weight-related demonstration but does not directly validate or overturn the primary compliance result.
The posted secondary comparison has substantial uncertainty, with an interval that includes both negative and positive mean differences. This broad range means the secondary estimate does not establish a clear direction of difference between the sequences. The outcome is also measured over a different timeframe from the primary endpoint, so the two results should not be treated as interchangeable measures of a single effect.
The registry record indicates 3 outcome measures, but only one secondary outcome is posted. It lists no reported analyses. As a result, the full set of measured or planned secondary outcomes and the relationship between them are not clear. The record does not report whether the secondary outcome was prespecified in an analysis plan, whether additional secondary measures were collected but not posted, or whether the displayed result was selected from a larger set of analyses.
Unadjusted secondary results generally carry less weight when multiple outcomes or comparisons may have been examined. General statistical explanation: testing multiple hypotheses increases the opportunity for apparently notable results to arise by chance, so interpretation depends on the prespecified outcome hierarchy and any multiplicity strategy. No adjustment or hierarchy is reported in the registry record. This does not make the BMI result invalid, but it means it should be considered supportive or exploratory rather than as a standalone confirmatory finding. Its uncertainty and the absence of further analysis details should be kept in view.
Secondary outcomes registered for this trial, for reference (only those above were recalculated):
- Change in Body Mass Index (BMI) at 30 Days
9. Statistical Interpretation
The posted primary results show a higher compliance percentage in the Dietician / Health Coach arm than in the Health Coach / Dietician arm: 87.0% compared with 76.0%. The unadjusted risk difference for Health Coach / Dietician relative to Dietician / Health Coach is -0.099. The confidence interval for this difference extends from -0.301 to 0.104, and the unadjusted p-value is 0.346. Thus, the posted estimate points toward lower compliance in the health-coach-first sequence, but the interval includes both a difference favoring that sequence and a difference favoring the dietician-first sequence. The result is imprecise enough that the direction and size of the underlying difference are not settled by these data.
The separate arm confidence intervals are also broad: 66.0% to 97.0% for Dietician / Health Coach and 54.0% to 90.0% for Health Coach / Dietician. Overlap between these intervals is not itself a formal test of the difference; the reported risk-difference interval and p-value address the arm comparison more directly. The p-value of 0.346 does not establish that the sequences are equivalent. It indicates that the observed data are not unusually incompatible with a null risk difference under the unadjusted comparison, while the confidence interval communicates the range of differences that remain compatible with the estimate and its uncertainty.
The secondary body mass index outcome is reported as mean change at 30 days. The arm means are -0.07 kg/m^2 for Dietician / Health Coach and -0.16 kg/m^2 for Health Coach / Dietician. The posted mean difference for Health Coach / Dietician relative to Dietician / Health Coach is -0.09 kg/m^2, with a confidence interval from -0.718 to 0.537 and an unadjusted p-value of 0.779. This estimate is also uncertain, and the interval includes values on both sides of no difference. The secondary result does not resolve the uncertainty in the primary compliance comparison because it addresses a different outcome and a different time point.
There are 3 outcome measures indicated in the registry record, but only one secondary outcome is posted, and no analyses are listed as reported analyses. The record does not identify a multiplicity adjustment or a prespecified hierarchy for interpreting outcomes. Consequently, the posted secondary result should be read as exploratory context rather than as independent confirmation of a primary finding. No sponsor analysis is reported for the primary result in the registry record; the posted comparison is the result available for interpretation.
These results describe measured visit-order completion and short-term body mass index change in a small randomized demonstration. They do not establish whether either sequence improves health, nor do they show that one order is definitively better. The estimates should be considered in light of their uncertainty, the available outcome population, and the limited analysis details in the registry record.
10. Statistical Methods Explained
The registry record describes a randomized, parallel comparison with no masking. In a randomized design, assignment is intended to make treatment groups comparable on average, including with respect to factors that may affect outcomes. Randomization does not guarantee that groups will be identical, especially in a small study, and its practical value depends on how assignment was generated and implemented. The registry record does not report the sequence-generation method, allocation concealment, or operational details of assigning visit order through the electronic health record.
What does the risk difference represent?
The primary outcome is binary at the participant level: a participant either completed both visits in the assigned order or did not. A risk, in this setting, is the proportion with the event of interest—in this case, completing the assigned sequence. The comparison reports a risk difference for Health Coach / Dietician relative to Dietician / Health Coach. Its estimate is -0.099. A negative value means the posted compliance proportion was lower in the first-named comparison arm than in the reference arm. The registry record also gives the two arm proportions used in the comparison: 0.759 and 0.857.
A risk difference expresses an absolute contrast between proportions. It differs from a risk ratio, which expresses a relative contrast, and an odds ratio, which compares odds rather than proportions. The registry record includes interval estimates for risk ratio and odds ratio as well: 0.885 with limits 0.686 and 1.142 for the risk ratio, and 0.524 with limits 0.135 and 2.037 for the odds ratio. These are alternative ways of expressing the same binary outcome comparison, not separate outcomes. All are unadjusted comparisons in the posted results.
How should the p-value be read?
The unadjusted p-value for the primary comparison is 0.346. A p-value describes how compatible the observed data are with a specified null model, assuming that model and the analysis assumptions hold. It is not the probability that the null hypothesis is true, nor does it measure the probability that a result will replicate. A p-value also does not quantify the size or practical importance of an effect. The confidence interval for the risk difference is useful alongside the p-value because it conveys the uncertainty and range of effects compatible with the observed estimate.
“Unadjusted” means the result is reported without a statistical adjustment indicated in the registry record. The record does not report covariate adjustment, a stratified analysis, or other modeling details. It also does not report a formal analysis plan, so it is not possible to determine from the registry record whether the posted comparison follows a prespecified analysis or whether other analyses were planned.
What does the secondary mean difference describe?
Change in body mass index at 30 days is a continuous outcome, summarized by an arm mean and standard error. The posted arm means are -0.07 kg/m^2 and -0.16 kg/m^2, and the mean difference for Health Coach / Dietician relative to Dietician / Health Coach is -0.09 kg/m^2. The reported confidence interval ranges from -0.718 to 0.537. A mean difference expresses the contrast between average changes in the two arms. The standard error describes the estimated sampling uncertainty around an arm mean; it is not the spread of individual participants’ values.
The comparison also reports a pooled standard deviation of 1.204 and a standardized effect estimate of -0.075. Standardization expresses a mean contrast relative to a measure of variability, which can help describe effect size without retaining the original measurement unit. These statistics are calculations based on the posted summaries, not a separately reported sponsor analysis. The record does not report the exact model used for the secondary comparison. Its unadjusted p-value is 0.779.
Why can binary outcomes be over-dispersed?
General statistical explanation: a simple binomial model treats outcomes as independent and assumes that the probability of the event is adequately represented by the model. If participants’ outcomes are more alike than the model allows—for example, because outcomes cluster within sites or providers—the observed variability can exceed the binomial expectation. This is called over-dispersion. It can make standard errors too small and confidence intervals too narrow if ignored. The registry record does not report whether participants were clustered by provider or whether an over-dispersion adjustment was considered, so no conclusion about that issue can be drawn for this study.
General statistical explanation: a confidence interval gives a range of values compatible with the data under the method’s assumptions. It is not a guarantee that the true value lies within the particular interval, and it does not show the probability that a treatment effect is beneficial. Interpretation depends on the validity of the data, outcome definition, analysis method, and assumptions used to form the interval.
What analysis details remain unknown?
The registry record lists no reported analyses and does not describe missing-data procedures, adjustments for multiple outcomes, interim analyses, or a statistical analysis plan. It reports that the primary outcome population excluded 3 patients who lacked health record data, but does not say whether all other randomized participants were analyzed according to assigned sequence for every outcome. The record also does not report whether the analyses were intention-to-treat, per-protocol, or based on another population definition. These omissions limit the ability to assess whether the displayed estimates reflect a prespecified analysis or how sensitive they might be to alternative assumptions.
11. Confidence Intervals and Precision
Precision concerns how narrowly an estimate is determined by the available data. The primary arm-level compliance intervals are 66.0% to 97.0% for Dietician / Health Coach and 54.0% to 90.0% for Health Coach / Dietician. Each interval covers a broad span of possible compliance rates. The primary comparison is more directly represented by the risk-difference interval, -0.301 to 0.104. That interval includes zero and values on either side, so the posted data do not pin down the direction of the difference. The point estimate, -0.099, is only one estimate within a range of substantially different possibilities.
Confidence intervals should not be read as a list of equally likely values or as proof that any one value in the interval is correct. They describe uncertainty under the method’s assumptions. Wider intervals generally indicate less precision, while narrower intervals indicate more precision. A confidence interval does not account automatically for unreported sources of bias, missing data, outcome misclassification, or analysis choices.
The secondary mean-difference interval, -0.718 to 0.537 kg/m^2, is also wide relative to its estimate of -0.09 kg/m^2 and includes values on both sides of zero. The posted evidence therefore leaves considerable uncertainty about the average difference in 30-day body mass index change. The arm standard errors are 0.25 and 0.2, while the record does not provide individual-level outcome distributions.
The sample sizes contributing to the primary outcomes are 28 and 29. Limited numbers of participants generally constrain precision, although the registry record does not provide a design-based precision target or the original sample-size justification. Confidence intervals make this uncertainty visible; the point estimates alone would not. Neither the primary nor secondary intervals support a definitive claim of equivalence between the visit sequences.
12. Planning a Similar Trial: Sample Size
This is a planning illustration, not a re-evaluation of the original trial. It asks: if a new study wanted to detect a difference like the one observed, or a smaller one, how large would it need to be? It is deliberately not “observed power” of the finished trial, which is not a meaningful quantity.
| Effect to detect | Participants per arm, 80% power | Participants per arm, 90% power |
|---|---|---|
| 100% of the observed effect | 250 | 334 |
| 75% of the observed effect | 423 | 566 |
| 50% of the observed effect | 901 | 1,206 |
The planning figures are participants per arm, not total events. They are illustrations based on the observed effect, not the original trial design. The stated assumptions are a two-sided significance level of 0.05, equal allocation between the two arms, a normal approximation, and an effect to detect set to the stated fraction of the observed effect. The registry record does not report the trial’s own original design assumptions or sample-size justification.
The 80% power column gives the estimated number of participants per arm for a future study to have an 80% chance of detecting the specified effect under those assumptions, if that effect and the model assumptions are correct. The 90% power column gives the corresponding estimate for a 90% chance. These are planning targets, not guarantees that a future study will produce a statistically significant result. They are also not estimates of the power achieved by this completed study.
Planning rows use 100%, 75%, and 50% of the observed effect. For the primary compliance outcome, the figures range from 250 participants per arm for 80% power and 334 for 90% power when planning for 100% of the observed effect, to 901 and 1,206 when planning for 50% of the observed effect. The intermediate 75% row gives 423 and 566 participants per arm. For the secondary body mass index outcome, the corresponding figures are 2,810 and 3,761 at 100%, 4,995 and 6,687 at 75%, and 11,238 and 15,044 at 50%.
Smaller effects are harder to distinguish from random variation, so planning to detect a smaller fraction of the observed effect requires more participants under the stated assumptions. These calculations use the observed effects as their starting point, which may be unstable in a small study. This is not observed power: it does not describe the probability that the completed study detected its actual effect, and it should not be used to replace interpretation of the estimates and confidence intervals.
13. How the Sample Size Is Derived
Power is the probability that a study of a given size rejects the null hypothesis when a specific true effect exists. It depends on four things: the effect to detect, the variability of the outcome, the significance level (here a two-sided α of 0.05, so z1−α/2 = 1.960) and the desired power (z1−β = 0.842 for 80% and 1.282 for 90%).
Formula
n is the number of participants in each arm, p0 and p1 are the control and treatment proportions, and p̄ is their average.
Worked calculation for this trial
- Control proportion p0 = 0.857; treatment proportion p1 = 0.759; average p̄ = 0.808.
- First term: z1−α/2 × √[2p̄(1−p̄)] = 1.96 × 0.5572 = 1.0920.
- Second term: z1−β × √[p0(1−p0) + p1(1−p1)] = 0.842 × 0.5528 = 0.4652.
- n = (1.0920 + 0.4652)2 / (-0.0985)2 = 249.8, rounded up to 250 participants per arm for 80% power.
The smaller-effect rows in the table above repeat the same calculation with the effect multiplied by 0.75 and 0.5 (for ratios, applied to the logarithm of the ratio).
R code
alpha <- 0.05
z_a <- qnorm(1 - alpha / 2) # two-sided
p0 <- 0.857142857142857 # control proportion
p1 <- 0.758620689655172 # treatment proportion
n_per_arm <- function(power, frac = 1) {
q1 <- p0 + (p1 - p0) * frac # smaller effect = fraction of the observed difference
pbar <- (p0 + q1) / 2
z_b <- qnorm(power)
ceiling(round((z_a * sqrt(2 * pbar * (1 - pbar)) + z_b * sqrt(p0 * (1 - p0) + q1 * (1 - q1)))^2 / (q1 - p0)^2, 6))
}
for (f in c(1, 0.75, 0.5))
cat(sprintf("%3.0f%% of observed effect: n per arm = %s (80%% power), %s (90%% power)\n", 100 * f,
format(n_per_arm(0.80, f), big.mark = ","), format(n_per_arm(0.90, f), big.mark = ",")))
# Check against R's built-in function (unrounded n per arm)
cat("power.prop.test:", round(power.prop.test(p1 = p0, p2 = p1, power = 0.80)$n, 1), "\n")
# Simulation: how often does the z-test reject at the 80%-power sample size?
set.seed(4514679)
n <- n_per_arm(0.80)
reps <- 20000
x0 <- rbinom(reps, n, p0); x1 <- rbinom(reps, n, p1)
ph <- (x0 + x1) / (2 * n)
z <- ((x1 - x0) / n) / sqrt(ph * (1 - ph) * 2 / n)
cat(sprintf("Simulated power at n = %s per arm: %.3f (%d replicates)\n", format(n, big.mark = ","), mean(abs(z) > z_a, na.rm = TRUE), reps))
Output from running the code
100% of observed effect: n per arm = 250 (80% power), 334 (90% power)
75% of observed effect: n per arm = 423 (80% power), 566 (90% power)
50% of observed effect: n per arm = 901 (80% power), 1,206 (90% power)
power.prop.test: 249.8
Simulated power at n = 250 per arm: 0.804 (20000 replicates)
The simulation draws many pairs of event counts at the planned sample size and counts how often the two-proportion z-test rejects the null. It should land close to the target power; at very small required sample sizes it can fall below it, because the normal approximation is rough there.
14. Multiplicity, Missing Data and Analysis Populations
The registry record indicates 3 outcome measures, reports one secondary outcome, and lists no reported analyses. It does not describe a prespecified testing hierarchy, a family of hypotheses, or any multiplicity adjustment. Therefore, it is not possible to determine how the posted primary and secondary results relate to all outcomes considered during the study, or whether additional comparisons were planned or conducted.
General statistical explanation: when several outcomes or comparisons are tested, the chance of at least one apparently unusual result can increase. A prespecified hierarchy or adjustment can help control how results are interpreted across multiple tests. Without information on these choices, unadjusted p-values should be read as applying to their individual comparisons, not as accounting for all possible testing. The record does not report whether the p-values were adjusted, so the posted values are described as unadjusted in the outcome comparisons.
The primary population is described as participants who had health record data, eliminating 3 patients. The record does not report why those records were unavailable, whether missingness differed by assigned arm, whether any other outcome data were missing, or whether imputation or sensitivity analyses were used. It also does not describe whether the primary analysis included all randomized participants or was restricted to those with usable records. These details matter because missing records can affect the estimated compliance proportions if availability is related to assignment or to whether visits were completed.
The registry record does not report an interim analysis plan, early stopping rules, or a statistical analysis plan. It also does not specify an intention-to-treat, per-protocol, or other analysis population. Consequently, the relationship between randomized assignment and the participants contributing to each posted estimate cannot be fully assessed. The information that is reported supports describing the posted arm results and comparisons, but does not support assumptions about unreported analysis procedures.
15. Safety Results
The registry reports serious adverse events as affected participants divided by participants at risk.
| Arm | Participants with serious adverse events | Participants at risk | Percentage |
|---|---|---|---|
| Dietician / Health Coach | 0 | 30 | 0.0% |
| Health Coach / Dietician | 0 | 30 | 0.0% |
Deaths
| Arm | Deaths (participants affected of at risk) |
|---|---|
| Dietician / Health Coach | 0 of 30 |
| Health Coach / Dietician | 0 of 30 |
Other (non-serious) adverse events
Participants with at least one non-serious event at or above the registry's 0% reporting threshold.
| Arm | Participants affected of at risk |
|---|---|
| Dietician / Health Coach | 0 of 30 |
| Health Coach / Dietician | 0 of 30 |
The registry record lists serious adverse event figures for both arms. For each arm, the number at risk for serious adverse events is 30, the number affected is 0, and deaths are 0. The record also lists 30 at risk for other adverse events and 0 affected in each arm. No serious adverse event terms are listed.
These are the adverse-event figures posted in the registry record. They should be read as counts within the registry’s stated at-risk groups, not as proof that no events occurred outside the way events were defined, collected, or recorded. The registry record does not describe the adverse-event collection period, definitions, severity assessment, or whether events were actively solicited. It also does not explain how the adverse-event group counts relate to the outcome denominators reported for the primary endpoint.
Zero reported affected participants means that no events are listed in these posted fields; it does not establish that a risk is impossible or that the study could rule out uncommon events. The participant numbers are limited, and the record provides no additional safety analysis. These figures are descriptive and do not support a comparative safety conclusion beyond the counts reported.
16. What the Result Does, and Does Not, Mean
The primary result is a comparison of how often participants completed two visits in the assigned order. Compliance was 87.0% in the Dietician / Health Coach arm and 76.0% in the Health Coach / Dietician arm. The estimated risk difference for the health-coach-first sequence relative to the dietician-first sequence is -0.099, with a 95% confidence interval from -0.301 to 0.104 and an unadjusted p-value of 0.346. The estimate points toward lower compliance in the health-coach-first arm, but the uncertainty is wide and includes values in both directions.
This result does not show that one sequence is definitively more effective, nor does the absence of a statistically notable difference establish that the sequences are equivalent. The data are compatible with a range of possible differences. The outcome is visit-order completion, so it should not be interpreted as a direct estimate of weight loss, long-term health, or patient benefit.
The secondary outcome concerns change in body mass index at 30 days. Its mean difference is -0.09 kg/m^2, with a confidence interval from -0.718 to 0.537 and an unadjusted p-value of 0.779. This result is also uncertain and concerns a different endpoint. It does not provide a firm basis for choosing between the sequences or for making claims about longer-term weight outcomes.
The study was a randomized, parallel demonstration project in an electronic health record setting. Its results offer information about the implementation of two visit sequences among participants with health record data. The registry record does not report the original design assumptions, detailed methods for assignment, analysis plan, missing-data procedures, or multiplicity strategy. These omissions limit how fully the estimates can be evaluated. The results are best understood as descriptive and comparative evidence from this small study, with precision and interpretation constrained by the information reported in the registry.
17. Why This Trial Matters Statistically
This study is statistically notable because its central question concerns whether a randomized comparison can be embedded in an electronic health record workflow and yield a measurable implementation outcome. Randomized studies in routine care settings depend on more than generating an assignment: the assigned options must be delivered and recorded well enough to support a meaningful comparison. Visit-order compliance provides a direct measure of whether participants completed the assigned sequence.
The registry record’s project description frames the work as developing informatics systems and statistical methods for supporting randomization in electronic health records for comparative effectiveness research. The two obesity-treatment sequences offer a demonstration of how assigned care sequences can be compared in that context. The posted results also show why confidence intervals matter: the primary arm estimates differ, but the estimated risk difference remains imprecise and compatible with a range of effects.
The study also illustrates the distinction between an implementation endpoint and a clinical endpoint. Completing visits in the intended order speaks to execution of the assigned sequence. The secondary body mass index measure addresses a different aspect of the intervention and is reported at a different time point. Keeping those questions separate avoids treating successful implementation as proof of a health effect, or treating an uncertain health measure as a direct test of the randomization process.
Finally, the registry record demonstrates the importance of complete reporting for statistical interpretation. Information about the original sample-size rationale, analysis plan, missing data, and multiplicity would help readers understand how the estimates were obtained and how broadly they can be interpreted. The reported outcomes remain useful as a record of the comparison, but the unreported design and analysis details limit the strength of conclusions that can be drawn.
18. Limitations
- The posted summary statistics are the only data these results rest on; patient-level data, full protocols and statistical analysis plans are not available.
- Recalculated results are simple, unadjusted calculations from posted summaries. They do not replace the sponsor's prespecified analysis.
- Participant counts were reconstructed from posted percentages and arm sizes, so small rounding differences are possible.
The enrollment figure is 60, while the primary outcome population is described as excluding 3 patients without health record data and the posted arm denominators are 28 and 29. The registry record does not provide a participant-flow account that reconciles these figures or explain the reasons for missing health record data. It also does not describe whether missingness could be related to visit completion or assigned sequence. This limits assessment of selection and missing-data effects.
The record does not report the original sample-size target, the design assumptions used to set it, or the statistical analysis plan. Planning illustrations based on observed effects do not fill that gap. There are no reported analyses in the registry record, and details such as covariate adjustment, analysis population, missing-data handling, and any interim analysis procedures are not given. The absence of these details prevents evaluation of whether the posted analyses were prespecified.
The primary endpoint depends on identifying whether both visits occurred in the correct order, but the registry record does not explain the operational definition, record-review process, or handling of timing deviations. No masking was used, and the record does not report whether outcome ascertainment involved subjective decisions. The clinical significance of completing the order is also not defined as an outcome in its own right.
The primary confidence interval is broad, and the secondary outcome interval is also broad. The study therefore offers limited precision for estimating differences between the sequences. The record indicates 3 outcome measures but posts only one secondary outcome and does not report a multiplicity approach. These reporting and precision limitations should be considered when interpreting the observed contrasts. The registry record also does not provide detailed participant characteristics, so the extent to which these results apply to other settings or patient groups cannot be assessed from the posted information.
19. Related Tutorials
20. Related Statistical Calculators
21. Sources
- ClinicalTrials.gov: EHR Embedded Comparative Effectiveness Studies--WWC — NCT04514679.
- Registry record last updated 2025-06-12; status last verified 2025-05.
- Results first posted 2025-06-12.
- Data retrieved from ClinicalTrials.gov on 2026-11-09.
- Group sizes, means, event counts and sponsor-reported p-values are reproduced as posted. Effect estimates, confidence intervals and p-values not attributed to the sponsor are independently calculated from the posted summary statistics.
Continue through the Clinical Biostats statistical pathway
Use this trial's endpoint as a starting point for deeper study of effect measures, confidence intervals, and sample-size planning.
22. Record Summary
NCT04514679 is a completed, randomized, parallel study with no masking, sponsored by Dartmouth-Hitchcock Medical Center. The project’s official title is “Randomize Everyone: Creating Valid Instrumental Variables for Learning Health Care Systems.” Enrollment is listed as 60, and the registry record lists obesity as a condition.
The comparison assigned patients to Dietician / Health Coach or Health Coach / Dietician. The primary outcome was completion of the two visits in the assigned order from baseline through approximately 2 months. The posted compliance percentages were 87.0% and 76.0%, respectively. The estimated risk difference for Health Coach / Dietician relative to Dietician / Health Coach was -0.099, with a confidence interval from -0.301 to 0.104 and an unadjusted p-value of 0.346.
Change in body mass index at 30 days was posted as a secondary outcome. The registry record lists no reported analyses, indicates 3 outcome measures, and does not report the original sample-size justification, a statistical analysis plan, or detailed missing-data procedures. The adverse-event fields list zero affected participants and zero deaths in both arms.