The Secure Messaging for Medication Reconciliation Tool (SMMRT) Trial: Complete Statistical Analysis

1. Trial at a Glance

This page covers the registry results for The Secure Messaging for Medication Reconciliation Tool (SMMRT) Trial (NCT02482025), a randomized na study. The primary outcome is Medication Discrepancies.

2. Clinical Question

3. Trial Design

The SMMRT Trial was a randomized, parallel-group trial with no masking. It is listed as completed, with enrollment of 240. The registry dates are 2018-11-16 for trial start and 2019-10-22 for primary completion. The sponsor was the VA Office of Research and Development.

The trial addressed medication discrepancies after hospital discharge. Its objectives included refining the tool through usability testing, evaluating usual care against additional support involving My HealtheVet training and SMMRT, and examining Veterans’ and staff members’ perceptions of effects on routine clinical practice and interactions with primary care providers. The registered outcome results presented for comparison are for usual care and SMMRT. A separate result for a training-only group is not reported in the registry record, so the posted estimates do not answer every comparison described in the objectives.

The primary outcome was recorded in an intention-to-treat population, and the results give group sizes of 122 and 118. The registry record does not report further details about the randomization procedure, allocation concealment, recruitment criteria, adherence to the intervention, or the statistical analysis plan. The trial’s original sample-size justification and design assumptions are not reported in the registry record. These omissions limit how fully the registered summary can be used to reconstruct the conduct and analysis of the trial.

4. Participant Flow and Accounting

The registry reports how participants moved through each stage of the study. All counts below are exactly as posted; missing counts are not inferred from other stages.

Overall Study

Analysis population for the primary outcome, as defined in the registry: Intention to treat.

5. Baseline Characteristics

The table reproduces the baseline characteristics posted in the registry, which describe the participants analysed at baseline and help show whether the randomized groups were comparable.

Population: Comparable.

6. Primary Endpoint

The registry lists the primary outcome as Medication Discrepancies, measured over 30 days after hospitalization.

The primary endpoint was the number of medication discrepancies identified 30 days after hospitalization. A discrepancy was defined as an unintentional difference between the medication a patient reported taking and the medication recorded in the medical record. The endpoint was assessed using a telephone interview and a review of the medical record, and the registry identifies the population as intention to treat.

This measure was relevant to the trial’s focus because medication lists can differ across the transition from hospital to home, and reconciliation seeks to identify and resolve such differences. Counting discrepancies provides a way to compare the recorded burden of mismatches between groups at the specified follow-up time. The endpoint is not itself a measure of adverse drug events, emergency visits, readmissions, or the quality of communication; those outcomes should not be inferred from the discrepancy count.

The registered summary treats the outcome as continuous and reports group means and standard deviations. A mean describes the average count in a group, while a standard deviation describes how dispersed individual counts were around that group average. The summary does not report additional detail about how discrepancies were classified, whether assessors were blinded, or how missing endpoint observations were handled.

7. Results Posted in the Registry

• SMMRT: mean 4.40 versus 6.40 on control; difference -2.00 (95% CI -3.00 to -1.00), standardized effect d = -0.50; z-test p = <0.001.

8. Secondary Efficacy Endpoints

The registry also posts results for secondary outcomes. For each outcome below, the endpoint type, the test applied to the posted summary statistics, the result and its interpretation are given in turn. Secondary endpoints are usually not adjusted for multiple testing, so they support rather than confirm the primary finding.

Hospital Utilization

• Endpoint type: Binary: the number of participants with the event, compared as proportions.
• Test: Unadjusted two-proportion z-test, with a Wald 95% confidence interval for the risk difference.
• Result:
  • SMMRT: 29.7% (35/118) versus 33.6% (41/122) on control; risk difference -3.9 percentage points (95% CI -15.7 to 7.8), risk ratio 0.88 (0.61–1.28); unadjusted two-proportion z-test p = 0.511.
• Interpretation:
  • The 95% confidence interval includes no difference, so these data do not show a statistically significant difference between SMMRT and the control. The interval gives the range of effects that remain compatible with the data.

The registry reports one secondary endpoint, hospital utilization within 30 days. Its unadjusted point estimate is in the direction of lower utilization in the SMMRT group, but the confidence interval includes no difference and the unadjusted p-value is 0.511. The overall pattern is therefore not as conclusive as the primary discrepancy comparison. The two endpoints address different outcomes: a medication-list discrepancy count and hospital utilization should not be treated as interchangeable measures.

The secondary result neither establishes a change in utilization nor rules out potentially meaningful differences. Its interval is compatible with values on both sides of no difference. The registered summary gives a useful indication of the direction and uncertainty of the comparison, but it does not report a formal analysis plan or adjusted analysis for this endpoint.

Unadjusted secondary findings generally carry less weight when several outcomes or comparisons are considered, because repeated testing can increase the chance of a chance finding. Here, the registry record lists two outcome measures overall, but it does not report a multiplicity strategy, a hierarchy of outcome testing, or formal analyses. The single posted secondary comparison should therefore be interpreted as descriptive and exploratory rather than as a confirmatory conclusion. It also cannot serve as a substitute for the primary endpoint: the secondary measure does not explain why the discrepancy averages differed, nor does the primary result establish a hospital-utilization effect.

Secondary outcomes registered for this trial, for reference (only those above were recalculated):

• Hospital Utilization

9. Statistical Interpretation

10. Statistical Methods Explained

What does a mean difference represent?

In general, a mean difference subtracts one group average from another, using the stated group order. Here, the posted contrast is SMMRT minus usual care: -2.0 medication discrepancies. The sign gives the direction of the contrast, while the size describes the gap between the two reported averages on the endpoint’s scale. This is a difference in average counts, not a relative percentage change and not a statement that every participant had fewer discrepancies.

The registry gives a standard deviation of 4.6 for usual care and 3.2 for SMMRT. In general, standard deviations describe the spread of individual observations around a group mean. They do not describe uncertainty in the mean difference by themselves. The mean difference’s confidence interval addresses uncertainty in the group contrast, subject to the assumptions behind its calculation.

How should the confidence interval and p-value be read?

In general, a confidence interval gives a range of estimates reasonably compatible with the data under the method used. The primary interval runs from -3.0 to -1.0 discrepancies, so it does not include a difference of zero. The reported unadjusted p-value is p < 0.001. A p-value measures how unusual data at least as incompatible with a zero-difference model would be under the specified testing assumptions; it is not the probability that the null hypothesis is true, nor does it measure the importance of the difference.

The registry record does not report the trial’s full statistical plan or formal analysis output. The comparison is an unadjusted recalculation based on the posted group summaries. Such a comparison does not account for covariates or other adjustments that might have been prespecified. The record does not report whether any adjusted analysis was conducted.

Why does the endpoint type matter?

In general, the statistical summary should match the kind of outcome being measured. Medication discrepancies are reported as a continuous measure summarized by means and standard deviations, even though each participant’s discrepancy count consists of discrete events. The registered comparison focuses on group averages, which makes the mean difference and its interval directly interpretable in units of discrepancies. The record does not report distributional diagnostics or alternative analyses of the count outcome.

Hospital utilization is a binary outcome in the secondary results: a participant either contributed to the reported count or did not. Its contrast is expressed as a risk difference, the difference between the proportions in the two groups. The interval for this secondary contrast includes zero. General statistical interpretation therefore differs from that of the primary mean difference, and each estimate needs to be read in the units appropriate to its outcome.

What does over-dispersion mean in general?

For count outcomes, over-dispersion means that the observed variability is greater than expected under a simple count model. When it is present and ignored, uncertainty estimates can be too small. Whether over-dispersion is relevant depends on the model and observed distribution; the registry record does not report a count model, distributional checks, or an over-dispersion assessment. The reported mean-and-standard-deviation summaries do not establish whether a particular model would be suitable.

Likewise, masking can affect measurement and assessment in some settings, but this trial is listed as having no masking. The registry does not report whether interviewers or medical-record reviewers were otherwise kept unaware of group assignment. General concerns about ascertainment should not be treated as proof that the reported discrepancy count was biased; they identify information that is not available in the registry record.

11. Confidence Intervals and Precision

The primary mean difference is -2.0 discrepancies, with a 95% confidence interval from -3.0 to -1.0. The interval expresses uncertainty about the estimated group contrast: values throughout that range are compatible with the posted comparison under its calculation assumptions. It excludes zero, meaning the interval does not include an average difference of exactly zero. Its width also shows that the estimate is not exact; the data remain compatible with a range of possible group differences.

The interval is narrower than the corresponding range for the secondary hospital-utilization risk difference, which extends from -0.157 to 0.078 and includes zero. That secondary estimate therefore allows values on either side of no difference. This is a statement about statistical precision, not about whether one endpoint is more important than another.

Precision depends on factors such as the amount of outcome variability, the number of observations, and the analysis method. The primary group standard deviations were 4.6 and 3.2, and the reported group sizes were 122 and 118. The registry record does not report missing-data handling or other analysis details that could affect uncertainty. The intervals and p-values in the posted comparisons are unadjusted recalculations from summary statistics; the registry record does not report formal sponsor analyses. They should be interpreted with that limitation in mind.

12. Planning a Similar Trial: Sample Size

This is a planning illustration, not a re-evaluation of the original trial. It asks: if a new study wanted to detect a difference like the one observed, or a smaller one, how large would it need to be? It is deliberately not “observed power” of the finished trial, which is not a meaningful quantity.

The planning figures are illustrations based on the observed effect, not the trial’s original design assumptions. The original sample-size justification is not reported in the registry record. For the continuous primary outcome, the figures are participants per arm. They are not total participants across both arms and are not event counts. The planning assumptions are a two-sided significance level of 0.05, equal allocation between two arms, and a normal approximation. The target effect is expressed as a stated fraction of the observed effect.

Read the 80% power column as the estimated number of participants per arm needed for an 80% chance of detecting the specified effect under those planning assumptions, if that effect were the true effect and the assumptions held. Read the 90% power column in the same way, but for a 90% chance. These are design-planning quantities, not guarantees about what a future study will find. The row for 100% uses the full observed effect as the planning target; the 75% and 50% rows use smaller fractions of that target.

Smaller effects generally require more participants to detect with the same power and significance level. The planning curve expresses this same relationship across effect sizes: moving toward a smaller target effect corresponds to a larger participant requirement. For the binary secondary outcome, the stated planning convention also reports participants per arm, not total events. The figures are planning illustrations and are not observed power.

13. How the Sample Size Is Derived

Power is the probability that a study of a given size rejects the null hypothesis when a specific true effect exists. It depends on four things: the effect to detect, the variability of the outcome, the significance level (here a two-sided α of 0.05, so z_1−α/2 = 1.960) and the desired power (z_1−β = 0.842 for 80% and 1.282 for 90%).

Formula

n is the number of participants in each arm, Δ is the difference in means to detect and σ is the common standard deviation.

Worked calculation for this trial

1. Observed difference Δ = -2.000; pooled standard deviation σ = 3.974.
2. n = 2 × (1.96 + 0.842)² × 3.974² / -2.000² = 62.0, rounded up to 62 participants per arm for 80% power.

The smaller-effect rows in the table above repeat the same calculation with the effect multiplied by 0.75 and 0.5 (for ratios, applied to the logarithm of the ratio).

R code

alpha <- 0.05
z_a   <- qnorm(1 - alpha / 2)          # two-sided
delta <- -2                      # observed difference in means
sigma <- 3.97388534131048                     # pooled standard deviation
n_per_arm <- function(power, frac = 1) {
  ceiling(round(2 * (z_a + qnorm(power))^2 * sigma^2 / (delta * frac)^2, 6))
}
for (f in c(1, 0.75, 0.5))
  cat(sprintf("%3.0f%% of observed effect: n per arm = %s (80%% power), %s (90%% power)\n", 100 * f,
              format(n_per_arm(0.80, f), big.mark = ","), format(n_per_arm(0.90, f), big.mark = ",")))

# Check against R's built-in function (t-based, so very slightly larger)
cat("power.t.test:", round(power.t.test(delta = delta, sd = sigma, power = 0.80)$n, 1), "\n")

# Simulation: exact two-sample t-test using simulated means and variances
set.seed(2482025)
n    <- n_per_arm(0.80)
reps <- 20000
m_diff <- rnorm(reps, delta, sigma * sqrt(2 / n))             # difference in sample means
s2     <- sigma^2 * rchisq(reps, 2 * (n - 1)) / (2 * (n - 1))   # pooled sample variance
tstat  <- m_diff / sqrt(s2 * 2 / n)
cat(sprintf("Simulated power at n = %s per arm: %.3f (%d replicates)\n", format(n, big.mark = ","), mean(abs(tstat) > qt(0.975, 2 * (n - 1))), reps))

Output from running the code

100% of observed effect: n per arm = 62 (80% power), 83 (90% power)
 75% of observed effect: n per arm = 111 (80% power), 148 (90% power)
 50% of observed effect: n per arm = 248 (80% power), 332 (90% power)
power.t.test: 62.9 
Simulated power at n = 62 per arm: 0.792 (20000 replicates)

The simulation draws many sets of group means and variances at the planned sample size and counts how often the two-sample t-test rejects the null. It should land close to the target power; at very small required sample sizes it can fall below it, because the normal approximation is rough there.

14. Multiplicity, Missing Data and Analysis Populations

The registry record describes more than one objective and reports a primary outcome alongside a secondary hospital-utilization outcome. It does not report whether the investigators prespecified an approach to multiplicity, such as prioritizing tests or adjusting for multiple comparisons. Without that information, the status of the secondary finding within the trial’s planned testing strategy is unclear. The primary and secondary outcomes should be considered separately, and the unadjusted secondary result should be interpreted cautiously.

The record reports zero analyses. It does not report a statistical analysis plan, model specifications, covariate adjustments, sensitivity analyses, or a formal sponsor analysis. The posted comparisons are unadjusted recalculations from the summary statistics, so they should not be mistaken for a complete account of the investigators’ intended or conducted analyses.

The primary outcome population is labeled intention to treat, but details about the analysis population are not reported in the registry record. The record does not state how many participants had missing outcome assessments, whether all randomized participants contributed data, or how missing data were handled. Reasons for missingness, withdrawals, exclusions after randomization, and any imputation method are also not reported in the registry record. These details matter because missing outcomes can affect both the estimated group contrast and its uncertainty, especially if missingness differs between groups or relates to the outcome.

It is also not reported in the registry record whether the endpoint assessment was blinded, whether additional outcome definitions were prespecified, or whether analyses departed from the original plan. These gaps do not show that the analysis was inappropriate; they limit what can be concluded from the registry summary alone.

15. Safety Results

The registry reports serious adverse events as affected participants divided by participants at risk.

Deaths

Other (non-serious) adverse events

Participants with at least one non-serious event at or above the registry's 0% reporting threshold.

The registry reports zero serious adverse events, zero other adverse events, and zero deaths in each group. The corresponding numbers at risk were 122 in usual care and 118 in SMMRT for both serious and other adverse events. No serious adverse-event terms are listed.

These entries describe what was recorded in the registry; they do not establish that no adverse events occurred outside the reporting process or that the intervention has no safety risks. The registry record does not report details about how adverse events were collected, the period of safety surveillance, event definitions, or whether ascertainment was systematic. The group counts should therefore be read as registry-reported figures, not as proof of the absence of harm.

16. What the Result Does, and Does Not, Mean

The primary result indicates that the reported average discrepancy count at 30 days was lower in the SMMRT group than in the usual-care group. The confidence interval does not include a zero difference, and the unadjusted p-value is below 0.001. This describes the registered comparison of average medication discrepancies for the groups and follow-up period reported.

It does not establish that every participant in the SMMRT group had fewer discrepancies, or that the difference led to fewer adverse drug events, emergency visits, or readmissions. Those outcomes cannot be inferred from the primary endpoint. The secondary hospital-utilization result is uncertain, with an interval that includes no difference; it does not provide conclusive evidence of a utilization difference.

The result also does not resolve all questions about implementation or generalizability. The registry describes Veterans and VA Boston usual care, and the intervention used a VA patient portal and secure messaging. The record does not report participant eligibility details, uptake of the portal, intervention adherence, missing-data handling, or adjustment methods. The posted summary should be understood in that context rather than as a finding that applies automatically to other settings or populations.

Randomization is an important design feature for comparing groups, but the registry summary does not provide enough information to assess every aspect of trial conduct and analysis. The results describe registered group-level outcomes; they are not medical advice and do not determine what approach is appropriate for an individual patient.

17. Why This Trial Matters Statistically

This trial addresses a statistical problem that occurs at transitions of care: medication lists in records and patients’ accounts may not match. The primary endpoint translates that problem into a count that can be compared between randomized groups. The reported mean difference and confidence interval show both the direction of the comparison and the uncertainty around its size, rather than relying on a p-value alone.

The registered outcomes also illustrate why a result for one endpoint should not be generalized to another. The primary discrepancy measure and the secondary hospital-utilization measure are distinct, and the latter has greater uncertainty in the posted comparison. Statistical interpretation requires attention to the scale, endpoint definition, and interval for each outcome.

At the same time, the registry record leaves important analysis details unreported. Missing-data procedures, the multiplicity approach, and formal analysis results are not available in the record. Distinguishing the reported summary comparison from a full prespecified analysis is important for clinical researchers assessing evidence and planning future studies. The sample-size illustrations can inform planning under their stated assumptions, but they do not recover the trial’s original design rationale.

18. Limitations

• The posted summary statistics are the only data these results rest on; patient-level data, full protocols and statistical analysis plans are not available.
• Recalculated results are simple, unadjusted calculations from posted summaries. They do not replace the sponsor's prespecified analysis.

Several trial-specific limitations follow from the information absent in the registry record. The intervention and comparator are described, but details about eligibility, recruitment, baseline characteristics, intervention adherence, and exposure to secure messaging are not reported in the registry record. Without those details, it is difficult to assess how representative the enrolled group was or how consistently the intervention was delivered.

The trial is listed as unmasked, and the record does not report whether outcome assessors were blinded to group assignment. The primary outcome relied on telephone interviews and medical-record review, but assessor procedures and discrepancy adjudication are not described. The record also does not report the number of participants with missing outcome data or how such data were handled.

The posted outcome results include usual care and SMMRT, while the objectives describe an additional training-only comparison. A result for that separate group is not reported in the registry record. The registered summary also lacks formal analysis output and a stated multiplicity strategy. Finally, the reported outcomes are limited to medication discrepancies and hospital utilization in the specified follow-up periods; other consequences of medication reconciliation cannot be inferred from these results.

19. Related Tutorials

20. Related Statistical Calculators

21. Sources

• ClinicalTrials.gov: The Secure Messaging for Medication Reconciliation Tool (SMMRT) Trial — NCT02482025.
• Registry record last updated 2020-11-19; status last verified 2020-10.
• Results first posted 2020-11-19.
• Data retrieved from ClinicalTrials.gov on 2026-11-09.
• Group sizes, means, event counts and sponsor-reported p-values are reproduced as posted. Effect estimates, confidence intervals and p-values not attributed to the sponsor are independently calculated from the posted summary statistics.

22. Record Summary

The Secure Messaging for Medication Reconciliation Tool (SMMRT) Trial was a completed randomized, parallel-group study with no masking and enrollment of 240. It compared usual care with usual care plus SMMRT. The primary outcome was medication discrepancies measured 30 days after hospitalization in an intention-to-treat population.

The registry reports a lower mean discrepancy count for the SMMRT group, with an unadjusted mean difference of -2.0 and a 95% confidence interval from -3.0 to -1.0. The secondary hospital-utilization comparison is uncertain. The record reports no adverse events or deaths in either group, while details about analysis methods, missing data, and the trial’s original design assumptions are not reported in the registry record.