Email Campaign to Increase Uptake of myGeisinger: Complete Statistical Analysis

1. Trial at a Glance

This page covers the registry results for Email Campaign to Increase Uptake of myGeisinger (NCT04332614), a registered phase not stated study. The primary outcome is Email Opened: Control vs. Less-is-better vs. Social Proof vs. Endowment.

2. Clinical Question

The study, titled “Email Campaign to Increase Uptake of myGeisinger,” was designed to test email messages intended to promote enrollment in Geisinger’s patient portal. The primary outcome was email opening, rather than portal enrollment itself. The registry also lists secondary comparisons of email opening, link clicking, and enrollment across message groups, including comparisons among the three focused-benefit message variants.

The population definition matters to the question: it includes patients whose emails were delivered and who had an electronic health record entry 10 days post-intervention. It is therefore narrower than everyone who might have been sent a message. The registry record does not report how many people met this definition in each group or the outcome results. It consequently does not establish whether any message was associated with more opens, clicks, or enrollments.

3. Trial Design

This was a behavioral email-message study sponsored by Geisinger Clinic. The registry lists an enrollment of 14099 and describes the study as completed. The recorded start date is 2020-04-06, and the primary completion date is 2020-04-13. The study’s stated purpose was to test different email messages to determine the most effective way of promoting enrollment in myGeisinger, Geisinger’s patient portal.

The control message was a standard marketing message encouraging activation and describing portal benefits. Other descriptions include simple messages focused on communicating with providers, scheduling and managing appointments online, or accessing medical information such as test results. The social proof message added information about how many other patients were using myGeisinger. The endowment and decision-staging message framed the account as something patients already had to activate, with the process broken into stages.

The registry lists the primary outcome as whether an email was opened, assessed 8-10 days post-intervention. Its defined population was patients whose email was delivered and who had an electronic health record record 10 days post-intervention. The registry also lists secondary outcomes involving email opens, link clicks, and enrollment. It records 12 outcome measures and 36 analyses.

Allocation is not stated in the registry record. The record therefore does not establish random assignment, concealment, or a like-for-like control comparison. Further details of the study protocol, including how messages were assigned and whether analyses adjusted for participant characteristics, are not reported in the registry record. These omissions limit what can be concluded about the comparability of groups and the interpretation of differences between them.

4. Participant Flow and Accounting

The registry reports how participants moved through each stage of the study. All counts below are exactly as posted; missing counts are not inferred from other stages.

~1 Week After Study Launch

~1 Month After Study Launch

Analysis population for the primary outcome, as defined in the registry: Patients who were delivered an email (i.e. email did not bounce back) and for whom there was a record in the EHR 10 days post-intervention.

5. Baseline Characteristics

The table reproduces the baseline characteristics posted in the registry, which describe the participants analysed at baseline and help show whether the randomized groups were comparable.

Population: Investigators did not have access to baseline demographic information for study participants.

Only the first 6 of 7 groups are shown.

6. Primary Endpoint

The registry lists the primary outcome as Email Opened: Control vs. Less-is-better vs. Social Proof vs. Endowment, measured over 8-10 days post-intervention.

The primary endpoint was whether an email was opened, recorded as a yes-or-no outcome. The registry identifies participants as the unit and specifies an assessment timeframe of 8-10 days post-intervention. The eligible population was patients whose email was delivered and who had a record in the electronic health record 10 days post-intervention.

Email opening is a proximal measure of response to a message: it indicates that an email was opened, but it does not by itself show that the recipient clicked a link, completed portal activation, or enrolled in myGeisinger. Those are distinct behaviors. The registry lists link clicking and enrollment among its secondary outcomes, but results for those outcomes are not reported in the registry record.

Using an outcome close to the message itself can help assess whether different communications attract attention. It is not equivalent to measuring the broader goal of portal enrollment. The population definition also shapes the interpretation: recipients whose email bounced back, or who did not have the specified electronic health record record, are not included in the primary population as defined. Counts for that population and the number of email opens in each group are not reported in the registry record.

7. Why No Re-analysis Is Shown

8. Statistical Interpretation

9. Statistical Methods Explained

The primary outcome is binary: each eligible participant is classified according to whether the email was opened. A basic description of a binary outcome is the number of participants with the event and the number without it. For a comparison between groups, analysts may report the event proportion in each group and a measure of the difference or relative contrast, together with uncertainty. The registry record does not report these outcome counts or effect estimates, so no such comparison can be interpreted for this study.

What does a confidence interval express?

In general, a confidence interval describes the uncertainty around an estimated effect under the assumptions of the analysis. A narrow interval indicates greater precision than a wide interval; an interval that includes values compatible with no difference does not, on its own, prove that groups are equivalent. Interpretation depends on the estimated effect, the interval, the outcome definition, and the study design. Confidence intervals for this study are not reported in the registry record.

What can a significance test tell us?

In general, a significance test evaluates how compatible the observed data are with a specified null model. A p-value is not the probability that the null hypothesis is true, nor does it measure the size or practical importance of an effect. The registry record does not report p-values or the statistical tests used. It is therefore not possible to characterize any comparison as statistically significant or not statistically significant.

Why does allocation matter?

Comparisons between groups are easiest to interpret causally when group assignment creates comparable groups and the analysis follows a prespecified plan. The registry record does not state the allocation method, and it cautions that the arms are not a like-for-like control comparison. Without further design information, a difference between groups, if one were reported, could reflect differences in who was included in each group as well as differences in the message. Adjustment for measured characteristics can sometimes address some imbalance, but it cannot guarantee that unmeasured differences have been removed. Whether any adjustment was used is not reported in the registry record.

How should the planning illustration be read?

In general, sample-size planning asks how many participants would be needed to detect an effect under stated assumptions. The assumptions described for the planning illustration are a two-sided significance level of 0.05, equal allocation between two arms, and a normal approximation. The target effect is expressed as a stated fraction of the observed effect. The illustration is based on the observed effect rather than being the trial’s own design calculation. It should not be read as observed power: observed power is not a substitute for an effect estimate and its uncertainty. No planning values or observed effect are reported in the registry record, so their numerical implications cannot be assessed.

What do the listed comparisons imply?

The registry lists 12 outcome measures and 36 analyses. The primary comparison concerns email opening across control, less-is-better, social proof, and endowment messages. Secondary comparisons address email opening, link clicking, and enrollment, including comparisons among the focused-benefit versions. Multiple comparisons are not inherently inappropriate, but they make it important to distinguish prespecified primary questions from exploratory analyses and to consider the possibility of chance-positive findings. The registry record does not report how the analysis plan defined that distinction or whether multiplicity was handled.

The outcome is a count of participants with a binary event, not a continuous measurement. General statistical models for binary outcomes can express differences in event probability or other contrasts, but the appropriate method depends on the design, estimand, and data structure. The registry record does not report the model, covariates, handling of clustering, or other analytic details. No particular model should therefore be attributed to this study.

10. Exact Estimates for Each Arm

No two-arm comparison is shown for this trial, but each arm's result can still be summarized with exact interval estimates computed from the posted counts. The Clopper-Pearson interval is exact and conservative; the Wilson interval is a common closer-to-nominal alternative; the Bayesian credible interval uses a uniform Beta(1, 1) prior. For small groups the three can differ noticeably.

R code and output

arm    <- c("Control", "Less-is-better", "Social Proof", "Endowment / Decision Staging")
n      <- c(2117, 6384, 2127, 2115)
events <- c(558, 1528, 375, 488)
for (i in seq_along(n)) {
  ci <- binom.test(events[i], n[i])$conf.int          # Clopper-Pearson exact interval
  cr <- qbeta(c(0.025, 0.975), 1 + events[i], 1 + n[i] - events[i])   # Bayesian, uniform Beta(1,1) prior
  wi <- (function(x, n) { p <- x / n; z <- qnorm(0.975); d <- 1 + z^2 / n
    c((p + z^2 / (2 * n) - z * sqrt(p * (1 - p) / n + z^2 / (4 * n^2))) / d, (p + z^2 / (2 * n) + z * sqrt(p * (1 - p) / n + z^2 / (4 * n^2))) / d) })(events[i], n[i])
  cat(sprintf("ARM|%s|%d|%d|%.1f|%.1f|%.1f|%.1f|%.1f|%.1f|%.1f\n", arm[i], n[i], events[i], 100 * events[i] / n[i],
              100 * ci[1], 100 * ci[2], 100 * wi[1], 100 * wi[2], 100 * cr[1], 100 * cr[2]))
}

ARM  Control  2117  558  26.4  24.5  28.3  24.5  28.3  24.5  28.3
ARM  Less-is-better  6384  1528  23.9  22.9  25.0  22.9  25.0  22.9  25.0
ARM  Social Proof  2127  375  17.6  16.0  19.3  16.1  19.3  16.1  19.3
ARM  Endowment / Decision Staging  2115  488  23.1  21.3  24.9  21.3  24.9  21.3  24.9

11. Multiplicity, Missing Data and Analysis Populations

The registry lists 12 outcome measures and 36 analyses, spanning email opening, link clicking, and enrollment. It also lists comparisons across the main message groups and comparisons among the focused-benefit message variants. With several outcomes and comparisons, there are multiple opportunities to observe an apparently notable result by chance. General approaches include prespecifying a primary comparison, adjusting for a family of tests, or clearly labeling secondary analyses as exploratory. Which approach was used is not reported in the registry record.

The primary population excludes emails that bounced back and is restricted to patients with an electronic health record record 10 days post-intervention. This definition helps specify who contributes to the primary outcome, but the registry record does not report group-specific numbers of delivered emails, eligible patients, or outcome observations. It also does not explain how missing outcome information or patients without the required record were handled. The extent and consequences of missing data therefore cannot be assessed.

The record does not report an analysis population such as intention-to-treat or per-protocol, nor does it explain whether participants were analyzed according to an assigned message or a message received. Because allocation is not stated, the relationship between the target population and the analyzed groups is also unclear. These omissions matter when interpreting an estimate: the population included, exclusions, and missing-data handling help define what the result represents.

The statistical analysis plan, including any prespecified hierarchy of outcomes, missing-data methods, and multiplicity procedure, is not reported in the registry record. The listed number of outcomes and analyses alone does not establish that a particular correction was required or omitted; it does make the unreported analysis details important for judging the strength of any eventual findings.

12. What the Result Does, and Does Not, Mean

The record establishes that this study examined email messages intended to encourage use of myGeisinger and that email opening was the primary outcome. It describes a control message and alternative messages focused on particular benefits, social proof, or endowment and decision staging. It also identifies secondary outcomes involving link clicks and enrollment. These are useful details for understanding what the study set out to assess.

The record does not report results for the primary or secondary outcomes. It therefore does not establish which message was associated with more email opens, clicks, or enrollments, nor does it quantify the precision of any comparison. The outcome counts, estimates, confidence intervals, and p-values are not reported in the registry record. No claim about the direction, size, or statistical certainty of a message effect can be made from these details alone.

Even if email-opening results were available, an open would describe a limited step in the path toward portal use. It would not establish that a recipient clicked through, completed enrollment, or used the portal. The registry’s secondary outcome definitions recognize some of these distinct steps, but their results are not reported.

The completed status indicates that the study is recorded as completed; it does not supply an outcome finding. Similarly, the listed enrollment of 14099 does not reveal the number included in the primary outcome population or the number assigned to each message group. The registry does not state the allocation method, and it cautions that the arms are not a like-for-like control comparison. These limits mean the record supports a description of the study question and its measurement, not a conclusion that one email strategy was more effective than another.

13. Why This Trial Matters Statistically

This study illustrates how a behavioral intervention can be evaluated using a measurable action close to the intervention itself. Email opening is relatively direct to define as a binary outcome, while link clicking and enrollment represent different downstream actions. Keeping those outcomes distinct is important: a message may prompt one action without necessarily prompting the next. The registry lists all three types of outcomes, but does not report their results.

It also illustrates why design information is essential to statistical interpretation. A comparison of message groups is not automatically a causal comparison. The registry does not state how participants were allocated and explicitly cautions that the arms are not a like-for-like control comparison. A statistical contrast cannot resolve questions about group comparability that depend on the design and data collection.

The number of listed outcomes and analyses highlights another recurring issue in clinical and behavioral research: interpretation depends on which comparisons were planned, how they relate to the primary question, and how multiplicity was addressed. The registry record does not report these analysis details. Statistical transparency is therefore necessary not only for evaluating numerical estimates, but also for understanding what those estimates were intended to test.

Finally, the record shows the difference between a registered study description and a report of results. Enrollment, timing, status, intervention descriptions, and endpoint definitions do not by themselves indicate the observed effect. Without outcome estimates and uncertainty, readers cannot judge the magnitude or precision of a comparison. The study is statistically informative as an example of a defined behavioral question, but the registry information recorded is insufficient to establish an outcome conclusion.

14. Limitations

• The posted summary statistics are the only data these results rest on; patient-level data, full protocols and statistical analysis plans are not available.
• Recalculated results are simple, unadjusted calculations from posted summaries. They do not replace the sponsor's prespecified analysis.

A central limitation is that the registry record does not report results for the primary outcome or the listed secondary outcomes. Without counts and estimates, it is impossible to describe observed differences or their precision. The record also does not report confidence intervals, p-values, or the analysis methods used.

Allocation is not stated, and the registry cautions that the arms are not a like-for-like control comparison. This limits causal interpretation and prevents an assessment of whether groups were comparable at baseline. Details about recruitment, message assignment, adherence, or participant characteristics are not reported in the registry record.

The primary endpoint captures email opening, not completion of portal enrollment. Although clicking and enrollment are listed as secondary outcomes, their results are unavailable in the registry record. The endpoint population is also restricted to delivered emails and people with an electronic health record record 10 days post-intervention. Counts excluded by these conditions, and the consequences of those exclusions, are not reported.

The registry lists 12 outcome measures and 36 analyses, but does not report the analysis plan, outcome hierarchy, or multiplicity handling. It also does not describe missing-data procedures or the analysis population. These omissions restrict assessment of selective reporting, robustness, and the relevance of any estimate to the intended population.

The study is recorded as completed, but completion does not resolve the missing-result and design details. The record lists zero serious and other adverse events in each group, with no adverse-event terms. These entries do not provide a detailed account of event ascertainment or establish the completeness of safety reporting.

15. Related Tutorials

16. Related Statistical Calculators

17. Sources

• ClinicalTrials.gov: Email Campaign to Increase Uptake of myGeisinger — NCT04332614.
• Registry record last updated 2021-05-03; status last verified 2021-04.
• Results first posted 2021-05-03.
• Data retrieved from ClinicalTrials.gov on 2026-11-09.
• Group sizes, means, event counts and sponsor-reported p-values are reproduced as posted. Effect estimates, confidence intervals and p-values not attributed to the sponsor are independently calculated from the posted summary statistics.

18. Record Summary

Geisinger Clinic sponsored the completed study “Email Campaign to Increase Uptake of myGeisinger,” which examined messages encouraging patients to use the myGeisinger portal. The registry lists an enrollment of 14099, a start date of 2020-04-06, and a primary completion date of 2020-04-13.

The primary outcome was whether an eligible patient opened an email 8-10 days post-intervention. The specified population had a delivered email and an electronic health record record 10 days post-intervention. The control message was a standard marketing message; other messages emphasized provider communication, scheduling, medical information access, social proof, or endowment and decision staging.

The registry lists 12 outcome measures and 36 analyses, including outcomes for email opening, link clicking, and enrollment. Allocation is not stated, and the record cautions that the arms are not a like-for-like control comparison. Outcome results and statistical analysis details are not reported in the registry record.