COVID-19 Vaccination Take-Up: Complete Statistical Analysis

1. Trial at a Glance

This page covers the registry results for COVID-19 Vaccination Take-Up (NCT04867174), a randomized na study. The primary outcome is Rate of COVID-19 Vaccination at 1 Month.

2. Clinical Question

3. Trial Design

The registry describes a randomized, parallel-design study with no masking and phase listed as NA. The enrollment is listed as 2701. The study began on 2021-05-24, had a primary completion date of 2021-08-15, and is marked completed. The sponsor is the National Bureau of Economic Research, Inc.

The study focused on COVID-19 vaccination and vaccination-related behaviors in a county-run Medicaid managed-care population. The investigators partnered with Contra Costa Health Services, the department of health in Contra Costa County, California. The registry describes randomization to a control condition with no vaccination message or to one of three message conditions: an emotional message, a safety and effectiveness message, or a consequences message. The emotional condition used a California state video about getting back to normal; the safety and effectiveness condition used a video about vaccine safety and effectiveness; and the consequences condition used a video about consequences of not getting vaccinated.

The registry also lists financial incentives, a convenient scheduling link, and race- and gender-concordant approaches as behavioral interventions. The primary endpoint description says this was a cross-randomization design and refers to incentives of $10 or $50 and a convenient link. Its description ends before fully specifying the assignment structure. The exact combinations, allocation procedures for each component, and complete intervention content are not reported in the registry record. The record also does not report the randomization sequence method, concealment procedures, or any masking beyond stating that masking was none.

Randomization is intended to make assigned groups comparable on average, while the parallel design means participants are assigned to a study condition rather than described as switching between conditions. These are general features of randomized parallel studies; they do not establish that every implementation detail is documented or that the groups were identical in this study.

4. Participant Flow and Accounting

The registry reports how participants moved through each stage of the study. All counts below are exactly as posted; missing counts are not inferred from other stages.

Before assignment: This is a cross randomization design. Subjects were randomized equally to control (no message) or an emotional message, a safety/effectiveness message or a consequence message. Each message arm was further randomized to financial incentives \[50%\] vs. no financial incentives \[50%\], with 25% randomized to a $10 incentive and 25% to a $50 incentive Each arm further randomized to receive a convenient scheduling link highlighted \[50%\] vs. not \[50%\].

Only the first 6 of 60 groups are shown.

Overall Study

Analysis population for the primary outcome, as defined in the registry: This is a cross randomization design. Subjects were randomized equally to control (no message) or an emotional message, a safety/effectiveness message or a consequence message. Each of these arms was interacted with a financial incentive of $10 or $50 and, separately, with a convenient link to a vaccine appointment scheduling system. In practice, this means there are 60 arms. See protocol for details. Since it's not feasible to show all arms concisely, we report details by message arm.

5. Baseline Characteristics

The table reproduces the baseline characteristics posted in the registry, which describe the participants analysed at baseline and help show whether the randomized groups were comparable.

Population: This is a cross randomization design. Subjects were randomized equally to control (no message) or an emotional message, a safety/effectiveness message or a consequence message. Each of these arms was interacted with a financial incentive of $10 or $50 and, separately, with a convenient link to a vaccine appointment scheduling system. In practice, this means there are 60 arms. See protocol for details. Since it's not feasible to show all arms concisely, we report details by message arm.

Only the first 6 of 61 groups are shown.

6. Primary Endpoint

The registry lists the primary outcome as Rate of COVID-19 Vaccination at 1 Month, measured over 1 month.

The primary endpoint is titled “Rate of COVID-19 Vaccination at 1 Month.” It is defined as the percent of the population with at least one COVID-19 vaccination, and its timeframe is 1 month. The measure therefore focuses on whether vaccination had occurred by the stated follow-up point, rather than on a continuous quantity such as number of doses or time spent in care. The registry classifies it as a count of participants, consistent with recording how many people met the vaccination criterion.

As a general statistical concept, a binary endpoint represents whether each participant meets a defined condition. Summarizing such an endpoint as a group rate or proportion can make randomized groups comparable on a common outcome scale. The estimate is most informative when its definition, follow-up window, denominator, and handling of missing vaccination status are clear. The registry record gives the endpoint description and timeframe, but does not report the observed counts or rates, the denominators used in the primary analysis, or how missing status was handled.

Vaccination status is relevant to the study’s stated aim of testing ways to increase COVID-19 vaccine uptake. The record also refers more broadly to other preventive behaviors, but these are not part of the primary endpoint description. The registry record does not report the operational source or verification process used to determine vaccination status.

7. Why No Re-analysis Is Shown

8. Statistical Interpretation

9. Statistical Methods Explained

What does randomization contribute?

In general, randomization assigns participants to study conditions by chance. Its purpose is to reduce systematic differences between groups at the point of assignment, including differences that investigators may not have measured. This helps support a causal interpretation of a between-group contrast, provided the assignment is implemented as intended and the subsequent analysis is appropriate. Randomization does not guarantee that groups will be identical, and it does not remove concerns about missing outcomes, deviations from assignment, or measurement quality.

The registry labels this study randomized and parallel, with no masking. A parallel study compares assigned groups over the study period rather than describing participants as crossing from one condition to another. The record does not report the sequence-generation or allocation-concealment methods. Nor does it give enough detail to reconstruct every assignment combination across the message, incentive, and other behavioral components. Those implementation details are not reported in the registry record.

How should the vaccination endpoint be understood?

The primary endpoint is a binary outcome at the participant level: at least one COVID-19 vaccination by the 1-month timeframe. In general, a binary outcome can be summarized as the number meeting the criterion and the corresponding proportion among those assessed. A comparison of proportions describes how often the criterion was met in one group relative to another. The registry identifies the endpoint as a count of participants and describes it as a percent, but does not post the group counts, analysis denominators, or rates. No effect estimate can be reconstructed from the endpoint definition alone.

For a binary outcome, the choice of effect measure affects interpretation. A difference in proportions expresses an absolute contrast, whereas a relative measure expresses a contrast relative to a reference group. These are general concepts, not reported analyses for this study. The registry record does not identify the effect measure used or provide a primary statistical model, so neither measure should be attributed to the study’s analysis.

What do uncertainty estimates convey?

As a general explanation, a confidence interval conveys statistical uncertainty around an estimate under the assumptions of the analysis. A narrower interval indicates greater precision for that estimate; an interval spanning a null value may be compatible with a range of possible effects, including little or no difference. A p-value summarizes how unusual the observed data would be under a specified null model. It is not a measure of clinical importance and does not give the probability that the null hypothesis is true. The registry record does not report a confidence interval or p-value for the primary vaccination endpoint.

Why can multiple comparisons matter?

When a study examines several intervention components, outcomes, or contrasts, the chance of at least one apparently notable result can increase if many hypotheses are tested. This is the general multiplicity concern. Approaches to address it can include prespecifying a primary comparison, adjusting inference across a defined family of tests, or clearly distinguishing confirmatory from exploratory analyses. The record lists two outcome measures, but does not report a multiplicity strategy, a statistical analysis plan, or analysis results. It is not possible to determine from the registry record how multiple outcomes or intervention contrasts were handled.

What does over-dispersion mean?

Over-dispersion is a general concern for count models when observed variability exceeds what a chosen model assumes. It can lead to uncertainty estimates that are too small if the model does not account for the extra variability. The primary endpoint is described as a participant count and percent, rather than as a repeated event count. The registry lists no dispersion value, and the posted record does not identify a model requiring a dispersion adjustment. Over-dispersion should therefore not be assumed to have been tested or addressed.

How should planning calculations be read?

General sample-size calculations estimate the amount of information needed to detect an assumed effect under stated design and analysis conditions. The registry’s planning assumptions describe a two-sided significance level of 0.05, equal allocation between two arms, and a normal approximation, with the effect to detect expressed as a fraction of the observed effect. Such calculations are planning illustrations, not observed power and not evidence that the study had a particular probability of detecting an effect. The observed effect and resulting planning values are not reported in the registry record. The registry also does not report the trial’s own sample-size justification or statistical plan.

10. Multiplicity, Missing Data and Analysis Populations

The registry lists two outcome measures, while the primary endpoint described in detail is vaccination at 1 month. It lists zero analyses. The record does not report a statistical analysis plan, prespecified hierarchy of outcomes, multiplicity adjustment, or the set of contrasts treated as confirmatory. Since the intervention description includes different messages and behavioral components, the multiplicity implications would depend on which comparisons were planned and reported. Those details are not reported in the registry record.

Missing outcome data can affect a binary endpoint if vaccination status is unavailable for some participants or if availability differs across assigned groups. As a general matter, interpretation depends on the amount and pattern of missingness and on the method used to handle it. The registry record does not report missing vaccination-status counts, reasons for missingness, imputation procedures, or sensitivity analyses. It also does not specify the analysis population, such as whether analyses followed assignment regardless of intervention receipt.

The registry’s enrollment figure and the at-risk counts in the adverse-event listings do not establish the denominator used for the primary endpoint analysis. The record does not report participant flow for the outcome analysis, exclusions, adherence, or crossover information. Accordingly, the available record does not allow an assessment of whether the primary analysis included everyone as randomized, or whether missing data or multiple comparisons influenced the reported findings. No such influence should be presumed; the relevant procedures and results are simply not reported in the registry record.

11. Safety Results

The registry reports serious adverse events as affected participants divided by participants at risk.

Deaths

Other (non-serious) adverse events

Participants with at least one non-serious event at or above the registry's 0% reporting threshold.

The registry lists serious adverse events, other adverse events, and deaths by message group. For the Control Arm, Emotional Message, Safety and Effectiveness Message, and Consequences Message, the posted serious-event count is 0, the other-event count is 0, and deaths are listed as 0. The corresponding at-risk counts are 729, 681, 663, and 628, respectively. These are the registry’s group-level safety figures; they should not be treated as results for the vaccination endpoint.

As a general statistical point, a reported count of zero means that no events in that category are recorded in the displayed data. It does not by itself establish that the intervention is free of risk or that an event could not occur. Interpreting safety also depends on how events were defined, collected, monitored, and reported, as well as the duration and completeness of follow-up. Those details are not reported in the registry record. The figures therefore describe the recorded safety counts, rather than proving the absence of harms.

12. What the Result Does, and Does Not, Mean

The record describes a randomized study intended to test ways of increasing COVID-19 vaccination uptake in a county-run Medicaid managed-care population. Its primary endpoint focuses on whether participants had at least one COVID-19 vaccination at 1 month. The message conditions and listed behavioral approaches make the study relevant to questions about how communication and access-related interventions might be evaluated in this population.

However, the registry record does not report the primary endpoint results, effect estimates, uncertainty intervals, or analysis details. It therefore does not establish whether any message or behavioral component was associated with higher or lower vaccination uptake, how large any difference was, or how precisely it was estimated. The study hypothesis should not be mistaken for a finding. Nor does a completed study status imply that a particular intervention effect was demonstrated.

As a general principle, the interpretation of randomized evidence depends on the contrast being estimated, the outcome definition, the analysis population, and the handling of missing data and multiple comparisons. The available record identifies some design features and the endpoint, but leaves these analysis details unreported. It also does not provide enough information to assess whether findings would apply beyond the described county Medicaid managed-care population.

The safety counts are a separate part of the record and do not answer whether vaccination uptake changed. The posted zero counts should be read as recorded event counts, not as proof that no risk existed. Overall, the registry record documents the study question and some design and safety information, but does not contain the results needed to judge the comparative vaccination outcome.

13. Why This Trial Matters Statistically

This trial is statistically notable because its stated question concerns more than a single message-versus-control contrast. The record describes an emotional message, safety and effectiveness information, and a consequences message, alongside financial incentives and other behavioral approaches. Studies with multiple components can address practical questions about communication and access, but their interpretation depends on defining which component contrasts are being estimated and whether combinations or interactions are part of the analysis.

Random assignment is a useful design feature for comparing intervention groups because, in general, it reduces systematic confounding at assignment. Yet the strength of any resulting inference depends on the reported results and methods. Here, the registry record identifies randomization and the primary endpoint but does not provide the vaccination outcome estimates or the full analysis plan. This makes the distinction between a scientifically relevant study question and an evaluable reported result especially important.

The population also matters for interpretation. The study concerns a county-run Medicaid managed-care population, and its partnership with Contra Costa Health Services links the question to a specific public-health and care-delivery setting. Whether results generalize to other populations or settings would require outcome findings and information about the people actually analyzed; neither can be assessed from the registry record alone.

Statistically, the record illustrates why endpoint definitions and transparent reporting are essential. A clear endpoint can define what was intended to be measured, but without group-level results, uncertainty estimates, and analysis details, readers cannot judge the size or precision of an effect. The record establishes the study’s design intent, not the comparative result.

14. Limitations

• The posted summary statistics are the only data these results rest on; patient-level data, full protocols and statistical analysis plans are not available.
• Recalculated results are simple, unadjusted calculations from posted summaries. They do not replace the sponsor's prespecified analysis.

A central limitation of the available record is that it does not report primary endpoint results. The vaccination counts or rates by group, effect estimates, confidence intervals, and p-values are not reported in the registry record. This prevents assessment of the direction, magnitude, and precision of any comparison.

The intervention description is incomplete for reconstructing the cross-randomization. Although the record lists message groups, financial incentives of $10 or $50, a convenient scheduling link, and race- and gender-concordant approaches, it does not fully specify how these components were assigned or combined. The details of the comparison definitions and analyses are not reported in the registry record.

The primary endpoint is described as the percent vaccinated by 1 month, but the record does not explain how vaccination status was ascertained, what denominator was used in the analysis, or how missing status was handled. It also does not report participant flow, adherence, exclusions, or analysis-population definitions. These omissions limit assessment of outcome completeness and the relationship between assignment and analysis.

The registry lists two outcome measures but no analyses, and it does not report a statistical analysis plan or multiplicity strategy. The enrollment figure alone cannot establish the precision of the endpoint comparison or whether the study was sized for its primary analysis. The registry’s planning assumptions are not a substitute for the trial’s own design justification, which is not reported in the registry record.

Finally, the safety listings report event counts but do not provide details about event definitions, ascertainment, or monitoring. These record-specific omissions limit what can be concluded from the posted information; they do not show that the study did not collect the information elsewhere.

15. Related Tutorials

16. Related Statistical Calculators

17. Sources

• ClinicalTrials.gov: COVID-19 Vaccination Take-Up — NCT04867174.
• Registry record last updated 2024-03-28; status last verified 2024-03.
• Results first posted 2024-03-28.
• Data retrieved from ClinicalTrials.gov on 2026-11-09.
• Group sizes, means, event counts and sponsor-reported p-values are reproduced as posted. Effect estimates, confidence intervals and p-values not attributed to the sponsor are independently calculated from the posted summary statistics.

18. Record Summary

NCT04867174, titled “COVID-19 Vaccination Take-Up,” is a completed, randomized, parallel study with no masking and phase listed as NA. The registry lists enrollment as 2701, a start date of 2021-05-24, and a primary completion date of 2021-08-15. The sponsor is the National Bureau of Economic Research, Inc.

The study concerns vaccination uptake in a county-run Medicaid managed-care population and includes no-message control and three message groups. The registry also lists financial incentives, a convenient scheduling link, and race- and gender-concordant approaches. The primary endpoint is the rate of COVID-19 vaccination at 1 month, defined as the percent with at least one vaccination.

The registry record lists two outcome measures and zero analyses. It does not report primary vaccination results, statistical estimates, or the analysis plan. Safety listings show zero recorded serious adverse events, other adverse events, and deaths for each of the four named message groups.