Text and Talk: A Multi-level Intervention to Increase Provider HPV Vaccine Recommendation Effectiveness: Complete Statistical Analysis

1. Trial at a Glance

This page covers the registry results for Text and Talk: A Multi-level Intervention to Increase Provider HPV Vaccine Recommendation Effectiveness (NCT05006833), a randomized na study. The primary outcome is Initiation of the HPV Vaccine.

2. Clinical Question

3. Trial Design

The registry describes a randomized, single-masked, factorial study. Its brief summary says it tests two interventions: parent-targeted text messages and brief clinician-targeted training. The intervention descriptions distinguish bundled and benefits approaches for both the clinician training and the parent messages. The arm descriptions also include deferred clinician training and no intervention text messages, creating combinations that allow the posted results to compare different approaches with a control condition.

The control condition is deferred clinician training, with training after the study, and no parent intervention text messages. The remaining arms combine active clinician training approaches or deferred training with bundled messages, benefits messages, or no messages. The registry’s use of a factorial model identifies the study’s broad design structure; the specific randomization unit, allocation procedure, and any stratification are not reported in the registry record.

The primary endpoint is initiation of the HPV vaccine, assessed at 24 months. The registry says the participant-level outcome is whether a participant received one or more doses and that percentages are estimated using predicted probabilities adjusted for clinic-level clustering. The registry also lists clinician recommendation frequency at 12 months and parent text-message acceptability as secondary outcomes. Results for the acceptability outcome are not reported in the registry record.

The study is listed as completed. Enrollment is 8043, the sponsor is the University of Florida, and the record lists a start date of 2021-11-18 and a primary completion date of 2025-02-25. The masking field says single, but who was masked and how masking was implemented are not reported in the registry record. The record does not give the original sample-size justification or the planned statistical analysis in enough detail to assess those design choices.

4. Participant Flow and Accounting

The registry reports how participants moved through each stage of the study. All counts below are exactly as posted; missing counts are not inferred from other stages.

Before assignment: 30 clinics (and clinicians within clinics) were randomized equally to 3 clinician training arms: 10 clinics in clinician bundled training, 10 clinics in clinician benefits training, and 10 clinics to no training.

Only the first 6 of 21 groups are shown.

Overall Study

Reasons for not completing this period:

5. Baseline Characteristics

The table reproduces the baseline characteristics posted in the registry, which describe the participants analysed at baseline and help show whether the randomized groups were comparable.

Only the first 6 of 22 groups are shown.

6. Primary Endpoint

The registry lists the primary outcome as Initiation of the HPV Vaccine, measured over 24 months.

The primary endpoint is initiation of the HPV vaccine, measured as the percentage of participants who received one or more doses during the 24-month timeframe. It is a binary participant-level outcome: each participant either meets the definition of initiation or does not. The endpoint is not a measure of completing a vaccine series, the timing of each dose, or the number of doses received.

Using a percentage makes the outcome interpretable as the estimated frequency of initiation in each study arm. Comparing arm-specific percentages can describe whether the estimated probability differs between an intervention combination and the control condition. The registry states that the percentages use predicted probabilities adjusted for clustering at the clinic level. This matters because participants treated at the same clinic may have more similar outcomes than participants from different clinics. Accounting for that clustering can affect the estimated uncertainty and the resulting confidence intervals.

The registry’s primary endpoint is relevant to the stated study aim of increasing HPV vaccination among 11- to 12-year-olds in Florida because it captures whether vaccination was initiated. The endpoint alone does not establish why a participant did or did not initiate vaccination, whether the intervention changed clinician behavior, or whether a vaccine series was completed. Details of outcome ascertainment, the handling of missing endpoint observations, and the analysis population are not reported in the registry record.

7. Results Posted in the Registry

• Adolescent Arm 1: CL= Bundled Training and P=Bundled Text: 44.3% (341/769) versus 49.3% (400/812) on control; risk difference -4.9 percentage points (95% CI -9.8 to -0.0), risk ratio 0.90 (0.81–1.00); unadjusted two-proportion z-test p = 0.050.
• Adolescent Arm 2: CL= Bundled Training and P=Benefits Text: 48.2% (378/785) versus 49.3% (400/812) on control; risk difference -1.1 percentage points (95% CI -6.0 to 3.8), risk ratio 0.98 (0.88–1.08); unadjusted two-proportion z-test p = 0.658.
• Adolescent Arm 3: CL= Bundled Training and P=No Text: 44.4% (342/770) versus 49.3% (400/812) on control; risk difference -4.8 percentage points (95% CI -9.8 to 0.1), risk ratio 0.90 (0.81–1.00); unadjusted two-proportion z-test p = 0.054.
• Adolescent Arm 4: CL= Benefits Training and P=Bundled Text: 44.8% (269/600) versus 49.3% (400/812) on control; risk difference -4.4 percentage points (95% CI -9.7 to 0.8), risk ratio 0.91 (0.81–1.02); unadjusted two-proportion z-test p = 0.100.

8. Secondary Efficacy Endpoints

The registry also posts results for secondary outcomes. For each outcome below, the endpoint type, the test applied to the posted summary statistics, the result and its interpretation are given in turn. Secondary endpoints are usually not adjusted for multiple testing, so they support rather than confirm the primary finding.

Frequency of Clinician Recommendation

• Endpoint type: Continuous: a mean value per participant, compared as a difference in means.
• Test: Unadjusted two-sample z-test for the difference in means, using the posted standard deviations.
• Result:
  • Clinician Arm 2- Benefits: mean 89.80 versus 94.70 on control; difference -4.90 (95% CI -13.39 to 3.59), standardized effect d = -0.40; z-test p = 0.258.
  • Clinician Arm 3 - Deferred Training: mean 90.60 versus 94.70 on control; difference -4.10 (95% CI -11.72 to 3.52), standardized effect d = -0.35; z-test p = 0.291.
• Interpretation:
  • The 95% confidence interval includes no difference, so these data do not show a statistically significant difference between Clinician Arm 2- Benefits and the control. The interval gives the range of effects that remain compatible with the data.
  • The 95% confidence interval includes no difference, so these data do not show a statistically significant difference between Clinician Arm 3 - Deferred Training and the control. The interval gives the range of effects that remain compatible with the data.

The secondary outcomes offer context about clinician recommendation behavior and parent response to the text messages, but they do not replace the primary participant-level vaccination outcome. The posted clinician recommendation percentages are generally high across the clinician groups, while the comparison estimates have confidence intervals spanning zero. Their unadjusted p-values do not indicate clear differences in the listed pairwise comparisons. These outcomes are measured at a different level and timeframe from vaccine initiation, so they should not be read as direct evidence that the primary endpoint changed.

Parent Text Message Acceptability is named as a secondary outcome, but its results are not reported in the registry record. The record also does not give a result for every possible combination of clinician and parent intervention for the secondary clinician outcome. The absence of posted results is not evidence that an outcome had no effect or that it was not measured.

In general, secondary outcomes can help explore potential pathways or implementation issues, but they usually carry less confirmatory weight than the primary endpoint. Their interpretation can be limited by smaller effective sample sizes, multiple comparisons, and the possibility that they were not the main basis for the study’s design. The registry does not report whether secondary analyses were prespecified, how many tests were planned, or whether multiplicity was addressed. The secondary findings should therefore be treated as exploratory context and not as a substitute for the primary result.

Secondary outcomes registered for this trial, for reference (only those above were recalculated):

• Frequency of Clinician Recommendation
• Parent Text Message Acceptability

9. Statistical Interpretation

10. Statistical Methods Explained

The registry describes the study as randomized and factorial. A factorial design studies combinations of intervention components. In general, this can support questions about the separate contributions of components and whether their effects depend on each other. However, the posted primary results are arm-specific estimates and comparisons with the control arm. Main-effect estimates, interaction estimates, and an analysis that combines arms to answer component-level questions are not reported in the registry record. The design label alone does not establish which of those analyses were planned or carried out.

What does a risk difference describe?

For a binary outcome, a risk difference compares the estimated probability of the outcome in one group with the estimated probability in another. A value below zero means the first group’s estimated probability is lower than the reference group’s; a value above zero means it is higher. A value of zero represents no difference in the estimated probabilities. This is a general interpretation of the measure, not a claim about the cause of any observed difference. The posted comparisons use the control arm as the reference and report risk differences with confidence intervals and unadjusted p-values.

Why account for clinic clustering?

The primary outcome description says that percentages are estimated with predicted probabilities adjusted for clustering at the clinic level. In general, observations from the same clinic may be correlated because participants share a care setting or clinical processes. Treating all observations as independent when they are correlated can make uncertainty appear smaller than it is. A cluster-adjusted analysis aims to account for this dependence when estimating effects and uncertainty. The registry does not report the model family, the number of clinics, the method used to estimate uncertainty, or how the clinic-level adjustment was implemented.

The displayed primary estimates are percentages, while the comparison results express differences on the risk-difference scale. The registry describes the percentages as predicted probabilities, so these should not be assumed to be simple raw proportions. The specific model and any covariates used to generate them are not reported in the registry record. The reported values should therefore be interpreted as the registry’s adjusted estimates, without inferring model details that are absent.

What does a confidence interval add?

In general, a confidence interval communicates uncertainty around an estimate. A narrower interval indicates greater precision on the scale being measured, while a wider interval indicates less precision. For a risk difference, an interval that includes zero is compatible with no difference under the analysis used, as well as with values on either side of zero. An interval that does not include zero is evidence against a zero difference under that analysis, but does not resolve other concerns such as multiple comparisons, bias, or model assumptions. The interval does not give the probability that the true effect lies within its bounds.

How should p-values be read?

A p-value describes how compatible the observed data are with a specified null model, assuming the analysis assumptions hold. It is not the probability that the null hypothesis is true, nor does it measure the size or clinical importance of an effect. The listed comparison p-values are explicitly unadjusted. Since several intervention arms are compared with the control, an isolated p-value does not account for the full set of comparisons unless a suitable multiplicity procedure is applied. Such a procedure is not reported in the registry record.

The statistical plan, including the primary hypothesis test, covariate adjustment, missing-data approach, analysis population, and any planned sensitivity analyses, is not reported in the registry record. The registry lists no reported analyses field entries, even though arm-level results and comparison statistics are posted. This limits the ability to determine whether the displayed estimates are the prespecified primary analysis, a subset of planned results, or an analysis conducted under another specification.

11. Confidence Intervals and Precision

The 95% confidence intervals show substantial uncertainty in the estimated initiation percentages. For the control arm, the estimate is 49.2%, and the interval extends from 40.0% to 60.4%. The intervention-arm intervals are also broad. For example, the bundled clinician training and benefits text arm has an estimate of 48.1%, with an interval from 40.8% to 56.8%; the benefits clinician training and benefits text arm has an estimate of 43.1%, with an interval from 35.7% to 52.1%. These intervals indicate that the point estimates should not be treated as precise values for the underlying probabilities.

Precision is also limited for the between-arm differences. Several intervals include zero and allow both lower and higher estimated initiation in the intervention arm relative to control. For example, the bundled clinician training and no text comparison has an interval from -0.098 to 0.001. The interval for benefits clinician training with benefits text messages is entirely below zero, from -0.115 to -0.010, but it still spans a range of possible magnitudes. An interval’s position and width both matter: excluding zero does not mean the effect size is estimated precisely.

In general, confidence intervals summarize sampling uncertainty under the fitted analysis and its assumptions. They do not automatically account for bias, missing outcomes, model misspecification, or the consequences of testing multiple comparisons. The primary endpoint description notes adjustment for clinic-level clustering, but details needed to evaluate that adjustment are not reported in the registry record. Thus, the posted intervals help describe statistical precision, while leaving uncertainty about analysis choices and other sources of uncertainty.

The sample sizes differ across the listed arms, but they should not be added or treated as a total number with outcome data. Enrollment and arm-level sample sizes describe different record fields, and the registry does not explain the relationship between them. Further detail about participant flow, outcome availability, or exclusions is not reported in the registry record.

12. Planning a Similar Trial: Sample Size

This is a planning illustration, not a re-evaluation of the original trial. It asks: if a new study wanted to detect a difference like the one observed, or a smaller one, how large would it need to be? It is deliberately not “observed power” of the finished trial, which is not a meaningful quantity.

The sample-size illustrations translate observed effects into planning scenarios. For the primary binary outcome, the figures are participants per arm, not a total across arms or a total number of events. The illustrations use a two-sided significance level of 0.05, equal allocation between the two arms, and a normal approximation. They consider detecting 100%, 75%, or 50% of the observed effect. The original trial design assumptions and the registry’s own sample-size justification are not reported in the registry record.

The 80% power column gives the estimated participants per arm for a hypothetical future study to have an 80% chance of detecting the specified effect, if the planning assumptions and effect size are correct. The 90% power column describes the corresponding planning target for a 90% chance. These are planning probabilities under a model, not guarantees for a particular study. Smaller effects require more participants to distinguish them from random variation under otherwise similar assumptions; the illustrations reflect this by showing larger requirements for smaller fractions of the observed effect.

The values use the observed effect from the relevant comparison as their starting point, rather than the trial’s original design target. Consequently, figures based on a small observed effect can become very large, and estimates derived from noisy trial results can be unstable. These calculations are not observed power. They are hypothetical planning illustrations, and they do not establish what the completed trial was designed to detect.

For continuous outcomes, the stated convention is also participants per arm. The general note in the registry distinguishes time-to-event planning figures, which would be total events, from the other planning figures, which are participants per arm. No time-to-event sample-size illustration is described in the posted results. The illustrations assume equal allocation and do not describe adjustment for clinic clustering, factorial structure, or multiplicity; whether those features were incorporated in the trial’s original planning is not reported in the registry record.

13. How the Sample Size Is Derived

Power is the probability that a study of a given size rejects the null hypothesis when a specific true effect exists. It depends on four things: the effect to detect, the variability of the outcome, the significance level (here a two-sided α of 0.05, so z_1−α/2 = 1.960) and the desired power (z_1−β = 0.842 for 80% and 1.282 for 90%).

Formula

n is the number of participants in each arm, p₀ and p₁ are the control and treatment proportions, and p̄ is their average.

Worked calculation for this trial

1. Control proportion p₀ = 0.493; treatment proportion p₁ = 0.443; average p̄ = 0.468.
2. First term: z_1−α/2 × √[2p̄(1−p̄)] = 1.96 × 0.7057 = 1.3831.
3. Second term: z_1−β × √[p₀(1−p₀) + p₁(1−p₁)] = 0.842 × 0.7048 = 0.5932.
4. n = (1.3831 + 0.5932)² / (-0.0492)² = 1,614.9, rounded up to 1,615 participants per arm for 80% power.

The smaller-effect rows in the table above repeat the same calculation with the effect multiplied by 0.75 and 0.5 (for ratios, applied to the logarithm of the ratio).

R code

alpha <- 0.05
z_a   <- qnorm(1 - alpha / 2)          # two-sided
p0 <- 0.492610837438424                          # control proportion
p1 <- 0.443433029908973                          # treatment proportion
n_per_arm <- function(power, frac = 1) {
  q1 <- p0 + (p1 - p0) * frac               # smaller effect = fraction of the observed difference
  pbar <- (p0 + q1) / 2
  z_b <- qnorm(power)
  ceiling(round((z_a * sqrt(2 * pbar * (1 - pbar)) + z_b * sqrt(p0 * (1 - p0) + q1 * (1 - q1)))^2 / (q1 - p0)^2, 6))
}
for (f in c(1, 0.75, 0.5))
  cat(sprintf("%3.0f%% of observed effect: n per arm = %s (80%% power), %s (90%% power)\n", 100 * f,
              format(n_per_arm(0.80, f), big.mark = ","), format(n_per_arm(0.90, f), big.mark = ",")))

# Check against R's built-in function (unrounded n per arm)
cat("power.prop.test:", round(power.prop.test(p1 = p0, p2 = p1, power = 0.80)$n, 1), "\n")

# Simulation: how often does the z-test reject at the 80%-power sample size?
set.seed(5006833)
n    <- n_per_arm(0.80)
reps <- 20000
x0 <- rbinom(reps, n, p0); x1 <- rbinom(reps, n, p1)
ph <- (x0 + x1) / (2 * n)
z  <- ((x1 - x0) / n) / sqrt(ph * (1 - ph) * 2 / n)
cat(sprintf("Simulated power at n = %s per arm: %.3f (%d replicates)\n", format(n, big.mark = ","), mean(abs(z) > z_a, na.rm = TRUE), reps))

Output from running the code

100% of observed effect: n per arm = 1,615 (80% power), 2,162 (90% power)
 75% of observed effect: n per arm = 2,876 (80% power), 3,850 (90% power)
 50% of observed effect: n per arm = 6,480 (80% power), 8,674 (90% power)
power.prop.test: 1614.9 
Simulated power at n = 1,615 per arm: 0.804 (20000 replicates)

The simulation draws many pairs of event counts at the planned sample size and counts how often the two-proportion z-test rejects the null. It should land close to the target power; at very small required sample sizes it can fall below it, because the normal approximation is rough there.

14. Multiplicity, Missing Data and Analysis Populations

The registry posts several comparisons of intervention arms with the control arm. When many comparisons are examined, the chance of finding at least one apparently unusual result can increase even if there are no true effects. In general, multiplicity procedures can control or otherwise address this issue, but they may affect the interpretation of p-values and confidence intervals. The registry does not report a multiplicity adjustment, a testing hierarchy, or a prespecified strategy for interpreting the several arm comparisons. The posted p-values are identified as unadjusted, so they should not be treated as multiplicity-adjusted confirmatory evidence.

The factorial design raises additional questions about the analysis target. The posted primary comparisons are arm-level comparisons with the control. The registry does not report whether the planned analysis also assessed overall clinician-training effects, overall text-message effects, or interactions between these components. It also does not report how any such analyses would be prioritized relative to the arm comparisons.

Missing-data methods and analysis populations are not reported in the registry record. The record does not describe whether all randomized participants were included in the primary analysis, how participants with unavailable vaccination information were handled, or whether imputation or sensitivity analyses were used. No participant-flow details are included in the registry record in the registry record. These omissions matter because missing outcomes can affect both the estimated percentages and the uncertainty around comparisons.

The primary endpoint description does identify predicted probabilities adjusted for clinic-level clustering, but the details of that model and the inferential procedure are not reported. The record does not specify covariates, handling of clinic-level variation beyond the stated adjustment, or whether the confidence intervals and p-values came from the same model. These are limits on the statistical auditability of the results, not proof that the analyses were conducted incorrectly.

15. What the Result Does, and Does Not, Mean

The posted results describe estimated HPV vaccine initiation by 24 months in the trial’s intervention combinations and control condition. The intervention-arm estimates range from 43.1% to 48.6%, while the control estimate is 49.2%. Most posted risk-difference intervals include zero. One listed comparison has an interval below zero, but the registry reports multiple unadjusted comparisons and does not describe a multiplicity plan. The results are therefore best read as a set of uncertain arm-level estimates rather than as a single definitive summary of the factorial study.

The findings do not establish that any intervention caused a change in vaccination, nor do they establish that an intervention had no effect. Estimates with intervals that include zero remain compatible with no difference and with effects in either direction. Even a comparison interval excluding zero does not by itself settle questions about bias, missing information, multiplicity, or the full analysis plan. The primary outcome is initiation, not completion of vaccination, and the posted result does not address the number of doses received after initiation.

The comparison results also do not resolve which component, if any, is associated with a difference. That would require clear estimates for the factorial components and their possible interaction, which are not reported in the registry record. Clinician recommendation frequency is a secondary measure, and acceptability results are not reported. Those measures cannot be used to infer a mechanism for the primary estimates from the information available.

In general, a randomized design can support causal comparisons when allocation, follow-up, outcome ascertainment, and analysis are appropriately carried out. The record identifies randomization, but does not report enough detail about those procedures to independently assess all of these conditions. The interpretation should remain descriptive: the registry reports particular adjusted arm percentages and pairwise comparisons, with substantial uncertainty and important analysis details not reported in the registry record.

16. Why This Trial Matters Statistically

This study is statistically useful because it places a behavioral intervention question in a factorial framework. Parent messaging and clinician training are distinct components, and the arm combinations make it possible to ask whether approaches differ when used together or separately. Such designs can be informative, but only when the analysis makes the intended component-level and interaction questions clear. Those results are not reported in the registry record, so the posted estimates chiefly support arm-by-arm description.

The primary outcome also has a meaningful statistical feature: it is measured among participants, but the registry says its estimated probabilities account for clustering at clinics. This highlights the difference between the number of individual observations and the amount of independent information. General methods that ignore shared clinic context can understate uncertainty. The record’s explicit mention of clinic adjustment is useful, although the implementation details needed for evaluation are absent.

The posted results demonstrate why point estimates, confidence intervals, and multiplicity need to be read together. Arm percentages appear relatively close, yet their intervals are broad; most pairwise intervals include zero. A nominal result in one comparison cannot summarize a multi-arm factorial trial on its own, especially when the adjustment strategy is not reported. The trial therefore illustrates a central biostatistical lesson: design labels and p-values are not enough. The estimand, analysis model, uncertainty, and number of comparisons all shape what can be concluded from clinical trial results.

17. Limitations

• The posted summary statistics are the only data these results rest on; patient-level data, full protocols and statistical analysis plans are not available.
• Recalculated results are simple, unadjusted calculations from posted summaries. They do not replace the sponsor's prespecified analysis.
• Participant counts were reconstructed from posted percentages and arm sizes, so small rounding differences are possible.

Several limitations are specific to the registry information for this trial. The primary endpoint is described as clinic-cluster-adjusted predicted probabilities, but the model specification, covariates, clinic structure, and method for calculating uncertainty are not reported in the registry record. That prevents a detailed assessment of how the adjustment may have influenced the arm estimates and comparisons.

The record lists a factorial model and multiple combinations of clinician training and parent messages, but does not report component-level main effects or interaction estimates. The listed comparisons therefore do not answer all questions that the factorial design could raise. The record also does not describe the randomization unit or allocation procedure, and it does not identify who was masked under the single-masked design.

Participant flow, missing vaccination outcomes, exclusions, and the primary analysis population are not reported in the registry record. These omissions make it difficult to understand how the arm-level samples relate to enrollment or how unavailable outcome data may have been handled. The record lists zero at-risk and affected counts, and zero deaths, for every adverse-event group. Because the at-risk fields are also zero, these entries should not be read as a complete account of safety surveillance or exposure. Details of adverse-event collection are not reported in the registry record.

Finally, the registry does not report the original sample-size rationale, a multiplicity strategy, or whether secondary analyses were prespecified. Parent text-message acceptability is named, but its results are not reported. These registry reporting gaps constrain interpretation; they do not establish that the study team did not conduct analyses or collect information that is absent from the record.

18. Related Tutorials

19. Related Statistical Calculators

20. Sources

• ClinicalTrials.gov: Text and Talk: A Multi-level Intervention to Increase Provider HPV Vaccine Recommendation Effectiveness — NCT05006833.
• Registry record last updated 2026-08-04; status last verified 2025-04.
• Results first posted 2026-08-04.
• Data retrieved from ClinicalTrials.gov on 2026-11-09.
• Group sizes, means, event counts and sponsor-reported p-values are reproduced as posted. Effect estimates, confidence intervals and p-values not attributed to the sponsor are independently calculated from the posted summary statistics.

21. Record Summary

NCT05006833 is a completed randomized, single-masked factorial trial sponsored by the University of Florida. It evaluates parent-targeted HPV vaccine text messages and brief clinician training among 11- to 12-year-olds living in Florida. Registry enrollment is 8043. The primary endpoint is initiation of HPV vaccination by 24 months, reported as clinic-cluster-adjusted predicted probabilities. The control condition is deferred clinician training with no intervention text messages.

Posted initiation estimates range from 43.1% to 49.2%, with broad confidence intervals. Most listed risk-difference intervals include zero; one unadjusted comparison has an interval below zero. The registry does not report the full statistical plan, multiplicity approach, missing-data methods, analysis population, or factorial component and interaction estimates. Clinician recommendation frequency is listed as a secondary outcome; parent text-message acceptability results are not reported in the registry record.