This page provides an independent statistical analysis and educational interpretation of publicly reported results. ClinicalTrials.gov provides the official trial registry record. Numerical trial results on this page are restricted to the results posted in the ClinicalTrials.gov record; calculations added by Clinical Biostats are labelled as such.
1. Trial at a Glance
This page covers the registry results for From Innovation to Solutions: Childhood Influenza Vaccination Planning (NCT01664793), a randomized na study. The primary outcome is Primary Outcome.
| Feature | Registry value |
|---|---|
| ClinicalTrials.gov identifier | NCT01664793 |
| Phase | NA |
| Condition | Influenza |
| Allocation | Randomized |
| Design model | Single Group |
| Masking | None (open label) |
| Enrollment | 87665 |
| Arms in the primary outcome | 2 |
| Lead sponsor | University of Pittsburgh |
| Results posted | Yes |
2. Clinical Question
Population
Children receiving care in participating primary care offices were the population of interest. The registry describes childhood influenza vaccination but does not report the children’s age range, eligibility criteria, or the number of participating offices. The recorded overall enrollment is 87665.
Intervention
Offices assigned to intervention received the 4 Pillars Immunization Toolkit, donated vaccines for early-season vaccination, staff education, and support in Year 1. The intervention was behavioral, and the registry describes the vaccine use as following national guidelines.
Comparator
Control offices received no assistance to increase influenza vaccination during Year 1 and followed usual-care guidelines. The brief summary says that control offices would receive the intervention in the second year.
Primary question
The primary question was whether the number of participants vaccinated by the end of Year 1 differed between intervention and control offices. The recorded outcome period is 3/1/2011-2/29/2012. The record reports group counts and a chi-squared p-value, but no comparative effect estimate or confidence interval.
This is a question about a practice-level strategy and the vaccination counts associated with its implementation, not a comparison of vaccine products. The intervention bundled several elements, so the registry does not isolate the contribution of any one component. It also does not report whether the primary comparison was adjusted for office-level clustering or other differences between participants or sites.
3. Trial Design
The study is titled “From Innovation to Solutions: Childhood Influenza Vaccination Planning.” Its official title describes strategies involving autodialers, early-autumn physicals, and vaccination assessment as a vital sign. The listed intervention, however, is the 4 Pillars Immunization Toolkit, with donated vaccines for early-season vaccination, staff education, and support described for intervention sites.
The brief summary describes a randomized cluster trial in which primary care offices were randomized to intervention or control. Intervention offices received the toolkit in Year 1, while control offices followed usual care and were to receive the intervention in the second year. This control-to-intervention sequence is relevant when interpreting the Year 1 comparison: the primary outcome timeframe is the first-year period, not the later period in which control sites were to receive support.
The registry lists allocation as randomized and masking as none. It also lists the study model as single group, which does not align clearly with the summary and arm descriptions that present intervention and control groups. The record does not explain this inconsistency. The number of offices, the randomization procedure, any stratification, and whether analysis accounted for clustering are not reported in the registry record.
Recorded enrollment was 87665, and status is completed. The study start is listed as 2010-10 and primary completion as 2013-02. The phase is listed as NA. The University of Pittsburgh is named as sponsor. These design details provide context, but the registry record does not state the protocol’s full statistical analysis plan or give further information on how participants were selected or followed.
4. Participant Flow and Accounting
The registry reports how participants moved through each stage of the study. All counts below are exactly as posted; missing counts are not inferred from other stages.
Overall Study
| Stage | Intervention Group | Control Group |
|---|---|---|
| STARTED | 49039 | 38626 |
| COMPLETED | 49039 | 38626 |
| NOT COMPLETED | 0 | 0 |
5. Baseline Characteristics
The table reproduces the baseline characteristics posted in the registry, which describe the participants analysed at baseline and help show whether the randomized groups were comparable.
Population: Children enrolled at family medicine practices and pediatric practices in the year preceding the intervention year for both intervention and control groups who were seen for a visit between 3/1/2010 and 2/28/2011.
| Characteristic | Unit | Intervention Group | Control Group | Total |
|---|---|---|---|---|
| Participants analysed | 43293 | 38306 | 81599 | |
| Age, Continuous | years | 7.3 (SD 4.8) | 7.1 (SD 5.2) | 7.2 (SD 5) |
| Sex: Female, Male: Female | Participants | 21560 | 18540 | 40100 |
| Sex: Female, Male: Male | Participants | 21733 | 19766 | 41499 |
| Region of Enrollment: United States | children at sites | 43293 | 38306 | 81599 |
6. Primary Endpoint
The registry lists the primary outcome as Primary Outcome, measured over 3/1/2011-2/29/2012.
| Element | Registry entry |
|---|---|
| Result type | Number |
| Measure of spread | not applicable |
| Unit | participants vaccinated out of all |
| Endpoint classification | not reduced to a single comparison |
The primary endpoint is described as influenza vaccination rates in each arm at the end of Year 1. Its unit is participants vaccinated out of all, and its statistical parameter is listed as a number. The registry gives the measurement period as 3/1/2011-2/29/2012. In the intervention group, 26430 participants were vaccinated out of 49040; in the control group, 19350 were vaccinated out of 38630.
This endpoint is used to describe whether vaccination occurred among participants in each group during the specified period. For a practice-level intervention intended to change vaccination processes, counts of vaccinated participants offer a direct summary of the outcome the strategy sought to increase. The registry’s endpoint description calls the outcome a rate, but the posted arm results are counts rather than rates. No rate estimate, between-group difference, ratio, or confidence interval is included in the record, so those quantities should not be inferred from the counts in this interpretation.
The endpoint is a group-level comparison of participant outcomes in offices assigned to different strategies. Because assignment was at the office level according to the study summary, the analysis needs to account appropriately for the relationship among participants treated at the same office. Whether the reported analysis did so is not reported in the registry record.
7. Why No Re-analysis Is Shown
8. Statistical Interpretation
The Year 1 results report 26430 vaccinated participants among 49040 in the intervention group and 19350 among 38630 in the control group. The registry reports a chi-squared analysis with p < 0.05. This is evidence that the observed group distributions were statistically distinguishable under the assumptions of that analysis. The registry does not report a numerical effect estimate or a confidence interval, so the size and precision of the between-group difference cannot be read from the posted result.
A p-value addresses how compatible the observed data are with a specified null model, under the analysis assumptions. It is not a measure of the size or practical importance of an effect, and it does not state the probability that the null hypothesis is true. Here, the result is reported only as p < 0.05; a more exact p-value is not reported in the registry record. Without an effect estimate and interval, the statistical result alone gives limited information about how large a difference the data support.
The study summary says offices were randomized, so the assignment unit was the primary care office, while vaccination was counted for individual participants. Participants within one office may share clinicians, procedures, or local context. In general, such within-office similarity can make participant outcomes less independent than a simple individual-level analysis assumes. The record names a chi-squared method but does not say whether the analysis accounted for office clustering. The p-value therefore has to be interpreted with that reporting gap in mind.
The arm counts describe the registered primary outcome for the first-year period. The registry does not provide baseline vaccination levels, site-level results, details about missing outcome data, or an analysis population definition. It is therefore not possible to judge from this record whether groups differed at baseline, how complete outcome ascertainment was, or whether all assigned offices and participants were included in the comparison.
The available evidence supports a limited description: the registry records different vaccinated-participant counts in the two groups and reports p < 0.05 from a chi-squared test. It does not support a precise estimate of the magnitude of the intervention-associated difference, nor does it establish which components of the bundled strategy were responsible. These are limits of the registry report, not proof that the underlying trial did not collect or analyze additional information.
9. Statistical Methods Explained
The registry names the chi-squared test for the primary comparison. In general, a chi-squared test evaluates whether categorical outcome counts differ across groups more than would be expected under a null model of no association. For this outcome, vaccination status is categorical and the groups are intervention and control. The test uses the observed counts to assess evidence against the null model; it does not by itself provide an estimate of the magnitude of the group difference.
What does the reported p-value mean?
The reported result is p < 0.05. In general, a p-value describes how unusual results at least as incompatible with the null model as the observed result would be if that model and the test assumptions were correct. It is not the probability that the intervention had no effect, and it does not quantify the chance that the result was due to chance. The record gives a threshold rather than an exact p-value, so the evidence cannot be characterized more finely from this report.
Effect estimates and confidence intervals answer different questions from a p-value. An effect estimate describes the direction and magnitude of a comparison, while a confidence interval expresses the uncertainty around that estimate under the model. Neither is reported for this primary result. The participant counts should therefore be read as descriptive outcome counts, not as a substitute for an unreported comparative estimate. No estimate or interval should be reconstructed from these counts in this account.
Why does office-level assignment matter?
General statistical principle: when assignment occurs by cluster, such as a primary care office, outcomes from participants in the same cluster may be correlated. For example, participants may share office procedures or staff. A standard analysis that treats every participant as fully independent can misstate uncertainty if this correlation is present and not accommodated. Cluster-aware methods adjust the analysis to reflect the assignment and outcome structure. The registry identifies a chi-squared test but does not report whether it included a cluster adjustment, how many offices took part, or the clustering approach. The validity of the reported p-value with respect to clustering cannot be assessed from the registry record alone.
The study’s fields also do not fully align: allocation is listed as randomized, the summary describes randomization of primary care offices, and the model is listed as single group even though intervention and control results are reported. The record does not explain this discrepancy. A reader should distinguish the description of the design in the summary and arm details from the separate model classification, rather than assuming the classification resolves how the study was conducted.
What can and cannot be inferred from the endpoint?
The endpoint is described as a vaccination rate, with units of participants vaccinated out of all. The posted results provide counts for each group, while the reported analysis gives only a p-value. In general, counts convey the number of recorded outcomes and the size of the corresponding group, but a comparative effect measure is useful for judging magnitude and precision. Since the registry does not report one, no rate, difference, ratio, or interval is stated.
The record lists one secondary outcome, “Effectiveness Score,” but its result is not reported in the registry record. The number and nature of analyses beyond the single reported analysis, the prespecified analysis plan, and any adjustment for multiple comparisons are not reported. These omissions matter because the interpretation of a single p-value depends partly on how many outcomes and analyses were considered and how those analyses were selected.
In general, missing outcome data can affect both the estimated group comparison and its uncertainty. The registry does not report missing-data counts, methods for handling missing observations, or an analysis population definition. It is therefore not possible to assess whether the displayed counts include all participants assigned to each group or how incomplete observations were handled. The reported enrollment and arm counts describe recorded totals but do not resolve those analysis details.
Finally, a statistically significant test does not establish the quality of outcome measurement, the completeness of follow-up, or the appropriateness of the model. Those issues require design and analysis information beyond the named test and result. The chi-squared finding should be understood as the reported statistical comparison, with the uncertainty and design details left unquantified in the registry record.
10. Analyses Reported by the Sponsor
| Method | Estimate | 95% CI | P-value |
|---|---|---|---|
| Chi-squared | not reported | < 0.050 |
The registry reports the sponsor’s primary analysis as a chi-squared test, with p < 0.05, comparing the intervention and control groups. It does not provide a test statistic, exact p-value, effect estimate, confidence interval, or more detailed description of the analysis. The posted participant counts are 26430 of 49040 in the intervention group and 19350 of 38630 in the control group.
A recalculated comparative estimate is not available for this count outcome in the registry record. The posted result therefore remains the basis for interpreting the statistical test, rather than a separate unadjusted recalculation. No discrepancy is flagged between a sponsor result and a recalculated result. If a discrepancy were present, the sponsor’s own analysis would take precedence; that situation is not indicated for this result.
The limited reporting means the sponsor’s result cannot be checked for details such as handling of office-level clustering, missing outcomes, or the analysis population. The record also does not include a confidence interval to show precision. Consequently, the posted p-value is best read as the sponsor-reported evidence from the named test, not as a complete account of the estimated effect or its uncertainty.
11. Multiplicity, Missing Data and Analysis Populations
The registry lists the primary vaccination outcome and one secondary outcome, Effectiveness Score. A result for that secondary outcome is not reported in the registry record. The record names one analysis for the reported outcome but does not give the full statistical analysis plan, state whether other analyses were performed, or explain how the primary outcome was selected. No method for adjusting for multiple outcomes or analyses is reported.
In general, when many outcomes or analyses are examined, the chance of at least one apparently unusual result can increase if each is assessed independently using the same decision threshold. Multiplicity procedures can address this issue, but the registry record does not say whether any were planned or used. The reported p < 0.05 should therefore be interpreted as the result for the named comparison, without assuming that it was or was not adjusted for other analyses.
The record does not describe missing-data counts, reasons for missingness, imputation, or other missing-data methods. It also does not define an intention-to-treat, per-protocol, or other analysis population. Although enrollment and outcome denominators are reported, those figures alone do not show whether every assigned participant was followed or how any unavailable outcomes were treated.
These reporting gaps do not demonstrate that the investigators failed to consider multiplicity or missingness. They mean that those decisions cannot be evaluated from the registry record. The record likewise does not report interim analyses, stopping rules, a sample-size justification, or the detailed prespecified analysis plan. Their presence or absence should not be inferred from the single posted analysis.
12. Safety Results
The registry reports serious adverse events as affected participants divided by participants at risk.
| Arm | Participants with serious adverse events | Participants at risk | Percentage |
|---|---|---|---|
| Intervention Group | 0 | 26432 | 0.0% |
| Control Group | 0 | 19352 | 0.0% |
Other (non-serious) adverse events
Participants with at least one non-serious event at or above the registry's 0% reporting threshold.
| Arm | Participants affected of at risk |
|---|---|
| Intervention Group | 0 of 26432 |
| Control Group | 0 of 19352 |
The registry lists zero serious adverse events and zero other adverse events for both groups. The recorded at-risk counts are 26432 in the intervention group and 19352 in the control group. These safety denominators differ from the primary outcome group sizes, which are 49040 and 38630, respectively; the record does not explain the distinction. Safety counts should therefore be read using their own listed denominators rather than assumed to describe the same analysis population as the vaccination outcome.
No serious adverse-event terms are listed. Deaths are recorded as null, which means a death count is not reported in the registry record; it should not be read as zero. Likewise, the absence of listed event terms does not establish that no events were assessed or occurred outside the reported entries. The registry does not give information about safety-event definitions, collection procedures, follow-up duration, or how events were attributed.
These entries are descriptive registry figures, not a complete safety analysis. They do not establish comparative safety, and the zero counts should not be used to make conclusions beyond the specific adverse-event fields reported.
13. What the Result Does, and Does Not, Mean
The registry record describes a randomized office-level strategy to increase childhood influenza vaccination and reports a Year 1 comparison between intervention and usual-care control groups. The intervention combined the 4 Pillars Immunization Toolkit with donated early-season vaccines, staff education, and support. The primary outcome was the number of participants vaccinated during 3/1/2011-2/29/2012.
The posted results record 26430 vaccinated participants among 49040 in the intervention group and 19350 among 38630 in the control group. The sponsor’s analysis is identified as a chi-squared test and reported as p < 0.05. This gives evidence of a difference in the observed group outcome distributions under the assumptions of the test. The registry does not report a comparative effect estimate or confidence interval, so the magnitude and precision of the contrast are not established by the posted result.
The finding does not identify which intervention component contributed to the group comparison. It does not, on its own, show that the result generalizes to settings beyond the participating offices, whose number and characteristics are not reported. Nor does it demonstrate that the analysis accounted for possible correlation among participants treated at the same office. Those questions require details absent from the registry record.
The result also does not provide a complete account of the study’s secondary outcomes, missing data, analysis populations, or multiplicity decisions. The secondary outcome Effectiveness Score is listed, but its result is not reported in the registry record. Safety entries report zero serious and other adverse events in the listed groups, while deaths are not reported. These safety entries do not substitute for a fuller account of event ascertainment and follow-up.
Overall, the record offers a statistically significant result as reported by the sponsor, alongside group counts, but leaves important information about effect size, uncertainty, and analytic handling unavailable. The appropriate reading is bounded by what is registered: the observed counts and the reported test result, not a broader claim about clinical impact or the mechanisms behind the difference.
14. Why This Trial Matters Statistically
This trial raises an important statistical issue: the intervention was assigned to primary care offices, while the endpoint was counted across participants. That distinction affects the structure of the data. When people share an office, their outcomes may be related; in general, an analysis should reflect the unit of randomization and any within-office dependence. The registry names a chi-squared test but does not say whether the analysis accounted for clustering, leaving a central feature of the design and inference unclear.
The record also illustrates the difference between statistical evidence and an estimate of effect. A thresholded p-value of p < 0.05 indicates that the reported test met its stated threshold, but it does not quantify the magnitude of the comparison or its precision. The participant counts help describe the observed outcomes, yet the absence of a comparative estimate and confidence interval makes it harder to assess how large a difference is compatible with the data.
A further point is the importance of consistency in registry descriptions. Randomized allocation and intervention/control arms appear alongside a single-group model classification. The office-level design is described in the summary, but details of office numbers and cluster-adjusted analysis are absent. These differences in reporting can make it difficult for readers to determine exactly how design features were represented in the analysis.
Finally, the intervention was a package of activities rather than one isolated procedure. Even if a group difference is present, the registry result cannot attribute that difference to autodialers, early-season vaccination, staff education, or another component individually. Clear reporting of design, analysis, effect estimates, uncertainty, and outcome completeness is essential for interpreting a cluster-based behavioral intervention.
15. Limitations
- The posted summary statistics are the only data these results rest on; patient-level data, full protocols and statistical analysis plans are not available.
- Recalculated results are simple, unadjusted calculations from posted summaries. They do not replace the sponsor's prespecified analysis.
The registry record leaves several trial-specific details unclear. It describes randomization of primary care offices and reports intervention and control groups, but the study model is listed as single group. The reason for that classification is not reported. The number of offices, the process used to randomize them, and any balancing or stratification procedures are also not reported in the registry record.
The primary result is reported as a chi-squared test with p < 0.05, without a comparative effect estimate or confidence interval. The registry does not say whether the test accounted for office-level clustering, even though assignment is described as occurring at the office level. It also does not explain whether the reported outcome counts were analyzed at participant or office level, or identify the precise population included in the analysis.
Outcome ascertainment and missing data are not described in enough detail to evaluate completeness. The registry does not report baseline vaccination information, missing-outcome counts, data collection procedures, or reasons participants may not have contributed outcome data. A secondary outcome, Effectiveness Score, is listed but its result is not reported in the registry record. The record does not describe multiplicity handling, interim analyses, or a sample-size justification.
The intervention includes several components, and the registry does not provide component-specific comparisons. The control group was described as receiving the intervention in the second year, but the primary result concerns Year 1; later-period outcomes and analyses are not reported in the posted results. The safety denominators differ from the primary outcome denominators, and the record does not explain why. These limitations constrain interpretation of the registered result; they do not establish what information may have been available in other trial documents.
16. Related Tutorials
17. Related Statistical Calculators
18. Sources
- ClinicalTrials.gov: From Innovation to Solutions: Childhood Influenza Vaccination Planning — NCT01664793.
- Registry record last updated 2014-11-24; status last verified 2014-11.
- Results first posted 2014-11-24.
- Data retrieved from ClinicalTrials.gov on 2026-11-09.
- Group sizes, means, event counts and sponsor-reported p-values are reproduced as posted. Effect estimates, confidence intervals and p-values not attributed to the sponsor are independently calculated from the posted summary statistics.
Continue through the Clinical Biostats statistical pathway
Use this trial's endpoint as a starting point for deeper study of effect measures, confidence intervals, and sample-size planning.
19. Record Summary
NCT01664793 is a completed study sponsored by the University of Pittsburgh. It examines a childhood influenza vaccination initiative, with the registry summary describing randomized assignment of primary care offices to intervention or control. The intervention group received the 4 Pillars Immunization Toolkit and associated support in Year 1; control offices followed usual care and were to receive the intervention in the second year.
The primary outcome is the number of participants vaccinated during 3/1/2011-2/29/2012. The registry reports 26430 of 49040 in the intervention group and 19350 of 38630 in the control group, with a chi-squared result of p < 0.05. No comparative effect estimate or confidence interval is reported. The record also lists Effectiveness Score as a secondary outcome without a reported result.
The registry identifies serious and other adverse-event counts as zero in both groups, while deaths are not reported. Important details—including cluster adjustment, missing-data handling, analysis populations, and the reason the model is classified as single group despite the two reported groups—are not reported in the registry record.