This page provides an independent statistical analysis and educational interpretation of publicly reported results. ClinicalTrials.gov provides the official trial registry record. Numerical trial results on this page are restricted to the results posted in the ClinicalTrials.gov record; calculations added by Clinical Biostats are labelled as such.
1. Trial at a Glance
This page covers the registry results for A Trial of Behavioral Economic Interventions Among Food Pantry Clients (NCT04011384), a randomized na study. The primary outcome is Mean Monthly Change in Number of Units of Healthier Foods.
| Feature | Registry value |
|---|---|
| ClinicalTrials.gov identifier | NCT04011384 |
| Phase | NA |
| Condition | Food Preferences |
| Allocation | Randomized |
| Design model | Parallel |
| Masking | Single |
| Enrollment | 286 |
| Arms in the primary outcome | 2 |
| Lead sponsor | Abramson Cancer Center at Penn Medicine |
| Results posted | Yes |
2. Clinical Question
Population
The trial concerned food pantry clients using an online ordering system. The registry describes a public health context that includes food insecurity and unhealthy dietary habits, and says that some participants were missing order data, especially near March 2020 at the start of the Covid-19 pandemic. The record does not provide further detail about eligibility criteria or the characteristics of participants represented in the primary outcome analysis.
Intervention
The behavioral economic intervention added several features to the online ordering platform: healthy-food shopping cart defaults, healthy placement choice architecture, traffic light nutrition labels, social norms messaging, and healthy swaps. The primary question concerns whether assignment to this platform was associated with a different average monthly change in the number of healthier-food units purchased per shopping trip.
Comparator
The control group used the typical web-based ordering platform, described in the registry as usual care. The comparison is between assignment to the platform with behavioral features and assignment to the usual platform. The record does not describe additional services or differences between the groups beyond these platform conditions.
Primary question
The primary endpoint was the mean monthly change from baseline, averaged across 3 months of participation, in units of healthier foods purchased per shopping trip. Healthier foods were labeled with a green traffic light, using the food pantry's definition of a unit. The clinical and behavioral question is whether the mean change differed between the randomized groups. The posted estimates describe that contrast, but do not by themselves establish why any difference arose or whether it would persist beyond the recorded participation period.
3. Trial Design
The registry describes a randomized, single-masked, parallel-group trial. The behavioral economic intervention was compared with a control condition using the typical web-based ordering system. The masking description is single, but the record does not report which people were masked or how masking was maintained. The trial is listed as completed, and its phase is recorded as not applicable.
The registry lists enrollment as 286. Its brief summary describes an overall objective to conduct a trial among 500 regular food pantry clients, while the recorded enrollment is 286. These are different figures serving different roles in the record: the 500 appears in the description of the proposal, and 286 is the listed enrollment. The registry does not explain this difference, so it should not be interpreted as evidence about the reason recruitment ended or the intended sample-size rationale.
The record gives a start date of 2019-07-05 and a primary completion date of 2022-12-21. The primary outcome was collected from the online system at baseline and then each month for the remaining 3 months of participation. Some participants were missing order data, especially near March 2020 at the start of the Covid-19 pandemic. The record does not specify how missing order records were handled in the primary analysis.
Several secondary outcomes are listed, including food-purchase measures, self-reported and biomarker measures of fruit and vegetable intake, body mass index, and blood pressure. The posted numerical secondary results cover only some of those outcomes. The registry record does not report the original sample-size justification, an analysis plan in sufficient detail to reproduce the primary comparison, or which participants were included in each analysis beyond the endpoint-specific counts.
4. Participant Flow and Accounting
The registry reports how participants moved through each stage of the study. All counts below are exactly as posted; missing counts are not inferred from other stages.
Before assignment: Due to the Covid-19 pandemic, we lost 47 participants who were consented (before March 2020), but had not yet completed their baseline appointment. 3 others were accidentally enrolled, but did not meet our enrollment criteria and were excluded before baseline. Finally 1 was excluded because they were not randomized to a condition.
Overall Study
| Stage | Control Group | Behavioral Economic Intervention Group |
|---|---|---|
| STARTED | 111 | 124 |
| COMPLETED | 109 | 124 |
| NOT COMPLETED | 2 | 0 |
Reasons for not completing this period:
| Reason | Control Group | Behavioral Economic Intervention Group |
|---|---|---|
| Withdrawal by Subject | 2 | 0 |
Analysis population for the primary outcome, as defined in the registry: Some participants were missing order data -- especially near March 2020 at the start of the Covid-19 pandemic.
5. Baseline Characteristics
The table reproduces the baseline characteristics posted in the registry, which describe the participants analysed at baseline and help show whether the randomized groups were comparable.
| Characteristic | Unit | Control Group | Behavioral Economic Intervention Group | Total |
|---|---|---|---|---|
| Participants analysed | 109 | 124 | 233 | |
| Age, Continuous | years | 66 (SD 12) | 66 (SD 15) | 66 (SD 14) |
| Sex: Female, Male: Female | Participants | 65 | 91 | 156 |
| Sex: Female, Male: Male | Participants | 44 | 33 | 77 |
| Ethnicity (NIH/OMB): Hispanic or Latino | Participants | 3 | 3 | 6 |
| Ethnicity (NIH/OMB): Not Hispanic or Latino | Participants | 105 | 121 | 226 |
| Ethnicity (NIH/OMB): Unknown or Not Reported | Participants | 1 | 0 | 1 |
| Race (NIH/OMB): American Indian or Alaska Native | Participants | 0 | 1 | 1 |
| Race (NIH/OMB): Asian | Participants | 6 | 7 | 13 |
| Race (NIH/OMB): Native Hawaiian or Other Pacific Islander | Participants | 0 | 0 | 0 |
| Race (NIH/OMB): Black or African American | Participants | 8 | 5 | 13 |
| Race (NIH/OMB): White | Participants | 88 | 109 | 197 |
| Race (NIH/OMB): More than one race | Participants | 3 | 2 | 5 |
| Race (NIH/OMB): Unknown or Not Reported | Participants | 4 | 0 | 4 |
| Language: English | Participants | 50 | 55 | 105 |
| Language: Russian | Participants | 56 | 66 | 122 |
| Language: Other language | Participants | 3 | 3 | 6 |
| Education: Less than high school | Participants | 5 | 6 | 11 |
| Education: High school degree or equivalent | Participants | 30 | 29 | 59 |
| Education: Some college or associates | Participants | 24 | 33 | 57 |
| Education: College or above | Participants | 49 | 53 | 102 |
| Education: Unknown | Participants | 1 | 3 | 4 |
| Children at home: No children at home | Participants | 87 | 105 | 192 |
| Children at home: 1 or more children at home | Participants | 19 | 17 | 36 |
| Children at home: Unknown | Participants | 3 | 2 | 5 |
6. Primary Endpoint
The registry lists the primary outcome as Mean Monthly Change in Number of Units of Healthier Foods, measured over Collected from the online system at baseline and then each month for the remaining 3 months of participation..
| Element | Registry entry |
|---|---|
| Result type | Mean |
| Measure of spread | Standard Deviation |
| Unit | units of healthy food |
| Endpoint classification | Continuous Endpoint |
The primary endpoint is the mean monthly change from baseline, averaged across 3 months of participation, in the number of units of healthier foods purchased per shopping trip. The endpoint uses online ordering records rather than relying solely on participants’ recollection. A healthier food is defined using the food pantry’s classification: foods labeled with a green traffic light. A unit is based on the pantry’s own definition.
Change from baseline is intended to describe how purchasing shifted during participation, while averaging over the recorded months provides a summary of that period. The measure is specific to purchases made through the online system and should not be treated as a complete measure of what participants ate, bought elsewhere, or had available to them. Nor does a change in purchases by itself establish a change in health.
The registry reports an average monthly change, not a count of all units purchased across the whole trial. It also notes that some order data were missing, particularly near March 2020 at the start of the Covid-19 pandemic. The record does not state whether the endpoint was calculated only among participants with complete monthly records, how partial records were incorporated, or whether any adjustment was made for missingness. Those details matter because the interpretation of an average change depends on which observations contributed to it and how incomplete records were treated.
7. Results Posted in the Registry
- Behavioral Economic Intervention Group: mean 0.53 versus -1.32 on control; difference 1.85 (95% CI -0.01 to 3.71), standardized effect d = 0.28; z-test p = 0.051.
Mean difference: Behavioral Economic Intervention Group
95% CI: -0.01 to 3.71
Recalculated from the posted summary statistics
8. Secondary Efficacy Endpoints
The registry also posts results for secondary outcomes. For each outcome below, the endpoint type, the test applied to the posted summary statistics, the result and its interpretation are given in turn. Secondary endpoints are usually not adjusted for multiple testing, so they support rather than confirm the primary finding.
| Secondary outcome | Type | Effect measure | Estimate (95% CI) | Planning size, 80% power |
|---|---|---|---|---|
| Mean Monthly Change in Calories From Healthier Foods Per Shopping Trip | Continuous Endpoint | Mean difference | 155.00 (-20.52 to 330.52) | 266 per arm |
| Mean Monthly Change in Calories From Less Healthy Foods Per Shopping Trip | Continuous Endpoint | Mean difference | -15.00 (-154.73 to 124.73) | 18,032 per arm |
| Self-reported Change in Fruit and Vegetable Intake | Continuous Endpoint | Mean difference | 0.09 (-0.26 to 0.44) | 3,213 per arm |
Mean Monthly Change in Calories From Healthier Foods Per Shopping Trip
- Endpoint type: Continuous: a mean value per participant, compared as a difference in means.
- Test: Unadjusted two-sample z-test for the difference in means, using the posted standard deviations.
- Result:
- Behavioral Economic Intervention Group: mean 31.00 versus -124.00 on control; difference 155.00 (95% CI -20.52 to 330.52), standardized effect d = 0.24; z-test p = 0.083.
- Interpretation:
- The 95% confidence interval includes no difference, so these data do not show a statistically significant difference between Behavioral Economic Intervention Group and the control. The interval gives the range of effects that remain compatible with the data.
Mean Monthly Change in Calories From Less Healthy Foods Per Shopping Trip
- Endpoint type: Continuous: a mean value per participant, compared as a difference in means.
- Test: Unadjusted two-sample z-test for the difference in means, using the posted standard deviations.
- Result:
- Behavioral Economic Intervention Group: mean 14.00 versus 29.00 on control; difference -15.00 (95% CI -154.73 to 124.73), standardized effect d = -0.03; z-test p = 0.833.
- Interpretation:
- The 95% confidence interval includes no difference, so these data do not show a statistically significant difference between Behavioral Economic Intervention Group and the control. The interval gives the range of effects that remain compatible with the data.
Self-reported Change in Fruit and Vegetable Intake
- Endpoint type: Continuous: a mean value per participant, compared as a difference in means.
- Test: Unadjusted two-sample z-test for the difference in means, using the posted standard deviations.
- Result:
- Behavioral Economic Intervention Group: mean 0.08 versus -0.01 on control; difference 0.09 (95% CI -0.26 to 0.44), standardized effect d = 0.07; z-test p = 0.615.
- Interpretation:
- The 95% confidence interval includes no difference, so these data do not show a statistically significant difference between Behavioral Economic Intervention Group and the control. The interval gives the range of effects that remain compatible with the data.
The listed secondary outcomes cover several kinds of measures: online purchases of healthier and less healthy foods, self-reported and biomarker measures of fruit and vegetable intake, and health measures including body mass index and blood pressure. Numerical results are posted for some, but not all, of the listed secondary outcomes. The pattern across the posted results is not uniformly aligned in direction or strength, and the estimates generally remain uncertain. This broad pattern does not provide a clear independent confirmation of the primary purchasing result.
Secondary outcomes can help assess whether a primary purchasing pattern is accompanied by changes in related behaviors or measures. They are not interchangeable with the primary endpoint. Online orders measure purchasing through the pantry system, while self-reported intake and biomarkers address different aspects of diet. Body mass index and blood pressure are further downstream measures. A finding on one type of measure does not automatically establish a corresponding change in another.
The secondary results should carry less weight as confirmatory evidence because the registry record does not report a multiplicity adjustment or a detailed hierarchy for interpreting them. In general, when many outcomes are examined, some estimates may appear notable by chance. Unadjusted p-values do not account for that broader set of comparisons. The record also does not report full analysis details for the secondary outcomes, and some posted outcomes have no reported analyses. This limits how much can be concluded from apparent similarities or differences among estimates.
The secondary measures therefore add context, but they do not resolve the primary result’s uncertainty. They also cannot establish that changes in purchasing translated into changes in intake or health. The most defensible overall reading is that the posted secondary evidence is exploratory unless a more complete prespecified analysis and multiplicity approach is available. The registry record does not report such details.
Secondary outcomes registered for this trial, for reference (only those above were recalculated):
- Mean Monthly Change in Calories From Healthier Foods Per Shopping Trip
- Mean Monthly Change in Calories From Less Healthy Foods Per Shopping Trip
- Self-reported Change in Fruit and Vegetable Intake
- Biomarker Change in Fruit and Vegetable Intake
- Change in Body Mass Index (BMI)
- Change in Systolic Blood Pressure
- Change in Diastolic Blood Pressure
9. Statistical Interpretation
The posted primary means are -1.32 units in the control group and 0.53 units in the behavioral economic intervention group. The reported mean difference, intervention relative to control, is 1.85 units. In descriptive terms, the group averages point in a more favorable direction for healthier-food purchasing in the intervention group over the measured period. The estimate does not, on its own, establish that the platform features caused that pattern or that the difference is important to participants.
The uncertainty around the estimated difference is substantial relative to the estimate. Its confidence interval extends from a value just below zero to positive values, and the unadjusted p-value is 0.051. Taken together, these statistics do not provide a clear separation from no difference under commonly used inferential conventions. The exact threshold used for the trial’s own primary decision is not reported in the registry record. A result near a conventional threshold should not be reduced to a yes-or-no label: the estimated direction and size, the interval of plausible values, the analysis method, and the study context all matter.
Variation among individuals was large relative to the difference between group means. The reported standard deviations are 7.77 units in the control group and 5.54 units in the intervention group. A standardized effect-size estimate is 0.276. That standardized value can help describe the contrast relative to the variation in the measure, but it does not substitute for understanding what a unit means in the pantry’s ordering system or whether a change of this size is meaningful in practice.
The primary comparison is based on the posted group summaries. The registry record does not provide a detailed model specification, covariate-adjustment strategy, or a full account of missing-data handling. The estimates should therefore be read as a summary of the observed reported results, not as a complete account of every analysis that may have been planned or conducted. The record indicates that order data were missing for some participants, with particular concern around the start of the Covid-19 pandemic. This limits certainty about how well the analyzed data represent all randomized participants.
Overall, the posted results show an estimated difference favoring the intervention group in healthier-food units purchased, with uncertainty that includes values close to no difference. They do not establish a definitive effect, and they do not answer whether any purchasing pattern led to changes in intake, biomarkers, body mass index, or blood pressure. Those outcomes require their own evidence and careful interpretation.
10. Statistical Methods Explained
The primary measure is continuous: it is a numerical amount of change in healthier-food units, and the registry reports a mean and standard deviation for each group. A mean gives the arithmetic average for the analyzed observations. It is useful for describing the central pattern, but can be influenced by unusually high or low values. The standard deviation describes how spread out individual observations are around the group mean. A larger standard deviation indicates more variability in the measured outcome, not greater uncertainty about the definition of the endpoint.
What does the mean difference represent?
The reported mean difference compares the intervention group with the control group. A positive estimate indicates that the intervention-group mean is higher on the change measure; it does not necessarily indicate that every participant increased purchasing or that the average difference is important in daily life. The recorded estimate is 1.85 units. It is a contrast between group averages, not a count of extra units purchased by every person, and it is not a direct measure of consumption.
Random assignment is intended to make the groups comparable, on average, with respect to factors that could affect the outcome. It does not guarantee that the realized groups are identical, and it does not remedy missing outcome information automatically. The record describes the allocation as randomized, but does not provide details about the randomization sequence, concealment, or the analysis population. Those features are relevant when assessing whether the observed comparison retains the advantages of randomization.
How should the p-value be read?
The unadjusted p-value is 0.051. In general, a p-value describes how compatible the observed data are with a specified null model, under the assumptions of the analysis. It is not the probability that the null hypothesis is true, the probability that the result happened by chance, or a measure of the effect’s importance. Its interpretation also depends on the analysis model and on whether the reported comparison was selected from several outcomes or analyses. The registry does not report a full primary analysis specification or a multiplicity strategy.
The word unadjusted matters: the reported value does not account for a stated multiplicity correction. It should not be read as if it were an adjusted confirmatory test when the record does not document such a procedure. In general, examining several outcomes can increase the chance that at least one result appears unusually strong even when there is no corresponding underlying effect. This is one reason secondary results without a stated multiplicity plan deserve cautious interpretation.
What does a confidence interval add?
A confidence interval expresses uncertainty about an estimated effect under the method used to calculate it. In general, it gives a range of values compatible with the data and assumptions at the stated confidence level; it is not a guarantee that the true effect lies inside that particular interval. For this primary comparison, the interval spans zero, so values corresponding to little or no difference remain compatible with the posted result, as do positive values. The interval is more informative than the point estimate alone because it conveys both direction and precision.
Why consider standardized effects?
The reported standardized effect-size estimate is 0.276. Standardization expresses a difference relative to the variability of the outcome, which can aid comparisons across measures with different units. Its interpretation remains dependent on the measure and setting. It does not make the endpoint more clinically meaningful, resolve missingness, or show whether the intervention’s components were responsible for the observed pattern. The reported pooled standard deviation is 6.704 units and is part of the scaling used for the standardized contrast.
The registry lists the endpoint dispersion as standard deviation. The posted summaries and comparisons support a simple unadjusted reading of the group means and their contrast, but do not reveal whether the trial’s primary inferential analysis used this same approach. In particular, the registry record does not report whether the analysis accounted for repeated measurements within participants, other covariates, or incomplete monthly ordering. Repeated observations from the same participant are generally correlated rather than independent. In general, analyses of repeated measures need to account for that structure if observations are modeled individually. The record does not provide enough information to determine whether or how such modeling was used.
For this endpoint, the recorded group sizes are 99 in the control group and 107 in the intervention group. These are the counts attached to the posted primary outcome summaries, not a statement that every randomized participant had complete data. The registry’s note about missing order data is a reason to distinguish enrollment from the observations contributing to an outcome estimate. Without details of the missing-data method, it is not possible to assess whether the reported means might be affected by differences between people with and without usable order records.
These methods concepts explain how to interpret the posted statistics; they do not reconstruct an unreported analysis plan. The registry record does not report a prespecified decision rule, a full model, or the treatment of missing records. Accordingly, inferential language should remain tied to the estimates and uncertainty actually recorded rather than implying that all design and analysis assumptions are known.
11. Confidence Intervals and Precision
The primary estimate is a mean difference of 1.85 healthier-food units, with a reported confidence interval from -0.006 to 3.706 and an unadjusted p-value of 0.051. The interval extends across zero, meaning that the posted data are compatible with a small difference in either direction as well as with a larger positive difference. The point estimate alone therefore gives an incomplete picture: it describes the center of the estimate, while the interval shows that its precision is limited.
Precision is affected by the amount of information in the outcome data and by how variable the measure is. The reported standard deviations are 7.77 units for the control group and 5.54 units for the intervention group, compared with a mean difference of 1.85 units. This provides context for why the group contrast is uncertain. It is not appropriate to infer a precise effect from the direction of the means alone.
A confidence interval is not a range of individual participant outcomes, and it does not mean that every value inside it is equally plausible. Its coverage interpretation relies on the statistical method and assumptions used to calculate it. The registry record does not report enough detail to assess all those assumptions, including how missing order data were handled. The noted data gaps, especially near March 2020 at the start of the Covid-19 pandemic, are relevant to confidence in the estimate as a representation of the randomized groups.
The interval and p-value are two related ways of conveying uncertainty, not independent confirmations. Neither establishes the importance of the observed difference. That judgment would require a meaningful threshold for change in this pantry-defined unit, and the registry record does not report such a threshold. The result is best read as an uncertain positive point estimate rather than as a precise or definitive estimate of the intervention’s effect.
12. Planning a Similar Trial: Sample Size
This is a planning illustration, not a re-evaluation of the original trial. It asks: if a new study wanted to detect a difference like the one observed, or a smaller one, how large would it need to be? It is deliberately not “observed power” of the finished trial, which is not a meaningful quantity.
| Effect to detect | Participants per arm, 80% power | Participants per arm, 90% power |
|---|---|---|
| 100% of the observed effect | 207 | 277 |
| 75% of the observed effect | 367 | 491 |
| 50% of the observed effect | 825 | 1,105 |
The planning figures are participants per arm, not total participants and not total events. The calculations are planning illustrations based on the observed effect, using a two-sided significance level of 0.05, equal allocation between the two arms, and a normal approximation. For each scenario, the effect to detect is the stated fraction of the observed effect: 100%, 75%, or 50%. These are assumptions for the illustration, not the trial’s original design assumptions; the registry record does not report the trial’s original sample-size justification.
The 80% power column describes the number of participants per arm estimated to give an 80% chance of detecting the specified effect under those assumptions, if that effect is truly present and the calculation’s conditions hold. The 90% power column gives the corresponding planning target for a higher chance of detection. Power is a property of a study design under a specified effect and model, not a guarantee that a particular study will produce a positive result.
The rows illustrate how planning needs change when the target effect is smaller. Detecting 75% or 50% of the observed effect requires more participants per arm than targeting the full observed effect. This is a general feature of power calculations: smaller differences are harder to distinguish from random variation, so more information is needed. The observed effect can itself be uncertain, especially when estimated from a study with limited precision, so it should not be treated as a dependable planning target without external justification.
These figures are not observed power. They are hypothetical sample-size illustrations and do not describe the probability that the completed trial’s result is true, nor do they repair uncertainty from missing data or unreported analysis choices. They should be used to understand how assumptions about the target effect influence planning, not to characterize the trial’s achieved evidence.
13. How the Sample Size Is Derived
Power is the probability that a study of a given size rejects the null hypothesis when a specific true effect exists. It depends on four things: the effect to detect, the variability of the outcome, the significance level (here a two-sided α of 0.05, so z1−α/2 = 1.960) and the desired power (z1−β = 0.842 for 80% and 1.282 for 90%).
Formula
n is the number of participants in each arm, Δ is the difference in means to detect and σ is the common standard deviation.
Worked calculation for this trial
- Observed difference Δ = 1.850; pooled standard deviation σ = 6.704.
- n = 2 × (1.96 + 0.842)2 × 6.7042 / 1.8502 = 206.2, rounded up to 207 participants per arm for 80% power.
The smaller-effect rows in the table above repeat the same calculation with the effect multiplied by 0.75 and 0.5 (for ratios, applied to the logarithm of the ratio).
R code
alpha <- 0.05
z_a <- qnorm(1 - alpha / 2) # two-sided
delta <- 1.85 # observed difference in means
sigma <- 6.70449578458292 # pooled standard deviation
n_per_arm <- function(power, frac = 1) {
ceiling(round(2 * (z_a + qnorm(power))^2 * sigma^2 / (delta * frac)^2, 6))
}
for (f in c(1, 0.75, 0.5))
cat(sprintf("%3.0f%% of observed effect: n per arm = %s (80%% power), %s (90%% power)\n", 100 * f,
format(n_per_arm(0.80, f), big.mark = ","), format(n_per_arm(0.90, f), big.mark = ",")))
# Check against R's built-in function (t-based, so very slightly larger)
cat("power.t.test:", round(power.t.test(delta = delta, sd = sigma, power = 0.80)$n, 1), "\n")
# Simulation: exact two-sample t-test using simulated means and variances
set.seed(4011384)
n <- n_per_arm(0.80)
reps <- 20000
m_diff <- rnorm(reps, delta, sigma * sqrt(2 / n)) # difference in sample means
s2 <- sigma^2 * rchisq(reps, 2 * (n - 1)) / (2 * (n - 1)) # pooled sample variance
tstat <- m_diff / sqrt(s2 * 2 / n)
cat(sprintf("Simulated power at n = %s per arm: %.3f (%d replicates)\n", format(n, big.mark = ","), mean(abs(tstat) > qt(0.975, 2 * (n - 1))), reps))
Output from running the code
100% of observed effect: n per arm = 207 (80% power), 277 (90% power)
75% of observed effect: n per arm = 367 (80% power), 491 (90% power)
50% of observed effect: n per arm = 825 (80% power), 1,105 (90% power)
power.t.test: 207.1
Simulated power at n = 207 per arm: 0.798 (20000 replicates)
The simulation draws many sets of group means and variances at the planned sample size and counts how often the two-sample t-test rejects the null. It should land close to the target power; at very small required sample sizes it can fall below it, because the normal approximation is rough there.
14. Multiplicity, Missing Data and Analysis Populations
Multiplicity arises when several hypotheses are tested or several outcomes are examined. In general, each additional opportunity to observe a striking result can increase the chance of a false-positive finding across the full set, even if each individual test is conducted appropriately. The registry lists a primary outcome and multiple secondary outcomes, but does not report a multiplicity adjustment, a testing hierarchy, or a plan for deciding which secondary findings would be considered confirmatory. The posted unadjusted p-values should therefore be interpreted as individual comparisons rather than as evidence corrected for the whole outcome set.
The registry reports nine outcome measures and lists seven secondary outcomes. Numerical secondary results are available for only some of the listed measures. The record also does not report a complete set of secondary analyses. Missing outcome analyses should not be taken as proof that an outcome was not measured, nor should the listed outcomes be assumed to have been analyzed in a particular way. The registry record does not explain why some results are absent.
Missing data are a separate concern from multiplicity. For the primary outcome, the registry notes that some participants lacked order data, especially near March 2020 at the start of the Covid-19 pandemic. It does not report whether missingness was handled through complete-case analysis, imputation, a model using available observations, or another method. In general, missingness can bias estimates if the chance of an observation being absent is related to the outcome or to participant characteristics. Without the analysis details and assumptions, the direction or size of any such effect cannot be determined.
The record does not report the analysis population definition, whether participants were analyzed according to randomized assignment, or whether exclusions occurred after randomization. It also does not provide the missingness pattern by group or a sensitivity analysis. These omissions limit assessment of how closely the reported estimates reflect the randomized comparison. They are reasons for caution, not proof that the analyses were biased or that the results are invalid.
15. Safety Results
The registry reports serious adverse events as affected participants divided by participants at risk.
| Arm | Participants with serious adverse events | Participants at risk | Percentage |
|---|---|---|---|
| Control Group | 0 | 109 | 0.0% |
| Behavioral Economic Intervention Group | 0 | 124 | 0.0% |
Deaths
| Arm | Deaths (participants affected of at risk) |
|---|---|
| Control Group | 0 of 109 |
| Behavioral Economic Intervention Group | 0 of 124 |
Other (non-serious) adverse events
Participants with at least one non-serious event at or above the registry's 0% reporting threshold.
| Arm | Participants affected of at risk |
|---|---|
| Control Group | 0 of 109 |
| Behavioral Economic Intervention Group | 0 of 124 |
The registry reports zero serious adverse events in the control group among 109 participants at risk and zero in the behavioral economic intervention group among 124 participants at risk. It also reports zero other adverse events in each group, with 109 and 124 participants at risk, respectively, and zero deaths in both groups. No serious adverse-event terms are listed.
These figures describe the events recorded in the registry, not proof that no adverse events occurred outside the reporting process or that the intervention is risk-free. The record does not explain adverse-event definitions, collection procedures, follow-up completeness, or whether events were assessed as related to the intervention. A zero count should therefore be read as a report of no events in these registry categories, with the limits of the record in mind. The study’s focus was a web-based ordering intervention, but that context does not remove the need to interpret safety reporting according to its documented scope.
16. What the Result Does, and Does Not, Mean
The primary result describes an uncertain difference in average change in healthier-food units purchased per shopping trip between people assigned to the behavioral economic platform and people assigned to the usual platform. The intervention-group mean was higher than the control-group mean on this change measure, and the estimated mean difference was 1.85 units. The confidence interval spans zero and the unadjusted p-value is 0.051. Thus, the posted data suggest a direction of difference but do not provide a precise estimate that clearly separates the groups from no difference.
This result does not show that every participant changed their purchasing in the same way. Group averages conceal individual variation, which is substantial in the reported standard deviations. Nor does the endpoint measure total diet or food consumption: it is based on online pantry orders and a pantry-defined unit. The record does not establish whether purchased foods were consumed, whether purchases from other sources changed, or whether the observed pattern affected nutrition or health.
The registry lists other outcomes, including fruit and vegetable intake, biomarkers, body mass index, and blood pressure. Those measures address distinct questions and cannot be inferred from the primary purchase outcome. The available secondary evidence does not eliminate uncertainty about the primary estimate, and the record does not report a multiplicity strategy for interpreting the broader set of outcomes.
The study also has important reporting limits. Some order data were missing, particularly near March 2020 at the start of the Covid-19 pandemic, and the registry does not state how those data were handled. The original sample-size justification and detailed statistical plan are not reported in the registry record. These limitations mean that the posted statistics should be understood as evidence about the reported outcome data and comparisons, rather than as a complete account of all analytic decisions or of the intervention’s broader consequences.
17. Why This Trial Matters Statistically
This trial is statistically interesting because its primary outcome is closely tied to the setting in which the intervention was delivered: purchases recorded through a food pantry’s online ordering system. That offers a direct behavioral measure of ordering within the platform, while also defining the limits of what the endpoint can represent. The outcome is not a direct measure of consumption, and its meaning depends partly on the pantry’s definition of a unit and healthier food.
The results also illustrate why a point estimate should be read together with its uncertainty. The average contrast is positive, but the confidence interval spans zero and the unadjusted p-value is 0.051. A directionally favorable estimate is not the same as precise evidence of a consistent difference. The reported variability in the endpoint further emphasizes that group averages can coexist with wide variation among participants.
The trial’s context makes the missing-order-data note especially relevant. Data were missing for some participants, particularly near March 2020 at the start of the Covid-19 pandemic. The registry does not document how that missingness was handled. This matters statistically because estimates from available records may not represent all participants equally if data availability is related to ordering behavior or other factors.
Finally, the multiple secondary outcomes show why outcome planning and interpretation matter. Purchasing, reported intake, biomarkers, and health measures answer related but distinct questions. Without a reported hierarchy or multiplicity approach, secondary comparisons should not be treated as a collection of independent confirmations. The study therefore offers a useful example of how design, endpoint choice, missing data, precision, and selective emphasis all shape the interpretation of a randomized trial.
18. Limitations
- The posted summary statistics are the only data these results rest on; patient-level data, full protocols and statistical analysis plans are not available.
- Recalculated results are simple, unadjusted calculations from posted summaries. They do not replace the sponsor's prespecified analysis.
The primary endpoint depends on online ordering records. It captures purchases through the pantry system, not all food acquisition or actual consumption. Its unit is defined by the pantry, and the record does not report a threshold for what size of change would be important to participants. These features constrain interpretation of the numerical difference beyond the ordering context.
Some order data were missing, especially near March 2020 at the start of the Covid-19 pandemic. The registry does not report the extent or group distribution of missingness, the method used to handle incomplete observations, or sensitivity analyses. It is therefore not possible to assess how much the missing records may have influenced the posted estimate.
The registry records enrollment as 286 while its brief summary describes a proposed trial among 500 regular food pantry clients. The record does not explain this difference. It also does not report the original sample-size justification, detailed eligibility criteria, or the rationale for the final enrollment. The planning illustrations based on the observed effect are not a substitute for those original design details.
Further limits include the absence of a detailed statistical analysis plan, model specification, masking details, and a stated multiplicity procedure. Although the trial is described as single-masked, the registry does not identify who was masked. The posted secondary results cover only some listed outcomes, and the record does not explain why other analyses are absent. These limitations restrict independent assessment of the full analytic process; they do not, by themselves, establish that the trial was conducted improperly.
19. Related Tutorials
20. Related Statistical Calculators
21. Sources
- ClinicalTrials.gov: A Trial of Behavioral Economic Interventions Among Food Pantry Clients — NCT04011384.
- Registry record last updated 2024-02-29; status last verified 2024-02.
- Results first posted 2024-02-29.
- Data retrieved from ClinicalTrials.gov on 2026-11-09.
- Group sizes, means, event counts and sponsor-reported p-values are reproduced as posted. Effect estimates, confidence intervals and p-values not attributed to the sponsor are independently calculated from the posted summary statistics.
Continue through the Clinical Biostats statistical pathway
Use this trial's endpoint as a starting point for deeper study of effect measures, confidence intervals, and sample-size planning.
22. Record Summary
The registry record describes a completed, randomized, single-masked, parallel-group trial of a web-based behavioral economic intervention among food pantry clients. The intervention platform included healthy defaults, placement features, traffic light nutrition labels, social norms messaging, and healthy swaps; the control group used the typical online ordering platform.
The primary outcome was average monthly change over 3 months in healthier-food units purchased per shopping trip. The posted estimate favored the intervention group, but its confidence interval spans zero and its unadjusted p-value is 0.051. Some order data were missing, especially near March 2020 at the start of the Covid-19 pandemic, and the registry does not report how missing data were handled.
Secondary outcomes include purchasing, dietary, biomarker, and health measures, but the registry does not report a complete set of numerical secondary analyses or a multiplicity strategy. The results are best interpreted as an uncertain comparison of recorded online purchasing, not as proof of changes in consumption or health.