← Clinical Trials
Underage Drinking NA Continuous Endpoint NCT03932097

REAL Parenting Alcohol and Substance Use App: Complete Statistical Analysis

This analysis examines a randomized trial of REAL Parenting digital materials versus active control materials, focusing on teens’ reported drinking frequency 12 months after baseline. It explains the posted comparison, its uncertainty, and the limits of what the registry record can establish.

Trial start: 2021-01-15  ·  Enrollment: 612  ·  Sponsor: Real Prevention, LLC
Scope of this record

This page provides an independent statistical analysis and educational interpretation of publicly reported results. ClinicalTrials.gov provides the official trial registry record. Numerical trial results on this page are restricted to the results posted in the ClinicalTrials.gov record; calculations added by Clinical Biostats are labelled as such.

1. Trial at a Glance

This page covers the registry results for REAL Parenting Alcohol and Substance Use App (NCT03932097), a randomized na study. The primary outcome is Frequency Past 3-month Drinking.

612
Enrollment
NA
2
Arms
In the primary outcome
0.05
Mean difference
REAL Parenting
-0.21–0.31
95% CI
Two-sided
FeatureRegistry value
ClinicalTrials.gov identifierNCT03932097
PhaseNA
ConditionUnderage Drinking, Alcohol Use, Underage
AllocationRandomized
Design modelParallel
MaskingSingle
Enrollment612
Arms in the primary outcome2
Lead sponsorReal Prevention, LLC
Results postedYes

2. Clinical Question

Population

The intervention was intended for parent-teen dyads connected with high school students. The primary drinking question was assessed only in teens. The registry record lists underage drinking as a condition and reports enrollment of 612.

Intervention

REAL Parenting was a digital parent-based intervention. It emphasized communication about drinking, risks of drinking and alcohol abuse, and added communication content about nicotine and marijuana risks.

Comparator

The active control consisted of National Institute on Alcohol Abuse and Alcoholism materials for parents about underage drinking. It was an active information comparison, not a no-materials control.

Primary question

At T5, 12 months after baseline, did the mean teen-reported score for drinking frequency in the past 3 months differ between REAL Parenting and active control materials? Higher scale scores represented more frequent drinking and a worse outcome.

The trial’s central statistical question is about a group difference at a specified follow-up, not whether each individual teen changed or whether the program altered every alcohol-related behavior. The primary measure asked teens how many days in the past 3 months they had at least one drink, with responses represented by an ordered score from 0 to 6. The posted means and comparison estimate describe the difference between the groups’ average scores at the follow-up assessment.

The registry record also lists secondary measures addressing marijuana use, combined alcohol and marijuana use, consequences of alcohol use, and nicotine use. Those outcomes broaden the behavioral scope, but they do not replace the stated primary endpoint. This distinction matters when interpreting a set of results: the primary comparison addresses the principal registered question, while secondary outcomes provide additional, generally less definitive evidence.

The clinical question should be read in the context of the comparator. Both groups received parent-oriented materials, but their content differed. The comparison therefore concerns REAL Parenting relative to the active control materials, rather than relative to no intervention or usual practice as separately defined in the registry record.

3. Trial Design

The registry record describes a randomized, single-masked, parallel-group trial comparing REAL Parenting with Active Control Materials. The listed enrollment is 612, and the study status is completed. The record gives a start date of 2021-01-15 and a primary completion date of 2022-01-15. Who was masked is not reported in the registry record, so the masking label cannot establish whether teens, parents, outcome assessors, or other people involved in the study were unaware of assignment.

The study summary describes inviting parent-teen dyads to participate, completing a baseline assessment, and providing parents with REAL Parenting or active control materials shortly afterward. The intervention was digital and addressed parent-teen or parent-young-adult communication about alcohol-related risks, with additional content about nicotine and marijuana. The active control was National Institute on Alcohol Abuse and Alcoholism material on underage drinking for parents. The record identifies the allocation as randomized but does not report the sequence-generation procedure, concealment process, or any stratification factors.

The primary outcome was assessed at T5, described as 12 months after baseline. It was a teen-reported measure of drinking frequency during the past 3 months. The primary analysis entries show means and standard errors for REAL Parenting and active control participants, along with an ANOVA comparison reported by the sponsor. The outcome-specific participant counts in the posted results are smaller than the enrollment figure; reasons for the difference and the analysis population definition are not reported in the registry record.

The record also includes three posted secondary outcome comparisons at the same 12-month follow-up. It lists 10 outcome measures and 10 analyses overall, but detailed results are posted for four secondary outcomes. The original trial’s sample-size assumptions, the planned handling of missing observations, and any interim analysis procedures are not reported in the registry record. The sample-size illustrations accompanying the statistical interpretation are calculations based on the observed effect, not evidence of the trial’s original design rationale.

4. Participant Flow and Accounting

The registry reports how participants moved through each stage of the study. All counts below are exactly as posted; missing counts are not inferred from other stages.

Before assignment: 1,155 parents were invited to participate in the study. 423 participants responded to the invitation. Of the 423 participants, 117 were not eligible for the study because teens were outside the age range (n=94), teens did not report an age (n=6), or teens were currently not enrolled in school (n=17). There were 306 parents eligible for the study. These parents were randomly assigned to treatment (REAL Parenting condition) or active control (NIAAA website condition) along with their teen (N=612).

Overall Study

StageREAL ParentingActive Control Materials
STARTED320292
T2 Parent Survey (3-Months Post-Baseline)126122
T2 Teen Survey (3-Months Post-Baseline)126122
T3 (6-Months Post-Baseline)124115
T4 (9-Months Post-Baseline)132106
T5 (12-Months Post-Baseline)127103
COMPLETED254206
NOT COMPLETED6686

Analysis population for the primary outcome, as defined in the registry: This question was only assessed in teens.

5. Baseline Characteristics

The table reproduces the baseline characteristics posted in the registry, which describe the participants analysed at baseline and help show whether the randomized groups were comparable.

Population: The overall number of baseline participants includes both parents and their teens. REAL Parenting includes 320 parents and teens (160 parent-teen dyads) and Active Control includes 292 participants (146 parent-teen dyads).

CharacteristicUnitREAL ParentingActive Control MaterialsTotal
Participants analysed320292612
Age, Continuous: Parentsyears45.64 (SD 7.18)46.35 (SD 6.62)45.96 (SD 6.91)
Age, Continuous: Teensyears16.31 (SD 0.95)16.42 (SD 0.79)16.36 (SD 0.92)
Sex/Gender, Customized: Teens – MaleParticipants7564139
Sex/Gender, Customized: Teens – FemaleParticipants8178159
Sex/Gender, Customized: Teens – Transgender man/trans man/female-to-male (FTM)Participants011
Sex/Gender, Customized: Teens – Transgender woman/trans woman/male0to-female (MTF)Participants101
Sex/Gender, Customized: Teens – Genderqueer/gender nonconforming neither exclusively male nor femaleParticipants224
Sex/Gender, Customized: Teens – Decline to answerParticipants112
Sex: Female, Male: Parents – FemaleParticipants9881179
Sex: Female, Male: Parents – MaleParticipants6265127
Ethnicity (NIH/OMB): Hispanic or LatinoParticipants413475
Ethnicity (NIH/OMB): Not Hispanic or LatinoParticipants117112229
Ethnicity (NIH/OMB): Unknown or Not ReportedParticipants202
Race (NIH/OMB): American Indian or Alaska NativeParticipants336
Race (NIH/OMB): AsianParticipants9918
Race (NIH/OMB): Native Hawaiian or Other Pacific IslanderParticipants000
Race (NIH/OMB): Black or African AmericanParticipants241135
Race (NIH/OMB): WhiteParticipants112111223
Race (NIH/OMB): More than one raceParticipants101121
Race (NIH/OMB): Unknown or Not ReportedParticipants213
Race/Ethnicity, Customized: Parents – White, Non-HispanicParticipants9097187
Race/Ethnicity, Customized: Parents – Black, Non-HispanicParticipants261339
Race/Ethnicity, Customized: Parents – Other, Non-HispanicParticipants101020
Race/Ethnicity, Customized: Parents – HispanicParticipants302151
Race/Ethnicity, Customized: Parents – 2+ Races, Non-HispanicParticipants459
Frequency past 3-month drinkingunits on a scale0.32 (SD 0.77)0.29 (SD 0.80)0.31 (SD 0.78)

6. Primary Endpoint

The registry lists the primary outcome as Frequency Past 3-month Drinking, measured over T5 (12-months post baseline).

ElementRegistry entry
Result typeMean
Measure of spreadStandard Error
Unitunits on a scale
Endpoint classificationContinuous Endpoint

The primary endpoint is the frequency of drinking during the past 3 months, assessed in teens at T5, 12 months after baseline. Teens were asked how many days in that period they had at least one alcoholic drink. Responses were encoded on a scale from 0 to 6: 0 represented 0 days, and 6 represented 30 days or more, with the intervening scores corresponding to the stated ranges of drinking days. Higher scores indicated more frequent drinking and a worse outcome.

This endpoint is used to represent recent drinking frequency at the specified follow-up. It targets a behavior relevant to the intervention’s alcohol-related communication content and allows the two randomized groups to be compared on a common outcome. The registry record describes the endpoint as a mean, so the posted summary compares average scores rather than reporting the proportion of teens in each response category.

There is an interpretive distinction between the coded score and an exact count of drinking days. The response options are ordered ranges, and the highest category is open-ended. As a general statistical consideration, a mean of such category scores summarizes the coded responses; it should not be read as a mean number of actual drinking days. The registry record does not report a separate analysis of category distributions or a clinical threshold for interpreting a particular mean difference.

The primary endpoint is one measured aspect of alcohol use, at one follow-up point. It does not by itself describe other possible outcomes, the full course of behavior between assessments, or whether any observed group difference would be important to participants or families.

7. Results Posted in the Registry

Mean difference: REAL Parenting

0.05

95% CI: -0.21 to 0.31

Recalculated from the posted summary statistics

Primary outcome by arm, with 95% confidence intervals
Primary outcome by arm with 95% confidence intervals0.20.30.40.50.60.7REAL Parenting (n=125)Active Control Materials (n=103)units on a scale (95% CI)
Treatment effect versus the control arm
Forest plot of mean difference vs control (95% CI)REAL Parenting-0.26-0.13-0.010.110.230.36mean difference vs control (95% CI)

8. Secondary Efficacy Endpoints

The registry also posts results for secondary outcomes. For each outcome below, the endpoint type, the test applied to the posted summary statistics, the result and its interpretation are given in turn. Secondary endpoints are usually not adjusted for multiple testing, so they support rather than confirm the primary finding.

Secondary outcomeTypeEffect measureEstimate (95% CI)Planning size, 80% power
Frequency of Past 3-month Marijuana UseContinuous EndpointMean difference-0.13 (-0.34 to 0.08)594 per arm
Consequences of Alcohol UseContinuous EndpointMean difference-0.01 (-0.93 to 0.91)1,894,569 per arm
Frequency of Past 3-month Nicotine UseContinuous EndpointMean difference-0.03 (-0.32 to 0.26)21,870 per arm

Frequency of Past 3-month Marijuana Use

Treatment effect versus the control arm
Forest plot of mean difference vs control (95% CI)REAL Parenting-0.37-0.28-0.18-0.080.020.11mean difference vs control (95% CI)

Consequences of Alcohol Use

Treatment effect versus the control arm
Forest plot of mean difference vs control (95% CI)REAL Parenting-1.07-0.65-0.220.200.631.05mean difference vs control (95% CI)

Frequency of Past 3-month Nicotine Use

Treatment effect versus the control arm
Forest plot of mean difference vs control (95% CI)REAL Parenting-0.37-0.23-0.100.040.170.31mean difference vs control (95% CI)

The posted secondary outcomes extend the assessment beyond drinking frequency to marijuana use, combined alcohol and marijuana use, consequences of alcohol use, and nicotine use. Across these measures, the estimates do not form a consistent pattern of clear separation between the intervention and active control groups. The posted comparisons are generally uncertain, and the secondary results do not provide a strong, coherent basis for a different overall reading from the primary outcome.

The secondary outcomes are related to the broader content of REAL Parenting, which addressed communication about alcohol, nicotine, and marijuana risks. Their relevance does not make them interchangeable with the primary endpoint: each captures a different behavior or consequence, and their scales and meanings differ. Together, they can provide context about the scope of measured outcomes, but they do not establish effects on behaviors that were not assessed or at times not reported.

Secondary results usually carry less weight than a prespecified primary result because multiple outcomes create more opportunities for apparently notable findings to arise by chance. When many comparisons are examined, an unadjusted p-value does not account for that multiplicity. The registry record lists 10 outcome measures and 10 analyses, while detailed secondary results are posted for four outcomes. Whether a multiplicity adjustment was planned or applied is not reported in the registry record.

Accordingly, the secondary comparisons are best treated as additional, exploratory context unless the registry record reports a prespecified testing hierarchy or adjustment strategy. Neither is reported. Their interpretation should account for both statistical uncertainty and the fact that the outcomes represent separate aspects of substance use rather than repeated measurements of one identical endpoint.

Secondary outcomes registered for this trial, for reference (only those above were recalculated):

9. Statistical Interpretation

Clinical Biostats interpretation

At the 12-month assessment, the mean primary drinking-frequency score was 0.5 in the REAL Parenting group and 0.45 in the active control group. The recalculated unadjusted mean difference, expressed as REAL Parenting minus active control, was 0.05. Its confidence interval ranged from -0.214 to 0.314, and the recalculated unadjusted p-value was 0.71. The sponsor’s reported ANOVA p-value was 0.70.

The estimated difference is close to zero on the coded scale, but the confidence interval is important to the interpretation. It includes zero and is compatible with differences in either direction. It also extends to values that are not especially close to zero on this scale. Thus the observed data do not pin down the group difference precisely enough to treat the point estimate as the only plausible value. The results do not establish that the groups are exactly equivalent.

Direction should be interpreted using the endpoint’s scoring rule. A higher score indicates more frequent reported drinking and is designated a worse outcome. The point estimate is slightly higher for REAL Parenting than for active control, while the confidence interval also includes possible values favoring REAL Parenting. The data therefore do not provide a precise directional conclusion for the primary outcome.

The reported p-values are not measures of the probability that the intervention works, nor do they quantify the size or importance of any difference. They describe how compatible the observed test statistic is with a null hypothesis under the assumptions of the analysis. A p-value that is not small does not prove that there is no effect. Precision and the range of effects compatible with the data are better conveyed by considering the estimate alongside its confidence interval.

The record reports a randomized comparison, which is a useful design for comparing assigned groups, but the posted primary outcome counts do not account for all listed enrollment. Reasons for missing outcome data, the analysis population, and the missing-data approach are not reported in the registry record. Those omissions limit assessment of possible bias and of how well the analysis preserves the randomized comparison.

Overall, the posted primary results show a small estimated mean-score difference with substantial uncertainty around its direction and magnitude. They are not conclusive evidence of a meaningful advantage for either set of materials, and they do not establish equivalence. Interpretation should remain specific to teen-reported drinking-frequency scores at the stated follow-up and to the comparison with the active control materials.

10. Statistical Methods Explained

The registry record reports an ANOVA for the primary comparison and gives group means with standard errors. The accompanying recalculation is an unadjusted comparison of the posted summary statistics. The following explanations describe what these statistics mean in general and how they apply to this trial’s reported outcome.

What does a mean difference describe?

A mean is the arithmetic average of the recorded scores within a group. The primary outcome is a coded frequency scale, so each participant’s response is represented by a score from 0 to 6. The mean difference compares the groups’ average scores at the assessment time. In the recalculation, the difference is defined as the REAL Parenting mean minus the active control mean. A positive estimate therefore means the REAL Parenting mean is higher on this scale; because higher scores indicate more frequent drinking, that direction corresponds to a worse outcome on the measure. A negative estimate points in the opposite direction.

A mean difference is not a count of additional drinking days. The score categories represent ranges of days, and the highest category represents 30 days or more. A difference in average score should therefore be interpreted in coded scale units, not translated into a specific number of days. The registry record does not report a category-by-category distribution or an alternative model for the ordered responses.

What does the standard error represent?

A standard error describes the statistical uncertainty in an estimated quantity, such as a group mean. It is not the spread of individual participants’ outcomes. In the posted primary results, the standard errors are 0.09 for REAL Parenting and 0.1 for active control. They help describe how precisely the sample means estimate the corresponding group averages under the assumptions of the analysis. Standard errors are also used when estimating the uncertainty of a difference between group means.

In general, a standard error tends to be smaller when an estimate is based on more information and when the underlying observations are less variable. It does not reveal whether observations are missing systematically, whether the model assumptions are appropriate, or whether a difference is clinically important. Those questions require other information and analyses.

How should the confidence interval be read?

A confidence interval gives a range of values that are reasonably compatible with the data under the statistical method used. The primary recalculated interval runs from -0.214 to 0.314 scale units. Because it includes zero, the results are compatible with no mean difference as well as differences in either direction. The interval is not a guarantee that the true value lies within its bounds, and it does not assign a probability to the particular values inside the interval.

Precision is reflected in the width of the interval: a narrower interval indicates less uncertainty about the estimate, while a wider interval indicates more. The interval should be considered together with the estimate and with the meaning of the scale. Statistical compatibility does not establish that every value in the interval is important in practice; the registry record does not report a threshold for a meaningful difference.

What does ANOVA test?

ANOVA, or analysis of variance, is a method for testing whether group means differ more than would be expected from random variation under a specified model. The registry record reports an ANOVA p-value of 0.70 for the primary outcome. The recalculated unadjusted p-value from the posted summary statistics is 0.71. These values are close, and the record marks the discrepancy flag as false.

In general, a p-value describes the compatibility of the observed result with a null hypothesis under the test’s assumptions. It is not the probability that the null hypothesis is true, and it does not indicate the probability that the intervention will have a particular effect. A large p-value does not demonstrate that groups are equivalent; that conclusion would require an equivalence framework and a prespecified margin, neither of which is reported in the registry record.

Why might outcome distributions matter?

In general, statistical methods based on means and standard errors can be sensitive to how observations are distributed and how the measure is constructed. Over-dispersion means that observed variability is greater than a model expects. When present, it can make standard errors too small if the model does not account for the extra variability, leading to confidence intervals that are too narrow and tests that are too optimistic. This is a general consideration, not a finding that over-dispersion occurred in this trial. The registry record does not report diagnostic information that would establish whether this concern applies.

The primary measure is formed from ordered response categories rather than exact drinking-day counts. Summarizing it by a mean is a recognizable way to compare average coded scores, but it treats changes in the score as numerical units. The widths of the underlying day ranges are not uniform, and one category is open-ended. That limits direct interpretation of the mean as a behavioral quantity and makes the actual category pattern potentially informative. Such a distribution is not reported in the registry record.

What is the role of the recalculation?

The recalculated estimate and confidence interval are simple, unadjusted re-calculations from the posted group summary statistics. They are not an adjusted model, do not account for unreported covariates, and do not substitute for the sponsor’s analysis. The sponsor’s ANOVA result is therefore presented alongside them. Because the record does not flag a discrepancy, the two reported p-values can be described as close without claiming that the analyses were identical.

The estimates describe the posted outcome data at the specified follow-up. They do not by themselves address missing-data bias, adherence to assigned materials, effects at other times, or outcomes not measured by the primary scale. Details of those analyses and of the original statistical plan are not reported in the registry record.

11. Confidence Intervals and Precision

The primary recalculated mean difference is 0.05 scale units, with a confidence interval from -0.214 to 0.314. The interval spans zero, so the data are compatible with no group difference, as well as with a difference in either direction. Its bounds also indicate that the estimate is not highly precise: a range of possible values remains compatible with the posted data under the calculation’s assumptions.

Confidence intervals are useful because they convey both the estimated direction and uncertainty, whereas a p-value alone does not show the range of plausible effects. The interval is not a probability distribution for the true difference and should not be read as assigning equal likelihood to every value between its bounds. It also does not determine whether a value is meaningful to families or clinicians. The registry record does not report a threshold for a minimally important difference on this coded scale.

The point estimate is slightly higher in the REAL Parenting group, but the interval also includes lower values. As higher scores indicate more frequent drinking, the point estimate alone should not be used to make a firm directional claim. The uncertainty encompasses both a possible difference favoring active control and one favoring REAL Parenting.

Precision also depends on the outcome’s measurement and the analysis population. The response scale uses ordered ranges rather than exact numbers of days, and the posted primary outcome counts do not explain why the numbers assessed differ from the listed enrollment. The handling and reasons for missing outcome data are not reported in the registry record. These limitations make it important to interpret the interval as uncertainty about the posted comparison, not as a complete description of every uncertainty in the trial.

12. Planning a Similar Trial: Sample Size

This is a planning illustration, not a re-evaluation of the original trial. It asks: if a new study wanted to detect a difference like the one observed, or a smaller one, how large would it need to be? It is deliberately not “observed power” of the finished trial, which is not a meaningful quantity.

Effect to detectParticipants per arm, 80% powerParticipants per arm, 90% power
100% of the observed effect6,4088,578
75% of the observed effect11,39115,249
50% of the observed effect25,62934,310
Sample size needed versus the size of the effect to detect
Sample size needed versus size of effect to detect5,00010,00020,00050,000100,00040%60%80%100%80% power90% powersize of effect to detect, as % of the effect observed (log vertical scale: participants per arm)
Treat these figures cautiously. The observed difference is imprecise: its confidence interval includes no difference at all. A sample size built on a noisy estimate can be far too small or too large, so the smaller-effect rows are the more cautious guide.
Method and assumptions. Two-sided α = 0.05, 1:1 allocation, two-sample comparison of means with the pooled standard deviation. Calculated for the comparison: REAL Parenting.

The sample-size illustrations are planning calculations based on the observed effect, not the trial’s original design assumptions. They give participants per arm for 80% power and participants per arm for 90% power. They do not give total trial enrollment or total event counts. The table’s rows consider planning for 100%, 75%, and 50% of the observed effect; the curve provides a visual way to understand how the required sample size changes with the effect being targeted.

The stated planning assumptions are a two-sided significance level of 0.05, equal allocation between the two arms, and a normal approximation. For each row, the effect to detect is the stated fraction of the observed effect. The 80% power column means the planned design would have an 80% chance of detecting an effect of that specified size at the stated significance level, if the assumptions hold. The 90% power column describes a higher chance of detecting that effect under the same kind of assumptions. These are planning probabilities, not probabilities that the intervention is effective.

Smaller effects are harder to distinguish from random variation, so planning to detect a smaller effect generally requires more participants. The table illustrates that relationship by showing larger requirements for the smaller fractions of the observed effect. Because the primary observed effect is close to zero, calculations based on that estimate can yield very large planning requirements. Such figures should not be treated as a recommendation or as evidence that the original trial was designed to detect those effects.

This is not observed power. It is a set of hypothetical planning illustrations calculated after observing the effect, and it does not describe the probability that the completed study detected an effect. The trial’s own original sample-size justification and design assumptions are not reported in the registry record.

13. How the Sample Size Is Derived

Power is the probability that a study of a given size rejects the null hypothesis when a specific true effect exists. It depends on four things: the effect to detect, the variability of the outcome, the significance level (here a two-sided α of 0.05, so z1−α/2 = 1.960) and the desired power (z1−β = 0.842 for 80% and 1.282 for 90%).

Formula

n = 2 (z1−α/2 + z1−β)2 σ2 / Δ2

n is the number of participants in each arm, Δ is the difference in means to detect and σ is the common standard deviation.

Worked calculation for this trial

  1. Observed difference Δ = 0.050; pooled standard deviation σ = 1.010.
  2. n = 2 × (1.96 + 0.842)2 × 1.0102 / 0.0502 = 6,407.2, rounded up to 6,408 participants per arm for 80% power.

The smaller-effect rows in the table above repeat the same calculation with the effect multiplied by 0.75 and 0.5 (for ratios, applied to the logarithm of the ratio).

R code

alpha <- 0.05
z_a   <- qnorm(1 - alpha / 2)          # two-sided
delta <- 0.05                      # observed difference in means
sigma <- 1.01014762786857                     # pooled standard deviation
n_per_arm <- function(power, frac = 1) {
  ceiling(round(2 * (z_a + qnorm(power))^2 * sigma^2 / (delta * frac)^2, 6))
}
for (f in c(1, 0.75, 0.5))
  cat(sprintf("%3.0f%% of observed effect: n per arm = %s (80%% power), %s (90%% power)\n", 100 * f,
              format(n_per_arm(0.80, f), big.mark = ","), format(n_per_arm(0.90, f), big.mark = ",")))

# Check against R's built-in function (t-based, so very slightly larger)
cat("power.t.test:", round(power.t.test(delta = delta, sd = sigma, power = 0.80)$n, 1), "\n")

# Simulation: exact two-sample t-test using simulated means and variances
set.seed(3932097)
n    <- n_per_arm(0.80)
reps <- 20000
m_diff <- rnorm(reps, delta, sigma * sqrt(2 / n))             # difference in sample means
s2     <- sigma^2 * rchisq(reps, 2 * (n - 1)) / (2 * (n - 1))   # pooled sample variance
tstat  <- m_diff / sqrt(s2 * 2 / n)
cat(sprintf("Simulated power at n = %s per arm: %.3f (%d replicates)\n", format(n, big.mark = ","), mean(abs(tstat) > qt(0.975, 2 * (n - 1))), reps))

Output from running the code

100% of observed effect: n per arm = 6,408 (80% power), 8,578 (90% power)
 75% of observed effect: n per arm = 11,391 (80% power), 15,249 (90% power)
 50% of observed effect: n per arm = 25,629 (80% power), 34,310 (90% power)
power.t.test: 6408.1 
Simulated power at n = 6,408 per arm: 0.801 (20000 replicates)

The simulation draws many sets of group means and variances at the planned sample size and counts how often the two-sample t-test rejects the null. It should land close to the target power; at very small required sample sizes it can fall below it, because the normal approximation is rough there.

14. Analyses Reported by the Sponsor

MethodEstimate95% CIP-value
ANOVAnot reported0.700

The sponsor’s primary analysis is reported as an ANOVA, with a p-value of 0.70 for the comparison of REAL Parenting and active control materials. The unadjusted recalculation from the posted group summary statistics gives a mean difference of 0.05 scale units, a confidence interval from -0.214 to 0.314, and a p-value of 0.71. The record does not flag a discrepancy between the reported result and the recalculation.

The recalculation is useful for expressing the comparison as an estimated difference with an uncertainty interval. It is a simple, unadjusted calculation from the posted means, standard errors, and group counts. It does not reconstruct an individual-level analysis or establish that the sponsor used the same assumptions, data handling, or model details. The sponsor’s reported ANOVA remains the registry’s own analysis, while the recalculation supplies a complementary summary of magnitude and precision.

The close p-values should not be taken to mean that the methods are identical or that every possible analytical choice would produce the same result. Details such as covariate adjustment, analysis population, and missing-data handling are not reported in the registry record. The available information supports describing the sponsor’s reported test and the straightforward summary-statistic recalculation, without inferring unreported analysis procedures.

15. Multiplicity, Missing Data and Analysis Populations

Multiplicity arises when several outcomes or statistical tests are considered. Even if each test uses a conventional significance threshold, examining many tests increases the chance of at least one seemingly notable result occurring by chance. A multiplicity strategy can address this by controlling error across a defined family of comparisons or by specifying an ordered testing plan. Whether the trial used such a strategy is not reported in the registry record.

The record lists 10 outcome measures and 10 analyses and posts results for the primary endpoint and four secondary outcomes. It does not report whether the posted secondary tests were prespecified, whether adjustments were made, or how the outcomes were grouped into testing families. For that reason, the secondary p-values should not be interpreted as if they were isolated, confirmatory tests independent of all other analyses. The primary result remains the clearest reference point for the stated main question, while secondary results provide supporting context with greater multiplicity concerns.

Missing-data information is also limited. The primary outcome counts differ between the two groups, and those outcome counts are below the listed enrollment of 612. The registry record does not explain the reasons for these differences, report the amount or pattern of missingness over time, or describe whether missing values were imputed or handled by another method. Without these details, it is not possible to assess whether the analyzed observations represent the randomized groups or whether missingness could have influenced the comparison.

The analysis populations are not reported in the registry record. It is therefore unclear from the posted information whether the primary analysis followed an intention-to-treat principle, included only participants with observed follow-up data, or used another defined population. The record also does not report protocol deviations or adherence information that would clarify how assignment and receipt of materials were handled analytically.

These are reporting limitations, not evidence that a particular procedure was or was not performed. They restrict evaluation of the robustness of the findings and reinforce the need to keep conclusions tied to the posted analyses and their stated uncertainty.

16. Safety Results

The registry reports serious adverse events as affected participants divided by participants at risk.

ArmParticipants with serious adverse eventsParticipants at riskPercentage
REAL Parenting03200.0%
Active Control Materials02920.0%

Deaths

ArmDeaths (participants affected of at risk)
REAL Parenting0 of 320
Active Control Materials0 of 292

Other (non-serious) adverse events

Participants with at least one non-serious event at or above the registry's 0% reporting threshold.

ArmParticipants affected of at risk
REAL Parenting0 of 320
Active Control Materials0 of 292
Do not overextend the safety comparison. Serious-adverse-event counts are safety observations, not efficacy outcomes, and should not be combined with the primary result into a single benefit-risk number.

The serious adverse event figures report zero serious adverse events in both groups. The registry lists 320 participants at risk in REAL Parenting and 292 at risk in Active Control Materials for the serious-event fields, with zero serious events affecting participants in either group. The record also lists zero deaths in both groups. No serious-event terms are listed.

These figures should be read as the safety information recorded for the trial, not as proof that no adverse experiences of any kind occurred. The registry also records zero other adverse events in both groups, but adverse-event reporting depends on how events were defined, collected, and entered. The record does not provide further detail about event surveillance, severity categories beyond the listed fields, or the relationship of any event to study materials.

The at-risk figures in the safety record are not the same as the outcome-specific analysis counts reported for the drinking endpoint. They describe the denominators attached to the safety entries. The posted information supports stating that no serious adverse events or deaths were recorded in these fields; it does not support broader claims about safety beyond the reporting captured in the registry record.

17. What the Result Does, and Does Not, Mean

The posted results estimate the difference between groups in teens’ average coded drinking-frequency scores at T5, 12 months after baseline. The primary mean difference is close to zero, and its confidence interval includes zero and values on both sides. The sponsor’s ANOVA p-value and the unadjusted recalculated p-value are not small. Taken together, these results do not provide a precise indication that one set of materials produced a different average score from the other in this comparison.

That interpretation is narrower than saying the intervention has no effect. A result that does not clearly distinguish the groups is not proof of equivalence, and the registry record does not report an equivalence design or an acceptable margin for declaring the groups sufficiently similar. The uncertainty interval leaves a range of possible differences, and missing-data and analysis-population details are not reported.

The result also does not answer every question about alcohol use. The endpoint is based on a teen’s report at one follow-up and uses ordered categories for drinking days in the past 3 months. It does not directly measure exact days, alcohol quantity, all consequences, or behavior at every point during follow-up. Secondary outcomes address some additional behaviors and consequences, but their results are subject to uncertainty and the unreported multiplicity approach.

The comparator matters. The finding concerns REAL Parenting relative to active control materials for parents about underage drinking. It is not a comparison with no intervention, and it should not be generalized to a different control condition without evidence. Nor does the posted outcome establish whether parents used the materials as intended or how communication changed, since those results are not reported in the information available.

In practical terms, the data support a cautious description of an uncertain group difference on the registered primary scale. They do not justify a claim that the intervention works, does not work, is equivalent to control, or changes all forms of substance use. Such broader conclusions would require additional evidence and fuller reporting of the analysis and outcome data.

18. Why This Trial Matters Statistically

This trial raises an important statistical issue for digital behavioral interventions: the estimated effect is interpreted relative to an active comparison, not an absence of information or contact. Statistical conclusions depend on that contrast. A mean difference between REAL Parenting and parent-oriented underage drinking materials answers a more specific question than a comparison with no materials would answer.

The study also illustrates why an effect estimate and its uncertainty matter alongside a p-value. The primary p-values do not indicate a clear separation, but that alone does not establish that the groups are identical. The confidence interval shows the range of differences compatible with the posted summary-statistic calculation, making the uncertainty visible and discouraging an overly simple yes-or-no reading.

Measurement is another central consideration. The primary endpoint uses ordered response categories to summarize drinking frequency. Reporting the mean makes a group comparison possible, but the score is not an exact count of drinking days, and the category structure affects interpretation. A more complete statistical picture could include outcome distributions and details of how the scale was analyzed; those are not reported in the registry record.

Finally, the registry record makes clear why transparent reporting matters. Outcome-specific counts differ from enrollment, while missing-data methods, analysis populations, multiplicity procedures, and original sample-size assumptions are not reported. These omissions do not negate the posted results, but they limit how confidently readers can assess robustness and generalizability. Clear reporting of the design, analysis choices, and uncertainty is essential when evaluating a behavioral intervention whose outcomes include multiple related measures.

19. Limitations

The primary outcome is a teen-reported, categorized measure of drinking frequency. Its scores represent ranges of days rather than exact counts, and the highest response category is open-ended. The registry record does not post the distribution across response categories, so readers cannot see whether the means reflect broadly similar response patterns or different patterns that average to similar scores.

The primary result is limited to T5, 12 months after baseline, and to the past-3-month drinking-frequency question. The record does not provide primary results at other follow-up times, so it cannot establish the trajectory of behavior across the full study period. Although the study summary refers to sustained effects across follow-up, the posted primary outcome analyzed is specifically the 12-month assessment.

The outcome-specific participant counts are lower than the listed enrollment, and counts differ between the two primary groups. Reasons for these differences, missing-data procedures, and analysis-population definitions are not reported in the registry record. This makes it difficult to judge whether loss of outcome information could have affected the comparison.

The control condition received parent materials about underage drinking, so findings apply to the contrast with that active control rather than to no intervention. The registry record also does not identify who was masked, describe the masking procedure, or report intervention adherence. It does not report a multiplicity strategy for the several outcomes and analyses, or the original sample-size justification. These reporting gaps constrain interpretation; they should not be treated as evidence that particular methods were absent.

20. Related Tutorials

21. Related Statistical Calculators

22. Sources

Continue through the Clinical Biostats statistical pathway

Use this trial's endpoint as a starting point for deeper study of effect measures, confidence intervals, and sample-size planning.

23. Record Summary

NCT03932097 is the completed randomized, single-masked, parallel-group study titled REAL Parenting Alcohol and Substance Use App. The registry lists enrollment of 612 and compares the digital REAL Parenting intervention with Active Control Materials, described as National Institute on Alcohol Abuse and Alcoholism materials on underage drinking for parents. The study began on 2021-01-15, with primary completion on 2022-01-15.

The primary outcome was teen-reported frequency of drinking in the past 3 months at T5, 12 months after baseline, expressed as a mean score on a scale from 0 to 6. The recalculated unadjusted mean difference was 0.05, with a confidence interval from -0.214 to 0.314; the sponsor reported an ANOVA p-value of 0.70. The interval includes zero and supports a cautious interpretation of an uncertain group difference rather than a conclusion of equivalence.

The record also posts secondary substance-use and consequence outcomes and safety fields showing zero recorded serious adverse events and deaths in both groups. Missing-data methods, analysis populations, multiplicity procedures, and the original sample-size assumptions are not reported in the registry record.

Clinical Biostats