Keep It Up! 2.0: A Comparison of Two Online HIV Intervention Programs for Young Men Who Have Sex With Men: Complete Statistical Analysis

1. Trial at a Glance

This page covers the registry results for Keep It Up! 2.0: A Comparison of Two Online HIV Intervention Programs for Young Men Who Have Sex With Men (NCT01836445), a randomized na study. The primary outcome is Participants Self-Reporting Condomless Anal Sex at Baseline and 3 Months.

2. Clinical Question

The study compares two behavioral online programs rather than an online program with no intervention. Therefore, the question is about the relative outcomes associated with the YMSM-specific program and general HIV knowledge content. The registry also describes planned follow-up contacts and surveys after program completion, with additional assessments over the course of a year. The primary outcome is focused on the baseline-to-three-month period, not on every follow-up or outcome listed for the study.

The study description presents reductions in condomless anal sex and new STI infections as hypotheses. Those statements describe the research question; they are not reported findings. Results for new STI infections, including any comparison between arms, are not reported in the registry record. The same is true of the primary self-reported sexual-risk outcome. Accordingly, the clinical question can be described, but whether the posted results support either hypothesis cannot be determined from this record.

3. Trial Design

The registry describes a randomized, double-masked, parallel-group study comparing Keep It Up! with HIV Knowledge Control. Random allocation means assignment to the two arms was intended to be determined by chance. A parallel design means participants were assigned to one program rather than receiving both programs in sequence. The record labels masking as double, but does not report which people or roles were masked, how masking was maintained, or whether it was successful.

The registry reports enrollment of 901 participants. The study description says that a total of 900 YMSM would be enrolled, so the description and the enrollment field state different totals. The record does not explain that difference. The sponsor is Northwestern University. The listed start was 2013-05, primary completion was 2017-03, and the study status is completed. The phase is recorded as NA.

Recruitment is described as occurring through community-partner clinics in Chicago, New York, and Atlanta. The intervention descriptions indicate that both study arms involved online behavioral content. Keep It Up! was developed specifically for young men who have sex with men; the control was described as general-audience HIV information that was not tailored and was non-interactive. Detailed intervention content beyond the excerpts in the registry record is not reported there.

The study description says participants would also take at-home urine and rectal tests for chlamydia and gonorrhea, and would be contacted for follow-up sessions and surveys after completing the program. These are descriptions of the planned study procedures. Completion rates, adherence to the assigned programs, and the number of participants contributing data at each follow-up are not reported in the registry record. The schedule and detailed procedures for those contacts are also not fully reported.

Although the record identifies randomization and the primary outcome, it does not report the allocation-generation procedure, concealment process, eligibility criteria in full, or the statistical plan. The sample-size justification and any interim analyses are not reported in the registry record. These omissions limit what can be inferred about how the trial was designed to estimate the comparison and how it was ultimately analyzed.

4. Participant Flow and Accounting

The registry reports how participants moved through each stage of the study. All counts below are exactly as posted; missing counts are not inferred from other stages.

Before assignment: Participants were randomized after completing a baseline assessment and STI testing.

Intervention Modules

3 Month Follow-up Assessment

6 Month Follow-up Assessment

12 Month Follow-up Assessment

Analysis population for the primary outcome, as defined in the registry: Enrolled participants who completed assessments on their sexual risk behaviors (e.g. condomless anal sex (CAS) acts) at baseline and 3 month follow-up.

5. Baseline Characteristics

The table reproduces the baseline characteristics posted in the registry, which describe the participants analysed at baseline and help show whether the randomized groups were comparable.

6. Primary Endpoint

The registry lists the primary outcome as Participants Self-Reporting Condomless Anal Sex at Baseline and 3 Months, measured over Baseline, 3 Months.

The primary outcome is titled Participants Self-Reporting Condomless Anal Sex at Baseline and 3 Months. It measures the change in self-reported condomless anal sex acts at the three-month follow-up, with baseline assessment serving as the starting reference. The population specified for this outcome consists of enrolled participants who completed assessments of sexual-risk behaviors at both baseline and three months.

This outcome is used to assess a behavioral target of the prevention programs. Self-report can capture behavior that is not directly observable in a study visit, but it depends on participants recalling and reporting their experiences. In general, self-reported outcomes can be affected by recall and by how comfortable participants feel disclosing sensitive information. These are general considerations, not evidence that reporting bias occurred in this trial.

The registry classifies the outcome as a count of participants and describes its kind as participants with an event. It does not provide the event definition, the exact response categories, the coding rule for change, or the arm-specific counts at either assessment. It also does not report whether the primary analysis compared follow-up values, changes from baseline, or another summary of the repeated assessments.

The outcome title and timeframe identify what was intended to be evaluated, but no numerical primary outcome result or statistical comparison is reported in the registry record. Thus, the endpoint can be described, while the size, direction, and precision of the observed between-arm contrast cannot be evaluated from the posted results.

7. Why No Re-analysis Is Shown

8. Statistical Interpretation

9. Statistical Methods Explained

The trial is described as randomized and parallel-group. In general, randomization aims to make the assigned groups comparable with respect to both measured and unmeasured characteristics, on average. It does not guarantee that the groups will be identical, and the validity of a randomized comparison also depends on how assignment was generated and concealed and on how participants were handled after assignment. The registry identifies random allocation but does not report the generation or concealment procedures.

What does the parallel-group design mean?

In a parallel-group trial, each participant is assigned to one study arm, and outcomes are compared across those arms. Here, the named arms are Keep It Up! and HIV Knowledge Control. The record labels the study double-masked. Masking can reduce the influence of expectations on behavior, outcome assessment, or analysis, depending on who is masked. The registry does not identify the masked parties or describe how masking worked for these online programs. Because the interventions differ in tailoring and interactivity, whether participants could distinguish their program assignments is not reported in the registry record.

How should the primary outcome be understood?

The primary outcome concerns self-reported condomless anal sex at baseline and three months. The stated outcome description is change in self-report at three months. The record identifies a count-of-participants outcome and classifies its kind as participants with an event. It does not explain how reported behavior was converted into an event, how change was calculated, or what statistical model was used. Those details matter because a count, a binary indicator, and a change score can require different analysis approaches. The primary analysis method is not reported in the registry record.

In general, an analysis of repeated measurements can account for the fact that observations from the same participant are related. A comparison of change can also depend on the baseline value and the way the outcome is summarized. These are general methodological considerations; the registry does not say whether the trial used a repeated-measures model, baseline adjustment, a change-score analysis, or another method. It would be inappropriate to assume a particular approach.

What does statistical precision tell readers?

An effect estimate describes the direction and size of a comparison on a particular outcome scale. A confidence interval conveys uncertainty around that estimate under the model and assumptions used. In general, greater precision means a narrower interval, while limited data or substantial variability can produce a wider interval. The registry information for the primary outcome does not include an estimate or confidence interval, so precision cannot be evaluated. A p-value, which summarizes how compatible the data are with a specified null hypothesis under a statistical model, is also not reported.

Statistical significance alone would not establish whether a difference matters in practice. Interpretation would also depend on the size and uncertainty of the estimate, the outcome’s meaning, and the study’s limitations. Since the primary estimate is not reported, neither statistical significance nor practical importance can be assessed.

What do the planning assumptions mean?

The record describes a planning illustration based on the observed effect, with a two-sided significance level of 0.05, equal allocation between arms, and a normal approximation. It says that the effect to detect is a stated fraction of the observed effect. Such a calculation is a planning illustration, not observed power and not a replacement for the trial’s results. Sample-size planning asks how much information might be needed to detect a specified effect under stated assumptions. It does not show that the study actually achieved that effect or had a particular probability of finding it.

The record does not report the trial’s own sample-size justification. The primary outcome has several categories or time points and is not reduced to one comparison in the posted analysis. The observed effect needed for the described illustration is not available in the primary-outcome information, so the planning assumptions cannot be connected to a reported primary contrast. The illustration’s numerical sample-size results are not described in the registry record.

Why can variability and over-dispersion matter?

For count outcomes, a statistical model may make assumptions about how much observations vary around their average. Over-dispersion means the observed variability is greater than the model expects. In general, ignoring over-dispersion can make uncertainty appear smaller than it is, which can affect confidence intervals and tests. This is a general explanation, not a claim about this trial: the registry classifies the primary endpoint as a count of participants but does not report a dispersion estimate, model, or assessment of over-dispersion.

More broadly, the registry lists 27 outcome measures and 30 analyses. When many outcomes or comparisons are examined, the chance of at least one apparently unusual result can increase. Statistical methods can address multiplicity, but the registry does not report whether any adjustment or hierarchy of outcomes was used. The number of listed outcomes alone does not establish that every possible comparison was conducted or that any particular adjustment was needed.

Interpretation of the trial would also depend on follow-up completeness, outcome definitions, adherence, and the analysis population. The registry identifies the primary population as participants who completed assessments at baseline and three months, but does not report how many participants met that description or how missing information was handled. No specific analysis method or sensitivity analysis can be attributed to the trial from the registry record.

10. Multiplicity, Missing Data and Analysis Populations

The registry lists 27 outcome measures and 30 analyses, including a primary outcome and secondary measures assessed at different times. Examining multiple outcomes or comparisons raises a general multiplicity issue: the more tests considered, the greater the chance that at least one result may appear statistically unusual by chance alone. In general, prespecified outcome hierarchies or statistical adjustments can help guide interpretation. Whether the investigators prespecified such a hierarchy or used any multiplicity adjustment is not reported in the registry record.

The secondary outcome list includes condom errors, HIV knowledge, motivation and behavioral skills, health-protective communication skills, and drug use before sex. The list also contains measures at more than one follow-up time. Results and analysis details for these measures are not reported in the registry record, so it is not possible to assess consistency across outcomes or time points.

The primary outcome population is described as enrolled participants who completed sexual-risk assessments at baseline and three months. The number who completed both assessments is not reported in the registry record. The handling of missing assessments, reasons for missingness, and any methods used to address missing data are also not reported. In general, missing data can affect interpretation if participants with observed follow-up differ from those without it, but the registry does not provide information to evaluate that possibility in this study.

The record does not identify whether the analysis followed an intention-to-treat approach, included only participants with complete assessments, or used another analysis population. It also does not report adherence-based analyses, sensitivity analyses, or interim analyses. These items should not be inferred from the randomized design or from the outcome population description.

11. Safety Results

The registry reports serious adverse events as affected participants divided by participants at risk.

Deaths

Other (non-serious) adverse events

Participants with at least one non-serious event at or above the registry's 0% reporting threshold.

Most frequent serious adverse event terms

Serious events with the highest arm-level rate: confidentiality breach 3.27.14 (2.6%), confidentiality breach 9.11.15 (0.2%).

The registry’s serious adverse event figures list 445 participants at risk in the Keep It Up! Intervention group, with 11 affected by serious adverse events, and 456 participants at risk in the HIV Knowledge Control group, with 13 affected. The record lists 0 deaths in each group. It also lists 0 other adverse events in each group. These are reported counts; they should not be read as comparative rates or as evidence about the primary behavioral outcome.

The registry identifies confidentiality breach terms among the top serious-event terms: Confidentiality Breach 3.27.14 at 2.6% and Confidentiality Breach 9.11.15 at 0.2%. The record does not explain the labels, the denominators used for those percentages, whether the terms refer to separate events or reporting categories, or how they relate to the group-level serious-event counts. Those details are not reported in the registry record.

Serious-event counts do not by themselves establish why events occurred or whether they were caused by a study program. The registry information presented does not provide event narratives, timing by participant, attribution, or comparative statistical analyses. The counts should therefore be interpreted descriptively, with the available information, rather than as a causal comparison of program safety.

12. What the Result Does, and Does Not, Mean

The registry record defines a randomized comparison between a YMSM-specific online HIV prevention program and a general HIV knowledge control. It identifies condomless anal sex self-report at baseline and three months as the primary outcome. These registry entries clarify the study question and its intended endpoint, but they do not supply the primary results.

The record does not report arm-specific primary outcome counts, a between-arm estimate, a confidence interval, or a p-value. It therefore cannot establish whether the outcome changed differently between the programs, the direction of any difference, or how precisely a difference was estimated. The research hypothesis in the study description must not be mistaken for an observed finding.

The absence of a reported result in the registry record is not evidence that there was no effect, nor is it evidence that an effect occurred. It means that the primary comparison cannot be judged from the information reported. The same caution applies to the listed secondary outcomes and the hypothesis concerning new STI infections: comparative results for those outcomes are not reported in the registry record.

Even if an estimate were available, interpretation would require attention to its uncertainty, the outcome definition, follow-up completeness, and the analysis population. In general, a statistically detectable difference does not automatically imply a practically important difference, while an imprecise estimate may leave a range of possibilities unresolved. The relevant estimate and uncertainty are not reported, so neither type of interpretation is possible for the primary outcome.

The study’s safety counts are separate from its behavioral outcome and do not answer whether the online programs changed reported sexual behavior. Nor do they establish the cause of serious events. The trial record supports describing the design, population, interventions, and outcome question, but not a conclusion about comparative effectiveness, STI outcomes, or causality.

13. Why This Trial Matters Statistically

This trial addresses a statistical challenge common in behavioral research: comparing two active forms of information rather than comparing a program with no content. The relevant contrast is therefore between a program developed specifically for young men who have sex with men and a general HIV knowledge program. Statistical interpretation must preserve that comparator; it is not a comparison with no intervention.

The primary outcome is based on participants’ reports of condomless anal sex at baseline and three months. This makes the definition of the reported behavior, the timing of measurement, and the handling of repeated observations important to understanding the analysis. The registry names the outcome and its assessment period but does not report the detailed coding or analysis method. That distinction illustrates why a clear outcome label is not a substitute for a fully described statistical analysis.

The record also lists multiple outcome measures and analyses. In general, a broad outcome set can provide information about several aspects of a behavioral program, but it can complicate interpretation when analysis priorities and multiplicity methods are unclear. The registry does not report whether an outcome hierarchy or adjustment was used. Readers therefore cannot determine how the listed analyses were prioritized or how results across outcomes should be interpreted together.

Finally, the registry reports enrollment and a primary analysis population definition but not the number with complete primary assessments or the missing-data approach. In longitudinal studies, follow-up completeness can influence how representative the analyzed group is of everyone randomized. The missingness information and analysis results are not reported, so this issue cannot be assessed for this study.

These features matter because the credibility of a trial conclusion depends not only on randomization, but also on transparent outcome definitions, analysis methods, precision, and follow-up. The registry record identifies the central question and broad design, while leaving important information needed to evaluate the primary statistical result unreported.

14. Limitations

• The posted summary statistics are the only data these results rest on; patient-level data, full protocols and statistical analysis plans are not available.
• Recalculated results are simple, unadjusted calculations from posted summaries. They do not replace the sponsor's prespecified analysis.

The primary limitation for statistical interpretation is that the registry record does not report the primary outcome results. There are no arm-specific counts, estimated contrast, confidence interval, or p-value in the information available. Without these, the direction, magnitude, and precision of the primary comparison cannot be evaluated.

The outcome is described as a change in self-reported behavior, but the event definition, response coding, and analysis method are not reported in the registry record. The number of participants who completed assessments at baseline and three months is also not reported. Missing-data handling, reasons for loss to follow-up, and the analysis population actually used are not reported, limiting assessment of attrition and potential selection effects.

The registry lists multiple outcomes and analyses but does not report the statistical plan, any multiplicity approach, or whether the primary endpoint and analysis were prespecified in detail. It does not report the sample-size justification or interim analyses. These omissions mean readers cannot compare the posted outcome information with a complete prespecified analysis strategy.

Information about the interventions is incomplete: the descriptions are abbreviated, so the full content and delivery details are not reported in the registry record. The record labels masking as double but does not specify who was masked or how masking was maintained. It also does not explain the difference between the enrollment figure of 901 and the study description’s stated total of 900.

The safety information is descriptive and lacks details about event attribution and the denominators for the listed percentages. Results for STI outcomes are not reported in the registry information. Together, these limitations restrict conclusions about outcome effects, implementation, safety attribution, and generalizability.

15. Related Tutorials

16. Related Statistical Calculators

17. Sources

• ClinicalTrials.gov: Keep It Up! 2.0: A Comparison of Two Online HIV Intervention Programs for Young Men Who Have Sex With Men — NCT01836445.
• Registry record last updated 2022-08-15; status last verified 2022-08.
• Results first posted 2018-07-11.
• Data retrieved from ClinicalTrials.gov on 2026-11-09.
• Group sizes, means, event counts and sponsor-reported p-values are reproduced as posted. Effect estimates, confidence intervals and p-values not attributed to the sponsor are independently calculated from the posted summary statistics.

18. Record Summary

NCT01836445, titled Keep It Up! 2.0: A Comparison of Two Online HIV Intervention Programs for Young Men Who Have Sex With Men, is a completed randomized, double-masked, parallel-group study sponsored by Northwestern University. The registry reports enrollment of 901. The listed study start is 2013-05, and primary completion is 2017-03.

The study compared Keep It Up!, a YMSM-specific online HIV prevention program, with HIV Knowledge Control, described as non-tailored, non-interactive HIV and STI information for a general audience. The primary outcome concerns self-reported condomless anal sex at baseline and three months among participants completing assessments at both times.

The registry identifies the design and primary question but does not report the primary outcome comparison, its uncertainty, or the analysis method. Missing-data procedures, multiplicity methods, and the sample-size justification are not reported in the registry record. The listed serious-event counts include 11 affected participants in the intervention group and 13 in the control group, with 0 deaths reported in each group.