Modifying (Phase I) and Evaluating (Phase II) Virtual Reality Job Interview Training for Youth in Transition: Complete Statistical Analysis

1. Trial at a Glance

This page covers the registry results for Modifying (Phase I) and Evaluating (Phase II) Virtual Reality Job Interview Training for Youth in Transition (NCT03514134), a randomized na study. The primary outcome is Change in Job Interview Skills.

2. Clinical Question

The trial addressed the challenge of preparing transition-age youth with ASD for employment, focusing on job interviews as a gateway to a job offer. The registry describes a broader project with an initial aim to adapt an existing training program for high school students with ASD, followed by a controlled evaluation in a high school setting. The registered primary outcome is change in job interview skills, measured using a role-play assessment.

The intervention is behavioral, and the comparison is not an absence of support: the control group received services as usual. The registry says these services could include vocational, daily living, or social skill training. The key contrast is therefore the added training alongside usual services versus usual services alone.

The record also lists job interview self-efficacy, change in job interview anxiety, and competitive employment as secondary outcomes. These measures broaden the questions beyond scored interview performance, but their inclusion does not establish that results were reported. The registry record does not report a single primary-outcome comparison suitable for describing the size or precision of an intervention effect.

3. Trial Design

The registry describes a randomized, single-masked, parallel-group trial. Participants were assigned to services as usual or to services as usual plus Virtual Reality Job Interview Training. The masking designation is single, but the registry record does not report which people involved in the trial were masked. It also does not report details of the randomization procedure, such as how the allocation sequence was generated or concealed.

The listed enrollment is 71. The trial status is completed. Its recorded start date is 2018-10-01, and primary completion is recorded as 2020-08-01. The phase field is listed as NA. The sponsor is the University of Michigan. Autism Spectrum Disorder is the listed condition.

The intervention group received the training in addition to usual services. The comparator group received community-based or school-based services as usual, which could include vocational skill training, daily living skill training, and social skill training. The registry's description of the broader project includes adapting the existing virtual reality program for high school students with ASD and evaluating its feasibility and effectiveness in a high school setting.

The primary outcome was collected at pre-test and post-test visits within approximately 1 month of active treatment. The assessment used a role-play job interview with trained actors, recorded on video and scored. The record lists five outcome measures and six analyses, but it does not report the full analysis plan or the results of a single primary-outcome comparison. Details such as the exact allocation ratio, follow-up beyond the stated assessment timeframe, and the identity of the masked party are not reported in the registry record.

4. Participant Flow and Accounting

The registry reports how participants moved through each stage of the study. All counts below are exactly as posted; missing counts are not inferred from other stages.

Overall Study

Reasons for not completing this period:

Analysis population for the primary outcome, as defined in the registry: Performance on 2 Mock Job Interviews collected at pre-test and post-test

5. Baseline Characteristics

The table reproduces the baseline characteristics posted in the registry, which describe the participants analysed at baseline and help show whether the randomized groups were comparable.

6. Primary Endpoint

The registry lists the primary outcome as Change in Job Interview Skills, measured over Collected at Pre-Test and Post-Test visits (within approximately 1 month of active treatment).

The primary endpoint is change in job interview skills, measured as a score on a scale. Participants acted out a job interview scenario with trained actors at pre-test and post-test. The interviews were video recorded and scored using the Mock Interview Rating Scale. The registry describes one mock interview at pre-test and one at post-test.

The scale has 14 items, with each item ranging from 1 (Unlikely) to 7 (Very Likely). Possible scores range from 14 to 98, with higher scores indicating better interview skills. The measure therefore captures performance in a structured, simulated interview rather than employment itself. The registry lists competitive employment separately as a secondary outcome.

A role-play measure is relevant to a training program focused on interview skills because it assesses performance in an interview-like task. Comparing pre-test and post-test scores is intended to capture change during the treatment period; comparing that change between the intervention and comparator groups would address whether the change differs between assigned groups. The registry record does not report the primary outcome as one single comparison across the listed assessment categories or time points, so the size and precision of that contrast cannot be described from the record.

7. Why No Re-analysis Is Shown

8. Statistical Interpretation

9. Statistical Methods Explained

The statistical information available for this trial is limited. The record identifies random allocation, a parallel-group model, single masking, a continuous primary measure summarized as a mean, and standard error as the listed dispersion. It does not report the statistical analysis plan or a single primary-outcome comparison. The concepts below explain how these methods are generally used; they do not imply that a particular analysis was performed for this trial.

What does random allocation contribute?

In general, random allocation assigns participants to study groups by chance. Its purpose is to reduce systematic differences between groups at the start of a trial, including differences that researchers may not have measured. It does not guarantee that groups will be identical, and it cannot prevent problems arising after assignment, such as missing outcome measurements or departures from the assigned intervention. The registry says the trial was randomized but does not report the sequence-generation method, allocation concealment, or an assessment of baseline comparability.

What does a parallel-group comparison estimate?

In a parallel-group design, participants remain assigned to one intervention group or comparator group during the study. A comparison can use post-test scores, change from pre-test to post-test, or a model that accounts for baseline scores. These approaches may estimate different quantities and can have different assumptions. The registry names change in job interview skills as the primary outcome but does not report which precise contrast or model was used. The fact that a measure was collected at pre-test and post-test does not establish which statistical approach was used to compare groups.

How should means and standard errors be understood?

In general, a mean describes the average score in a group or set of observations. A standard error describes the estimated sampling variability of an estimate, such as a mean; it is not the same as the spread of individual scores. A standard error can be used to quantify uncertainty around an estimate, but its interpretation depends on the estimate and the analysis that produced it. The registry identifies the primary measure as a mean with standard error as its dispersion, but does not report arm-specific values or an estimated treatment contrast.

A confidence interval is a general way to express uncertainty around an estimated effect. Its width reflects both the estimate's precision and the confidence level used. A narrow interval generally indicates greater precision than a wide interval, although an interval alone does not resolve design or measurement limitations. No confidence interval for the primary comparison is reported in the registry record.

Why does the repeated assessment matter?

The endpoint was assessed at pre-test and post-test, with visits within approximately 1 month of active treatment. Repeated measurements allow assessment of change over time, but a change within one group is not by itself the randomized treatment contrast. The more direct comparative question is whether the change differs between the assigned groups, or whether their post-test scores differ under a specified analysis. Which form was used is not reported in the registry record. The outcome description also says there are several categories or time points and that they were not reduced to one comparison, so a unique primary contrast cannot be reconstructed from the available record.

What can be said about planning calculations?

The planning assumptions describe illustrations based on an observed effect, not the trial's own design justification. The assumptions specify a two-sided significance level of 0.05, equal allocation between the groups, and a normal approximation; the effect to detect is expressed as a stated fraction of the observed effect. Such calculations are planning illustrations, not observed power and not evidence that the trial was adequately or inadequately powered. The registry record does not report the original sample-size justification or the planning effect used by the trial investigators. No planning estimate can replace a reported outcome contrast with its uncertainty.

Why can over-dispersion matter in general?

Over-dispersion is a general concern when observed variability is greater than a statistical model assumes. If a model understates variability, standard errors may be too small and uncertainty may appear less than it is. Appropriate methods depend on the outcome type and model. The primary measure is described as a continuous score, but the registry record does not identify a model or report any assessment of over-dispersion. There is therefore no basis to say whether it arose or how it was handled.

Overall, the design labels and endpoint description do not reveal the analysis choices needed to reproduce or fully appraise a treatment estimate. The registry record does not report the model specification, covariate adjustments, handling of missing outcome data, or an inferential result for a single primary comparison.

10. Multiplicity, Missing Data and Analysis Populations

The registry lists five outcome measures and six analyses. The primary outcome is change in job interview skills, while the listed secondary outcomes include job interview self-efficacy, change in job interview anxiety, and competitive employment. The primary outcome description refers to pre-test and post-test assessments, and the record notes that it has several categories or time points and was not reduced to one comparison. That makes it important to know which comparisons were designated as primary and how multiple outcomes or time points were handled.

Multiplicity is a general statistical issue that arises when several hypotheses or comparisons are tested. As the number of tests increases, the chance of at least one apparently unusual result can increase if each test is judged separately. Researchers may address this through a prespecified hierarchy, adjustment, or another plan. The registry record does not report whether any multiplicity strategy was prespecified or used, nor does it identify how the six analyses relate to the outcome measures.

The record does not report missing-data counts for the primary outcome, reasons for missing assessments, or the method used to handle missing data. It also does not report an analysis population definition, such as whether results were analyzed according to randomized assignment or based on treatment received. These details affect interpretation because exclusions or incomplete follow-up can alter the composition of the groups being compared.

Without the analysis plan and outcome-level results, it is not possible to determine whether all prespecified comparisons were reported, how missing assessments influenced estimates, or whether the primary analysis retained participants according to their randomized assignment. These matters are not reported in the registry record.

11. Safety Results

The registry reports serious adverse events as affected participants divided by participants at risk.

Deaths

Other (non-serious) adverse events

Participants with at least one non-serious event at or above the registry's 0% reporting threshold.

The registry lists serious adverse-event figures by group. For Services as Usual, 23 participants are listed as at risk, with 0 affected by a serious adverse event and 0 deaths. For Services as Usual + Virtual Reality Job Interview Training, 48 participants are listed as at risk, with 0 affected by a serious adverse event and 0 deaths. No serious adverse-event terms are listed.

These entries describe what was recorded in the registry; they do not establish that no adverse experiences of any kind occurred. The figures specifically concern the listed serious-event and death counts. The record also gives zero for the other adverse-event affected count in each group, but does not provide further event descriptions in the registry record for this analysis.

Zero recorded serious adverse events is not proof that an intervention is free from risk. Safety interpretation depends on the definition and monitoring of events, follow-up, reporting completeness, and the amount of information available. The record does not provide details about safety monitoring or event-collection procedures, so the counts should be read as registry entries rather than as a comprehensive safety assessment.

12. What the Result Does, and Does Not, Mean

The trial record documents a randomized comparison of virtual reality job interview training added to usual services versus usual services alone for high school students with ASD. It identifies a role-play measure of job interview skills as the primary outcome and describes assessments at pre-test and post-test. These design and measurement details clarify the question the trial was intended to address.

They do not, however, establish what the trial found. The registry record does not report a single primary-outcome comparison, an estimated difference between groups, or a measure of uncertainty around such a difference. Without those results, the direction and size of any change cannot be described, and statistical precision cannot be assessed. The absence of a reported comparison is not evidence that the groups were the same, nor is it evidence that the intervention had an effect.

The primary measure concerns performance in a structured mock interview, not employment itself. Competitive employment is listed as a secondary outcome, but its result is not reported in the available record. A score on an interview rating scale and an employment outcome represent different aspects of the transition-to-work question, and one should not be treated as a substitute for the other.

The comparator also matters: participants assigned to the comparison group received services as usual, which could include vocational, daily living, or social skill training. The trial therefore concerns the added value of the virtual reality program in that services context. It does not directly answer how the program compares with receiving no support.

The safety entries list no serious adverse events or deaths in either group, but those records do not support a broad claim about safety. In statistical terms, the available registry entries allow a clear description of the intended comparison and outcome measurement, but not a conclusion about effectiveness, clinical importance, or the precision of the primary result.

13. Why This Trial Matters Statistically

This trial is statistically relevant because it evaluates a behavioral intervention in a population for whom the registry describes a need for employment-transition support. It also uses a comparator that reflects services as usual, rather than treating the alternative as no services. That makes the intended contrast more directly about the added contribution of the virtual reality program within a usual-services setting.

The outcome is measured through a structured mock interview scored on a defined scale. Such an outcome is closer to the skill targeted by the training than a broad employment outcome, while competitive employment is separately listed as a secondary measure. Keeping those outcomes conceptually distinct is important: performance during an assessment and obtaining employment are related but not identical questions.

The design is randomized and parallel-group, but the registry record does not report enough statistical detail to evaluate the primary estimate. In particular, the primary outcome has multiple categories or time points and was not reduced to one comparison. The absence of a single reported contrast illustrates why a stated endpoint and a completed trial are not, by themselves, enough to support an effect estimate. Readers need the group-level results, a defined comparison, and information about uncertainty to interpret a trial statistically.

The record also highlights the distinction between planning and evidence. Planning calculations based on an observed effect can illustrate sample-size requirements under stated assumptions, but they are not observed power and cannot stand in for the trial's results. A sample-size justification and an outcome analysis answer different questions. The former concerns design planning; the latter concerns what the data show.

Finally, the safety counts demonstrate the importance of interpreting zero-event entries narrowly. They report no listed serious events or deaths, but do not independently establish comprehensive safety. Overall, the statistical importance of this record lies both in its randomized design and in the limitations of the reported analysis details.

14. Limitations

• The posted summary statistics are the only data these results rest on; patient-level data, full protocols and statistical analysis plans are not available.
• Recalculated results are simple, unadjusted calculations from posted summaries. They do not replace the sponsor's prespecified analysis.

The primary limitation for interpreting results is that the registry record does not report a single primary-outcome comparison. It describes the measure and assessment occasions but says the outcome has several categories or time points and was not reduced to one comparison. No group-level primary scores, estimated effect, confidence interval, or p-value are reported in the registry record.

The record does not report the statistical analysis plan, exact model, analysis population, or method for handling missing data. It also does not identify the party masked under the single-masking designation, or describe sequence generation and allocation concealment. These omissions limit assessment of how the randomized comparison was conducted and analyzed.

The primary measure is a scored mock interview. The record describes the rating scale and trained actors, but does not report information about assessor masking, scoring reliability, or whether scoring procedures were consistent across groups and visits. The timeframe is within approximately 1 month of active treatment; longer-term primary outcome follow-up is not reported in the registry record.

The enrollment is listed as 71, but the registry entries do not give a sample-size justification or report whether all enrolled participants contributed primary-outcome data. The adverse-event figures list participants at risk and event counts, but do not provide further safety-monitoring details. These limitations mean that neither the precision of the treatment estimate nor the completeness of outcome and safety ascertainment can be evaluated from the record.

15. Related Tutorials

16. Related Statistical Calculators

17. Sources

• ClinicalTrials.gov: Modifying (Phase I) and Evaluating (Phase II) Virtual Reality Job Interview Training for Youth in Transition — NCT03514134.
• Registry record last updated 2023-06-22; status last verified 2023-06.
• Results first posted 2023-06-22.
• Data retrieved from ClinicalTrials.gov on 2026-11-09.
• Group sizes, means, event counts and sponsor-reported p-values are reproduced as posted. Effect estimates, confidence intervals and p-values not attributed to the sponsor are independently calculated from the posted summary statistics.

18. Record Summary

NCT03514134 is a completed randomized, single-masked, parallel-group trial sponsored by the University of Michigan. The listed enrollment is 71, and the condition is Autism Spectrum Disorder. The recorded start date is 2018-10-01, with primary completion on 2020-08-01. The phase field is NA.

The trial compared Virtual Reality Job Interview Training added to services as usual with services as usual. The primary outcome was change in job interview skills, assessed using the Mock Interview Rating Scale at pre-test and post-test visits within approximately 1 month of active treatment. Listed secondary outcomes include job interview self-efficacy, change in job interview anxiety, and competitive employment.

The registry record does not report a single primary-outcome comparison. Serious adverse events and deaths are listed as 0 in both groups. These registry entries describe the available registry entries but do not establish an intervention effect or a comprehensive safety conclusion.