This page provides an independent statistical analysis and educational interpretation of publicly reported results. ClinicalTrials.gov provides the official trial registry record. Numerical trial results on this page are restricted to the results posted in the ClinicalTrials.gov record; calculations added by Clinical Biostats are labelled as such.
1. Trial at a Glance
This page covers the registry results for Supported Employment in Patient Aligned Care Teams (NCT02400736), a randomized na study. The primary outcome is Steady Worker.
| Feature | Registry value |
|---|---|
| ClinicalTrials.gov identifier | NCT02400736 |
| Phase | NA |
| Condition | Mental Disorder, Substance Use Disorders, Posttraumatic Stress Disorder |
| Allocation | Randomized |
| Design model | Parallel |
| Masking | None (open label) |
| Enrollment | 119 |
| Arms in the primary outcome | 2 |
| Lead sponsor | VA Office of Research and Development |
| Results posted | Yes |
2. Clinical Question
Population
The study enrolled unemployed or under-employed Veterans with at least one mental disorder who were followed in a primary care Patient Aligned Care Team. The primary-outcome population is described as 119 randomized Veterans. The conditions listed in the registry include mental disorder, substance use disorders, posttraumatic stress disorder, major depressive disorder, and anxiety disorder.
Intervention
Individual Placement and Support (IPS) is a supported-employment approach. The registry describes competitive employment, participant choice without exclusion on eligibility grounds, and integration of employment support with the PACT treatment team as features of the intervention.
Comparator
The comparison was Treatment as Usual Vocational Rehabilitation/Transitional Work (TAU-VR). The record describes usual vocational rehabilitation as including pre-vocational counseling, community-based supported employment, or, most commonly, transitional work assignments. The description of those assignments is incomplete in the registry record.
Primary question
The primary question was whether the groups differed in the number of participants classified as steady workers over 12 months. A steady worker held a competitive job for 50% or more of follow-up, defined as 26 or more weeks out of 52 weeks. This outcome focuses on sustained competitive employment rather than simply obtaining a job.
3. Trial Design
The registry describes a randomized, parallel-group trial comparing IPS with Treatment as Usual Vocational Rehabilitation/Transitional Work. It lists 119 participants and identifies the study as completed. The start date was 2015-08-03, and primary completion was 2019-09-30. The study was unmasked: the registry lists no masking. The record does not report further details about the randomization procedure, such as how the allocation sequence was generated or concealed.
Participants were Veterans with at least one mental disorder who were unemployed or under-employed and followed in a primary care Patient Aligned Care Team. The study addressed delivery of IPS within that primary-care setting. The listed diagnoses include mental disorder, substance use disorders, posttraumatic stress disorder, major depressive disorder, and anxiety disorder. The registry frames the question in the context of access to vocational rehabilitation for Veterans facing unemployment and mental illness.
The intervention was behavioral IPS, while the comparator was usual vocational rehabilitation or transitional work. The registry describes IPS as supporting competitive employment, respecting client choice, and integrating the employment service with the PACT team. TAU-VR could include pre-vocational counseling, community-based supported employment, or transitional work. The full description of the comparator is not reported in the registry record, so the precise services received by participants cannot be reconstructed from that description alone.
The primary outcome was assessed over 12 months. The registry lists 11 outcome measures and 11 analyses, while seven secondary outcomes are identified in the record. Details about the original sample-size justification, the planned statistical analysis in full, interim analyses, and handling of missing outcome data are not reported in the registry record.
4. Participant Flow and Accounting
The registry reports how participants moved through each stage of the study. All counts below are exactly as posted; missing counts are not inferred from other stages.
Overall Study
| Stage | Individual Placement and Support (IPS) | Treatment as Usual Vocational Rehabilitation/Transitional Work (TAU-VR) |
|---|---|---|
| STARTED | 58 | 61 |
| COMPLETED | 47 | 53 |
| NOT COMPLETED | 11 | 8 |
Reasons for not completing this period:
| Reason | Individual Placement and Support (IPS) | Treatment as Usual Vocational Rehabilitation/Transitional Work (TAU-VR) |
|---|---|---|
| Lost to Follow-up | 6 | 6 |
| Withdrawal by Subject | 2 | 1 |
| Moved from the area | 3 | 1 |
Analysis population for the primary outcome, as defined in the registry: 119 randomized Veterans who were followed in a primary care Patient Aligned Care Team, who had a diagnosis of at least one mental disorder, and were unemployed or under-employed.
5. Baseline Characteristics
The table reproduces the baseline characteristics posted in the registry, which describe the participants analysed at baseline and help show whether the randomized groups were comparable.
Population: 140 Veterans were consented and 119 Veterans were randomized and included in the analysis. Two were randomized randomized in error and were not included in the analysis. For the 19 not randomized, 4 had no qualifying DSM-5 diagnosis, 5 changed their mind, 3 were unlikely to complete, 2 were employed; 3 were not eligible for vocational rehabilitation services; 1 declined competitive employment; 1 was enrolled in another interventional study.
| Characteristic | Unit | Individual Placement and Support (IPS) | Treatment as Usual Vocational Rehabilitation/Transitional Work (TAU-VR) | Total |
|---|---|---|---|---|
| Participants analysed | 58 | 61 | 119 | |
| Age, Continuous: Age | years | 37.8 (SD 8.4) | 38.6 (SD 8.4) | 38.2 (SD 8.4) |
| Sex: Female, Male: Female | Participants | 9 | 12 | 21 |
| Sex: Female, Male: Male | Participants | 49 | 49 | 98 |
| Ethnicity (NIH/OMB): Hispanic or Latino | Participants | 1 | 1 | 2 |
| Ethnicity (NIH/OMB): Not Hispanic or Latino | Participants | 57 | 60 | 117 |
| Ethnicity (NIH/OMB): Unknown or Not Reported | Participants | 0 | 0 | 0 |
| Race (NIH/OMB): American Indian or Alaska Native | Participants | 1 | 0 | 1 |
| Race (NIH/OMB): Asian | Participants | 0 | 0 | 0 |
| Race (NIH/OMB): Native Hawaiian or Other Pacific Islander | Participants | 0 | 0 | 0 |
| Race (NIH/OMB): Black or African American | Participants | 42 | 45 | 87 |
| Race (NIH/OMB): White | Participants | 15 | 16 | 31 |
| Race (NIH/OMB): More than one race | Participants | 0 | 0 | 0 |
| Race (NIH/OMB): Unknown or Not Reported | Participants | 0 | 0 | 0 |
| Mental Disorder: Major depression, past | Participants | 49 | 48 | 97 |
| Mental Disorder: Major depression, current | Participants | 43 | 41 | 84 |
| Mental Disorder: Agoraphobia | Participants | 7 | 13 | 20 |
| Mental Disorder: Panic Disorder (lifetime) | Participants | 24 | 27 | 51 |
| Mental Disorder: Post-traumatic stress disorder | Participants | 46 | 44 | 90 |
| Mental Disorder: Alcohol Use Disorder, past year | Participants | 29 | 26 | 55 |
| Mental Disorder: Substance Used Disorder | Participants | 29 | 27 | 56 |
6. Primary Endpoint
The registry lists the primary outcome as Steady Worker, measured over 12 months.
| Element | Registry entry |
|---|---|
| Result type | Count Of Participants |
| Measure of spread | not applicable |
| Unit | Participants |
| Endpoint classification | Binary Endpoint |
The primary endpoint, Steady Worker, counts participants who met a sustained-employment definition during the 12-month follow-up. A participant qualified by holding a competitive job for 50% or more of the follow-up, specified in the registry as 26 or more weeks out of 52 weeks. The endpoint is therefore a binary classification: each participant either meets the stated threshold or does not.
This threshold captures more than whether someone entered employment. It focuses on maintaining competitive work for a substantial portion of follow-up, which is relevant to a study of vocational rehabilitation and employment stability. It also gives the comparison an interpretable outcome: the number meeting the definition in each randomized group.
The endpoint does not describe every aspect of employment. It does not, by itself, show when a participant first obtained a job, how many weeks were worked by those who fell short of the threshold, or how much income was earned. Those topics appear among the secondary outcomes. Nor does the binary classification preserve all variation in the duration or quality of employment. The registry definition specifies competitive employment and a duration threshold; other details about ascertainment and endpoint adjudication are not reported in the registry record.
7. Results Posted in the Registry
- Individual Placement and Support (IPS): 44.8% (26/58) versus 24.6% (15/61) on control; risk difference 20.2 percentage points (95% CI 3.5 to 37.0), risk ratio 1.82 (1.08–3.08); unadjusted two-proportion z-test p = 0.020.
Risk difference: Individual Placement and Support (IPS)
95% CI: 3.5 to 37.0 percentage points
Recalculated from the posted summary statistics
8. Secondary Efficacy Endpoints
The registry also posts results for secondary outcomes. For each outcome below, the endpoint type, the test applied to the posted summary statistics, the result and its interpretation are given in turn. Secondary endpoints are usually not adjusted for multiple testing, so they support rather than confirm the primary finding.
| Secondary outcome | Type | Effect measure | Estimate (95% CI) | Planning size, 80% power |
|---|---|---|---|---|
| Weeks Worked in a Competitive Job | Continuous Endpoint | Mean difference | 14.00 (8.24 to 19.76) | 21 per arm |
| Time to First Competitive Job | Continuous Endpoint | Mean difference | -8.90 (-13.12 to -4.68) | 28 per arm |
| Income Earned From Competitive Jobs | Continuous Endpoint | Mean difference | 4,501 (338.58 to 8,663) | 103 per arm |
Weeks Worked in a Competitive Job
- Endpoint type: Continuous: a mean value per participant, compared as a difference in means.
- Test: Unadjusted two-sample z-test for the difference in means, using the posted standard deviations.
- Result:
- Individual Placement and Support (IPS): mean 38.60 versus 24.60 on control; difference 14.00 (95% CI 8.24 to 19.76), standardized effect d = 0.87; z-test p = <0.001.
- Interpretation:
- The 95% confidence interval excludes no difference, so the mean is statistically higher for Individual Placement and Support (IPS) than for the control. This is an unadjusted, secondary comparison, so it is suggestive rather than confirmatory.
Time to First Competitive Job
- Endpoint type: Continuous: a mean value per participant, compared as a difference in means.
- Test: Unadjusted two-sample z-test for the difference in means, using the posted standard deviations.
- Result:
- Individual Placement and Support (IPS): mean 8.70 versus 17.60 on control; difference -8.90 (95% CI -13.12 to -4.68), standardized effect d = -0.75; z-test p = <0.001.
- Interpretation:
- The 95% confidence interval excludes no difference, so the mean is statistically lower for Individual Placement and Support (IPS) than for the control. This is an unadjusted, secondary comparison, so it is suggestive rather than confirmatory.
Income Earned From Competitive Jobs
- Endpoint type: Continuous: a mean value per participant, compared as a difference in means.
- Test: Unadjusted two-sample z-test for the difference in means, using the posted standard deviations.
- Result:
- Individual Placement and Support (IPS): mean 11,106 versus 6,605 on control; difference 4,501 (95% CI 338.58 to 8,663), standardized effect d = 0.39; z-test p = 0.034.
- Interpretation:
- The 95% confidence interval excludes no difference, so the mean is statistically higher for Individual Placement and Support (IPS) than for the control. This is an unadjusted, secondary comparison, so it is suggestive rather than confirmatory.
The posted secondary outcomes cover several aspects of employment and participant experience: duration of competitive work, time to first competitive job, income from competitive jobs, income from all sources, and changes in symptom and self-esteem measures. The results available for the continuous comparisons include weeks worked, time to first competitive job, and competitive-job income. Taken together, these outcomes address whether the groups differed not only in meeting the sustained-employment threshold but also in other measured dimensions of work.
The pattern in the posted employment results is broadly consistent with the primary comparison: the group contrasts for weeks worked, time to first competitive job, and competitive-job income favor IPS on their respective measures. These measures are related to the primary result, but they are not interchangeable. Meeting a duration threshold, working a given number of weeks, reaching a first job sooner, and earning income each describe a different feature of participants' employment experiences. The secondary outcomes therefore add context rather than simply repeating the primary endpoint.
Secondary findings carry less weight than a prespecified primary outcome when many outcomes are assessed and no adjustment for multiple testing is reported. With multiple comparisons, the chance of obtaining at least one apparently notable result by chance can increase. The registry record does not report whether the secondary outcomes were ranked in advance, whether any multiplicity procedure was planned, or how missing observations affected each analysis. The p-values should therefore be interpreted as unadjusted evidence for their particular comparisons, not as a single confirmatory test across all outcomes.
The registry lists seven secondary outcomes, while the available posted secondary details are not complete for every listed measure. Results for income from all sources, symptom change, self-esteem change, and income among participants who held a competitive job are not available in the record as described. Their estimates and analyses are not reported in the registry record. The absence of those results from the posted details should not be read as evidence of no effect.
Secondary outcomes registered for this trial, for reference (only those above were recalculated):
- Weeks Worked in a Competitive Job
- Time to First Competitive Job
- Income Earned From Competitive Jobs
- Symptom Checklist-90-Revised Change From Baseline to Month 12.
- Income Earned From All Sources
- Rosenberg Self-Esteem Scale Change From Baseline to 12 Months.
- Income Earned From Competitive Jobs in Participants Who Held a Competitive Job
9. Statistical Interpretation
For the primary outcome, 26 of 58 participants in the IPS group were classified as steady workers, compared with 15 of 61 in the TAU-VR group. The registry reports an odds ratio of 2.49, with a 95% confidence interval from 1.14 to 5.43, and p = 0.02. The unadjusted re-calculation gives a risk difference of 0.202, with a 95% confidence interval from 0.035 to 0.370. These are different ways of expressing the observed randomized-group contrast: the odds ratio compares odds, while the risk difference expresses the difference in the proportion meeting the definition.
The estimates point toward a higher observed frequency of steady-worker classification in the IPS group. The confidence interval for the risk difference ranges from a relatively small positive difference to a larger one, so the precise magnitude remains uncertain. The confidence interval for the odds ratio also spans a range of effect sizes. The reported p-value is evidence against the null hypothesis of equal odds under the stated analysis, but it does not measure the probability that the null hypothesis is true or establish the size or practical importance of the effect.
The trial was randomized, which supports interpreting the comparison as a study of assigned groups, but the registry says the study was unmasked. The record does not report enough detail to assess all aspects of allocation, outcome measurement, adherence, or missing-data handling. The result concerns the defined employment endpoint and the enrolled population; it should not automatically be generalized to Veterans outside the described setting or to other employment outcomes.
Secondary outcomes provide related but distinct information. The posted results include differences in weeks worked, time to first competitive job, and income from competitive jobs. Their pattern is broadly aligned with the primary endpoint, but each measures a different dimension of work and is subject to its own analysis and uncertainty. Secondary findings are less definitive when multiple outcomes are tested without a reported multiplicity adjustment. The registry record does not report such an adjustment, so those results are best treated as additional, exploratory context rather than as independent confirmation of the primary result.
Overall, the posted primary result indicates a difference between the randomized groups in meeting the registry's steady-worker definition. It does not establish that every participant benefited, explain the mechanisms behind the observed difference, or establish effects beyond the measured outcomes and study population.
10. Statistical Methods Explained
The registry reports a chi-squared analysis for the primary comparison and presents the result as an odds ratio. A chi-squared test evaluates whether the distribution of participants across outcome categories differs between groups. For a binary outcome such as Steady Worker, those categories are meeting or not meeting the threshold. The odds ratio summarizes the relative odds of meeting the definition in one group compared with the other. It is not the same quantity as a difference in proportions, and it should not be read as a ratio of probabilities.
The posted primary analysis gives an odds ratio of 2.49, a 95% confidence interval from 1.14 to 5.43, and p = 0.02. The simple unadjusted re-calculation also describes the outcome using a risk difference, with an estimate of 0.202 and a 95% confidence interval from 0.035 to 0.370. A risk difference is an absolute contrast between the proportions meeting the endpoint in the groups. It is often easier to relate to the observed outcome frequencies than an odds ratio, while the odds ratio is the measure used in the registry's reported primary analysis. Both estimates are based on the same group-level outcome counts, but they use different scales.
A confidence interval describes a range of estimates compatible with the data under the method's assumptions. It reflects statistical uncertainty, not a range that necessarily contains the true effect with certainty. A 95% interval is a long-run property of the interval-generating procedure when its assumptions hold; it is not a statement that a particular hypothesis has a 95% probability of being true. The width of an interval is useful for judging precision, while its position relative to the null value helps convey whether the data are compatible with no group difference under the selected method.
The registry also reports t-tests for some continuous secondary outcomes and a mixed models analysis for time to first competitive job. A t-test compares group means, using the observed variability and sample sizes to assess how compatible the mean difference is with a null difference. The posted means and standard deviations describe the observed distributions at group level. A p-value from a t-test does not convey the magnitude of the difference, so it is useful to consider the mean difference and its confidence interval where available. The registry does not post confidence intervals for those sponsor-reported secondary analyses.
The registry's mixed models analysis is identified for time to first competitive job, but details of the model specification are not reported in the registry record. Mixed models generally represent outcomes using a model that accounts for structure in the observations; the exact structure, assumptions, covariates, and treatment of incomplete observations cannot be inferred from the method label alone. The posted comparison and the reported p-value should therefore be read without assuming a particular model formulation.
General statistical point: over-dispersion occurs when outcome variation is greater than a statistical model expects. It can make standard errors and confidence intervals too small if the model does not account for the extra variability. The primary outcome is a participant count for a binary classification, and no dispersion estimate is posted. The registry record does not report a model-based over-dispersion assessment or a correction for it. This general issue should not be taken to imply that over-dispersion occurred in this trial.
The re-calculations are simple, unadjusted comparisons based on posted summary statistics. They are not a substitute for the sponsor's analysis, which may use a distinct analysis specification. For the primary outcome, the re-calculated odds ratio is close to the registry's reported value at the displayed precision, and the posted p-values agree. The trial's own full statistical analysis plan, adjustment strategy, and additional implementation details are not reported in the registry record.
What does randomization contribute?
Random allocation is intended to make the comparison between groups less vulnerable to systematic differences in participant characteristics at the start of the study. It does not guarantee that groups are identical in a finite trial, and it does not eliminate concerns about missing outcomes, deviations from assigned interventions, measurement, or selective reporting. The registry identifies randomization but does not report its sequence-generation or concealment procedures.
What does unmasked mean for interpretation?
The registry lists no masking. Participants and study personnel may therefore have known which intervention was assigned, although the record does not give enough detail to determine who knew what during each stage. For an employment outcome, awareness could potentially influence engagement, service delivery, or reporting. This is a general consideration, not evidence that such bias occurred. Further information on outcome ascertainment and assessor masking is not reported in the registry record.
11. Confidence Intervals and Precision
The primary risk-difference estimate is 0.202, with a 95% confidence interval from 0.035 to 0.370. This interval is entirely above zero, the value corresponding to no difference on the risk-difference scale. Its span nevertheless allows a range of possible effect magnitudes, from a comparatively modest positive contrast to a substantially larger one. The point estimate is not the only plausible value supported by the data.
The registry reports the primary odds ratio as 2.49, with a 95% confidence interval from 1.14 to 5.43. This interval is above one, the null value for an odds ratio, but also includes a broad range of possible odds ratios. Because odds and probabilities are different scales, this interval cannot be read directly as a range of absolute differences in the proportion classified as steady workers. The separate risk-difference interval supplies that absolute-scale perspective.
Precision depends on the amount of information in the data and on the variability of the outcome. The group sizes were 58 and 61 participants. With these sample sizes and the observed outcome counts, the intervals still leave uncertainty about the exact size of the primary contrast. Statistical evidence against the null does not remove that uncertainty.
The registry does not report confidence intervals for the sponsor's secondary analyses. The posted re-calculations include intervals for selected secondary comparisons, but these should not be confused with the sponsor's own model-based estimates. For secondary findings, the number of outcomes considered also matters when judging how much weight to place on an isolated result. The registry record does not report a multiplicity adjustment.
12. Planning a Similar Trial: Sample Size
This is a planning illustration, not a re-evaluation of the original trial. It asks: if a new study wanted to detect a difference like the one observed, or a smaller one, how large would it need to be? It is deliberately not “observed power” of the finished trial, which is not a meaningful quantity.
| Effect to detect | Participants per arm, 80% power | Participants per arm, 90% power |
|---|---|---|
| 100% of the observed effect | 86 | 115 |
| 75% of the observed effect | 148 | 197 |
| 50% of the observed effect | 319 | 427 |
The planning figures are illustrations based on the observed effects, not the original trial's design calculations. The assumptions listed for these illustrations are a two-sided significance level of 0.05, equal allocation between the two arms, and a normal approximation. The effect to detect is defined as the stated fraction of the observed effect. The original sample-size justification and its assumptions are not reported in the registry record.
For the outcomes with planning figures, the entries are participants per arm, rather than a total number of participants across both groups. The general convention in the record is that time-to-event planning calculations give a total number of events using the Schoenfeld approach; other calculations give participants per arm. The posted planning figures for these comparisons are expressed per arm.
The 80% power column represents the planning sample size per arm associated with an 80% chance of detecting the specified effect under the stated assumptions, if that effect and those assumptions were correct. The 90% power column uses a higher target chance of detection and therefore generally requires a larger planned sample size for the same effect. These are design-planning quantities, not a measure of the certainty of the observed result.
Each row uses a different fraction of the observed effect. Smaller effects require more participants to detect, all else equal, because a smaller signal is harder to distinguish from random variation. For example, the primary-outcome illustration lists 86 participants per arm for 80% power when planning for 100% of the observed effect, 148 when planning for 75%, and 319 when planning for 50%. These figures show how strongly the required sample size depends on the effect chosen for planning.
This is not observed power. It is a planning illustration derived from the observed effect, not an assessment of whether the completed trial had adequate power or a replacement for the trial's original design rationale.
13. How the Sample Size Is Derived
Power is the probability that a study of a given size rejects the null hypothesis when a specific true effect exists. It depends on four things: the effect to detect, the variability of the outcome, the significance level (here a two-sided α of 0.05, so z1−α/2 = 1.960) and the desired power (z1−β = 0.842 for 80% and 1.282 for 90%).
Formula
n is the number of participants in each arm, p0 and p1 are the control and treatment proportions, and p̄ is their average.
Worked calculation for this trial
- Control proportion p0 = 0.246; treatment proportion p1 = 0.448; average p̄ = 0.347.
- First term: z1−α/2 × √[2p̄(1−p̄)] = 1.96 × 0.6732 = 1.3195.
- Second term: z1−β × √[p0(1−p0) + p1(1−p1)] = 0.842 × 0.6578 = 0.5537.
- n = (1.3195 + 0.5537)2 / (0.2024)2 = 85.7, rounded up to 86 participants per arm for 80% power.
The smaller-effect rows in the table above repeat the same calculation with the effect multiplied by 0.75 and 0.5 (for ratios, applied to the logarithm of the ratio).
R code
alpha <- 0.05
z_a <- qnorm(1 - alpha / 2) # two-sided
p0 <- 0.245901639344262 # control proportion
p1 <- 0.448275862068966 # treatment proportion
n_per_arm <- function(power, frac = 1) {
q1 <- p0 + (p1 - p0) * frac # smaller effect = fraction of the observed difference
pbar <- (p0 + q1) / 2
z_b <- qnorm(power)
ceiling(round((z_a * sqrt(2 * pbar * (1 - pbar)) + z_b * sqrt(p0 * (1 - p0) + q1 * (1 - q1)))^2 / (q1 - p0)^2, 6))
}
for (f in c(1, 0.75, 0.5))
cat(sprintf("%3.0f%% of observed effect: n per arm = %s (80%% power), %s (90%% power)\n", 100 * f,
format(n_per_arm(0.80, f), big.mark = ","), format(n_per_arm(0.90, f), big.mark = ",")))
# Check against R's built-in function (unrounded n per arm)
cat("power.prop.test:", round(power.prop.test(p1 = p0, p2 = p1, power = 0.80)$n, 1), "\n")
# Simulation: how often does the z-test reject at the 80%-power sample size?
set.seed(2400736)
n <- n_per_arm(0.80)
reps <- 20000
x0 <- rbinom(reps, n, p0); x1 <- rbinom(reps, n, p1)
ph <- (x0 + x1) / (2 * n)
z <- ((x1 - x0) / n) / sqrt(ph * (1 - ph) * 2 / n)
cat(sprintf("Simulated power at n = %s per arm: %.3f (%d replicates)\n", format(n, big.mark = ","), mean(abs(z) > z_a, na.rm = TRUE), reps))
Output from running the code
100% of observed effect: n per arm = 86 (80% power), 115 (90% power)
75% of observed effect: n per arm = 148 (80% power), 197 (90% power)
50% of observed effect: n per arm = 319 (80% power), 427 (90% power)
power.prop.test: 85.7
Simulated power at n = 86 per arm: 0.800 (20000 replicates)
The simulation draws many pairs of event counts at the planned sample size and counts how often the two-proportion z-test rejects the null. It should land close to the target power; at very small required sample sizes it can fall below it, because the normal approximation is rough there.
14. Analyses Reported by the Sponsor
| Method | Estimate | 95% CI | P-value |
|---|---|---|---|
| Chi-squared | Odds Ratio (OR) 2.49 | 1.14 to 5.43 | 0.020 |
For the primary outcome, the registry reports the sponsor's chi-squared analysis as an odds ratio of 2.49, with a 95% confidence interval from 1.14 to 5.43 and p = 0.02. The simple unadjusted re-calculation gives an odds ratio of 2.492, with a confidence interval from 1.143 to 5.432, and p = 0.02. The estimates are closely aligned at the precision displayed. The re-calculation also expresses the contrast as a risk difference, which is a different effect scale and is not a replacement for the sponsor's odds ratio.
For secondary comparisons, the posted recalculations and sponsor-reported p-values are not identical in every case. The record identifies t-tests for weeks worked and income from competitive jobs, and a mixed models analysis for time to first competitive job. The recalculations are simple, unadjusted comparisons, whereas the registry gives the sponsor's analysis method labels and p-values. The registry record does not provide enough model detail to reconcile every difference.
The registry does not flag a discrepancy between the primary reported analysis and the recalculation. The sponsor's analysis remains the primary reported result; a summary-statistic recalculation is best understood as a complementary check rather than as a replacement analysis. The full analysis plan and its implementation details are not reported in the registry record.
15. Multiplicity, Missing Data and Analysis Populations
The registry lists 11 outcome measures and 11 analyses, including the primary outcome and several secondary measures. When a study evaluates multiple outcomes, the chance of at least one nominally notable finding arising by chance can rise. Statistical methods can address multiplicity, for example by adjusting the threshold or otherwise controlling error across a defined family of tests. The registry record does not report whether such an approach was planned or used. As a result, the primary outcome is the clearest confirmatory focus in the posted results, while secondary p-values should be treated more cautiously.
The record identifies analysis methods for the posted comparisons but does not provide the complete statistical analysis plan. It does not report whether analyses were prespecified in detail, how covariates were selected, whether subgroup analyses were planned, or how departures from the original analysis plan were handled. These omissions limit assessment of how closely the reported analyses follow an intended plan.
Analysis populations are also not described in sufficient detail. The primary-outcome population is described as 119 randomized Veterans, and the outcome counts are reported by randomized group. However, the registry record does not specify whether the primary analysis followed an intention-to-treat principle, how participants were handled if they did not receive their assigned service, or whether any participants were excluded from particular analyses.
Missing-data procedures are not reported in the registry record. The posted group counts do not explain whether all randomized participants contributed outcome information, how unavailable follow-up was handled, or whether any imputation or sensitivity analysis was performed. Missingness can affect interpretation if the availability of outcome data is related to group assignment or to employment outcomes. Without the relevant details, the direction or size of any such influence cannot be assessed.
The registry does not report interim analyses, stopping rules, or the original sample-size justification. These are not grounds to assume that a particular procedure was or was not used. They remain unreported elements of the study's statistical design.
16. Safety Results
The registry reports serious adverse events as affected participants divided by participants at risk.
| Arm | Participants with serious adverse events | Participants at risk | Percentage |
|---|---|---|---|
| Individual Placement and Support (IPS) | 6 | 58 | 10.3% |
| Treatment as Usual Vocational Rehabilitation/Transitional Work (TAU-VR) | 4 | 61 | 6.6% |
Deaths
| Arm | Deaths (participants affected of at risk) |
|---|---|
| Individual Placement and Support (IPS) | 1 of 58 |
| Treatment as Usual Vocational Rehabilitation/Transitional Work (TAU-VR) | 0 of 61 |
Other (non-serious) adverse events
Participants with at least one non-serious event at or above the registry's 0% reporting threshold.
| Arm | Participants affected of at risk |
|---|---|
| Individual Placement and Support (IPS) | 17 of 58 |
| Treatment as Usual Vocational Rehabilitation/Transitional Work (TAU-VR) | 22 of 61 |
Most frequent serious adverse event terms
| Event term | Organ system | Individual Placement and Support (IPS) | Treatment as Usual Vocational Rehabilitation/Transitional Work (TAU-VR) |
|---|---|---|---|
| Suicidal Ideation | Psychiatric disorders | 2 of 58 | 1 of 61 |
| Seizure | Nervous system disorders | 1 of 58 | 0 of 61 |
| Kidney Stone | Renal and urinary disorders | 1 of 58 | 0 of 61 |
| Gun Shot Wound | Skin and subcutaneous tissue disorders | 1 of 58 | 0 of 61 |
| Peptic Ulcer | Gastrointestinal disorders | 0 of 58 | 1 of 61 |
| Death | Cardiac disorders | 1 of 58 | 0 of 61 |
| Cancer | Neoplasms benign, malignant and unspecified (incl cysts and polyps) | 0 of 58 | 1 of 61 |
| Chest Pain | Cardiac disorders | 1 of 58 | 0 of 61 |
| Hysterectomy | Reproductive system and breast disorders | 0 of 58 | 1 of 61 |
Most frequent non-serious adverse event terms
| Event term | Organ system | Individual Placement and Support (IPS) | Treatment as Usual Vocational Rehabilitation/Transitional Work (TAU-VR) |
|---|---|---|---|
| Muscular Skeletal Pain or Problem | Musculoskeletal and connective tissue disorders | 8 of 58 | 9 of 61 |
| Infection | Infections and infestations | 3 of 58 | 6 of 61 |
| Neurological | Nervous system disorders | 2 of 58 | 4 of 61 |
| Accidents, insects, dog bites | Injury, poisoning and procedural complications | 2 of 58 | 3 of 61 |
| Mental health | Psychiatric disorders | 2 of 58 | 1 of 61 |
| GI, including dental | Gastrointestinal disorders | 1 of 58 | 2 of 61 |
| Pmonary | Respiratory, thoracic and mediastinal disorders | 0 of 58 | 3 of 61 |
| Eye | Eye disorders | 1 of 58 | 1 of 61 |
| Cardiovascular | Cardiac disorders | 1 of 58 | 1 of 61 |
| Reproductive system | Reproductive system and breast disorders | 1 of 58 | 1 of 61 |
Serious events with the highest arm-level rate: suicidal ideation (3.4%), seizure (1.7%), kidney stone (1.7%), gun shot wound (1.7%), death (1.7%).
The registry reports serious adverse events in 6 of 58 participants in the IPS group and 4 of 61 participants in the TAU-VR group. It reports other adverse events in 17 of 58 and 22 of 61 participants, respectively. The death counts were 1 in the IPS group and 0 in the TAU-VR group. These are counts of participants with events as reported in the registry; they do not, by themselves, describe event timing, duration, severity beyond the serious-event classification, or causal relation to an intervention.
The listed serious-event terms include suicidal ideation, seizure, kidney stone, gun shot wound, death, and chest pain. The registry gives the corresponding listed percentages as 3.4%, 1.7%, 1.7%, 1.7%, 1.7%, and 1.7%. These terms should be read as the registry's reported event categories, not as evidence that the intervention caused them. Event-level detail, relatedness assessments, and a formal between-group safety analysis are not reported in the registry record.
Differences in adverse-event counts are descriptive. The record does not provide enough information to establish whether event ascertainment was comparable across groups or to evaluate the statistical uncertainty of these counts. The figures therefore should not be interpreted as a definitive comparative safety assessment.
17. What the Result Does, and Does Not, Mean
The primary result describes the randomized groups' observed performance on a specific employment endpoint: being classified as a steady worker, defined by holding a competitive job for 50% or more of the 12-month follow-up. The posted counts and estimates indicate that the IPS group had a higher observed proportion meeting this definition than the TAU-VR group. The confidence intervals express uncertainty about the magnitude of the group contrast, and the registry's reported p-value indicates that the observed data are not very compatible with equal odds under the stated chi-squared analysis.
This result does not mean that every participant assigned to IPS met the steady-worker definition, or that no participant assigned to TAU-VR did so. It does not establish the effect for each person, identify which participants might have had different employment experiences, or explain why the group outcomes differed. Nor does it show that the result applies to Veterans who do not match the study's described population or who receive services outside the primary-care PACT setting.
The endpoint captures sustained competitive employment according to a defined threshold. It does not fully represent other dimensions such as job quality, participant preferences beyond the IPS approach's stated emphasis on choice, long-term employment beyond follow-up, or income from all sources. Although the registry lists related secondary outcomes, the posted primary result should not be treated as a result for every employment or mental-health outcome.
The trial was randomized but unmasked. Randomization is an important feature for comparing assigned groups, while lack of masking and the unreported details about allocation, measurement, and missing data leave aspects of study conduct difficult to assess from the registry alone. The result is a statistical comparison, not a guarantee of a particular outcome for an individual and not medical advice. It supports a conclusion about the observed study data and endpoint, while leaving uncertainty about effect size, broader applicability, and unreported outcomes.
18. Why This Trial Matters Statistically
This trial addresses a statistical question that is also practical for service delivery: whether supported employment embedded in a primary-care team is associated with different employment outcomes from usual vocational rehabilitation or transitional work among Veterans with mental disorders. Its randomized parallel-group design makes the assigned-group comparison more informative than a comparison based only on people who chose different services, while the registry's unmasked status and incomplete methods detail remain important considerations.
The primary endpoint is defined in concrete terms, using sustained competitive employment over a 12-month follow-up rather than a vague measure of vocational progress. The reported counts make the endpoint directly interpretable, and the registry presents both an odds ratio and a p-value. The unadjusted re-calculation adds an absolute-scale risk difference and its confidence interval. Considering both scales helps distinguish a relative comparison from the absolute difference in the outcome frequency.
The secondary outcomes broaden the statistical picture by addressing work duration, time to first competitive job, and earnings, among other listed measures. Their pattern can help readers understand whether the primary endpoint sits alongside other employment measures, but multiple outcomes and incomplete reporting limit how strongly those findings can reinforce the primary result.
The record also illustrates why a result should be read alongside its precision and design context. The primary estimate is positive, yet the confidence interval leaves uncertainty about its magnitude. The sample-size illustrations show how planning needs depend on the effect a future study aims to detect, but they are not a substitute for the original trial's unreported design justification. Overall, the study offers a defined randomized comparison while leaving important questions about analysis details, missing data, and generalizability unresolved in the registry record.
19. Limitations
- The posted summary statistics are the only data these results rest on; patient-level data, full protocols and statistical analysis plans are not available.
- Recalculated results are simple, unadjusted calculations from posted summaries. They do not replace the sponsor's prespecified analysis.
Several limitations are specific to interpreting this trial from the registry record. The study was unmasked, and the record does not state whether outcome assessors were masked or how employment information was collected and verified. Because the primary endpoint depends on whether participants held competitive jobs and for how long, the unreported details of ascertainment matter for judging the possibility of measurement differences between groups.
The intervention and comparator descriptions are not equally complete. IPS is described through its focus on competitive work, client choice, and integration with the PACT team, while the TAU-VR description is truncated after introducing transitional work. The range and intensity of services in the comparator group, and any overlap between services, are therefore not fully clear from the registry record.
The population is described as Veterans receiving primary care in a PACT who had at least one mental disorder and were unemployed or under-employed. The record does not provide detailed baseline characteristics or enough information to judge how representative participants were of other Veterans with mental disorders, other care settings, or people facing different employment circumstances. The applicability of these findings beyond the enrolled population is consequently uncertain.
Missing-data handling, the analysis population, allocation concealment, the original sample-size justification, and the full statistical analysis plan are not reported in the registry record. The registry also does not describe interim analyses or a multiplicity strategy. These omissions restrict assessment of bias, precision planning, and the confirmatory status of secondary results. Finally, the primary endpoint is a threshold-based classification over 12 months; it does not convey the full range of employment experiences, and the incomplete posting of some listed secondary outcomes leaves parts of the outcome picture unavailable.
20. Related Tutorials
21. Related Statistical Calculators
22. Sources
- ClinicalTrials.gov: Supported Employment in Patient Aligned Care Teams — NCT02400736.
- Registry record last updated 2022-03-09; status last verified 2022-02.
- Results first posted 2021-03-30.
- Data retrieved from ClinicalTrials.gov on 2026-11-09.
- Group sizes, means, event counts and sponsor-reported p-values are reproduced as posted. Effect estimates, confidence intervals and p-values not attributed to the sponsor are independently calculated from the posted summary statistics.
Continue through the Clinical Biostats statistical pathway
Use this trial's endpoint as a starting point for deeper study of effect measures, confidence intervals, and sample-size planning.
23. Record Summary
The registry record describes a completed randomized, unmasked, parallel-group study of Individual Placement and Support compared with Treatment as Usual Vocational Rehabilitation/Transitional Work. The primary population comprised unemployed or under-employed Veterans with at least one mental disorder followed in a primary care PACT. The primary endpoint was Steady Worker over 12 months, defined as holding competitive employment for 50% or more of follow-up.
The registry reports 26 of 58 IPS participants and 15 of 61 TAU-VR participants meeting the primary definition. Its reported analysis gives an odds ratio of 2.49, a 95% confidence interval from 1.14 to 5.43, and p = 0.02. The unadjusted re-calculation gives a risk difference of 0.202, with a 95% confidence interval from 0.035 to 0.370.
Secondary results cover employment duration, time to first job, income, symptoms, and self-esteem. Details on multiplicity, missing-data handling, analysis populations, and the original sample-size justification are not reported in the registry record.