Concept Mapping as a Scalable Method for Identifying Patient-Important Outcomes: Complete Statistical Analysis

1. Trial at a Glance

This page covers the registry results for Concept Mapping as a Scalable Method for Identifying Patient-Important Outcomes (NCT02792777), a registered phase not stated study. The primary outcome is Comprehensiveness of Interviews as Compared to One Concept Mapping Group.

2. Clinical Question

This is a methods study rather than a like-for-like comparison of clinical treatments. The registry record does not state how participants were allocated to methods, and it does not describe a randomized comparison. Consequently, the central question is about the yield and coverage of two approaches to identifying outcomes, not about a clinical effect of one treatment versus another.

3. Trial Design

This completed study, NCT02792777, is sponsored by Thomas Jefferson University. Its title is “Concept Mapping as a Scalable Method for Identifying Patient-Important Outcomes.” The registry lists enrollment of 148. The study started in 2016-01, its primary completion date is 2018-01-29, and its status is COMPLETED.

The investigators planned to compare interviews with concept mapping as ways to identify outcomes important to patients for use in research. Interview participants were to be recruited from an emergency department acute care visit, a post-acute care visit within 1 week of hospital discharge, and a routine primary care visit. The target within each healthcare setting is 30 patients. The registry description for the interview cohort is incomplete, so it does not establish a complete recruitment target for that arm.

For concept mapping, participants were to be recruited from existing clinical and research databases into three separate groups. Each group had a target of 20 patients, and the total recruitment goal for this cohort is 60 people. The registry record does not report how many participants were actually recruited into each method or setting, whether participants completed the planned activities, or how any departures from the targets were handled.

Participation time is described as 1 day for interviewed patients and 3 days for concept mapping patients. Secondary outcomes concern comparing the methods in terms of time and cost. The registry does not report results for those outcomes. Allocation is not stated, and the arms are not a like-for-like control comparison. Thus, differences in the outcome sets could reflect the methods, the participants or settings involved, or other unreported aspects of implementation.

4. Participant Flow and Accounting

The registry reports how participants moved through each stage of the study. All counts below are exactly as posted; missing counts are not inferred from other stages.

Overall Study

Reasons for not completing this period:

5. Baseline Characteristics

The table reproduces the baseline characteristics posted in the registry, which describe the participants analysed at baseline and help show whether the randomized groups were comparable.

6. Primary Endpoint

The registry lists the primary outcome as Comprehensiveness of Interviews as Compared to One Concept Mapping Group, measured over Interviewed patients participated for 1 day; Concept mapping patients participated for 3 days.

The primary outcome concerns the number of reported patient-important outcomes and the extent to which outcomes identified through interviews are represented in the list generated by the initial concept mapping group. For interviews, investigators planned to analyze transcripts from one healthcare setting using qualitative content analysis. One code was “goals”; ideas coded to goals and relevant to patients’ diabetes care were to be extracted as patient-important outcomes.

The investigators then planned to determine the proportion of interview-identified outcomes present in the list produced during the initial concept mapping group’s brainstorming session. They also planned to identify outcomes unique to each method. This makes the endpoint a comparison of outcome coverage and distinct contributions, rather than a conventional clinical measure such as a change in symptoms or a laboratory value.

Comparing coverage can help assess whether one method identifies outcomes that the other also captures, while reviewing unique outcomes can reveal whether either approach contributes ideas not found through the other. These are useful questions when selecting methods for outcome identification. However, the registry record labels the measure as a number while its description also specifies a proportion. It does not provide a numerical endpoint result, a definition of how overlapping ideas were judged equivalent, or details on how coding disagreements were resolved.

The timeframe states that interviewed patients participated for 1 day and concept mapping patients participated for 3 days. The registry record does not report a completed measurement result for the primary outcome.

7. Why No Re-analysis Is Shown

8. Statistical Interpretation

9. Statistical Methods Explained

The study combines qualitative outcome identification with a comparison of the sets of ideas produced by two methods. The registry record describes the planned procedures, but it does not report a completed analysis or numerical primary result. The concepts below explain how such a comparison is generally interpreted; they do not add unreported findings about this study.

How are outcomes identified from interviews?

Qualitative content analysis organizes text into meaningful categories using a coding approach. In this study, interview transcripts from one healthcare setting were to be analyzed, with “goals” among the codes. Ideas coded to goals and relevant to patients’ diabetes care were to be extracted as patient-important outcomes. Coding translates participants’ words into an organized list while retaining their connection to the topic under study. The registry record does not describe the coding guide, the number of coders, how coding disagreements would be handled, or how the final list would be reviewed.

In general, a coding rule needs to distinguish between different expressions of the same idea and genuinely different outcomes. If two statements are treated as equivalent, they may contribute to a shared outcome category; if treated separately, they may expand the list. This is especially important when the final comparison asks whether ideas from one method are present in another method’s list. The registry record does not report the rules used for making those judgments.

What does overlap measure?

The planned primary comparison asks what proportion of outcomes identified in interviews from one setting are present in the list generated during the initial concept mapping group’s brainstorming session. In general, an overlap proportion describes coverage of one set by another set. It does not, by itself, show that either set is complete. A high overlap could coexist with outcomes missing from both lists, while a lower overlap could reflect differences in wording, coding, participants, or the types of ideas encouraged by each method. The registry record does not report the proportion or the underlying outcome lists.

The plan also calls for identifying unique outcomes from each method. This complements an overlap measure: overlap addresses shared content, whereas unique items indicate contributions not matched in the other list. A unique item is not automatically more important or more credible. Its interpretation depends on the criteria for identifying patient importance and on the reliability of the matching process, neither of which is fully described in the registry record.

Why might statistical precision be difficult to assess?

For a numerical estimate, a confidence interval generally describes the range of values compatible with the data under the analysis assumptions. A narrow interval indicates greater statistical precision than a wide interval, but it does not resolve problems of measurement, bias, or representativeness. The registry record does not report a confidence interval or a numerical estimate for the primary endpoint, so its precision cannot be assessed from this record.

General statistical methods also depend on the structure of the observations. Here, the primary comparison is based on outcome ideas and their classification, not on a conventional participant-level continuous or binary clinical response. The record does not specify a statistical model, a unit of analysis for uncertainty, or a method for accounting for clustering within concept mapping groups. It would therefore be inappropriate to infer an analysis method or calculate uncertainty from the enrollment figure alone.

Over-dispersion is a general concern when observed variation exceeds what a statistical model assumes. It can make standard errors too small if the model fails to allow for that extra variation. Whether over-dispersion is relevant depends on the data and model. Because the registry record does not report the analysis, model, outcome counts, or variability, there is no basis for asserting that over-dispersion was present or that a particular adjustment was used.

How should planning calculations be read?

In general, sample-size calculations describe how much information a future design might need under stated assumptions; they are not a measure of what a completed study achieved. Any planning illustration should not be confused with observed power. The registry record describes illustrative assumptions involving a two-sided significance level of 0.05, equal allocation between arms, a normal approximation, and detection of a stated fraction of an observed effect. It does not report the resulting planning values or an observed primary effect in the information available for this analysis. Those assumptions therefore cannot be used to claim that the study had adequate power or to reconstruct an effect.

More broadly, the record’s enrollment and target recruitment descriptions answer different questions. Enrollment is listed as 148, while the arm descriptions give recruitment targets for interview settings and concept mapping groups. Targets are planning descriptions, not evidence of achieved recruitment or analysis sample sizes. The registry record does not give the participant flow needed to connect enrollment to the primary analysis.

10. Multiplicity, Missing Data and Analysis Populations

The registry record lists five outcome measures and identifies time and cost comparisons as secondary outcomes. It lists no analyses. It does not report a statistical analysis plan, a hierarchy among outcomes, or a procedure for handling multiple comparisons. When several outcomes or analyses are considered, multiplicity is a general concern because the chance of finding an apparently notable result can rise as the number of statistical tests increases. Without a reported analysis plan or results, the implications of multiplicity for this study cannot be assessed.

Missing-data handling is not reported in the registry record. The record does not state whether participants withdrew, whether interview transcripts or concept mapping contributions were incomplete, how partial contributions would be used, or how missing outcome information would be handled. In a qualitative comparison, incomplete text or incomplete group participation may affect which ideas enter the outcome lists; the record does not say whether such cases occurred or how they were addressed.

The analysis populations are also not reported. Although enrollment is listed as 148, the registry record does not give the number assigned to each method, the number contributing analyzable data, or the number included in the primary comparison. Allocation is not stated, and the record does not describe whether analysis would follow an intention-to-treat principle or another inclusion rule. These omissions prevent assessment of participant flow, exclusions, and the relationship between enrollment and the reported outcome.

11. What the Result Does, and Does Not, Mean

The registry record describes a study intended to assess how interviews and concept mapping contribute to identifying patient-important outcomes for research. It specifies an approach for extracting outcomes from interview transcripts, comparing them with outcomes from a concept mapping brainstorming session, and identifying unique contributions. It also identifies time and cost as secondary topics.

What the record does not establish is the comparative result. It contains no reported primary outcome value, no list of identified outcomes, and no reported comparison of time or cost. It therefore does not show whether the methods produced substantial overlap, whether either method yielded unique outcomes, or how precisely any difference was estimated. The status COMPLETED indicates the recorded study status, not that numerical results are included in the registry record.

Even if outcome lists were available, the interpretation would depend on the procedures used to define, code, and match ideas. The record does not provide those details. A difference in lists would not alone establish that the method caused the difference because allocation is not stated and recruitment sources differ. Nor would an outcome’s appearance in one list alone establish its relative importance or validity.

The study is about methods for outcome identification, not clinical treatment effects. It cannot support conclusions about diabetes care or the effect of a treatment. The enrollment entry, recruitment targets, and participation durations describe the study context; they are not evidence that either method was more comprehensive, efficient, or less costly. The appropriate conclusion from the registry record is limited to the planned comparison and its reported design information. The central comparative findings are not reported in the registry record.

12. Why This Trial Matters Statistically

Researchers need outcomes that reflect what matters to patients when designing studies and evaluating care. The choice of outcome-identification method can shape which ideas enter a research program. This study matters statistically because it frames that choice as a comparison of coverage and unique contributions rather than assuming that a method’s output is complete or interchangeable with another method’s output.

The planned comparison highlights several measurement issues. Outcome lists depend on how participant statements are coded, when two ideas are treated as equivalent, and which participants and settings contribute. These decisions affect the observed overlap and the count of unique items. Reporting them transparently is important for assessing whether a comparison reflects the methods or the process used to classify their outputs.

The record also raises an efficiency question by naming time and cost as secondary outcomes and describing different participation durations for the two approaches. Those durations alone are not a complete measure of efficiency: a full comparison would need clearly defined resource measures and reported results. The registry record does not provide those results.

Finally, this study illustrates why a completed registry entry should not be treated as a report of findings when outcomes and analyses are absent. Enrollment and design information can explain what investigators intended to compare, but cannot establish the size, precision, or direction of a result. The distinction between a planned methods evaluation and a reported result is essential when interpreting evidence about how patient-important outcomes are identified.

13. Limitations

• The posted summary statistics are the only data these results rest on; patient-level data, full protocols and statistical analysis plans are not available.
• Recalculated results are simple, unadjusted calculations from posted summaries. They do not replace the sponsor's prespecified analysis.

The registry description leaves important details of the primary outcome operationalization unclear. It labels the measure as a number of reported patient-important outcomes, while the outcome narrative describes determining the proportion of interview-identified outcomes present in a concept mapping list. The record does not explain how these descriptions relate, how the denominator for the proportion would be defined, or how duplicate and overlapping ideas would be treated.

The comparison also uses interview transcripts from one healthcare setting for the stated primary analysis, while interview recruitment was planned across three settings. The registry record does not identify which setting would contribute to that analysis or explain how it was selected. Concept mapping uses an initial group for the described comparison, while the broader concept mapping cohort was planned across three groups. The record does not explain how the initial group relates to the other groups in the primary endpoint analysis.

Recruitment sources and participation formats differ between methods. Interviews draw on care settings, while concept mapping participants were to be recruited through existing clinical and research databases. The record does not report participant characteristics or establish comparability between the groups. Allocation is not stated, limiting causal interpretation of any difference between methods.

Additional limitations are the absence of reported outcome values, analysis details, missing-data procedures, and analysis populations. The registry record lists no analyses, so the precision and robustness of the planned comparison cannot be assessed. The interview arm description is incomplete, and the registry does not establish achieved recruitment by arm or setting. Finally, the record’s entries for adverse events do not provide context for interpreting safety; safety is not the central question in this methods comparison.

14. Related Tutorials

15. Related Statistical Calculators

16. Sources

• ClinicalTrials.gov: Concept Mapping as a Scalable Method for Identifying Patient-Important Outcomes — NCT02792777.
• Registry record last updated 2019-11-15; status last verified 2019-10.
• Results first posted 2019-10-21.
• Data retrieved from ClinicalTrials.gov on 2026-11-09.
• Group sizes, means, event counts and sponsor-reported p-values are reproduced as posted. Effect estimates, confidence intervals and p-values not attributed to the sponsor are independently calculated from the posted summary statistics.

17. Record Summary

NCT02792777 is a completed study sponsored by Thomas Jefferson University, titled “Concept Mapping as a Scalable Method for Identifying Patient-Important Outcomes.” The registry lists enrollment of 148, a start date of 2016-01, and primary completion on 2018-01-29.

The study planned to compare interviews with concept mapping as methods for identifying outcomes relevant to patients’ diabetes care. The primary outcome description focuses on the overlap between outcomes drawn from interview transcripts from one healthcare setting and outcomes generated during the initial concept mapping group’s brainstorming session, as well as outcomes unique to each method. Time and cost are listed as secondary outcome topics.

The record describes recruitment plans and participation durations but does not report a primary result, secondary results, analysis details, or analysis populations. Allocation is not stated. The registry record therefore documents the intended methods comparison but does not establish which approach identified more outcomes or how precisely the approaches differed.