Choosing a Panel for the Study at Hand
A reference panel is useful only to the extent that its haplotypes help recover untyped genotypes in the study sample. A large panel may offer extensive variant coverage but represent the study participants poorly; a smaller panel may be a closer match but omit variants the study needs. Panel choice is therefore a study-specific decision, not a universal ranking of resources.
The earlier material on reference panel size and coverage established why panel size and the variants represented matter. Here the task is to compare candidate panels across several considerations at once. The comparison should begin with requirements that a panel must meet, then weigh the advantages among panels that remain eligible. A score can make trade-offs explicit, but it cannot turn uncertain judgments into objective facts or guarantee imputation accuracy.
Screen for Requirements Before Comparing Preferences
First specify what the study needs. For example, decide which study participants or ancestry groups must be adequately represented, which genomic regions and variants are priorities, and what input data the imputation workflow can use. Compatibility includes whether the panel and study data can be compared in the relevant genomic regions and whether their genotype representations can be made consistent. A panel that cannot be used with the study data is not rescued by a high score on other features.
Some requirements are hard constraints: failure makes a panel unsuitable for the intended analysis. Others are preferences that can be traded off. The exact boundary depends on the study. If all major study groups need representation, a panel that lacks one group may fail a hard constraint. If one group is the primary target and others are outside the analysis, the same feature may instead be reflected as a lower ancestry-representation score.
For eligible candidates, define a scoring rubric before looking at the final ranking. A convenient approach is to score each criterion from 0 to 1, where higher values mean more suitable for this study. The numbers need a documented meaning. For instance, variant coverage might be scored according to a predeclared measure of how well the panel covers the study's priority variants, rather than a vague impression that coverage is “good.” Avoid counting the same advantage twice: if a coverage score already reflects the availability of target variants, do not add a second score that simply rewards those same variants.
This additive score makes an assumption: a deficit on one criterion can be compensated by strengths on others in proportion to the weights. That may be reasonable for preferences, but it is inappropriate for a true requirement. It also assumes the scores are meaningfully comparable and that the chosen weights represent the study's priorities. If either assumption is doubtful, show the separate criteria rather than relying on a single total.
Worked Comparisons
Worked Example: Ranking Two Eligible Panels
Suppose a hypothetical study has already established that both candidate panels meet its minimum compatibility requirements. The study team assigns the following scores, from 0 to 1, and weights.
| Criterion | Panel E score | Panel F score | Weight |
|---|---|---|---|
| Ancestry representation | 0.90 | 0.65 | 0.35 |
| Priority-variant coverage | 0.80 | 0.95 | 0.35 |
| Useful sample size | 0.70 | 0.95 | 0.10 |
| Compatibility | 0.95 | 0.90 | 0.20 |
The weights sum to \(0.35+0.35+0.10+0.20=1.00\). The scores are
Under these particular scores and priorities, Panel E ranks above Panel F, \(0.855\) versus \(0.835\). The difference is small, and the table shows why: F scores better on coverage and size, while E scores better on ancestry representation and compatibility. The result supports a conditional choice, not a claim that E is intrinsically the better panel for every study.
Worked Example: A High Score Does Not Override a Requirement
Consider a different hypothetical study that requires both ancestry group A and ancestry group B to be represented above a minimum level set in its analysis plan. Panel G meets both requirements. Panel H has strong overall variant coverage and a large sample, but its representation of group B is below the specified minimum. The team assigns preference scores to the eligible panels using the following illustrative weights: ancestry 0.40, coverage 0.30, size 0.15, and compatibility 0.15.
Suppose Panel G scores 0.85, 0.70, 0.65, and 0.90 on those four criteria. Its score is
Suppose Panel H scores 0.60, 0.98, 0.95, and 0.90. Its arithmetic total would be
That numerical result does not make H an acceptable choice: it fails the stated minimum for group B and is therefore ineligible. The score is intended to rank eligible candidates, not erase a requirement. If the minimum were not genuinely non-negotiable, the team would need to revise its analysis plan and justify treating ancestry representation as a trade-off instead.
Check Whether the Choice Is Robust
Scores and weights involve judgment, so report whether a modest change in priorities changes the ranking. This is a form of sensitivity analysis: recalculate the ranking under plausible alternative inputs and identify which assumptions drive the decision. It is especially useful when two candidates have similar totals or the scores are uncertain. Do not present the sensitivity calculation as a formal probability that one panel is best.
Worked Example: Sensitivity to the Coverage Priority
Return to Panels E and F in the first example. Suppose the study team decides that coverage deserves weight 0.50 and uses weights 0.25 for ancestry, 0.50 for coverage, 0.10 for size, and 0.15 for compatibility. These weights sum to 1.00. Using the same criterion scores, the revised totals are
With this revised priority, F ranks higher, \(0.8675\) versus \(0.8375\). The ranking changes because F's advantage in coverage receives more weight. A defensible report would state both the criteria and the reason for assigning coverage greater importance, rather than presenting the revised winner without its assumptions.
Diagnostics, Alternatives, and Limits
Before settling on a choice, inspect each criterion score and its basis. Ask whether the ancestry description is relevant to the actual study participants, whether the coverage measure emphasizes the variants and regions that matter, and whether “sample size” reflects useful haplotypes rather than only a headline count of individuals. A larger nominal sample need not provide a proportionate increase in useful information if the added samples do not address the study's needs. Check compatibility with the intended study input and workflow rather than inferring it from the panel's size or reputation.
Warning signs include scores with no documented rubric, weights chosen after seeing which panel wins, a total that changes dramatically under small plausible adjustments, and a panel that ranks first only because a failed requirement was treated as an ordinary preference. Also examine missing or uncertain information: an unknown criterion should not quietly be assigned a high score. Record the uncertainty, use a conservative range where appropriate, or restrict the comparison to what can be assessed.
The weighted score is not the only useful comparison. A side-by-side table of the separate criteria preserves detail and is often preferable when weights cannot be agreed upon. A dominance check is another simple alternative: if one eligible panel is at least as good as another on every criterion and strictly better on at least one, the dominated panel need not be preferred under any nonnegative weights. If neither dominates, the choice depends on priorities and may require a documented, study-specific decision.
These approaches have limits. Scores can compress important differences, and the additive formula permits compensation that may not be scientifically acceptable. Panel selection also does not establish the eventual quality of imputed genotypes; that requires evaluating the imputation results using appropriate methods. A choice based on available information should therefore be documented with its eligibility rules, criteria, weights, data limitations, and reasons for preferring the selected candidate. The purpose is a transparent and defensible choice, not a claim of certainty.
A Short Justification for the Weighted Score
The formula follows directly from the intended meaning of the weights. If a panel receives score \(x_{jk}\) on criterion \(k\), its contribution to the total is that score multiplied by the criterion's importance \(w_k\). Adding contributions gives \(S_j=\sum_k w_kx_{jk}\). When the weights sum to 1 and the scores lie from 0 to 1, the total also lies from 0 to 1: since \(0\leq x_{jk}\leq1\), each \(w_kx_{jk}\) lies between 0 and \(w_k\), so \(0\leq S_j\leq\sum_k w_k=1\). This bound does not make the score a probability; it only follows from the chosen scale and arithmetic.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- Why should a hard compatibility requirement be checked before computing a weighted score?
- A panel has scores 0.80 for ancestry representation and 0.70 for coverage. The corresponding weights are 0.60 and 0.40. Calculate its score.
- What does it mean, and what does it not mean, if an eligible panel has a higher weighted score than another?
- Name one reason that a panel's nominal sample size may not be enough to establish that it is the better choice.
- What should a team do if its preferred panel changes under a plausible change in weights?
Solutions. (1) A hard requirement defines eligibility; a high score on other criteria cannot make a panel usable if it fails that requirement. (2) \(0.60(0.80)+0.40(0.70)=0.48+0.28=0.76\). (3) It ranks higher under the stated scores and weights; it does not mean the panel has a 76% chance of success, nor does it guarantee better imputation. (4) The added samples may not improve representation of the study's target groups or provide proportionate useful haplotype information. (5) Report the ranking under both assumptions, identify which priority drives the change, and justify the weights or present the trade-off without claiming a uniquely determined winner.