Designing Hyperparameter Grids for PRS Methods

Why Grid Design Matters

A PRS method can produce different scores under different choices of p-value threshold, shrinkage strength, or prior settings. Tuning requires comparing candidate settings using validation data, but every extra candidate can add substantial computation, particularly when a method is fitted repeatedly within a validation procedure. A grid that is too coarse can miss useful regions; one that is too dense can spend resources comparing settings that are effectively indistinguishable.

Grid design is therefore a balance between coverage and resolution. Coverage means including a plausible range of settings. Resolution means having enough candidates within that range to locate meaningful changes in validation performance. The aim is not to make the grid large, but to make it informative and computationally feasible.

The choice of validation data and protection against test-set reuse are addressed in the earlier material on choosing score hyperparameters without reusing test data and nested validation. Here, assume that the validation design is fixed and that each candidate is evaluated using the same prespecified procedure. The discovery data, LD inputs, phenotype definition, covariates, sample eligibility rules, and validation splits should also be held fixed across candidates unless one of these is itself the intended object of study. Changing them along with the hyperparameters makes comparisons difficult to interpret.

Build the Grid Around the Parameter

Different PRS parameters have different scales and meanings. A p-value threshold \(t\) determines which variants are eligible for a score; changing \(t\) can change the number of variants substantially. A shrinkage parameter may control how strongly estimated effects are pulled toward zero, but its numerical direction and scale depend on the method. A prior may involve several settings, such as a probability \(q\) that an effect is nonzero and a variance \(\tau^2\) for nonzero effects. A useful grid reflects those distinctions rather than applying one arbitrary sequence to every parameter.

For p-value thresholds, include values spanning orders of magnitude when plausible choices do so. A list such as \(5\times10^{-8}, 10^{-6}, 10^{-4}, 0.01, 0.1, 1\) samples multiplicatively spaced regions more usefully than six equally spaced numbers between zero and one. Include a permissive endpoint only when it corresponds to a meaningful candidate, such as allowing all available variants. Record how many variants pass each threshold in the discovery data: that count helps show what the grid actually changes.

For a positive shrinkage strength \(\lambda\), a logarithmic grid is often sensible when the effect of changing \(\lambda\) is relative rather than additive. This is not a universal rule: inspect the method's parameterization and use its documented interpretation. In some procedures, larger values mean stronger shrinkage; in others, the supplied parameter may instead represent a quantity such as a retained signal fraction. Do not compare numerical values across different methods as though they had the same meaning.

For prior settings, first identify which combinations are valid for the model. If \(q\) denotes a nonzero-effect probability and \(\tau^2\) a variance conditional on a nonzero effect, it may be important to vary them jointly: the same variance can have different consequences under different values of \(q\). A full Cartesian grid can become large quickly. Use a modest set of scientifically plausible combinations, and avoid combinations that the model does not permit or that are incompatible with the scale used to define the prior.

Budget the Fits Before Running Them

A grid's computational cost is determined not only by its number of candidates but also by the validation procedure. If there are \(K\) candidate configurations and each is evaluated in \(F\) validation folds, the basic number of candidate-fold evaluations is \(KF\). If each evaluation requires a separate model fit, this is also the number of fits. When the evaluation is nested, outer repetitions multiply the work: with \(O\) outer folds, a simple count is \(OKF\) inner candidate-fold evaluations, before counting any final refits or method-specific iterations.

This count is a planning tool, not a guarantee of runtime. Some settings may reuse intermediate computations, while others may require expensive fitting. If computation is limited, reduce redundant candidates or use a staged search: first assess a coarse grid, then refine a small region using the permitted training and validation data. Any refinement must follow the validation design; the outer test data must not influence which candidates are added or selected.

Use Validation Patterns, Not Just the Highest Entry

Let \(V(\theta)\) denote the prespecified validation criterion, with larger values defined to be better. The numerical maximum is a natural candidate, but it can be misleading when it is an isolated peak surrounded by poorer values. A useful grid assessment asks whether high values persist across nearby settings, whether the best value lies at the edge of the grid, and whether the pattern is plausible for the method.

One transparent descriptive rule is to define a practical tolerance \(\delta\) in advance and call configurations near-optimal when they satisfy \(V(\theta)\geq V_{\max}-\delta\), where \(V_{\max}\) is the highest observed validation value. The tolerance is in the units of the validation criterion and should represent a difference small enough not to change the practical decision. It is not a probability statement, a test of equality, or a substitute for evaluating validation uncertainty. In particular, a difference smaller than \(\delta\) does not prove that the settings perform identically.

A robust choice is often one that belongs to a broad near-optimal region, rather than a lone point with the largest recorded value. If several candidates are practically indistinguishable, a simpler or more stable setting may be preferable, provided that preference is specified transparently. When the maximum occurs at a grid boundary, the grid may not cover the useful region; expand or refine it using permitted data rather than declaring the boundary best by default.

Joint Prior Grids and Their Interpretation

When prior settings act together, inspect the results in combinations rather than selecting each parameter independently. For instance, selecting the best \(q\) while holding an arbitrary \(\tau^2\) fixed can miss an interaction: the preferred nonzero-effect probability may depend on the conditional effect variance. A compact table or heat map can reveal whether performance changes smoothly across combinations, whether a ridge of good values exists, or whether one isolated combination dominates.

Diagnostics, Alternatives, and Limitations

Several patterns indicate that a grid needs attention. A best candidate at the smallest or largest value can signal an inadequate range. A sharp, isolated peak may reflect a genuinely sensitive method, but it can also result from instability in the validation results; do not respond by claiming certainty from a single maximum. A flat surface suggests that extra grid density may add little decision value. Unexpected discontinuities can arise from implementation errors, inconsistent variant sets, or changes in preprocessing across candidates. Check that candidates were fitted successfully and that all comparisons use the same validation design.

An exhaustive grid is easy to audit and useful when only a few settings are involved, but its cost grows multiplicatively with the number of parameters. A staged coarse-to-fine grid can use computation more efficiently when refinement is confined to allowed training and validation data. Random search samples a larger space without evaluating every combination and can be useful when there are many parameters, but it may leave important regions unexplored and can be harder to reproduce unless the sampling plan and random seed are recorded. More automated search strategies can reduce manual grid construction, but they do not remove the need to define valid ranges, control data reuse, or inspect the resulting search region.

No grid can compensate for a poorly chosen validation design, incompatible LD information, or a mismatch between the phenotype and the intended prediction target. Nor does the best observed candidate guarantee best performance in a new cohort. The purpose of the grid is to make a disciplined comparison over a defensible set of choices; conclusions remain limited by the candidates included and the validation data used.