Converting Rank Preferences into Utility Weights

From Preference Order to Candidate Weights

A stakeholder may be able to say that avoiding a severe adverse event matters more than improving symptom control, and that symptom control matters more than convenience. This ranking is useful: it rules out weights that contradict the stated order. But it does not say how much more important one concern is than another. Converting ranks into numbers therefore requires a rule that adds assumptions beyond the ranking itself.

The distinction matters when clinical outcomes are combined into a utility for comparing treatment options. In an additive representation, \(U(a)=\sum_{j=1}^{m} w_j u_j(a)\), \(u_j(a)\) is the utility score for option \(a\) on criterion \(j\), and \(w_j\) is its nonnegative weight. The weights sum to one when a normalized weighted average is desired. This representation assumes that the criterion scores have been defined on compatible scales and that an additive combination is appropriate. A ranking of criterion importance can inform the order of the \(w_j\), but does not itself supply the \(u_j\), their scales, or the gaps between weights.

Keep two kinds of ranking separate. A stakeholder might rank criteria, such as efficacy, toxicity, and treatment burden, by importance; a conversion rule can turn that ordering into candidate criterion weights. Alternatively, a stakeholder might rank clinical outcome profiles, such as “response without severe toxicity” and “no response with mild toxicity.” That profile ranking orders the profiles but does not automatically specify either criterion weights or the utility differences between profiles. The elicitation target must be explicit before numbers are assigned.

Rank-Order Centroid Weights

A commonly used rank-only conversion is the rank-order centroid (ROC) rule. Suppose \(m\) criteria have been ranked from most to least important. Write \(w_{(1)}\geq w_{(2)}\geq\cdots\geq w_{(m)}\geq0\), where parentheses indicate rank position, and impose \(\sum_{j=1}^{m}w_{(j)}=1\). The ROC weight at rank \(j\) is:

The rule has a precise interpretation: it is the centroid, or coordinate-wise average, of the region of all nonnegative weights that sum to one and respect the stated ranking. Equivalently, it averages across that feasible region under a uniform-volume assumption. That assumption is a mathematical convention, not a fact established by the stakeholder’s answers.

The formula can be justified directly. Set \(w_{(m+1)}=0\) and define the successive differences \(d_k=w_{(k)}-w_{(k+1)}\). The ordering constraints imply \(d_k\geq0\), and \(w_{(j)}=\sum_{k=j}^{m}d_k\). Summing the weights gives \(\sum_{j=1}^{m}w_{(j)}=\sum_{k=1}^{m}k d_k=1\). Now let \(y_k=k d_k\). The \(y_k\) are nonnegative and sum to one. The transformation from the ordered weights to the \(y_k\) has a constant Jacobian, so uniform volume on the feasible ordered-weight region corresponds to uniform volume on this simplex. By symmetry, each \(y_k\) has mean \(1/m\). Thus \(E[d_k]=E[y_k]/k=1/(mk)\), and:

The ROC rule therefore gives one defensible representative of the weights consistent with the ranking. It does not show that a stakeholder actually holds those exact weights, nor that uniform volume is the only reasonable assumption about the unknown weights.

Other Rank-Based Mappings

ROC is not the only possible conversion. For \(m\) criteria, a simple rank-sum rule assigns \(w_{(j)}=2(m+1-j)/[m(m+1)]\). It also produces nonnegative weights summing to one and preserves the ranking, but spreads weight differently. A reciprocal-rank rule, which assigns scores proportional to \(1/j\) before normalization, is another possibility. These are alternative conventions, not competing estimates that can be settled from the ranks alone.

A ranking may instead be about ordered outcome levels. For example, if four hypothetical clinical profiles are ranked best to worst, one possible score mapping is \((m-r)/(m-1)\), where \(r\) is the profile’s rank from 1 to \(m\). It assigns the best profile 1, the worst 0, and evenly spaced intermediate scores. That is a convenient ordinal scoring rule, but equal spacing is an added assumption: the ranking alone does not establish that the utility difference between ranks 1 and 2 equals the difference between ranks 3 and 4. Nor can the rule supply criterion weights unless the ranking was specifically about criteria.

What the Ranking Can and Cannot Determine

A complete strict ranking of criterion importance implies an ordering constraint on the weights: if criterion 1 is ranked above criterion 2, then a rank-consistent weighting requires \(w_1\geq w_2\). With nonnegative weights summing to one, these constraints define a set of possible vectors, not a single vector. If the ranking is weak, ties permit equal weights at those positions but still do not determine their common magnitude relative to other criteria.

The distinction is practical. Candidate weights can be useful for an initial analysis, transparent communication, or exploring how a ranking might affect comparisons. They should not be described as directly measured cardinal preferences. When a dose comparison depends on a particular difference between weights, the analyst should check whether that conclusion persists under other weight vectors consistent with the elicited order. The ranking may support a robust conclusion, but it cannot by itself establish that conclusion.

Assumptions, Diagnostics, and Practical Choices

Before using a rank-based conversion, check that the ranking concerns the quantities to which the weights will be attached. If stakeholders ranked clinical profiles but the model applies weights to separate criteria, the mapping needs additional elicitation or a clearly stated modeling bridge. Also check that the criterion scores have meaningful directions and compatible scales, and that an additive utility representation is appropriate for the intended use. A well-calculated set of weights cannot correct a mismatch between the elicited ranking and the model.

Common warning signs include ties silently broken into strict ranks, missing criteria treated as least important, and group disagreement hidden by reporting a single averaged ranking. For tied ranks, one transparent option is to average the conversion weights for the rank positions occupied by the tie and assign that average to each tied criterion. For incomplete rankings, do not fill missing positions without stating an assumption. For multiple stakeholders, retain individual rankings or report how an explicit aggregation rule handles disagreement; an aggregate ordering can conceal distinct preference patterns.

A practical comparison is between rank-based rules and direct elicitation of cardinal tradeoffs. Rank-based rules require relatively little information and can be useful when respondents can order concerns but cannot quantify differences. Their limitation is precisely that they impose a numerical pattern not identified by those answers. More detailed elicitation can ask about relative importance or changes in outcomes, but that requires additional judgments. The next methodological step should be motivated by the decision context, rather than treating any rank conversion as a uniquely correct answer.

The main diagnostic is sensitivity: compare the decision using more than one plausible conversion or using several weight vectors that respect the ranking. If the preferred option changes, report that the ranking alone does not resolve the comparison. If the choice remains the same across the examined vectors, that is evidence of robustness to those particular assumptions, not proof that all stakeholders would make the same choice.