Weighting Indications by Reliability

Why Weight Indications by Reliability?

When several analyses produce different rate indications, taking their simple average gives each result equal influence, whether it is based on stable data or on a small, volatile sample. Reliability weighting provides a transparent way to give more influence to indications considered more dependable and less influence to those considered less dependable.

The calculation is straightforward; defining what the weights mean is the essential judgment. A weight is a relative influence assigned to an indication. It is not automatically a percentage of premium, a measure of exposure, or a probability that the indication is correct. The analyst should identify what each indication measures, confirm they are comparable, and state how the weights were chosen.

Definition: Given comparable indications \(I_1,\ldots,I_n\) and nonnegative weights \(w_1,\ldots,w_n\), the reliability-weighted indication is

I_{\text{sel}}=\frac{\sum_{j=1}^{n}w_j I_j}{\sum_{j=1}^{n}w_j}, \qquad \sum_{j=1}^{n}w_j>0.

The normalized weight of indication \(j\) is \(\alpha_j=w_j/\sum_{k=1}^{n}w_k\). Thus \(I_{\text{sel}}=\sum_{j=1}^{n}\alpha_j I_j\), where the normalized weights sum to 1.

The indications must be on a common basis before they are combined. For example, a percentage rate change should not be averaged directly with a dollar loss cost or a rate level factor. If the input is a percentage change, the selected result is also a percentage change. If the input is a rate level factor, the result is a rate level factor.

Interpreting and Normalizing Weights

Only relative weights matter in the weighted-average formula. Multiplying every weight by the same positive constant leaves the result unchanged. Weights of 6, 3, and 1 therefore give the same selection as weights of 0.6, 0.3, and 0.1. The normalized weights in the first case are 0.60, 0.30, and 0.10.

A larger weight gives an indication more influence on the selection. If the indications are all expressed in the same units and normalized weights are nonnegative, the selected indication lies between the smallest and largest input indications. This is a useful arithmetic check. A result outside that range means that the weights or calculation are not being used as a weighted average.

Weights may be set directly to reflect a documented assessment of data quality and stability. For instance, an analysis based on a longer, more relevant experience period might receive more weight than one based on sparse or less comparable data. Such weights are not uniquely determined by the formula. They should be explained rather than presented as an objective statistical fact when they are based on professional assessment.

Worked Example: Combining Three Rate Changes

In a hypothetical commercial equipment program, three analyses indicate rate changes of 7%, 11%, and 5%. The analyst assigns reliability weights of 5, 3, and 2, respectively. Calculate the selected rate change and check the result.

Method. Use a reliability-weighted average of the three percentage indications. The weights total \(5+3+2=10\), so their normalized values are \(5/10=0.5\), \(3/10=0.3\), and \(2/10=0.2\).

Calculation. The weighted contributions are \(0.5(7\%)=3.5\%\), \(0.3(11\%)=3.3\%\), and \(0.2(5\%)=1.0\%\). Therefore,

I_{\text{sel}}=3.5\%+3.3\%+1.0\%=7.8\%.

The same result follows from the unnormalized weights: \((5(7\%)+3(11\%)+2(5\%))/10=(35+33+10)\%/10=7.8\%\). The input range is 5% to 11%, and 7.8% is within it, as required.

Common mistake avoided. The simple average, \((7\%+11\%+5\%)/3=7.67\%\) to two decimals, gives each indication equal influence and ignores the stated reliability weights. It is not the requested selection.

Why the Weighted Average Is a Natural Blend

A weighted average can be understood as a value that stays close to the indications, with larger weights penalizing distance more heavily. Consider the function \(Q(I)=\sum_{j=1}^{n}w_j(I-I_j)^2\), where each \(w_j\) is nonnegative and at least one is positive. The value of \(I\) minimizing \(Q(I)\) is the weighted average.

To establish this, differentiate with respect to \(I\) and set the derivative equal to zero:

\frac{dQ}{dI} =2\sum_{j=1}^{n}w_j(I-I_j) =2\left(I\sum_{j=1}^{n}w_j-\sum_{j=1}^{n}w_jI_j\right)=0.

Because \(\sum_j w_j>0\), solving gives \(I=\sum_j w_jI_j/\sum_j w_j\). Also, the second derivative is \(2\sum_j w_j>0\), so this stationary value is the unique minimum. This justification explains the role of the weights: discrepancies from a heavily weighted indication count more in the squared-distance criterion.

This is a mathematical interpretation of the calculation, not proof that a particular set of weights is appropriate. The formula will faithfully combine the assigned inputs, but it cannot make poorly chosen weights reliable.

Inverse-Variance Weights: A Statistical Special Case

When indications are treated as estimates of one common underlying quantity, inverse-variance weighting provides a specific statistical rule for choosing weights. Suppose each indication \(I_j\) is unbiased for the same value, has known variance \(\sigma_j^2>0\), and the estimates are independent. Among linear combinations whose weights sum to 1, the minimum-variance combination assigns weight proportional to \(1/\sigma_j^2\).

The intuition is that a less variable estimate carries more information and should receive more influence. This method is not automatically appropriate simply because a variance can be calculated. The estimates must target the same quantity, and the independence assumption matters. If indications share data or assumptions, their errors may be correlated, and inverse-variance weights calculated as if they were independent may overstate how much information the combined result contains.

Worked Example: Applying Inverse-Variance Weights

Two hypothetical analyses estimate the same rate change: 4% and 10%. Their estimated error variances are 4 and 9 squared percentage points, respectively. Assume both estimates are unbiased and independent. Find the inverse-variance weighted indication and its standard error.

Method. Inverse-variance weights are proportional to \(1/\sigma_j^2\). The raw weights are \(1/4\) and \(1/9\). Multiplying both by 36 preserves their relative sizes and gives convenient weights 9 and 4. Their sum is 13, so the normalized weights are \(9/13\) and \(4/13\).

Indication. The weighted contributions are \((9/13)(4\%)=36/13\%\) and \((4/13)(10\%)=40/13\%\). Thus,

I_{\text{sel}}=\frac{36+40}{13}\% =\frac{76}{13}\% =5.8462\%.

Standard error. For independent estimates with normalized weights \(\alpha_j\), the variance of their weighted sum is \(\sum_j \alpha_j^2\sigma_j^2\). Here it is \((9/13)^2(4)+(4/13)^2(9)=324/169+144/169=468/169=36/13\) squared percentage points. The standard error is \(\sqrt{36/13}=1.6641\) percentage points.

Check and condition. The indication, 5.8462%, is between 4% and 10% and closer to 4%, which has the smaller variance. The standard error calculation relies on the stated independence and unbiasedness assumptions; without them, this uncertainty result does not follow from these variances alone.

How Weight Changes Affect the Selection

A weight change shifts the selection toward or away from the indication whose weight changed. This effect can be seen directly by comparing the old and new normalized weights. The indication values themselves do not change; only their relative influence does.

Worked Example: Reducing the Influence of a Volatile Indication

For a hypothetical specialty liability program, three comparable analyses indicate rate changes of 6%, 9%, and 3%. Based on their relative stability, the assigned weights are 6, 3, and 1. Calculate the selected change, then double the third weight and calculate the revised selection. Explain the direction of the shift.

Initial selection. The weights total \(6+3+1=10\). The weighted contributions are \(6(6\%)=36\), \(3(9\%)=27\), and \(1(3\%)=3\), in weight-percentage units. Therefore,

I_{\text{initial}}=\frac{36+27+3}{10}\%=6.6\%.

Revised selection. Doubling the third weight changes the weights to 6, 3, and 2, which total 11. The weighted sum is \(36+27+2(3)=69\), so

I_{\text{revised}}=\frac{69}{11}\%=6.2727\%.

The selection falls by \(6.6\%-6.2727\%=0.3273\) percentage points. This direction is sensible because the increased weight belongs to the 3% indication, which is below the initial selection. As a check on the size of the effect, equal weights would produce \((6\%+9\%+3\%)/3=6.0\%\); the revised result lies between 3% and 9%.

Common mistake avoided. Doubling a weight does not double that indication’s percentage value. It changes its relative share of the total weight from \(1/10\) to \(2/11\); the selected indication must then be recalculated using the new total weight.

Practical Checks and Common Traps

A weighted selection is useful only when its inputs and weights are interpretable. Before reporting the result, confirm that all indications refer to the same coverage, period, and quantity, and that weights correspond to the intended reliability comparison. If an indication has already been adjusted or combined, check that its construction does not unintentionally duplicate the influence of another source.

Do not mix unlike measures. Convert inputs to a common basis before averaging; a percentage change and a dollar amount cannot be combined as though they were interchangeable indications.
Do not omit the denominator. If weights are 5, 3, and 2, the weighted sum must be divided by 10. Otherwise the result is a weighted sum, not a weighted average.
Do not mistake weights for probabilities. A weight of 0.6 means 60% influence after normalization; it does not mean there is a 60% chance that the indication is correct.
Do not infer reliability from the size of an indication. A larger or smaller rate change is not, by itself, evidence that its estimate is more or less reliable.
Do not apply inverse-variance weights mechanically. Their standard statistical interpretation requires a common target, unbiased estimates, and appropriate treatment of dependence between estimates.

For transparency, report the indications, their weights, the weight total or normalized weights, and the resulting selection. A reader should be able to reproduce the calculation and see how the weights affect the result. The weighted average is a clear aggregation method, not a substitute for checking whether its inputs and weights are defensible.

Key takeaway: Reliability weighting combines comparable indications in proportion to their assigned influence. Normalize or divide by the total weight, explain the basis for the weights, and verify that the selected result lies within the range of the indications when weights are nonnegative.

Check Your Understanding

For each problem, calculate the requested selection before checking the answer.

A hypothetical property portfolio has rate-change indications of 5%, 8%, and 11%, with reliability weights 2, 3, and 1. Calculate the weighted indication. (Answer: 7.5%)
Two analyses of the same hypothetical coverage indicate 6% and 14%, with assigned weights 4 and 1. Calculate the weighted selection and state which indication has greater influence. (Answer: 7.6%; the 6% indication has greater influence.)
Three hypothetical methods give indications of 2.5%, 7.5%, and 10%, with weights 5, 3, and 2. Calculate the normalized weights and the selected rate change. (Answer: normalized weights 0.5, 0.3, and 0.2; selected change 5.5%.)
Two independent, unbiased hypothetical indications are 3% and 9%, with variances 1 and 4 squared percentage points. Use inverse-variance weights to calculate the selected indication and its standard error. (Answer: 4.2%; standard error 0.8944 percentage points.)
Three indications are 4%, 8%, and 12%, initially weighted 1, 2, and 1. Calculate the initial selection, then increase the first indication’s weight from 1 to 3 and recalculate. By how many percentage points does the selection change? (Answer: initial selection 8%; revised selection 6.6667%; it decreases by 1.3333 percentage points.)

What You'll Learn