Temperature, Loss Scale, and Weight Concentration

Why Temperature and Loss Scale Matter

Exponential weights turn empirical losses into a probability distribution over candidate predictors. A candidate with lower empirical risk receives more weight, but the size of the difference matters: a small risk gap can have little effect or a large effect depending on the temperature. The numerical scale used to record losses matters for the same reason. Risks measured in different units cannot be compared through the exponential formula without accounting for that scale.

The main point is that temperature controls sensitivity to loss differences, not the ordering of candidates. We will make that distinction precise, calculate its consequences, and show how relative entropy measures the weights’ concentration relative to a prior distribution.

Definition and Basic Identities

Consider a finite set of \(M\) candidates, indexed by \(j=1,\ldots,M\). Let \(\hat R_j\) be candidate \(j\)'s empirical risk, and let \(\pi_j>0\) be its prior weight, with \(\sum_{j=1}^M \pi_j=1\). For inverse temperature \(\eta\geq 0\), define the normalized exponential weights by

The denominator normalizes the weights so that they sum to one. When \(\eta>0\), the temperature is \(\tau=1/\eta\). Thus a low temperature means a large inverse temperature and greater sensitivity to risk gaps. At \(\eta=0\), the weights equal the prior: \(\hat w_j(0)=\pi_j\).

For any two candidates \(j\) and \(k\), division cancels the common normalizing constant:

This identity separates two influences. The prior ratio sets the starting odds, while the empirical-risk difference changes those odds exponentially. If \(j\) has smaller empirical risk than \(k\), increasing \(\eta\) increases the odds in favor of \(j\).

Loss Rescaling and Temperature

Suppose every empirical risk is multiplied by the same constant \(c>0\), as when changing from one loss unit to another. With inverse temperature \(\eta'\), the weights based on the rescaled risks are proportional to \(\pi_j\exp(-\eta'c\hat R_j)\). They are exactly the original weights at inverse temperature \(\eta\) when \(\eta'c=\eta\).

Adding a common constant to every empirical risk also leaves the weights unchanged, because it multiplies every unnormalized weight by the same factor. These two facts give a useful general expression. For any \(a>0\) and any real \(b\), the weights formed from risks \(a\hat R_j+b\) at inverse temperature \(\eta'\) equal the original weights at \(\eta\) whenever \(\eta'a=\eta\). The additive shift \(b\) has no effect on the normalized weights.

How Concentration Changes

A direct measure of separation from the prior is the relative entropy

It is zero when the weights equal the prior and increases as the weights move away from it. The next result shows exactly how this measure changes with inverse temperature. All risks here are finite real numbers, and all prior weights are positive.

Proof. Write \(Z(\eta)=\sum_j\pi_j\exp(-\eta\hat R_j)\). Since this is a finite sum of differentiable functions and \(Z(\eta)>0\), its logarithm \(A(\eta)=\log Z(\eta)\) is differentiable. Differentiating gives

Differentiating the weights directly yields

Therefore

From the definition of the weights, \(\log(\hat w_j/\pi_j)=-\eta\hat R_j-\log Z(\eta)\). Summing this identity against \(\hat w_j\) gives \(D=-\eta\bar R-A\). Differentiate and use \(A'=-\bar R\):

The variance is nonnegative, which proves the claimed monotonicity. If the risks are not all equal and \(\eta>0\), the variance is positive because every weight is positive; in that case the relative entropy is strictly increasing. \(\square\)

This theorem quantifies concentration relative to the chosen prior, including when the prior is nonuniform. For a uniform prior, \(D(\hat w\Vert\pi)=\log M-H(\hat w)\), where \(H(\hat w)=-\sum_j\hat w_j\log\hat w_j\) is Shannon entropy. Consequently, with a uniform prior, Shannon entropy is nonincreasing as \(\eta\) increases. With a nonuniform prior, it is relative entropy from the prior—not necessarily Shannon entropy—that has the general monotonicity guarantee.

The Low-Temperature Limit

The pairwise-ratio identity also describes the limiting behavior as \(\eta\) grows. Let \(\hat R_{\min}=\min_j\hat R_j\), and let \(S=\{j:\hat R_j=\hat R_{\min}\}\) be the set of empirical minimizers. Factoring \(\exp(-\eta\hat R_{\min})\) out of the normalizing constant gives

For \(j\in S\), the exponential factor in this expression equals one. For \(j\notin S\), the gap \(\hat R_j-\hat R_{\min}\) is positive, so that factor tends to zero as \(\eta\) tends to infinity. Since there are only finitely many candidates, the denominator tends to \(\sum_{k\in S}\pi_k>0\). It follows that

Thus low temperature concentrates on the empirical minimizers. If there is a unique minimizer, its weight tends to one. If several candidates tie, the limiting weights among them are proportional to their prior masses, rather than necessarily equal.

Common Mistake and Practical Use

A common mistake is to say, “Multiplying all losses by a constant does not change which candidate is best, so it does not change the weights.” The first clause is correct; the conclusion is not. The normalized weights depend on the products \(\eta\hat R_j\), not only on the ranking. Scaling losses by \(c\) while keeping \(\eta\) fixed changes the effective inverse temperature from \(\eta\) to \(c\eta\). The weights are preserved only if the inverse temperature is also divided by \(c\).

This distinction matters when choosing a temperature or comparing implementations that use different conventions for loss scale. A temperature that produces moderate concentration for a loss recorded between zero and one may produce near-hard selection if the same loss is recorded on a scale one hundred times larger. Before interpreting weights, check the loss units, the inverse temperature, and the prior. Pairwise odds provide a quick diagnostic: for a risk gap \(\Delta>0\), the odds in favor of the lower-risk candidate are multiplied by \(\exp(\eta\Delta)\), relative to the prior odds.