Diagnosing Temperature and Normalization Errors

Why Small Notational Choices Matter

Exponential weights can look deceptively simple: multiply each candidate’s prior weight by an exponential of its loss, then normalize. Yet four choices determine what that calculation means: whether loss is summed or averaged, whether the exponent uses a temperature or its reciprocal, whether prior weights are included, and whether the loss satisfies the assumptions behind the claimed guarantee. A mistake in any one of them can change the weights substantially or make a valid bound inapplicable.

This tutorial uses a finite dictionary of \(M\) candidates. Candidate \(j\) incurs loss \(\ell_{t,j}\) at round \(t\), and its cumulative loss through round \(n\) is \(L_{n,j}=\sum_{t=1}^n \ell_{t,j}\). Its average empirical loss is \(\hat R_n(j)=L_{n,j}/n\). The prior weights \(\pi_1,\ldots,\pi_M\) satisfy \(\pi_j\geq 0\) and \(\sum_j\pi_j=1\).

Sum Versus Average: The Factor of \(n\)

The two loss conventions are equivalent only when their inverse temperatures are adjusted. Since \(L_{n,j}=n\hat R_n(j)\), the summed-loss exponent \(-\eta_{\mathrm{sum}}L_{n,j}\) equals the averaged-loss exponent \(-\eta_{\mathrm{avg}}\hat R_n(j)\) exactly when \(\eta_{\mathrm{avg}}=n\eta_{\mathrm{sum}}\). Keeping the same numerical value of \(\eta\) while switching conventions changes the distribution over candidates.

A related convention error is reversing temperature and inverse temperature. Increasing \(\eta\) concentrates weight more strongly on low-loss candidates; increasing \(T=1/\eta\) makes weights more diffuse. If a formula is written as \(\exp(-L/T)\), its coefficient is \(1/T\), not \(T\). Checking the exponent directly is safer than relying on the word “temperature.”

The Prior Belongs in the Numerator

The prior is not an optional decoration. It determines the initial mass assigned to each candidate, and that mass affects the normalized weights and the complexity term in standard oracle bounds. A candidate with \(\pi_j=0\) receives zero Gibbs weight for every finite loss and cannot be recovered by normalization. In a finite-dictionary bound involving \(\log(1/\pi_j)\), a zero prior also makes the penalty infinite.

For positive prior weights, define the normalizing constant \(Z_n=\sum_j\pi_j e^{-\eta L_{n,j}}\). The identity below makes the prior’s role precise: the normalized exponential weights minimize empirical loss plus a divergence penalty relative to that prior.

Proof. From the definition of \(q_{n,j}\), for each \(j\), \(\log(q_{n,j}/\pi_j)=-\eta L_{n,j}-\log Z_n\). For any probability vector \(\rho\), \(\operatorname{KL}(\rho\|q_n)=\operatorname{KL}(\rho\|\pi)+\eta\sum_j\rho_jL_{n,j}+\log Z_n\). Rearranging gives \(\sum_j\rho_jL_{n,j}+\eta^{-1}\operatorname{KL}(\rho\|\pi)=-\eta^{-1}\log Z_n+\eta^{-1}\operatorname{KL}(\rho\|q_n)\). The final term is nonnegative and equals zero when \(\rho=q_n\). This proves both the minimum value and its minimizer. \(\square\)

What a Standard Exponential-Weights Bound Requires

The normalization calculation alone is always valid for finite losses and \(\eta>0\). A prediction guarantee needs more. One common deterministic guarantee uses exp-concavity: for each outcome \(y\), the map \(a\mapsto \exp[-\eta\ell(y,a)]\) must be concave on the convex prediction set. The aggregated prediction must be the corresponding convex combination of candidate predictions, and the same loss function and value of \(\eta\) must be used in the concavity condition and in the weight update. This is a substantive assumption, not a consequence of merely having exponential weights.

Proof. Concavity and the definition of \(a_t\) give \(e^{-\eta\ell(y_t,a_t)}\geq\sum_jq_{t,j}e^{-\eta\ell(y_t,a_{t,j})}\). Taking negative logarithms and dividing by \(\eta>0\) yields \(\ell(y_t,a_t)\leq-\eta^{-1}\log\sum_jq_{t,j}e^{-\eta\ell(y_t,a_{t,j})}\). Let \(W_1=1\) and \(W_{t+1}=\sum_j\pi_j\exp[-\eta\sum_{s=1}^t\ell(y_s,a_{s,j})]\). The update gives \(q_{t,j}=\pi_j\exp[-\eta\sum_{s<t}\ell(y_s,a_{s,j})]/W_t\), so the right side of the preceding loss inequality equals \(-\eta^{-1}\log(W_{t+1}/W_t)\). Summing over \(t\) telescopes to aggregate loss at most \(-\eta^{-1}\log W_{n+1}\). Since \(W_{n+1}\geq\pi_j\exp[-\eta\sum_{t=1}^n\ell(y_t,a_{t,j})]\), taking negative logarithms and dividing by \(\eta\) proves the result. \(\square\)

Loss-Range Assumptions and Scaling

Some guarantees instead use a stated bounded-loss condition, such as \(0\leq\ell\leq1\), or a range of width \(B\). These conditions must be checked for the actual loss entering the formula. A result proved for unit-range losses cannot simply be applied to losses that may be 100 times larger with the same numerical inverse temperature. The exponential weights then respond to a 100-fold larger exponent.

A constant shift of all candidate losses at a given observation does not alter normalized weights: multiplying every unnormalized weight by the same positive factor cancels in the denominator. Rescaling is different. Replacing \(\ell\) by \(c\ell\), \(c>0\), preserves weights only if \(\eta\) is replaced by \(\eta/c\), since \((\eta/c)(c\ell)=\eta\ell\). Any theorem’s assumptions and constants must also be reconsidered after rescaling; preserving a numerical distribution does not automatically preserve the theorem’s stated hypotheses.

A Practical Diagnostic Checklist

When an exponential-weights expression or guarantee appears inconsistent, check the calculation in a fixed order. First identify the loss being exponentiated: a single-round loss, cumulative loss, or average empirical loss. Next identify the coefficient multiplying that loss, rather than trusting whether the author calls it temperature or inverse temperature. Then confirm that every prior mass is present and that the normalizer sums over the full candidate set. Finally, inspect the exact assumption used by the guarantee: exp-concavity on the prediction domain, a bounded range, or another stated condition.

These checks matter beyond arithmetic. The prior determines how much complexity is charged to a candidate, the loss convention determines the effective strength of the update, and range conditions determine whether a theorem applies at all. A weight vector can be normalized perfectly and still be based on the wrong exponent or an unjustified guarantee.

Common Mistakes

• Using one inverse temperature for both conventions. Since \(L_n=n\hat R_n\), equivalent summed- and average-loss formulas require \(\eta_{\mathrm{avg}}=n\eta_{\mathrm{sum}}\).
• Confusing temperature with inverse temperature. In \(\exp(-L/T)\), the coefficient of loss is \(1/T\). A smaller \(T\) produces stronger concentration.
• Dropping the prior. The update is proportional to \(\pi_j e^{-\eta L_j}\), not just \(e^{-\eta L_j}\), unless the prior is explicitly uniform.
• Assuming normalization proves a regret guarantee. Normalization is algebra; a prediction guarantee also needs its stated loss and aggregation assumptions.
• Reusing a unit-range result after rescaling. Changing the loss scale changes both the exponent and the range condition. Adjusting \(\eta\) may preserve weights, but does not by itself verify the theorem’s assumptions.