Conjugate Updating from Likelihood Kernels

Why Update From Likelihood Kernels?

Bayesian updating combines prior information about an unknown parameter with information in observed data. The central calculation is often simpler than it first appears: write the prior density and likelihood as functions of the parameter, discard factors that do not depend on that parameter, and multiply the remaining kernels. Recognizing the matching terms reveals both the posterior distribution and the data summaries that matter.

This tutorial develops that calculation for exponential-family likelihoods. Exposure is treated explicitly because its location in the likelihood determines whether the sufficient statistic is weighted. A common error is to attach exposure weights to a statistic when the stated likelihood does not contain them.

Definition: A likelihood kernel is a likelihood viewed as a function of the unknown parameter, with any positive factor independent of that parameter omitted. Two kernels are proportional if they differ only by such a factor. Proportionality is enough for identifying a posterior density after it is normalized.

The Exponential-Family Kernel and Its Conjugate Prior

Suppose conditionally independent observations \(Y_1,\ldots,Y_n\) have a one-parameter exponential-family likelihood of the form

f(y_j\mid\theta,w_j) =h(y_j,w_j)\exp\{\theta T(y_j)-w_j A(\theta)\},

where \(\theta\) is the unknown parameter, \(w_j\) is a known nonnegative exposure, and \(T(y_j)\) is the statistic contributed by observation \(j\). Assume the support of the observations does not depend on \(\theta\), and that the model and prior are defined so the relevant densities are proper. The function \(h\) has no dependence on \(\theta\); \(A\) is the log-partition function in this likelihood representation.

For this specific form, \(T(y_j)\) is not multiplied by \(w_j\). Conditional independence therefore gives the likelihood kernel

L(\theta\mid y_1,\ldots,y_n) \propto \exp\left\{\theta\sum_{j=1}^n T(y_j) -A(\theta)\sum_{j=1}^n w_j\right\}.

A matching conjugate prior has density, with respect to \(d\theta\), proportional to \(\exp\{\eta\theta-\tau A(\theta)\}\). Its normalizing constant depends on the hyperparameters but not on \(\theta\).

\pi(\theta\mid\eta,\tau) =C(\eta,\tau)\exp\{\eta\theta-\tau A(\theta)\}, \qquad C(\eta,\tau)^{-1} =\int_{\Theta}\exp\{\eta\theta-\tau A(\theta)\}\,d\theta.

The integral must be finite for the prior to be proper. After observing the data, the posterior kernel has the same functional form, with updated hyperparameters. The update is obtained by adding the coefficients of \(\theta\) and \(A(\theta)\) in the two kernels.

Theorem: Under the likelihood form above and a proper conjugate prior, the posterior density is proportional to \(\exp\{\eta'\theta-\tau'A(\theta)\}\), where

\eta'=\eta+\sum_{j=1}^n T(y_j), \qquad \tau'=\tau+\sum_{j=1}^n w_j.

The posterior is proper when the integral defining \(C(\eta',\tau')^{-1}\) is finite. The data enter this update through the total of the statistics and the total exposure. In this representation, the total of the statistics is \(\sum_j T(y_j)\), not \(\sum_j w_jT(y_j)\).

Why the Update Works

The justification is direct multiplication. Conditional independence makes the joint likelihood the product of the individual likelihoods. Factors \(h(y_j,w_j)\) do not depend on \(\theta\), so they can be omitted from the likelihood kernel. The exponents then add:

\begin{aligned} \pi(\theta\mid\eta,\tau)L(\theta\mid y_1,\ldots,y_n) &\propto \exp\{\eta\theta-\tau A(\theta)\} \prod_{j=1}^n \exp\{\theta T(y_j)-w_j A(\theta)\}\\ &= \exp\left\{ \left(\eta+\sum_{j=1}^nT(y_j)\right)\theta -\left(\tau+\sum_{j=1}^nw_j\right)A(\theta) \right\}. \end{aligned}

This is the prior kernel’s functional form with updated coefficients. It also explains why the summaries are sufficient for this update: once \(\sum_j T(y_j)\) and \(\sum_j w_j\) are known, the data-dependent part of the posterior kernel is determined. This statement concerns the displayed likelihood representation; a different placement of exposure in the likelihood changes the statistic summary.

Worked Example: Updating From Abstract Likelihood Contributions

Suppose a proper prior has kernel \(\exp\{1.5\theta-4A(\theta)\}\). Three conditionally independent observations have likelihood contributions, up to factors not depending on \(\theta\),

\exp\{2\theta-3A(\theta)\},\qquad \exp\{-\theta-2A(\theta)\},\qquad \exp\{4\theta-A(\theta)\}.

Find the posterior kernel parameters and the two data summaries used in the update. Assume the posterior kernel is proper.

Method. Match the contributions to \(\exp\{\theta T(y_j)-w_jA(\theta)\}\). Their statistic contributions are \(2,-1,4\), and their exposures are \(3,2,1\).

Sum the statistics. The total is \(2+(-1)+4=5\).

Sum the exposures. The total is \(3+2+1=6\).

Update the coefficients. Starting from \(\eta=1.5\) and \(\tau=4\), obtain \(\eta'=1.5+5=6.5\) and \(\tau'=4+6=10\). Thus the posterior kernel is \(\exp\{6.5\theta-10A(\theta)\}\).

Check. Adding the three likelihood exponents gives \(5\theta-6A(\theta)\); adding the prior exponent gives \(6.5\theta-10A(\theta)\), as required. The common mistake avoided here is weighting the three statistic contributions by their exposures even though the stated likelihood already shows their coefficients.

A Concrete Example: Grouped Binary Outcomes

A grouped Bernoulli model illustrates the unweighted-statistic form when exposure is the number of trials. Let \(\theta\) be the log odds of success, so a single trial has success probability \(e^\theta/(1+e^\theta)\). If a group contains \(w_j\) trials and \(s_j\) successes, its likelihood kernel is

\exp\{s_j\theta-w_j\log(1+e^\theta)\}.

This is the stated exponential-family form with \(T(y_j)=s_j\) and \(A(\theta)=\log(1+e^\theta)\). Exposure is the number of trials, but the statistic contribution is the number of successes, not exposure times the number of successes.

Worked Example: Updating a Log-Odds Prior From Grouped Trials

A hypothetical risk study records three groups of binary trials. The groups have \((w_j,s_j)=(5,2),(3,1),(4,3)\), where \(w_j\) is the number of trials and \(s_j\) is the number of successes. A prior density for the log odds \(\theta\) is proportional to \(\exp\{2\theta-5\log(1+e^\theta)\}\). Find the posterior kernel and check that the prior and posterior are proper.

Method. For a grouped Bernoulli likelihood, add the success counts to obtain the statistic summary and add the trial counts to obtain total exposure.

Statistic summary. The total number of successes is \(2+1+3=6\).

Exposure summary. The total number of trials is \(5+3+4=12\).

Update. Here \(\eta=2\), \(\tau=5\), and \(A(\theta)=\log(1+e^\theta)\). Therefore \(\eta'=2+6=8\) and \(\tau'=5+12=17\), giving posterior kernel

\exp\{8\theta-17\log(1+e^\theta)\}.

Propriety check. As \(\theta\) tends to negative infinity, the prior kernel behaves like \(e^{2\theta}\), which is integrable in that tail. As \(\theta\) tends to positive infinity, it behaves like \(e^{(2-5)\theta}=e^{-3\theta}\), also integrable. For the posterior, the corresponding tail behaviors are \(e^{8\theta}\) as \(\theta\) tends to negative infinity and \(e^{(8-17)\theta}=e^{-9\theta}\) as \(\theta\) tends to positive infinity. Both are integrable, so the posterior is proper.

Check. The total trial count is \(12\), while the total success count is \(6\), and the posterior coefficients are \(2+6=8\) and \(5+12=17\). The common mistake avoided is multiplying each success count by its trial count; the likelihood exponent contains \(s_j\theta\), not \(w_js_j\theta\).

When the Statistic Is Exposure-Weighted

Not every model places exposure only in the log-partition term. A different likelihood convention is

f(y_j\mid\theta,w_j) =h(y_j,w_j)\exp\{w_j\theta T(y_j)-w_jA(\theta)\}.

For this model, the exponent contains \(w_jT(y_j)\theta\). If the prior kernel is still \(\exp\{\eta\theta-\tau A(\theta)\}\), multiplication gives \(\eta'=\eta+\sum_j w_jT(y_j)\) and \(\tau'=\tau+\sum_jw_j\). This is a different likelihood specification from the first one. The correct statistic summary must be read from the actual coefficient of \(\theta\) in the likelihood.

Worked Example: Updating With an Exposure-Weighted Statistic

Suppose the stated likelihood contributions are of the form \(\exp\{w_j\theta T(y_j)-w_jA(\theta)\}\). The two observations have \((w_1,T(y_1))=(2,3)\) and \((w_2,T(y_2))=(5,-1)\). A proper prior has kernel \(\exp\{\theta-4A(\theta)\}\). Find the posterior coefficients, assuming the resulting kernel is proper.

Method. Because the likelihood explicitly contains \(w_j\theta T(y_j)\), calculate the exposure-weighted statistic for each observation.

Weighted statistic total. The first contribution is \(2\cdot3=6\); the second is \(5\cdot(-1)=-5\). The total is \(6-5=1\).

Exposure total. The total exposure is \(2+5=7\).

Update. The prior has \(\eta=1\) and \(\tau=4\), so \(\eta'=1+1=2\) and \(\tau'=4+7=11\). The posterior kernel is \(\exp\{2\theta-11A(\theta)\}\).

Check. The likelihood’s coefficient of \(\theta\) is \(6-5=1\), and its coefficient of \(-A(\theta)\) is \(7\). Adding the prior terms gives \(2\theta-11A(\theta)\). The common mistake avoided is using the unweighted total \(3+(-1)=2\) despite the weights appearing in the likelihood.

Conditions, Interpretation, and Exam Traps

Conjugacy is a property of the pairing between a likelihood kernel and a prior family. It does not mean that every prior is conjugate, nor does it guarantee that an improper posterior becomes proper. Before using an update, check the model form, the independence assumption, and the finiteness of the prior and posterior normalizing integrals.

Likelihood contribution	Statistic summary added to \(\eta\)	Exposure summary added to \(\tau\)
\(\exp\{\theta T(y_j)-w_jA(\theta)\}\)	\(\sum_j T(y_j)\)	\(\sum_j w_j\)
\(\exp\{w_j\theta T(y_j)-w_jA(\theta)\}\)	\(\sum_j w_jT(y_j)\)	\(\sum_j w_j\)

The table is a reminder to derive the summaries from the displayed likelihood, not from a general intuition that “more exposure means a larger statistic.” Some models use exposure to scale both terms; others do not. If an observation has zero exposure, it contributes no \(A(\theta)\) term. Whether it contributes a statistic term depends on the exact likelihood form and the data-generating model; a valid zero-exposure observation in an exposure-scaled model ordinarily contributes neither.

Do not weight by habit. In the first likelihood form, use \(\sum_jT(y_j)\). Use \(\sum_jw_jT(y_j)\) only when the likelihood’s coefficient of \(\theta\) includes \(w_j\).
Keep exposure and count distinct. In grouped binary data, the number of trials determines the log-partition contribution; the number of successes determines the statistic contribution.
Do not discard parameter-dependent factors. A factor can be omitted from a kernel only if it does not depend on \(\theta\).
Check normalizability. Matching the kernel form identifies a candidate posterior; the updated hyperparameters must still give a finite normalizing integral.

These calculations are aligned with the SOA Exam ASTAM (Advanced Short-Term Actuarial Mathematics) syllabus. In an exam problem, write the likelihood kernel first, mark the coefficient of each parameter-dependent term, and only then update the prior hyperparameters. That short discipline makes exposure conventions visible and prevents a plausible-looking but incorrect weighted sum.

Check Your Understanding

For each problem, identify the likelihood form before calculating the update. Assume any posterior described as proper has a finite normalizing constant.

A prior kernel is \(\exp\{3\theta-2A(\theta)\}\). Two independent observations have kernels \(\exp\{4\theta-5A(\theta)\}\) and \(\exp\{-2\theta-A(\theta)\}\). Find the posterior coefficients. (Answer: \(\eta'=5,\ \tau'=8\).)
Under the likelihood form \(\exp\{\theta T(y_j)-w_jA(\theta)\}\), two observations have \((T,w)=(2,4)\) and \((5,1)\). A prior has \((\eta,\tau)=(1,3)\). Find the two data summaries and posterior coefficients. (Answer: statistic total \(7\), exposure total \(5\); \(\eta'=8,\ \tau'=8\).)
For a grouped binary model with success counts \(3,2\) and trial counts \(6,5\), a log-odds prior kernel has \((\eta,\tau)=(1,4)\). Find the posterior kernel coefficients. (Answer: \(\eta'=6,\ \tau'=15\), so the kernel is proportional to \(\exp\{6\theta-15\log(1+e^\theta)\}\).)
A likelihood is explicitly given by \(\exp\{w_j\theta T(y_j)-w_jA(\theta)\}\). The observations have \((w,T)=(3,2)\) and \((4,-1)\); the prior has \((\eta,\tau)=(2,6)\). Find the posterior coefficients, and state why the statistic total is not \(2+(-1)\). (Answer: weighted statistic total \(3(2)+4(-1)=2\), exposure total \(7\); \(\eta'=4,\ \tau'=13\), because exposure multiplies \(T(y_j)\) in the stated likelihood.)
For the unweighted-statistic likelihood form, a grouped binary study has trial counts \(4,3,5\) and success counts \(1,2,4\). The prior has \((\eta,\tau)=(2,6)\). A candidate solution uses \(4(1)+3(2)+5(4)=30\) as the statistic total. Identify the error and give the correct posterior coefficients. (Answer: the likelihood coefficient of \(\theta\) is the unweighted success total \(1+2+4=7\), not \(30\); \(\eta'=9,\ \tau'=18\).)

What You'll Learn