Likelihood Contributions From One Observation

From an Observed Response to a Likelihood

A probability model describes what responses could occur and how their distribution depends on an unknown parameter. Once one response has been observed, a different question becomes useful: which parameter values make this particular observation more or less compatible with the model? The likelihood contribution answers that question by evaluating the model’s mass function or density at the observed response.

The key change is the role of the response. Before observation, $X$ is a random variable. After observing a value $y$, we hold $y$ fixed and regard the mass or density evaluated at $y$ as a function of the parameter. The likelihood contribution is not a new probability model for $y$; it is a way to compare parameter values in light of the observation.

Definition: Let $X$ have a specified discrete probability mass function or continuous probability density $f(x;\theta)$, indexed by a parameter $\theta$ in parameter space $\Theta$. After observing $X=y$, the likelihood contribution from this one observation is the function $$ L_y(\theta)=f(y;\theta), \qquad \theta\in\Theta, $$ where $y$ is fixed and $\theta$ varies. When $L_y(\theta)>0$, the corresponding log-likelihood contribution is $\ell_y(\theta)=\log L_y(\theta)$.

For a discrete response, $f(y;\theta)$ is the probability mass assigned to the observed value. For a continuous response, it is the density evaluated at that value. In either case, the response value is substituted into the model; the result is then viewed as a function of $\theta$. The notation $L_y$ emphasizes that changing the observation would generally give a different function.

The model’s parameter space matters. A parameter value may make an observed response impossible under the model. In that case, the model assigns the response mass zero in the discrete case, or the specified density value zero in the continuous case, and the likelihood contribution is zero. A likelihood value therefore cannot be interpreted without reference to the model and the observed response.

Discrete Responses: Evaluate the Mass

Suppose the response is discrete, with probability mass function $f(x;\theta)$. If the observed response is $y$, its single-observation likelihood contribution is obtained directly by substitution: $L_y(\theta)=f(y;\theta)$. The observation is not being resampled as $\theta$ changes; only the mass assigned to the already observed value changes.

Worked Example: A Bernoulli Observation

Let $X\sim\operatorname{Bernoulli}(p)$, where $p\in[0,1]$, so that $P(X=1)=p$ and $P(X=0)=1-p$. Suppose the observed response is $y=1$.

The mass at the observed value is $f(1;p)=p$. Thus the likelihood contribution is $$ L_1(p)=p,\qquad 0\leq p\leq 1. $$

For example, $L_1(0.7)=0.7$, while $L_1(0.2)=0.2$. Under this model, the observed success receives greater mass at $p=0.7$ than at $p=0.2$. At $p=0$, the contribution is zero because that parameter setting assigns no mass to observing $1$; at $p=1$, it is one.

If instead the observed response were $y=0$, substitution would give $L_0(p)=f(0;p)=1-p$. The two observations therefore produce different likelihood functions, even though they come from the same Bernoulli model.

Worked Example: A Poisson Count

Let $X\sim\operatorname{Poisson}(\lambda)$, with parameter space $\lambda>0$ and mass function $f(x;\lambda)=e^{-\lambda}\lambda^x/x!$ for nonnegative integer $x$. Suppose the observed count is $y=2$.

Substitute $x=2$ into the mass function: $$ L_2(\lambda)=f(2;\lambda) =e^{-\lambda}\frac{\lambda^2}{2!} =\frac{\lambda^2e^{-\lambda}}{2},\qquad \lambda>0. $$

At $\lambda=1$, this is $L_2(1)=e^{-1}/2\approx0.184$. At $\lambda=2$, it is $L_2(2)=2e^{-2}\approx0.271$. These calculations compare the mass assigned to the fixed observed count $2$ at two candidate parameter values.

The log-likelihood contribution is $$ \ell_2(\lambda)=\log\left(\frac{\lambda^2e^{-\lambda}}{2}\right) =2\log\lambda-\lambda-\log 2. $$ Its derivative is $2/\lambda-1$, which is positive for $0<\lambda<2$ and negative for $\lambda>2$. Therefore the likelihood increases up to $\lambda=2$ and decreases after it; its maximum over $\lambda>0$ occurs at $\lambda=2$. This maximization uses only the contribution from the single observed count.

Continuous Responses: Evaluate the Density

For a continuous response, the same construction uses the density specified by the model. If the observation is $y$, fix that value and evaluate $f(y;\theta)$ as $\theta$ varies. The calculation is a substitution into the density formula, not a change to the underlying random variable.

Worked Example: One Normal Measurement

Suppose $X\sim N(\mu,0.25)$, so the variance is known to be $0.25$ and $\mu\in\mathbb R$ is unknown. The density is $$ f(x;\mu)=\frac{1}{\sqrt{2\pi(0.25)}}\exp\left(-\frac{(x-\mu)^2}{2(0.25)}\right). $$ Suppose the observed measurement is $y=1.5$.

Holding $1.5$ fixed and substituting it for $x$ gives $$ L_{1.5}(\mu) =\frac{1}{\sqrt{0.5\pi}}\exp\left(-\frac{(1.5-\mu)^2}{0.5}\right),\qquad \mu\in\mathbb R. $$

At $\mu=1.5$, the exponential factor is $1$, so $L_{1.5}(1.5)=1/\sqrt{0.5\pi}\approx0.798$. At $\mu=1$, the squared difference is $0.25$, and the exponent is $-0.25/0.5=-0.5$. Thus $$ L_{1.5}(1)=\frac{e^{-0.5}}{\sqrt{0.5\pi}}\approx0.484. $$

The observed value is the same in both evaluations. The parameter changes, which changes the density assigned at that value. In this model, the contribution is largest at $\mu=1.5$, as the following argument verifies.

Result: In a normal model with known variance $\sigma^2>0$, unrestricted mean parameter $\mu\in\mathbb R$, and one observed value $y$, the likelihood contribution is maximized at $\mu=y$.

Proof. The contribution is $$ L_y(\mu)=\frac{1}{\sqrt{2\pi\sigma^2}} \exp\left(-\frac{(y-\mu)^2}{2\sigma^2}\right). $$ For every real $\mu$, $(y-\mu)^2\geq0$, so $-(y-\mu)^2/(2\sigma^2)\leq0$. The exponential is therefore at most $1$, with equality exactly when $(y-\mu)^2=0$, or $\mu=y$. The constant factor is positive and does not depend on $\mu$. Hence the unique maximum occurs at $\mu=y$. $\square$

Support and Zero Contributions

Every probability model specifies which response values are possible at each parameter setting. If a fixed observation is outside the model’s allowable response set at some setting, its likelihood contribution must be zero there. This is important for models whose allowable values depend on the parameter: the likelihood may change not only in height but also in where it is nonzero.

Result: Suppose a model specifies an allowable response set $\mathcal Y_\theta$ for each $\theta$, and assigns mass or density value $f(y;\theta)=0$ whenever $y\notin\mathcal Y_\theta$. For a fixed observation $y$, any parameter value satisfying $y\notin\mathcal Y_\theta$ has $L_y(\theta)=0$.

Proof. By definition, $L_y(\theta)=f(y;\theta)$. If $y\notin\mathcal Y_\theta$, the stated model specification gives $f(y;\theta)=0$. Substitution into the likelihood definition yields $L_y(\theta)=0$. $\square$

For instance, if a discrete model allows only nonnegative integer responses, observing a negative value gives zero contribution at every parameter setting in that model. In a model with a parameter-dependent allowable set, the same observed value might be possible for some settings and impossible for others. The observation should be checked against the model’s support before interpreting or maximizing its likelihood.

Log-Likelihoods and a Common Mistake

Likelihood contributions are nonnegative. When a contribution is positive, taking its logarithm often turns a complicated expression into a simpler one. For example, products of factors within a single mass or density formula become sums after taking logs. The log-likelihood is still a function of the parameter for the fixed observation; it is not a probability distribution over parameter values.

Result: For a likelihood contribution $L_y(\theta)\geq0$, define $\ell_y(\theta)=\log L_y(\theta)$ when $L_y(\theta)>0$, and set $\ell_y(\theta)=-\infty$ when $L_y(\theta)=0$. Then the parameter values maximizing $L_y$ are exactly those maximizing $\ell_y$.

Proof. The logarithm is strictly increasing on the positive real numbers. Thus, for any two parameter values $\theta_1,\theta_2$ with positive contributions, $L_y(\theta_1)\geq L_y(\theta_2)$ if and only if $\log L_y(\theta_1)\geq\log L_y(\theta_2)$. A zero contribution is below every positive contribution, and its assigned log value $-\infty$ is likewise below every finite log-likelihood value. If every contribution is zero, every parameter value maximizes both functions under the stated extended-value convention. The maximizing sets therefore agree. $\square$

Common mistake: “The likelihood at $p=0.7$ is $0.7$, so there is a 70% probability that $p=0.7$ is the true parameter.” This reverses the role of the parameter and the observation. In the Bernoulli example, $0.7$ is the mass assigned to the fixed observed success under the candidate setting $p=0.7$; it is not a probability assigned to that parameter value. The correction is to describe likelihoods comparatively: the observed outcome receives a larger or smaller model contribution at different candidate settings.

Key takeaway: For one observation, construct the likelihood contribution by fixing the realized response $y$ and evaluating the model’s mass or density $f(y;\theta)$. The resulting function varies with $\theta$, and a setting that makes the observation impossible receives contribution zero.

Check Your Understanding

For each question, keep the observed response fixed while varying the model parameter.

If $X\sim\operatorname{Bernoulli}(p)$ and the observed value is $0$, what is the likelihood contribution as a function of $p$?
If $X\sim\operatorname{Poisson}(\lambda)$ and the observed count is $3$, write the likelihood contribution $L_3(\lambda)$.
In a normal model with known variance and unrestricted mean, which mean maximizes the contribution from an observed value $y$?
What happens to a likelihood contribution when the observed value is outside the allowable response set at a parameter value?
Why does taking the logarithm of a positive likelihood contribution preserve which parameter values maximize it?