Likelihoods From Continuous Densities

Why Continuous Likelihoods Need Care

For a discrete observation, the probability of the observed value can be part of a likelihood. A continuous observation works differently: the probability of any one exact value is zero. Yet continuous data still provide information about an unknown parameter. The relevant quantity is the density evaluated at the observed value, combined across observations according to the sampling model.

The distinction matters in practical calculations. A density value can be greater than 1, depends on the measurement units, and is not the probability that an observation equals the recorded number. A likelihood uses density values to compare parameter values for the data that were observed; it does not turn those values into probabilities of exact observations.

Definition: Suppose $X_1,\ldots,X_n$ are independent observations from a continuous distribution with density $f(x;\theta)$, where $\theta$ is an unknown parameter. For observed values $x_1,\ldots,x_n$, the likelihood function is the joint density evaluated at those observations: $$ L(\theta;x_1,\ldots,x_n)=\prod_{i=1}^{n} f(x_i;\theta). $$ The observations are held fixed while $\theta$ varies. The density must be a valid density for every allowed value of $\theta$, and each observed value must lie in its support for the likelihood to be positive.

The product formula assumes independence. If the observations are dependent, the likelihood must instead be formed from their joint density, which need not be a product of marginal densities. The likelihood is a function of the parameter for the data at hand; the density, by contrast, is a function of possible data values once the parameter is fixed.

Density Is Not Point Probability

For any continuous random variable with density $f$, the probability of an exact value is obtained by integrating over a single point, an interval of length zero. Thus, for every fixed $x$,

P(X=x)=\int_x^x f(t)\,dt=0.

This does not say that $f(x)$ must be zero. The density describes probability per unit of measurement locally. For a small interval around $x$, its probability is approximately the density at $x$ multiplied by the interval width, provided the density is continuous there. More precisely, if $f$ is continuous at $x$, then

P(x-\varepsilon/2<X<x+\varepsilon/2) =\int_{x-\varepsilon/2}^{x+\varepsilon/2}f(t)\,dt \approx f(x)\varepsilon

for small positive $\varepsilon$. The approximation follows because over a sufficiently short interval a continuous density is close to its value at the interval's center. The integral is an actual probability; the density value alone is not.

Worked Example: Constructing a Likelihood and an Interval Probability

Suppose $X_1,X_2$ are independent with density $f(x;\theta)=(\theta+1)x^\theta$ for $0<x<1$, and zero otherwise, where $\theta>-1$. The observed values are $x_1=1/4$ and $x_2=1/2$. Construct the likelihood and evaluate it at $\theta=0$ and $\theta=1$. Then compare these density-based likelihoods with the probability that both observations fall in intervals of width $0.1$ centered at their recorded values.

Validity of the model. For $\theta>-1$, the density is nonnegative on $(0,1)$, and

\int_0^1(\theta+1)x^\theta\,dx =(\theta+1)\frac{1}{\theta+1}=1.

Likelihood construction. Independence gives the product of the two density values:

L(\theta)=\big[(\theta+1)(1/4)^\theta\big] \big[(\theta+1)(1/2)^\theta\big] =(\theta+1)^2(1/8)^\theta.

At $\theta=0$, the individual density values are $1$ and $1$, so $L(0)=1\cdot1=1$. At $\theta=1$, they are $2(1/4)=0.5$ and $2(1/2)=1$, so $L(1)=0.5\cdot1=0.5$.

Interval probabilities. The specified intervals are $(0.2,0.3)$ and $(0.45,0.55)$. At $\theta=0$, the density is 1 throughout $(0,1)$; each interval has probability $0.1$, so the probability that both independent observations fall in their respective intervals is $0.1(0.1)=0.01$.

At $\theta=1$, the density is $2x$. The two interval probabilities are

\int_{0.2}^{0.3}2x\,dx=0.3^2-0.2^2=0.05, \qquad \int_{0.45}^{0.55}2x\,dx=0.55^2-0.45^2=0.10.

Their joint probability is $0.05(0.10)=0.005$. These are probabilities for intervals, not for the exact recorded pair. The exact pair has probability zero under either parameter value. The method avoids the common mistake of reporting a density product as the probability of observing those exact values.

Check. The likelihood values have ratio $L(1)/L(0)=0.5$, and the interval probabilities have ratio $0.005/0.01=0.5$. In this example the short intervals preserve that ratio because the densities vary linearly over them in a way that yields the stated integrals.

Building and Reading a Likelihood

A reliable construction separates the data from the parameter. First write the sampling model, including its support and parameter restrictions. Next substitute the observed values into the density. If the sample is independent, multiply the resulting terms. The observed values then remain fixed: only the parameter varies in the likelihood function.

State the model.
Give the density, its support, the allowed parameter values, and whether observations are independent.

Insert the observations.
For each observed value, evaluate the density using the same candidate parameter.

Form the joint density.
For independent observations, multiply all evaluated densities. Keep the observations fixed and display the result as a function of the parameter.

Evaluate carefully.
When comparing parameter values, compute each density first and then their product. A likelihood value is not itself a probability of the observed sample.

The reason the product is appropriate under independence is the joint-density factorization. For any possible vector $(x_1,\ldots,x_n)$, independence gives joint density $f_{X_1,\ldots,X_n}(x_1,\ldots,x_n;\theta)=\prod_i f(x_i;\theta)$. Evaluating this joint density at the recorded vector gives the likelihood definition. The factorization is a consequence of independence, not a general rule for every collection of observations.

Worked Example: Comparing Candidate Likelihood Values

A hypothetical monitoring system records three independent readings $x_1=0.2$, $x_2=0.8$, and $x_3=0.8$, each with density $f(x;\theta)=1+\theta(2x-1)$ for $0<x<1$, where $-1\leq\theta\leq1$. Find the likelihood at $\theta=0.5$ and $\theta=-0.5$, and state which candidate gives the larger likelihood for these data.

Check the density model. On $(0,1)$, $2x-1$ ranges from $-1$ to $1$, so for $-1\leq\theta\leq1$, the density is nonnegative. Its integral is

\int_0^1[1+\theta(2x-1)]\,dx =1+\theta[x^2-x]_0^1=1.

At $\theta=0.5$. Since $2(0.2)-1=-0.6$, the first density is $1+0.5(-0.6)=0.7$. For each $0.8$ reading, $2(0.8)-1=0.6$, so its density is $1+0.5(0.6)=1.3$. Therefore,

$$ L(0.5)=0.7(1.3)(1.3)=0.7(1.69)=1.183. $$

At $\theta=-0.5$. The density at $0.2$ is $1+(-0.5)(-0.6)=1.3$. Each density at $0.8$ is $1+(-0.5)(0.6)=0.7$. Hence

$$ L(-0.5)=1.3(0.7)(0.7)=1.3(0.49)=0.637. $$

Thus $L(0.5)>L(-0.5)$ for this observed sample. This comparison does not establish that $0.5$ maximizes the likelihood over the entire parameter space; it only compares the two stated candidates. A common error is to interpret $1.183$, which exceeds 1, as an impossible probability. It is a product of density values, not a probability, so it need not be at most 1.

Measurement Units and Density Values

A density depends on the unit in which the variable is measured. If $Y=cX$ for a positive constant $c$, then the density transformation rule is $g(y;\theta)=f(y/c;\theta)/c$. The factor $1/c$ ensures that probabilities over corresponding intervals are unchanged. For a sample of $n$ independent observations, changing all measurements this way multiplies the likelihood by $c^{-n}$, the same factor for every candidate parameter. Consequently, likelihood values change with units, while ratios comparing two parameter values for the same data do not.

Worked Example: Converting a Likelihood to Percentage Units

Suppose $X_1,X_2$ are independent with density $f(x;\alpha)=(\alpha+1)(\alpha+2)x^\alpha(1-x)$ for $0<x<1$, where $\alpha>-1$. A hypothetical quality-control study records proportions $x_1=0.25$ and $x_2=0.50$. Construct the likelihood at $\alpha=0$ and $\alpha=1$. Then express the observations as percentage points $y_i=100x_i$ and verify the likelihood at $\alpha=1$ using the transformed density.

Check the density. For $\alpha>-1$,

\int_0^1(\alpha+1)(\alpha+2)x^\alpha(1-x)\,dx =(\alpha+1)(\alpha+2)\left(\frac{1}{\alpha+1}-\frac{1}{\alpha+2}\right)=1.

Likelihood in proportion units. At $\alpha=0$, the density is $2(1-x)$. Its values at $0.25$ and $0.50$ are $2(0.75)=1.5$ and $2(0.50)=1$, giving $L(0)=1.5$. At $\alpha=1$, the density is $6x(1-x)$. The values are $6(0.25)(0.75)=1.125$ and $6(0.50)(0.50)=1.5$. Thus $L(1)=1.125(1.5)=1.6875$.

Likelihood in percentage-point units. Since $Y=100X$, the transformed density is $g(y;\alpha)=f(y/100;\alpha)/100$, for $0<y<100$. At $\alpha=1$, $y_1=25$ and $y_2=50$. Therefore, $g(25;1)=1.125/100=0.01125$ and $g(50;1)=1.5/100=0.015$. The product is $0.01125(0.015)=0.00016875$, which equals $1.6875/10000$, as required for two observations.

The likelihood in percentage units is numerically smaller because each density has been divided by 100. The parameter comparison is unchanged: the same factor $100^{-2}$ applies at every $\alpha$. The common mistake is to compare density magnitudes from different measurement units as though they were probabilities on a common scale.

Common Exam Traps and the Practical Meaning

Continuous likelihoods are a bridge between a specified probability model and observed measurements. In applications, the raw measurement may be a time, amount, proportion, or laboratory reading. The likelihood calculation itself requires no special interpretation of those units, but the density and its numerical value must match the units used for the data.

Replacing density by point probability. For a continuous observation, $P(X=x)=0$. Use the density at $x$ in a likelihood, and use an integral over an interval when an actual probability is requested.
Forgetting the sampling assumptions. Multiplication is justified for independent observations with the stated common density. Dependence requires the joint density.
Using a formula outside its support. If an observed value lies outside the density's support, that density value is zero. The full likelihood is then zero for that parameter value.
Assuming a density must be at most 1. A density is probability per unit of measurement; it can exceed 1 even though probabilities, obtained by integration, cannot exceed 1.
Letting the observations vary during comparison. A likelihood compares parameter values for the same fixed observations. It is not a probability distribution over parameter values.

Key takeaway: For independent continuous data, evaluate the model density at each observed value and multiply. Interpret the result as a likelihood, not as the probability of the exact observations. Probabilities come from integrating the density over sets of positive width.

Check Your Understanding

For each problem, state the likelihood calculation and distinguish it from any exact-point probability requested.

Let $X_1,X_2$ be independent uniform observations on $(0,1)$, with observed values $0.3$ and $0.7$. Find their likelihood and the probability of observing that exact ordered pair. (Answer: likelihood $1$; exact-pair probability $0$.)
For independent observations $0.25,0.25$ from $f(x;\theta)=1+\theta(2x-1)$, $-1\leq\theta\leq1$, calculate the likelihoods at $\theta=0.4$ and $\theta=-0.4$, then compare them. (Answer: $L(0.4)=0.64$, $L(-0.4)=1.44$; the latter is larger.)
For independent observations $1/3$ and $2/3$ from $f(x;\alpha)=(\alpha+1)x^\alpha$, $0<x<1$, $\alpha>-1$, calculate the likelihood at $\alpha=0$ and $\alpha=1$. (Answer: $L(0)=1$, $L(1)=8/9$.)
Three independent observations from a uniform density on $(0,1)$ are $0.2,0.4,0.9$. Transform the measurements to $Y=10X$, and give the likelihood in both units. Explain why the values differ. (Answer: in $X$-units the likelihood is $1$; in $Y$-units it is $0.1^3=0.001$, due to the unit-conversion factor.)

What You'll Learn