Why an Upper Sampling Threshold Changes the Likelihood
Suppose a hypothetical claims study records a loss only when it is at or below a known upper threshold. A loss above that threshold does not appear in the data: the observer does not see its value or even a record that it occurred. This is right truncation. It differs from observing a loss and recording only that it exceeded a limit; in the truncated sample, losses above the threshold are absent.
The observed losses therefore do not follow the original loss distribution without adjustment. Each recorded value comes from the population of losses that passed the inclusion rule. The likelihood contribution must account for that condition. For continuous losses, this means dividing the original density at the observed value by the probability of being included.
The formula assumes the recorded value is exact, the threshold is known, and inclusion is governed by the stated rule. If the threshold varies across records, use the threshold applicable to each record. In a discrete model, the event \(X\le c\) must include all support points at or below \(c\); equality can have positive probability. The denominator is not the probability of exceeding the threshold.
Deriving the Conditional Contribution
For a continuous loss, the conditional distribution function for values \(x\le c\) follows directly from conditional probability:
Differentiating this conditional distribution function with respect to \(x\) gives the conditional density \(f_X(x)/F_X(c)\). The same result follows by dividing the probability of a small interval around \(x\) by the probability of inclusion. In a parametric model with parameter \(\theta\), the contribution is thus \(f_\theta(x)/F_\theta(c)\). The denominator depends on \(\theta\), so it cannot be dropped when estimating parameters.
If \(x_1,\ldots,x_n\) are independently observed, with known thresholds \(c_1,\ldots,c_n\), and the sample is modeled conditional on each observation being included, the likelihood is the product of the corresponding conditional contributions. In a discrete model, replace density values by probability masses and \(F_\theta(c_i)\) by \(P_\theta(X\le c_i)\).
These are conditional likelihoods: they describe the observed values given that they passed their inclusion rules. If the study also supplies information about how many losses were screened out, that additional information belongs to a model for the sampling process; it is not represented by pretending that the missing losses equal the threshold. The appropriate likelihood depends on what the data collection process actually records.
Worked Example: A Conditional Probability Under an Exponential Model
A hypothetical study has loss \(X\sim\operatorname{Exponential}(2)\), where 2 is the rate, so \(f_X(x)=2e^{-2x}\) and \(F_X(x)=1-e^{-2x}\) for \(x\ge0\). Only losses at most 1 are recorded. Find the conditional probability that a recorded loss is at most 0.5, and the conditional probability that it is between 0.5 and 1.
Method. Condition each event on inclusion, \(X\le1\). The inclusion probability is \(F_X(1)=1-e^{-2}=0.8646647\).
For the first probability, \(F_X(0.5)=1-e^{-1}=0.6321206\). Therefore,
For the second probability, the numerator is \(P(0.5<X\le1)=F_X(1)-F_X(0.5)=e^{-1}-e^{-2}=0.2325442\). Hence,
The two conditional probabilities sum to \(0.7311+0.2689=1.0000\), as they should. Probabilities are rounded to four decimal places. Common mistake avoided: using \(1-e^{-1}=0.6321\) as the first conditional probability without dividing by the inclusion probability. That is the unconditional probability \(P(X\le0.5)\), not the probability among recorded losses.
Likelihoods When Thresholds Vary
A threshold may differ by record because, for example, hypothetical files are screened at different predetermined amounts. In that case the likelihood denominator differs by observation. Applying one common denominator to all records changes the model. A continuous observation must also satisfy \(x_i\le c_i\); if it does not, the stated observation and inclusion rule are inconsistent.
Worked Example: Two Losses With Different Thresholds
Suppose \(X\sim\operatorname{Exponential}(1)\), with rate 1. In a hypothetical sample, the exact recorded losses are \(x_1=0.3\) and \(x_2=0.8\), and their respective upper thresholds are \(c_1=0.6\) and \(c_2=1.2\). Calculate their conditional likelihood contributions and the joint conditional likelihood.
Method. For an exponential rate of 1, \(f_X(x)=e^{-x}\) and \(F_X(c)=1-e^{-c}\). Both records satisfy their inclusion conditions: \(0.3\le0.6\) and \(0.8\le1.2\).
For the first record, \(e^{-0.3}=0.7408182\) and \(1-e^{-0.6}=0.4511884\), so
For the second record, \(e^{-0.8}=0.4493290\) and \(1-e^{-1.2}=0.6988058\), so
Multiplying the exact contributions gives
The product of the rounded individual contributions, \(1.6420(0.6430)\), is approximately \(1.0568\); the small difference from \(1.0558\) reflects rounding each factor. The joint result is computed from the unrounded factors. Common mistake avoided: using \(1-e^{-0.6}\) as the denominator for both observations. The second loss was selected using the threshold 1.2, so its denominator is \(1-e^{-1.2}\).
Conditional Maximum Likelihood for a Discrete Model
For a discrete loss, the conditional probability mass function is especially useful when the observed data are counts. Conditional maximum likelihood estimates the model parameters by maximizing the probability of the observed values after conditioning on their inclusion. The denominator can change the relationship between the observed proportions and the original model probabilities.
Worked Example: Estimating a Discrete Loss Parameter
Let a hypothetical loss \(X\) take values 1, 2, or 3, with \(P_\theta(X=1)=\theta\) and \(P_\theta(X=2)=P_\theta(X=3)=(1-\theta)/2\), where \(0<\theta<1\). Only losses at most 2 are recorded. Of 12 recorded losses, 8 equal 1 and 4 equal 2. Find the conditional maximum likelihood estimate of \(\theta\).
Method. The probability of inclusion is
Thus the conditional probability that a recorded loss equals 1 is \(q=P_\theta(X=1\mid X\le2)=2\theta/(1+\theta)\), and the conditional probability that it equals 2 is \(1-q=(1-\theta)/(1+\theta)\). The likelihood for the observed counts, omitting the parameter-free binomial coefficient, is \(q^8(1-q)^4\). As a function of \(q\), this is maximized at the observed proportion \(\widehat q=8/12=2/3\). Solving for \(\theta\),
Check: at \(\theta=1/2\), the original probabilities at 1 and 2 are \(1/2\) and \(1/4\), and the inclusion probability is \(3/4\). Their conditional probabilities are therefore \((1/2)/(3/4)=2/3\) and \((1/4)/(3/4)=1/3\), matching the observed proportions. Common mistake avoided: equating \(\theta\) directly to \(8/12\). The observed proportion estimates the conditional probability \(q\), not the original probability \(\theta\).
Parameter Estimation and Exam Checks
For a parameterized continuous model, the log-likelihood makes the adjustment visible:
When differentiating, both terms depend on \(\theta\). In particular, the score includes the derivative of \(-\log F_\theta(c_i)\); maximizing only the unadjusted density terms generally estimates the wrong model. In the discrete example, the same issue appeared through the conditional probability \(2\theta/(1+\theta)\), rather than the original mass \(\theta\).
Before calculating, check the sampling rule and the event being conditioned on. The formulas here apply when a record is included exactly when its loss is no greater than its known threshold. A threshold is not itself an observed loss, and an absent loss is not a zero. Also check that the data lie within the threshold and that the denominator uses the same model and threshold as the numerator. These checks are particularly important when a likelihood contains several different thresholds.
Check Your Understanding
For each problem, use the conditional model given the stated upper sampling threshold. Probabilities are reported to four decimal places and estimates to four decimal places where needed.
- A hypothetical loss has \(X\sim\operatorname{Exponential}(3)\), where 3 is the rate. Only values at most \(2/3\) are recorded. Find \(P(X\le1/3\mid X\le2/3)\). (Answer: \(0.7311\))
- For \(X\sim\operatorname{Exponential}(1)\), a recorded loss is \(x=0.4\) with threshold \(c=1\). Write its conditional likelihood contribution and evaluate it to four decimal places. (Answer: \(e^{-0.4}/(1-e^{-1})\approx1.0604\))
- A discrete loss has probabilities \(P_\theta(X=1)=\theta\), \(P_\theta(X=2)=P_\theta(X=3)=(1-\theta)/2\). Only values at most 2 are recorded, and 6 of 10 recorded losses equal 1. Find the conditional maximum likelihood estimate of \(\theta\). (Answer: \(\widehat\theta=3/7\approx0.4286\))
- For \(X\sim\operatorname{Exponential}(2)\), only losses at most 1 are observed. Find \(P(0.25<X\le0.75\mid X\le1)\). (Answer: \((e^{-0.5}-e^{-1.5})/(1-e^{-2})\approx0.4434\))
- In the same three-point discrete model in question 3, only losses at most 2 are observed. If 9 of 15 recorded losses equal 2, find the conditional maximum likelihood estimate of \(\theta\). (Answer: \(\widehat\theta=1/4=0.2500\))