Tutorials › VEE Mathematical Statistics › Exponential Samples and the Sample Minimum

Home  /  Tutorials  /  VEE Mathematical Statistics / Sufficiency and Optimal Estimation

VEE Mathematical Statistics · Sufficiency and Optimal Estimation · Tutorial 504 of 880

Exponential Samples and the Sample Minimum

Distinguish the information in an exponential sample’s minimum from the sufficient statistic for its rate.

Advanced 10 min read

What You'll Learn

  • Write the joint likelihood for independent exponential observations under the rate parameterization.
  • Apply the factorization theorem to identify a sufficient statistic for the rate.
  • Derive the distribution of the sample minimum and its rate-based likelihood.
  • Distinguish sufficiency in the full sample from inference using only the minimum.
  • Use likelihood ratios to diagnose a proposed sufficient statistic.

Why the Sample Minimum Needs a Careful Check

The minimum of a sample is often informative: it records the earliest observed event, and its distribution depends directly on the exponential rate. But being informative is not the same as being sufficient. For a full independent sample from an exponential distribution with unknown rate, the factorization theorem identifies the sample sum as sufficient. The minimum alone is not sufficient for that rate.

This distinction matters in estimation. If the only recorded datum is the minimum, its distribution can be used to estimate the rate in that reduced-data setting. If the full sample is observed, however, replacing the sample by its minimum discards rate-relevant information. The calculations below establish both claims.

Model and definition: Let \(X_1,\ldots,X_n\) be independent and identically distributed exponential observations with unknown rate \(\lambda>0\), using density \(f_\lambda(x)=\lambda e^{-\lambda x}\) for \(x\geq 0\), and \(f_\lambda(x)=0\) otherwise. A statistic \(T(X_1,\ldots,X_n)\) is sufficient for \(\lambda\) if the conditional distribution of the full sample given \(T\) does not depend on \(\lambda\). Equivalently, the factorization theorem says \(T\) is sufficient if the joint density can be written as \(g_\lambda(T(x))h(x)\), where \(h\) does not depend on \(\lambda\).

The Joint Likelihood and the Sufficient Statistic

Independence means that the joint density is the product of the individual densities. For a sample \(x_1,\ldots,x_n\), it is

$$ f_\lambda(x_1,\ldots,x_n) =\prod_{i=1}^{n}\lambda e^{-\lambda x_i} =\lambda^n e^{-\lambda\sum_{i=1}^{n}x_i} \prod_{i=1}^{n}\mathbf{1}_{\{x_i\geq 0\}}. $$

As a function of \(\lambda\), this is the likelihood. The data enter its parameter-dependent part through \(\sum_i x_i\), not through the smallest observation. Define the sample sum \(S=\sum_{i=1}^{n}X_i\). Then the likelihood factors into a function of \(S\) and \(\lambda\), multiplied by an indicator that does not depend on \(\lambda\):

$$ f_\lambda(x_1,\ldots,x_n) =\underbrace{\lambda^n e^{-\lambda S}}_{g_\lambda(S)} \underbrace{\prod_{i=1}^{n}\mathbf{1}_{\{x_i\geq0\}}}_{h(x_1,\ldots,x_n)}. $$

The factorization theorem therefore establishes that \(S\) is sufficient for \(\lambda\). This result does not say that the minimum is sufficient. To test the minimum, use the likelihood ratio criterion: if two possible samples have the same value of a proposed sufficient statistic, the ratio of their joint densities must be independent of the parameter.

Theorem: For an independent sample from the exponential rate model above, \(S=\sum_{i=1}^{n}X_i\) is sufficient for \(\lambda\). The sample minimum \(M=X_{(1)}=\min(X_1,\ldots,X_n)\) is not sufficient for \(\lambda\) when the full sample is observed and \(n\geq2\).

Worked Example: Factoring a Three-Observation Likelihood

Worked Example: Factoring a Three-Observation Likelihood

A hypothetical monitoring system records three independent waiting times, in hours, from an exponential model with unknown rate \(\lambda\) per hour. The observed waiting times are \(0.4\), \(1.1\), and \(1.5\) hours. Find the likelihood, identify a sufficient statistic by factorization, and explain whether the observed minimum is sufficient for the rate.

Method. Multiply the three densities, collect the terms involving \(\lambda\), and apply the factorization theorem. The calculation concerns waiting times only; no interest or mortality assumption is involved.

Step 1: Calculate the sample sum and minimum.

$$ S=0.4+1.1+1.5=3.0\text{ hours},\qquad M=\min(0.4,1.1,1.5)=0.4\text{ hours}. $$

Step 2: Write the likelihood. All three observations are nonnegative, so the support indicator equals 1. Thus

$$ L(\lambda)=\lambda^3 e^{-\lambda(0.4+1.1+1.5)} =\lambda^3e^{-3\lambda},\qquad \lambda>0. $$

Step 3: Apply factorization. The parameter-dependent factor is \(\lambda^3e^{-\lambda S}\), with \(S=3.0\). Therefore the sample sum is sufficient for \(\lambda\).

Step 4: Check the proposed minimum. The likelihood depends on the full sample through its sum, and the minimum does not determine that sum. For example, the nonnegative sample \((0.4,1.1,2.5)\) has the same minimum, \(0.4\), but sum \(4.0\), giving likelihood \(\lambda^3e^{-4\lambda}\). The ratio of the likelihood for the observed sample to that for this alternative sample is \(e^\lambda\), which depends on \(\lambda\). Thus the minimum is not sufficient.

Check and common mistake avoided. The likelihood has the required form in \(S\), confirming the factorization. Treating the minimum as sufficient merely because it is an extreme observation would ignore the parameter-dependent likelihood ratio.

What the Sample Minimum Does Tell You

Although the minimum is not sufficient for the full sample, its distribution is simple. For \(m\geq0\), the event \(M>m\) occurs exactly when every observation exceeds \(m\). Independence gives

$$ P_\lambda(M>m) =\prod_{i=1}^{n}P_\lambda(X_i>m) =(e^{-\lambda m})^n =e^{-n\lambda m}. $$

Therefore the cumulative distribution function and density of \(M\) are

$$ F_M(m)=1-e^{-n\lambda m},\qquad f_M(m)=n\lambda e^{-n\lambda m},\qquad m\geq0. $$

So \(M\) has an exponential distribution with rate \(n\lambda\). This distribution is useful when a data collection process retains only the earliest event. In that reduced-data experiment, the observed datum is \(M\), and its likelihood is \(n\lambda e^{-n\lambda m}\). That statement does not turn \(M\) into a sufficient statistic for the original full sample: sufficiency is assessed relative to what data were observed.

Worked Example: Estimating the Rate From a Recorded Minimum

A hypothetical system observes four independent exponential waiting times with common unknown rate \(\lambda\) per hour, but stores only their minimum. The recorded minimum is \(0.5\) hours. Find the maximum likelihood estimate based on the stored minimum, and compare it with the estimate that would be obtained from the full observations \(0.5,0.7,1.2,1.6\) hours.

Method. First use the density of the minimum as the likelihood for the reduced data. Then calculate the full-sample likelihood estimate to show what information the discarded observations provide.

Step 1: Likelihood from the minimum alone. Here \(n=4\) and \(m=0.5\), so

$$ L_M(\lambda)=4\lambda e^{-4\lambda(0.5)} =4\lambda e^{-2\lambda}. $$

The log-likelihood is \(\ell_M(\lambda)=\log 4+\log\lambda-2\lambda\). Differentiating and setting the derivative equal to zero gives

$$ \frac{d\ell_M}{d\lambda}=\frac{1}{\lambda}-2=0 \quad\Longrightarrow\quad \widehat{\lambda}_M=\frac{1}{2}=0.5\text{ per hour}. $$

The second derivative is \(-1/\lambda^2<0\), so this stationary point is a maximum.

Step 2: Likelihood from the full sample. The four observations sum to \(0.5+0.7+1.2+1.6=4.0\) hours. The full-sample likelihood is

$$ L(\lambda)=\lambda^4e^{-4\lambda}. $$

Its log-likelihood is \(4\log\lambda-4\lambda\). Differentiating gives \(4/\lambda-4=0\), hence \(\widehat{\lambda}=1.0\) per hour. The second derivative, \(-4/\lambda^2\), is negative, confirming a maximum.

Check and common mistake avoided. The minimum-only estimate is \(0.5\) per hour, whereas the full-sample estimate is \(1.0\) per hour. These differ because they use different data and likelihoods. Using the minimum-only likelihood while claiming to have used the full sample confuses the reduced-data model with the original sampling model.

Why the Minimum Fails the Sufficiency Test

The likelihood ratio criterion gives a direct proof. For two nonnegative sample points \(x\) and \(y\) with the same minimum,

$$ \frac{f_\lambda(x)}{f_\lambda(y)} =\frac{\lambda^n e^{-\lambda\sum_i x_i}} {\lambda^n e^{-\lambda\sum_i y_i}} =e^{-\lambda(\sum_i x_i-\sum_i y_i)}. $$

If their sums differ, this ratio depends on \(\lambda\). Since samples can have the same minimum but different sums whenever \(n\geq2\), the ratio condition fails. This rules out sufficiency of the minimum for the full sample.

The same conclusion follows from conditional distributions. For two independent exponential observations, suppose the first is the smaller and its value is \(m\). The memoryless property gives the excess of the second above \(m\) an exponential distribution with rate \(\lambda\). Its conditional distribution depends on \(\lambda\), so conditioning on the minimum does not remove all parameter dependence from the sample.

Worked Example: A Likelihood-Ratio Test of the Minimum

For two independent exponential waiting times with unknown rate \(\lambda\) per hour, compare the samples \((1,2)\) and \((1,3)\), each in hours. Use their likelihood ratio to decide whether the minimum can be sufficient. Evaluate the ratio at \(\lambda=0.5\) and \(\lambda=1\) per hour.

Method. The samples have the same minimum. If that minimum were sufficient, their likelihood ratio would not vary with the rate.

Step 1: Compute their sums. The first sum is \(1+2=3\) hours; the second is \(1+3=4\) hours. Each sample has minimum \(1\) hour.

Step 2: Form the likelihood ratio. Their likelihoods are \(L_1(\lambda)=\lambda^2e^{-3\lambda}\) and \(L_2(\lambda)=\lambda^2e^{-4\lambda}\). Thus

$$ \frac{L_1(\lambda)}{L_2(\lambda)} =\frac{\lambda^2e^{-3\lambda}}{\lambda^2e^{-4\lambda}} =e^\lambda. $$

Step 3: Evaluate at the specified rates. At \(\lambda=0.5\), the ratio is \(e^{0.5}\approx1.6487\). At \(\lambda=1\), it is \(e^1\approx2.7183\). The ratio changes with \(\lambda\).

Conclusion and check. The same minimum corresponds to a parameter-dependent likelihood ratio, so the minimum is not sufficient. The calculation also checks the general result: the differing sums, not the common minimum, determine the changing ratio. The common mistake is to compare only the minima and overlook the rest of the likelihood.

How to Keep the Two Roles Separate

The sample minimum has a useful marginal distribution, while the sample sum carries the full-sample likelihood information about the rate. These statements are compatible: many statistics can be informative about a parameter without being sufficient. For example, the minimum’s density depends on \(\lambda\), but it does not summarize all the rate-dependent information in the original observations.

QuestionCorrect result for the exponential rate model
What is the full-sample likelihood?\(\lambda^n e^{-\lambda\sum_i x_i}\), apart from the nonnegative-support indicator.
Which statistic is sufficient for \(\lambda\)?The sample sum \(S=\sum_iX_i\), by factorization.
What is the distribution of the minimum?\(M\) is exponential with rate \(n\lambda\).
Is the minimum sufficient for \(\lambda\) when the full sample is observed?No, for \(n\geq2\); likelihood ratios for equal minima can depend on \(\lambda\).
When is a minimum-based likelihood appropriate?When the recorded data consist only of the minimum, using its density as the reduced-data model.

A frequent exam trap is to infer sufficiency from a statistic’s simple distribution or from the fact that it contains some information about the parameter. Instead, state the sampling model, write the full joint likelihood, and check whether the proposed statistic captures every parameter-dependent feature. This distinction is aligned with the SOA VEE Mathematical Statistics syllabus.

Key takeaway: For an independent sample from an exponential distribution with unknown rate, the factorization theorem gives the sample sum as sufficient. The sample minimum has an exponential distribution with rate \(n\lambda\), but it is not sufficient for \(\lambda\) when the full sample of size at least two is observed.

Check Your Understanding

For each problem, distinguish the full-sample likelihood from a likelihood based only on the minimum.

  1. Three independent exponential waiting times have rate \(\lambda\) per day. Their observed values are \(0.2,0.8,1.0\) days. Find the sample sum and minimum, and state which is sufficient for the rate. (Answer: \(S=2.0\) days; \(M=0.2\) days; the sample sum is sufficient, while the minimum is not for the full sample.)
  2. A sample of five independent exponential observations has unknown rate \(\lambda\) per hour. Find the survival function and density of its minimum. (Answer: \(P(M>m)=e^{-5\lambda m}\) and \(f_M(m)=5\lambda e^{-5\lambda m}\) for \(m\geq0\).)
  3. Two possible samples from the same rate model are \((0.5,1.0)\) and \((0.5,2.0)\), in hours. Find the likelihood ratio of the first sample to the second and evaluate it at \(\lambda=1\) per hour. What does this imply about the minimum? (Answer: The ratio is \(e^\lambda\), which equals \(e\approx2.7183\) at \(\lambda=1\); the common minimum is not sufficient.)
  4. A system retains only the minimum of four independent exponential waiting times. The recorded minimum is \(0.25\) hours. Find the maximum likelihood estimate of \(\lambda\) based only on the retained minimum. (Answer: \(\widehat{\lambda}=1/(4(0.25))=1.0\) per hour.)
  5. Four full-sample observations are \(0.25,0.50,0.75,1.50\) hours. Find the full-sample maximum likelihood estimate of \(\lambda\), then compare it with the minimum-only estimate for the minimum \(0.25\) hours from a sample of size four. (Answer: The sum is \(3.0\) hours, so the full-sample estimate is \(4/3.0=1.3333\) per hour; the minimum-only estimate is \(1.0\) per hour. They differ because the likelihoods use different amounts of data.)

Previous

Next

Back to Tutorials
Clinical Biostats