The Gamma Distribution of the kth Arrival

From Exponential Gaps to an Arrival-Time Distribution

Knowing the distribution of the first arrival does not by itself give the distribution of the second, third, or later arrivals. The key is to add the waiting times between successive arrivals. Each gap has an exponential distribution, and the gaps are independent; their sum has a Gamma distribution. This gives direct formulas for probabilities about when a specified arrival occurs.

Consider a Poisson process \((N_t)_{t\ge0}\) on a probability space \((\Omega,\mathcal{F},\mathbb{P})\), with state space \(\mathbb{N}_0=\{0,1,2,\ldots\}\), rate \(\lambda>0\), and natural filtration \((\mathcal{F}_t)_{t\ge0}\). Set \(S_0=0\), and let \(S_k\) be the time of the \(k\)th arrival, for a positive integer \(k\). The interarrival time \(\tau_j=S_j-S_{j-1}\) is the wait between arrivals \(j-1\) and \(j\). From the earlier results on exponential interarrival times, \(\tau_1,\tau_2,\ldots\) are independent, each with density \(\lambda e^{-\lambda t}\) for \(t>0\). Thus \(S_k=\tau_1+\cdots+\tau_k\).

Definition: The Gamma distribution with positive integer shape \(k\) and rate \(\lambda>0\) has density \(f(t)=\frac{\lambda^k t^{k-1}e^{-\lambda t}}{(k-1)!}\) for \(t>0\), and density \(0\) for \(t\le0\). Its rate parameter is \(\lambda\); equivalently, its scale parameter is \(1/\lambda\). The density of the \(k\)th arrival time will be this Gamma density.

Deriving the Density

The first arrival time is exponential, which is the Gamma density with shape \(1\). To obtain the density for later arrivals, add one independent exponential gap at a time. For independent variables with densities, the density of their sum at \(t>0\) is the integral of the product of their densities over possible values of the first summand from \(0\) to \(t\). Applying this convolution formula gives the following result.

Theorem: For a Poisson process of rate \(\lambda>0\), the \(k\)th arrival time \(S_k\), for each positive integer \(k\), has density \(f_{S_k}(t)=\frac{\lambda^k t^{k-1}e^{-\lambda t}}{(k-1)!}\) for \(t>0\). Its cumulative distribution function is \(F_{S_k}(t)=0\) for \(t\le0\), and, for \(t>0\), \(F_{S_k}(t)=1-e^{-\lambda t}\sum_{r=0}^{k-1}\frac{(\lambda t)^r}{r!}\).

Proof. We prove the density formula by induction on \(k\). For \(k=1\), \(S_1=\tau_1\), whose density is \(\lambda e^{-\lambda t}\), as required. Suppose the formula holds for some \(k\ge1\). The next gap \(\tau_{k+1}\) is independent of \(S_k\), because \(S_k\) is a function of \(\tau_1,\ldots,\tau_k\) and the gaps are independent. The convolution formula therefore gives, for \(t>0\),

\begin{aligned} f_{S_{k+1}}(t) &=\int_0^t f_{S_k}(s)\lambda e^{-\lambda(t-s)}\,ds\\ &=\int_0^t \frac{\lambda^k s^{k-1}e^{-\lambda s}}{(k-1)!}\lambda e^{-\lambda(t-s)}\,ds\\ &=\frac{\lambda^{k+1}e^{-\lambda t}}{(k-1)!}\int_0^t s^{k-1}\,ds\\ &=\frac{\lambda^{k+1}t^k e^{-\lambda t}}{k!}. \end{aligned}

This is the claimed density for shape \(k+1\). To obtain the cumulative distribution function, put \(x=\lambda t\). For \(t>0\), integrate the density and substitute \(u=\lambda s\):

F_{S_k}(t)=\frac{1}{(k-1)!}\int_0^x u^{k-1}e^{-u}\,du.

For a nonnegative integer \(m\), integration by parts gives \(\int_0^x u^m e^{-u}\,du=-x^m e^{-x}+m\int_0^x u^{m-1}e^{-u}\,du\) when \(m\ge1\), and the integral is \(1-e^{-x}\) when \(m=0\). Repeatedly using this identity yields \(\int_0^x u^{k-1}e^{-u}\,du=(k-1)![1-e^{-x}\sum_{r=0}^{k-1}x^r/r!]\). Substituting \(x=\lambda t\) proves the stated CDF. For \(t\le0\), \(S_k>0\) almost surely because each exponential gap is positive almost surely, so \(F_{S_k}(t)=0\). This proves both formulas. \(\square\)

The density integrates to one: the CDF formula tends to \(1\) as \(t\) tends to infinity, since \(e^{-x}x^r\) tends to zero for each fixed nonnegative integer \(r\). In particular, the formula describes a probability distribution, not just a function proportional to a density. It also shows that \(S_k\) has no point masses: its distribution is obtained by integrating a density. Therefore, for any fixed \(t\), \(\mathbb{P}(S_k<t)=\mathbb{P}(S_k\le t)\); similarly, probabilities of intervals do not depend on whether a fixed endpoint is included.

Using the CDF for Arrival-Time Probabilities

Once \(k\) and \(\lambda\) are specified, set \(x=\lambda t\) in the CDF. The result is a finite sum, so probabilities can be computed exactly at many convenient parameter values. For an interval with endpoints \(a<b\), the probability that the \(k\)th arrival occurs in \((a,b]\) is \(F_{S_k}(b)-F_{S_k}(a)\). The absence of point masses means that changing either endpoint from strict to inclusive does not change this probability.

Worked Example: The Second Arrival by Time One

Suppose the Poisson process has rate \(\lambda=2\). Find the probability that the second arrival occurs by time \(1\), and check the answer by integrating the density.

For \(k=2\), the density is \(f_{S_2}(t)=\frac{2^2t^{2-1}e^{-2t}}{(2-1)!}=4te^{-2t}\) for \(t>0\). The CDF formula gives

\mathbb{P}(S_2\le1) =1-e^{-2}\left(1+\frac{2^1}{1!}\right) =1-3e^{-2}.

For an independent check, an antiderivative of \(4te^{-2t}\) is \(-(2t+1)e^{-2t}\), as differentiation gives \(4te^{-2t}\). Hence

\int_0^1 4te^{-2t}\,dt =\left[-(2t+1)e^{-2t}\right]_0^1 =1-3e^{-2}.

The integral and CDF agree, and the probability is between \(0\) and \(1\) because \(e^2>3\).

Worked Example: The Third Arrival in an Interval

Suppose \(\lambda=1\). Find the probability that the third arrival occurs after time \(1\) and by time \(2\), that is, \(1<S_3\le2\). Show the calculation both from the CDF and by integrating the density.

For \(k=3\), \(F_{S_3}(t)=1-e^{-t}(1+t+t^2/2)\) for \(t>0\). Substituting the two endpoints separately gives

\begin{aligned} F_{S_3}(2)&=1-e^{-2}\left(1+2+\frac{2^2}{2}\right)=1-5e^{-2},\\ F_{S_3}(1)&=1-e^{-1}\left(1+1+\frac{1^2}{2}\right)=1-\frac{5}{2e}. \end{aligned}

Therefore the required interval probability is

\mathbb{P}(1<S_3\le2)=F_{S_3}(2)-F_{S_3}(1)=\frac{5}{2e}-5e^{-2}.

The density is \(f_{S_3}(t)=t^2e^{-t}/2\). An antiderivative is \(-e^{-t}(1+t+t^2/2)\), since differentiating it gives \(t^2e^{-t}/2\). Thus the integral over the interval is

\int_1^2 \frac{t^2e^{-t}}{2}\,dt =\left[-e^{-t}\left(1+t+\frac{t^2}{2}\right)\right]_1^2 =\frac{5}{2e}-5e^{-2},

which verifies the CDF calculation. The result is positive because \(e>2\), so \(e/2>1\) and \(5/(2e)>5/e^2\).

Worked Example: The Fourth Arrival by Time Two

For a process with rate \(\lambda=1\), compute the probability that the fourth arrival has occurred by time \(2\). Then check the formula by integrating its density.

Here \(k=4\), \(t=2\), and \(\lambda t=2\). The CDF gives

\begin{aligned} \mathbb{P}(S_4\le2) &=1-e^{-2}\left(1+\frac{2}{1!}+\frac{2^2}{2!}+\frac{2^3}{3!}\right)\\ &=1-e^{-2}\left(1+2+2+\frac{4}{3}\right) =1-\frac{19}{3}e^{-2}. \end{aligned}

The density is \(t^3e^{-t}/6\). An antiderivative is \(-e^{-t}(1+t+t^2/2+t^3/6)\); differentiation gives \(t^3e^{-t}/6\). Integrating from \(0\) to \(2\) gives

\int_0^2 \frac{t^3e^{-t}}{6}\,dt =1-e^{-2}\left(1+2+\frac{2^2}{2}+\frac{2^3}{6}\right) =1-\frac{19}{3}e^{-2}.

This matches the CDF expression. It is positive because \(e^2>1+2+2^2/2+2^3/6=19/3\), as follows from the positive terms in the exponential series beyond those displayed.

Mean and Variance of the Arrival Epoch

The sum representation also gives the moments without integrating the Gamma density. For an exponential gap \(\tau_j\), direct integration gives \(\mathbb{E}[\tau_j]=\int_0^\infty t\lambda e^{-\lambda t}\,dt=1/\lambda\), and \(\mathbb{E}[\tau_j^2]=\int_0^\infty t^2\lambda e^{-\lambda t}\,dt=2/\lambda^2\), using integration by parts. Therefore \(\operatorname{Var}(\tau_j)=2/\lambda^2-(1/\lambda)^2=1/\lambda^2\).

Result: For each positive integer \(k\), \(\mathbb{E}[S_k]=k/\lambda\) and \(\operatorname{Var}(S_k)=k/\lambda^2\).

Proof. Since \(S_k=\sum_{j=1}^k\tau_j\), linearity of expectation gives \(\mathbb{E}[S_k]=\sum_{j=1}^k\mathbb{E}[\tau_j]=k/\lambda\). The gaps are independent, so their pairwise covariances are zero. Expanding the variance of the sum therefore gives \(\operatorname{Var}(S_k)=\sum_{j=1}^k\operatorname{Var}(\tau_j)=k/\lambda^2\). All the moments used here are finite by the exponential integrals above, so these expectations and variances are well-defined. \(\square\)

For example, when \(\lambda=3\), the mean time to the third arrival is \(3/3=1\), and its variance is \(3/3^2=1/3\). The Gamma density supplies the full distribution, while these two summaries give its typical time scale and spread.

Common Mistake: Losing a Factor of the Rate

A frequent error is to write the shape-two density as \(\lambda t e^{-\lambda t}\), leaving out one factor of \(\lambda\). For \(\lambda=2\), this wrong expression is \(2te^{-2t}\). Its integral over all positive times is \(1/2\), not \(1\): an antiderivative is \(-(t+\tfrac12)e^{-2t}\), so the integral is \(\lim_{b\to\infty}[-(b+\tfrac12)e^{-2b}]-[-\tfrac12]=\tfrac12\). It cannot be a probability density.

The correct shape-two density is \(\lambda^2te^{-\lambda t}\), which for \(\lambda=2\) is \(4te^{-2t}\). Its integral over positive times is \(1\), as seen by taking the limit in the antiderivative \(-(2t+1)e^{-2t}\). The power \(\lambda^k\) and factorial \((k-1)!\) in the general density are both needed for normalization.

The Gamma law is useful whenever the timing of a particular arrival matters, for example when assessing a deadline or summarizing the delay until a specified number of events has occurred. Keep the question tied to the arrival epoch \(S_k\): the formula above gives its distribution directly from the exponential gaps, without requiring a separate calculation of a process count.

Check Your Understanding

Use the density or CDF derived above, and check any probability calculation by substituting its endpoints into the formula.

For \(\lambda=2\), write the density and CDF of \(S_1\). What feature of the formula identifies it as exponential?
For \(\lambda=1\), compute \(\mathbb{P}(S_2\le2)\). Show the substitution into the CDF formula.
For \(\lambda=2\), express \(\mathbb{P}(1/2<S_2\le1)\) exactly, using the CDF at both endpoints.
Use the independent-gap representation to find the mean and variance of \(S_5\) when \(\lambda=2\).
Explain why the expression \(\lambda t e^{-\lambda t}\) cannot be the shape-two density for general \(\lambda\). Check its integral for \(\lambda=2\).