Consistency of Particle Estimates for Filtering Expectations

What Consistency Guarantees

A particle filter approximates a filtering distribution by a finite collection of simulated states and weights. The practical question is not only how to calculate an estimate, but what happens to that estimate as the particle count grows. Consistency gives a precise answer for a fixed observation time: under stated conditions, the particle estimate of the expectation of a bounded function of the state converges in probability to the filtering expectation.

This is an asymptotic statement, not a promise that a small particle set is accurate. It does not by itself control the error at a chosen particle count, or guarantee stability over an indefinitely long sequence of observations. We establish the fixed-time result for a standard bootstrap particle filter and identify which parts of the argument matter for interpreting pharmacokinetic concentration estimates.

Write \(M_k(x,dx')\) for the Markov transition kernel from time \(k-1\) to \(k\), and \(g_k(x')\) for the nonnegative observation likelihood at \(x'\), with the observed data held fixed. For a measurable function \(h\), write \(M_kh(x)=\int h(x')M_k(x,dx')\). The filtering recursion is

This formula requires a positive denominator. It first predicts the state using the transition kernel, then reweights the prediction according to the observation likelihood.

Particle Approximation and Assumptions

At time \(k-1\), suppose the particle filter has a resampled collection \(Z_{k-1}^1,\ldots,Z_{k-1}^N\). Propagate each particle independently through \(M_k\), producing \(Y_k^i\). The propagated empirical measure is \(\nu_k^N=N^{-1}\sum_{i=1}^N\delta_{Y_k^i}\). Weight the propagated particles by \(g_k(Y_k^i)\). If the total weight is positive, the weighted filtering estimate is

When this denominator is positive, multinomial resampling draws \(N\) particles independently from the normalized weights. These resampled particles form the collection for the next propagation step. A finite-sample convention is essential: if \(\nu_k^N(g_k)=0\), define the displayed weighted estimate to be \(0\), and set all resampled particles equal to \(Y_k^1\). This convention need not be scientifically meaningful on that event; it makes the algorithm-defined quantities well-defined. Under the assumptions below, the event becomes negligible as \(N\) grows.

Boundedness is a convenient sufficient condition: it keeps the conditional variances in the proof finite. It is not the only possible route to consistency, but relaxing it requires additional moment and integrability conditions. The positive-normalizer condition ensures that the exact filtering distribution is defined at each step.

Consistency Result and Proof

We prove the result by induction. The key steps are convergence of propagated empirical averages, stability of the ratio when its limiting denominator is positive, and concentration of the resampling average around the weighted estimate.

At time zero, the weak law of large numbers gives \(N^{-1}\sum_i h(Z_0^i)\xrightarrow{p}\eta_0(h)\) for every bounded measurable \(h\). Now assume that at time \(k-1\), the resampled empirical average of every bounded measurable function converges to its expectation under \(\eta_{k-1}\). For bounded \(h\), conditional on the particles at \(k-1\), the propagated draws are independent and

Because \(M_kh\) is bounded, the induction hypothesis makes the conditional mean converge in probability to \(\eta_{k-1}(M_kh)\). To see that the random average itself has the same limit, for any \(\epsilon>0\) split the deviation into the conditional-mean deviation and the difference from that mean. Conditional Chebyshev's inequality gives

Both terms tend to zero. Thus \(\nu_k^N(h)\xrightarrow{p}\eta_{k-1}(M_kh)\). In particular, use this result with \(h=g_k\varphi\) and \(h=g_k\). The numerator and denominator of the particle ratio converge in probability to \(\eta_{k-1}(M_k(g_k\varphi))\) and \(c_k>0\), respectively.

The denominator issue cannot be omitted. Even if \(c_k>0\), a finite-particle denominator can be zero with positive probability. Since \(\nu_k^N(g_k)\xrightarrow{p}c_k>0\), however,

On the event that the denominator is positive, the ratio converges in probability to the ratio of its limits: for example, this follows by restricting to denominators greater than \(c_k/2\), whose probability tends to one, and applying the algebra of differences of ratios. On the zero-denominator event the estimate was defined to be \(0\); because that event has probability tending to zero, this arbitrary finite value does not change convergence in probability. This proves consistency of the weighted estimate.

For resampling, conditional on the propagated particles and positive total weight, the resampled states are independent draws from the weighted empirical distribution. Their sample average of \(\varphi\) has conditional mean \(\widehat{\eta}_k^N(\varphi)\) and conditional variance at most \(\|\varphi\|_\infty^2/N\). Conditional Chebyshev's inequality shows that this average differs from the weighted estimate by a quantity converging to zero in probability. On the zero-weight event the resampled average follows the stated convention; that event has probability tending to zero. The resampled average therefore converges to \(\eta_k(\varphi)\), completing the induction. \(\square\)

Worked Concentration Examples

Interpretation, Diagnostics, and Limits

The theorem applies directly to bounded functions of a concentration state. If the state concentration is known to lie in a bounded range, \(\varphi(C)=C\) is bounded and the result says that the particle estimate of the filtered mean concentration converges in probability to the model's filtering mean. If concentration is unbounded, the theorem as stated does not cover the raw mean; a suitable extension needs assumptions controlling the tails and moments. A bounded function such as a concentration indicator \(\mathbf{1}\{C>c\}\) remains covered.

Several practical checks follow from the proof. A very small empirical total weight means the ratio can be unstable even when its limiting denominator is positive. A zero total weight makes the unextended ratio undefined and should be handled explicitly in software. A low effective sample size after weighting signals that few particles dominate; resampling may reduce duplicated low-weight particles but cannot create state diversity that was lost during propagation. These are finite-sample warnings, not contradictions of fixed-time consistency.

The result also has clear boundaries. It assumes a fixed finite number of observation times; it does not establish a uniform error bound as time grows. It addresses Monte Carlo approximation for the specified transition and likelihood, not bias from an approximate SDE transition, a misspecified observation model, or numerical error in a discretization. Exact linear-Gaussian filtering and Kalman methods can be preferable when their assumptions apply; particle filters address nonlinear or non-Gaussian cases but introduce sampling error. The earlier material on Filtering Distributions for Nonlinear State-Space Models describes the filtering target, while this result concerns convergence of its particle approximation.