Choosing and Scaling ABC Summary Statistics

Why Summary Choice and Scale Matter

In approximate Bayesian computation (ABC), simulated and observed data are compared through a chosen set of summaries. That choice determines which features of the data can influence calibration. Scaling then determines how strongly each summary discrepancy contributes to the comparison. These are distinct design decisions: a scaling rule can balance units, but it cannot restore information discarded by an uninformative summary.

Let \(y_{\mathrm{obs}}\) denote the observed QSP data, \(Y(\theta)\) data simulated at parameter vector \(\theta\), and \(S(\cdot)\) a vector-valued summary function. Rejection ABC compares \(S(Y(\theta))\) with \(S(y_{\mathrm{obs}})\), accepting a simulated parameter when a chosen distance is sufficiently small. The aim here is to design that summary vector and distance so the comparison represents scientifically relevant differences in the data.

Definition: An ABC summary statistic is a function \(S(y)\) that maps a data set to one or more features used in place of the full data during ABC comparison. A summary discrepancy is the difference \(S(Y(\theta))-S(y_{\mathrm{obs}})\). A distance combines these discrepancies into a measure of how close the simulated summaries are to the observed summaries.

If the full data likelihood is unavailable, ABC can still be useful, but its inference is tied to the summaries. In the limiting case of an increasingly strict match, ABC approximates inference conditional on the selected summaries. It recovers the full-data posterior only when the summaries retain all information relevant to the parameters, as a sufficient statistic would. In complex QSP models, such sufficiency is uncommon; practical design therefore requires explicit attention to what information is retained.

Choosing Summaries for the Scientific Question

Start with the calibration target. A summary is useful when it responds to features of the data that distinguish plausible parameter values or answer a prespecified scientific question. For a longitudinal biomarker, candidate summaries might include baseline, early change, peak response, time to peak, area under the curve (AUC), trough, and recovery rate. The appropriate set depends on the model, observation schedule, measurement process, and parameters of interest; no list is universally best.

Prefer summaries with a clear link to the system’s dynamics. For example, an early slope may be informative about a rapid activation rate, while a late plateau may help distinguish maximal effect from turnover. This link is a hypothesis to investigate, not a guarantee: parameters can act jointly, and several parameter combinations may yield similar summaries. The earlier material on posterior geometry and identifiability is relevant here because a summary design cannot resolve a parameter combination that the observed experiment does not distinguish.

A practical design process includes the following checks:

Represent the time course. Include features from different phases when the data support them. A peak alone can discard timing and recovery information.
Respect the observation process. Calculate summaries from simulated and observed data using the same sampling times, transformations, censoring rules, and preprocessing. A feature that cannot be calculated comparably in both sets is not a valid discrepancy.
Match the target. If inference concerns a parameter or mechanistic quantity, ask whether changes in that target alter the candidate summaries over scientifically plausible parameter values.
Avoid unnecessary duplication. Several summaries may encode almost the same feature. Redundancy can overweight that feature unless dependence is handled in the distance.
Check simulation-based recovery. Simulate data at known parameter values and examine whether the summaries and ABC procedure can recover meaningful distinctions among those values. This tests the complete design rather than relying on intuition alone.

A summary’s marginal variability is not the same as its informativeness. A feature can vary greatly because of observation noise while changing little with \(\theta\); another can have a modest range but strongly separate parameter values. Similarly, a summary highly correlated with another may add little new information. Use scientific interpretation and simulation checks together.

Worked Example: Retaining Time-Course Information

Suppose a hypothetical QSP study records a biomarker at baseline and at several times after dosing. The calibration target is the rate of biomarker recovery, while the model also has a parameter controlling maximal suppression. Candidate summaries are baseline \(B\), minimum value \(M\), time to minimum \(T_{\min}\), and late recovery slope \(R\).

A summary set containing only \(M\) is poorly matched to the recovery-rate target: two simulations can reach the same minimum but recover at different rates. Adding \(R\) preserves a feature directly related to recovery. \(T_{\min}\) can contribute information about timing, while \(B\) helps identify whether a difference in the minimum reflects suppression relative to baseline or a different starting level.

For a concrete hypothetical comparison, suppose the observed summaries are \(B=100\) units, \(M=40\) units, \(T_{\min}=2\) days, and \(R=8\) units per day. Two simulated parameter settings produce \((100,40,2,3)\) and \((100,43,2,8)\). The first matches baseline, minimum, and timing but misses recovery by \(8-3=5\) units per day. The second differs in minimum by \(43-40=3\) units but matches recovery. If recovery is the stated target, including \(R\) allows that distinction to affect ABC; using only \(M\) and \(T_{\min}\) would not.

This example does not establish that \(R\) alone identifies a recovery parameter. Noise, other parameters, and the sampling schedule may also affect it. The candidate summaries should be tested across simulations, including plausible parameter combinations, before treating them as adequate.

Scaling Discrepancies Across Units

A raw Euclidean distance adds squared discrepancies. If one summary is measured in thousands and another in fractions, the numerical units can make one term dominate regardless of its scientific importance. A basic remedy is to divide each discrepancy by a scale \(a_j>0\), chosen for summary \(j\):

d_{\mathrm{scaled}}^2 = \sum_{j=1}^{k} \left( \frac{S_j(Y(\theta))-S_j(y_{\mathrm{obs}})}{a_j} \right)^2.

The scale must have the same units as its summary. A common choice is a scientifically meaningful measurement-error scale; another is a standard deviation estimated from suitable simulations. Robust scales, such as a median absolute deviation with an appropriate conversion factor, can be useful when simulated values have strong outliers. The chosen scale should be positive and stable. A scale near zero can make tiny discrepancies overwhelming, while a scale that is too large can make a feature nearly irrelevant.

Standardizing by a standard deviation makes discrepancies comparable in units of variability, but it does not make them equally informative about \(\theta\). Nor does it mean that all summaries deserve equal scientific weight. Scales should reflect the intended comparison, not simply be selected to produce a convenient acceptance rate.

Worked Example: Standardizing Different Units

Suppose a hypothetical calibration uses AUC in concentration-days and a peak concentration in concentration units. The observed summaries are AUC \(=120\) concentration-days and peak \(=30\) concentration units. A simulation gives AUC \(=132\) concentration-days and peak \(=33\) concentration units. Let the prespecified scales be \(a_{\mathrm{AUC}}=20\) concentration-days and \(a_{\mathrm{peak}}=2\) concentration units.

The raw squared Euclidean discrepancy is \((132-120)^2+(33-30)^2=12^2+3^2=153\). This number is dominated by the AUC term partly because its units and numerical magnitude differ from those of peak concentration.

After scaling, the standardized discrepancies are \(12/20=0.6\) for AUC and \(3/2=1.5\) for peak. Thus, \(d_{\mathrm{scaled}}^2=0.6^2+1.5^2=0.36+2.25=2.61\), and \(d_{\mathrm{scaled}}=\sqrt{2.61}\approx1.616\). On these chosen scales, the peak mismatch is larger. This conclusion is conditional on the scales: changing them changes the relative contribution, so their scientific justification should be recorded.

Accounting for Correlated Summaries

Separate standardization does not address correlation. If two summaries tend to move together, an ordinary standardized Euclidean distance can count a shared departure more than once. A covariance-adjusted, or Mahalanobis, distance uses the covariance matrix \(\Sigma\) of the summary vector to account for both scale and linear dependence:

d_{\mathrm{Mah}}^2 = \Delta^\mathsf{T}\Sigma^{-1}\Delta, \qquad \Delta=S(Y(\theta))-S(y_{\mathrm{obs}}).

Here \(\Sigma\) must be positive definite for the inverse to exist. It may be estimated from replicated observations, a measurement-error model, or a clearly specified set of simulations. The choice of covariance source matters: variability across a broad prior predictive distribution includes parameter variation as well as observation noise, and may not represent the local comparison of interest.

Justification: If \(\Sigma=LL^\mathsf{T}\) is a Cholesky factorization, define the whitened discrepancy \(z=L^{-1}\Delta\). Then \(z^\mathsf{T}z=\Delta^\mathsf{T}L^{-\mathsf{T}}L^{-1}\Delta=\Delta^\mathsf{T}\Sigma^{-1}\Delta\). Thus, the Mahalanobis distance is the ordinary squared Euclidean distance after transforming discrepancies to account for covariance. This adjustment does not add information missing from the summaries.

Worked Example: A Covariance-Adjusted Distance

Consider two hypothetical summaries with covariance \(\Sigma=\begin{pmatrix}4&3\\3&9\end{pmatrix}\). The variances are \(4\) and \(9\), and the covariance is \(3\). This matrix is positive definite because its leading diagonal entry is \(4>0\) and its determinant is \(4(9)-3(3)=27>0\). Suppose the simulated-minus-observed discrepancy is \(\Delta=(2,3)^\mathsf{T}\).

The inverse is \(\Sigma^{-1}=\frac{1}{27}\begin{pmatrix}9&-3\\-3&4\end{pmatrix}\), since the inverse of a two-by-two matrix uses its determinant and swapped, sign-adjusted entries. Therefore, \[ d_{\mathrm{Mah}}^2 = \frac{1}{27}(2,3) \begin{pmatrix}9&-3\\-3&4\end{pmatrix} \begin{pmatrix}2\\3\end{pmatrix}. \] The matrix-vector product is \((9,6)^\mathsf{T}\), so the quadratic form is \(\frac{1}{27}\{2(9)+3(6)\}=\frac{36}{27}=\frac{4}{3}\). Hence \(d_{\mathrm{Mah}}=\sqrt{4/3}\approx1.155\).

For comparison, separate standardization by the marginal standard deviations \(2\) and \(3\) gives discrepancies \(1\) and \(1\), and a squared distance of \(2\). The covariance-adjusted squared distance is \(1.333\), reflecting the positive dependence between the summaries. Neither value is universally preferable: the covariance-adjusted calculation is appropriate only if this covariance is relevant and reliably estimated.

Diagnostics, Alternatives, and Limitations

Several signs suggest that summary design or scaling is failing:

Parameter recovery is systematically poor. If simulations generated at distinct parameter values yield indistinguishable summary distributions, those summaries may not distinguish the target values.
One term dominates unexpectedly. Inspect each scaled contribution to the distance. A dominant component may reflect a scale near zero, a unit error, or a deliberately important target; establish which before proceeding.
Small changes in the simulation set change the result. This can indicate unstable scale or covariance estimates, especially with highly correlated summaries or few simulations.
The summaries fit while important trajectory features do not. Matching selected summaries does not guarantee that the full data or unselected features are reproduced. Examine the simulated trajectories and use posterior predictive checks as a complementary assessment.

Alternatives depend on the problem. A diagonal standardized distance is simple and transparent when dependence is weak or covariance estimation is unreliable. A Mahalanobis distance accounts for linear dependence but can be unstable in high dimensions, particularly when the number of summaries is large relative to the simulations used to estimate covariance. Regularization or dimension reduction may stabilize calculations, but changes the effective weighting and should be documented and validated. Feature selection or scientifically designed transformations can reduce redundancy, but may discard useful information. No distance formula compensates for an inadequate experimental design or summaries that fail to encode the target signal.

Finally, scaling and tolerance are separate choices. Scaling defines how discrepancies are measured; a tolerance defines how close a simulated summary must be for the ABC procedure. A sensible scale does not by itself determine an appropriate tolerance. Keep the summaries, transformations, scales, and covariance estimate fixed and explicit when comparing calibration results, so differences are not mistakenly attributed to parameter inference when the discrepancy definition has changed.

Key takeaway: Choose summaries for the data features and mechanistic targets they preserve. Then scale discrepancies using defensible measurement or simulation variability, account for dependence only when its covariance is reliably characterized, and verify the entire design with simulated parameter-recovery checks.

Check Your Understanding

Answer each question, then compare with the brief solutions.

Why can a summary with large prior-predictive variability be less informative about a parameter than a less variable summary?
Observed AUC is \(80\) units and a simulation gives \(90\) units. If the scale is \(5\) units, what is the standardized discrepancy?
What additional feature might be needed if a biomarker summary set contains only the minimum response but the target concerns recovery?
When is a Mahalanobis distance not directly usable as written?
Does a small ABC distance guarantee that the full simulated trajectory matches the observed trajectory? Why or why not?

Solutions. (1) Variability can reflect noise or variation unrelated to the target; informativeness depends on how the summary changes with the parameter and separates plausible parameter values. (2) \((90-80)/5=2\) scale units. (3) A recovery feature such as a late slope, recovery time, or appropriately defined late-time change, provided the observation schedule supports it. (4) If the covariance matrix is singular or not positive definite, its inverse does not exist; estimation may also be too unstable to justify the unregularized distance. (5) No. ABC compares the selected summaries, so unrepresented trajectory features can differ even when those summaries match.