I-Optimality And Integrated Prediction Variance

Why Average Prediction Variance Is a Design Objective

A design can estimate a model precisely at some prediction settings and poorly at others. If the scientific goal is to predict across a defined range of doses, times, or covariate values, it may be more useful to control uncertainty across that range than to optimize uncertainty in the parameters themselves. I-optimality formalizes this goal: it chooses a design to make prediction variance, averaged over a specified target domain, as small as possible.

The target domain matters. An average is only meaningful after specifying which prediction settings count and how they are weighted. A domain may be a continuous interval with uniform weight, or a collection of important settings with unequal weights. Changing those weights can change the preferred design, even when the model and candidate design settings are unchanged.

Definition and Assumptions

Begin with a model whose mean response at setting \(x\) is \(f(x)^\mathsf{T}\beta\), where \(f(x)\) is a vector of known model terms and \(\beta\) is an unknown parameter vector. For example, a linear mean model with an intercept and slope has \(f(x)=(1,x)^\mathsf{T}\). An approximate design \(\xi\) assigns nonnegative proportions \(w_i\) to candidate settings \(x_i\), with \(\sum_i w_i=1\).

For the standard linear-model formulation, assume independent observations with a common error variance \(\sigma^2\), a correctly specified mean model, and a design that identifies all model terms. The per-observation information matrix is \(M(\xi)=\sum_i w_i f(x_i)f(x_i)^\mathsf{T}\). It must be nonsingular, so \(M(\xi)^{-1}\) exists. With \(N\) observations allocated according to the design, the covariance matrix of the least-squares estimator is \(\sigma^2 M(\xi)^{-1}/N\). Thus the variance of the estimated mean response at \(x\) is \(\sigma^2 f(x)^\mathsf{T}M(\xi)^{-1}f(x)/N\).

Let \(\nu\) be a specified probability measure over the prediction domain \(\mathcal{X}\). It describes how target settings are weighted, and its total weight is one. Define \(q(x;\xi)=f(x)^\mathsf{T}M(\xi)^{-1}f(x)\), the scaled prediction variance that omits the common factor \(\sigma^2/N\).

For a nonlinear model, the same idea applies locally. Replace \(f(x)\) with the gradient of the model mean with respect to its parameters at a specified nominal parameter value, and use the appropriate local information matrix. The resulting criterion is an average of local prediction variances; its interpretation depends on that nominal value and on the information model used to construct the matrix.

From Pointwise Variance to an Integrated Criterion

The criterion can also be written in matrix form. Define the target moment matrix \(W=\int_{\mathcal{X}} f(x)f(x)^\mathsf{T}\,d\nu(x)\). The trace identity \(\operatorname{tr}(AB)=\operatorname{tr}(BA)\) gives \(\operatorname{tr}(M^{-1}f(x)f(x)^\mathsf{T})=f(x)^\mathsf{T}M^{-1}f(x)\). Integrating both sides yields \(I(\xi)=\operatorname{tr}(M(\xi)^{-1}W)\).

This form is useful for calculation: the design enters through \(M(\xi)\), while the target domain enters through \(W\). The formula also makes clear that I-optimality is not simply another name for minimizing parameter variances. The target matrix weights parameter uncertainty according to how it affects prediction across the domain.

How I-Optimality Differs from G-Optimality

G-optimality minimizes the largest scaled prediction variance over a specified domain: it focuses on the worst point. I-optimality minimizes the average scaled prediction variance under a target measure. A point with very high variance can dominate a worst-case criterion even if it receives negligible weight in the target average. Conversely, a design may have low average variance while leaving a small portion of the domain poorly predicted.

The following example makes the distinction explicit. Both criteria use the same linear model and candidate settings, but the average uses a target measure that places most weight on one setting.

The Target Distribution Can Change the Allocation

In the same two-setting example, the I-optimal allocation can be derived directly. Suppose the target measure assigns weight \(r\) to \(x=-1\) and \(1-r\) to \(x=1\). If \(p\) is the design proportion at \(x=-1\), with \(0<p<1\), then \(I(p)=r/p+(1-r)/(1-p)\). Differentiating gives \(I'(p)=-r/p^2+(1-r)/(1-p)^2\). Setting the derivative to zero gives \(r/p^2=(1-r)/(1-p)^2\). Taking positive square roots and solving for \(p\) yields \(p=\sqrt{r}/(\sqrt{r}+\sqrt{1-r})\).

This stationary point is a minimum: the second derivative is \(2r/p^3+2(1-r)/(1-p)^3\), which is positive for \(0<p<1\) when \(0<r<1\). The criterion diverges at either boundary in that case, so the interior minimum is the global minimum.

Choosing and Checking an I-Optimal Design

Before optimizing, specify the prediction domain and target measure from the scientific purpose of the design. A uniform measure is appropriate only when equal weighting across the domain matches the intended use. If predictions at particular times, doses, or patient covariates matter more, their weights should reflect that purpose. The target should not be chosen after seeing which design it favors.

Several checks help reveal when an I-optimal calculation may be misleading:

• Check rank. If \(M(\xi)\) is singular, the full prediction-variance formula using \(M^{-1}\) does not apply. A numerical optimizer cannot repair a design that fails to identify required model terms.
• Inspect pointwise variance. Report or plot \(q(x;\xi)\) across the domain as well as its average. A small integral can conceal an unacceptable high-variance region with little target weight.
• Check the target weights and numerical integration. For continuous domains, verify that the integration method adequately represents regions where prediction variance changes quickly. For discrete target points, confirm that the weights are nonnegative and sum to one.
• Assess model and variance assumptions. The stated variance formula relies on the information model being appropriate. Heteroscedastic errors, correlated observations, or a misspecified mean require an information matrix and prediction-variance calculation that reflect those features.
• Consider local dependence in nonlinear models. A local information matrix based on nominal parameter values can produce a design whose prediction precision changes when the parameters differ from those values.

I-optimality is not a guarantee of low prediction error under model misspecification, nor does it include residual variation in a future observation unless that is explicitly added. Under constant error variance, the variance for predicting a new observation includes an additional \(\sigma^2\) beyond the variance of the estimated mean. Because that term is constant across settings and designs in the homoscedastic model, it does not change the design ranking; under other error structures it may not be constant.

The main alternative relevant here is G-optimality: use it when controlling the worst-case prediction variance is the priority. I-optimality is appropriate when average precision under a defensible target distribution is the priority. Neither criterion is intrinsically preferable; the scientific objective and the consequences of poor prediction in different parts of the domain determine which is suitable.