G-Optimality And Worst-Case Prediction Variance

Why Focus on the Worst Prediction?

A design can estimate parameters precisely overall and still predict poorly at a particular part of the scientific domain. For example, observations may be concentrated near the middle of a dose range, leaving predictions at the low or high end much less precise. G-optimality addresses this by asking: what is the largest parameter-estimation variance of the predicted mean anywhere in the domain of interest? A G-optimal design minimizes that largest variance.

The domain is part of the criterion, not a minor detail. It must be stated before designs are compared: it might be a range of sampling times, doses, covariate values, or a finite set of regimens. A design that protects predictions over one domain need not do so over another.

Definition and Calculation

Let $\theta$ be a $p$-dimensional parameter vector, and let $m(z,\theta)$ be the model's mean response at a prediction setting $z$. Suppose the fitted parameter estimate has approximate covariance matrix $I^{-1}$, where $I$ is the total Fisher information matrix for the design. Define the prediction sensitivity vector as the gradient of the mean response with respect to the parameters, evaluated at nominal parameters $\theta_0$:

g(z)=\left.\frac{\partial m(z,\theta)}{\partial\theta}\right|_{\theta=\theta_0}.

A first-order approximation to the variance of the estimated mean response at $z$ is $v(z)=g(z)^T I^{-1}g(z)$. This is the parameter-estimation component of prediction uncertainty; it is not, by itself, the variance of a future observed response.

Definition: For a stated prediction domain $\mathcal{Z}$, the G-optimality criterion is the largest parameter-estimation prediction variance over that domain: $$ G(\xi)=\sup_{z\in\mathcal{Z}} g(z)^T I(\xi)^{-1}g(z). $$ Here $\xi$ denotes a design and $I(\xi)$ is its total information matrix. A G-optimal design minimizes $G(\xi)$ among the allowed designs. The information matrix must be nonsingular for this inverse-based criterion to be finite and defined.

Some presentations use an information matrix per observation rather than total information. If $M(\xi)$ is per-observation information and the design has $n$ independent observations, then $I(\xi)=nM(\xi)$, so $G(\xi)=\sup_{z\in\mathcal{Z}}g(z)^TM(\xi)^{-1}g(z)/n$. When every candidate design has the same $n$, minimizing either version gives the same design. If a common residual variance factor has been excluded from $M$, the corresponding prediction variance also includes that factor.

The criterion concerns the variance of the estimated mean response. For a future observation with conditional residual variance $\sigma^2(z)$, the approximate prediction-error variance additionally includes that residual component. The distinction matters: G-optimality as defined here targets uncertainty due to estimating the mean model, not all sources of variability in a future outcome.

Why the Quadratic Form Gives Prediction Variance

The calculation follows from a first-order approximation. Near $\theta_0$, a small estimation error $\hat{\theta}-\theta_0$ changes the fitted mean approximately by $g(z)^T(\hat{\theta}-\theta_0)$. The variance of a linear combination $a^T\hat{\theta}$ is $a^T\operatorname{Cov}(\hat{\theta})a$. Taking $a=g(z)$ and using the approximate covariance $I^{-1}$ gives:

\begin{aligned} \hat m(z)-m(z,\theta_0)&\approx g(z)^T(\hat{\theta}-\theta_0),\\ \operatorname{Var}\{\hat m(z)\}&\approx g(z)^T\operatorname{Cov}(\hat{\theta})g(z) \approx g(z)^TI^{-1}g(z). \end{aligned}

This justification depends on the local covariance approximation being reasonable and the mean function being adequately approximated by its first-order change around $\theta_0$. It is not an exact finite-sample result for every nonlinear model.

Worked Example: Checking Two Prediction Settings

Suppose a hypothetical model has total information matrix $I=\begin{pmatrix}10&2\\2&4\end{pmatrix}$. Its prediction domain consists of two settings, A and B, with sensitivity vectors $g(A)=(3,0)^T$ and $g(B)=(2,1)^T$. Calculate the prediction variance at each setting and the G criterion on this two-point domain.

The determinant of $I$ is $10(4)-2(2)=36$, so the inverse is:

I^{-1}=\frac{1}{36} \begin{pmatrix} 4&-2\\ -2&10 \end{pmatrix}.

At A, the quadratic form is:

v(A)=g(A)^TI^{-1}g(A) =\frac{1}{36}\{4(3)^2\} =\frac{36}{36}=1.

At B, both diagonal terms and the off-diagonal term contribute:

v(B)=g(B)^TI^{-1}g(B) =\frac{1}{36}\{4(2)^2-4(2)(1)+10(1)^2\} =\frac{16-8+10}{36} =\frac{18}{36}=0.5000.

Thus the largest variance on the stated domain is $G=1$, attained at A. The calculation makes clear why retaining the cross-product term is essential: omitting it would give the wrong variance at B. The result describes approximate uncertainty in the estimated mean response at each setting, in squared response units.

Finding the Maximum Across a Domain

For a finite domain, calculate the variance at each specified setting and take the largest value. For a continuous domain, the maximum may occur between observed or supported design points. It is therefore not enough to evaluate variance only at the settings where observations were collected. One must optimize the variance over the entire stated prediction domain, using analytic optimization when feasible or a sufficiently thorough numerical search otherwise.

A useful special case also shows how design allocation can control the worst case. Consider a linear mean model $m(x)=\theta_0+\theta_1x$ on $-1\leq x\leq1$. Use an approximate design that allocates weight $w$ to $x=-1$ and $1-w$ to $x=1$, where $0<w<1$. The per-observation information matrix is:

M(w)= \begin{pmatrix} 1&1-2w\\ 1-2w&1 \end{pmatrix}.

It is nonsingular for $0<w<1$. The prediction sensitivity is $f(x)=(1,x)^T$. The following example derives the worst-case variance and its minimizing allocation.

Worked Example: Balancing Endpoint Allocations

For the linear model and endpoint design just defined, find the allocation $w$ that minimizes the largest per-observation prediction variance over $[-1,1]$. For a design with $n$ observations and known homoscedastic residual variance $\sigma^2$, state the corresponding variance of the estimated mean response.

Write $c=1-2w$. Since $1-c^2=4w(1-w)$, inversion gives $M(w)^{-1}=(1-c^2)^{-1}\begin{pmatrix}1&-c\\-c&1\end{pmatrix}$. Therefore:

q_w(x)=f(x)^TM(w)^{-1}f(x) =\frac{1+x^2-2cx}{1-c^2}.

For fixed $w$, this is a quadratic function of $x$ with positive coefficient $1/(1-c^2)$. It is convex, so its maximum on the closed interval $[-1,1]$ occurs at an endpoint. Substitution gives $q_w(-1)=1/w$ and $q_w(1)=1/(1-w)$. Hence the largest value is $\max\{1/w,1/(1-w)\}=1/\min(w,1-w)$. Since $\min(w,1-w)\leq 1/2$, this maximum is at least 2, with equality only when $w=1/2$.

The minimizing allocation assigns half the observations to each endpoint, and its worst-case per-observation variance factor is 2. With $n$ observations and residual variance $\sigma^2$, the approximate variance of the estimated mean response at the worst point is $\sigma^2(2/n)$. For example, if $n=40$ and $\sigma^2=4$ response-units squared, this is $4(2/40)=0.2000$ response-units squared. This calculation assumes the stated linear mean model, independent observations, equal residual variance, and the specified endpoint design.

Comparing Designs and Checking the Calculation

The next example compares two hypothetical per-observation information matrices for the same linear prediction model and domain. It illustrates that the G criterion is a worst-case comparison: the design with the smaller largest variance is preferred for this objective, even if another design performs well at some individual settings.

Worked Example: Comparing Two Candidate Designs

For $m(x)=\theta_0+\theta_1x$ on $[-1,1]$, suppose two equal-cost candidate designs have per-observation information matrices $M_A=\begin{pmatrix}1&0.2\\0.2&0.4\end{pmatrix}$ and $M_B=\begin{pmatrix}1&0\\0&1\end{pmatrix}$. Find each design's largest prediction-variance factor.

Both matrices are positive definite: their determinants are $0.4-0.04=0.36>0$ and $1>0$, respectively. For A, $M_A^{-1}=\frac{1}{0.36}\begin{pmatrix}0.4&-0.2\\-0.2&1\end{pmatrix}$. Thus:

q_A(x)=f(x)^TM_A^{-1}f(x) =\frac{0.4-0.4x+x^2}{0.36}.

This quadratic is convex, so its maximum on $[-1,1]$ is at an endpoint. The endpoint values are $q_A(-1)=1.8/0.36=5$ and $q_A(1)=1.0/0.36=2.778$. Therefore $G_A=5$. For B, $M_B^{-1}$ is the identity matrix, so $q_B(x)=1+x^2$, whose maximum on the interval is $2$. Hence $G_B=2$, and design B has the lower worst-case variance factor for this domain. With $n$ observations and residual variance $\sigma^2$, the respective worst-case mean variances are $5\sigma^2/n$ and $2\sigma^2/n$. This preference is specific to the stated domain, model, and criterion.

Assumptions, Diagnostics, and Limitations

A G-optimality calculation requires more than an invertible matrix and a maximization. The information matrix must represent the design and model being studied, and its inverse must be an adequate approximation to parameter-estimation covariance. In nonlinear models, both information and sensitivities are generally evaluated at nominal parameter values. The resulting design is local: substantial departures from those values can change the information, the sensitivities, and which prediction settings are most uncertain.

Before accepting a result, check that the domain matches the scientific prediction question, that the information matrix is symmetric and nonsingular, and that the computed quadratic forms are nonnegative. For a continuous domain, inspect the full variance profile or use a reliable optimizer with boundary checks; evaluating only the design support can miss the maximum. Recalculate selected quadratic forms independently, including all off-diagonal terms, and confirm that units are consistent. If a variance multiplier such as $\sigma^2$ has been omitted from the information convention, restore it before reporting a variance in response units.

G-optimality is not a universal measure of design quality. D-optimality targets generalized parameter precision, A-optimality targets average parameter variance, E-optimality targets the weakest information direction, and c-optimality targets a specified parameter contrast. G-optimality instead protects the largest mean-prediction variance over a defined domain. These objectives can favor different designs. Nor does a low G criterion guarantee good prediction outside the chosen domain, eliminate model misspecification, or account automatically for residual variability in future observations.

Common mistake: checking only where observations are placed. This can miss a larger prediction variance elsewhere in the domain. Optimize over the full prediction domain.
Common mistake: treating parameter-estimation variance as future-outcome variance. The former concerns uncertainty in the estimated mean; a future observation also varies around that mean.
Common mistake: interpreting a local result as robust to all parameter values. Reassess the calculation if plausible parameter values differ materially from the nominal values.
Common mistake: using an inverse when information is singular. A singular matrix indicates that the inverse-based calculation is undefined; it is not a finite prediction-variance result.

Key takeaway: G-optimality minimizes the largest approximate variance of an estimated mean response over a prespecified domain. Compute that variance as $g(z)^TI^{-1}g(z)$, verify the maximum over the whole domain, and interpret the result under the model and information assumptions used.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions.

For a two-point domain, the prediction variances are 0.12 and 0.35 response-units squared. What is the G criterion, and where is it attained?
Why is it generally incorrect to add residual variance when the stated target is parameter-estimation variance of the mean response?
For $q(x)=1+x^2$ on $[-1,1]$, identify the maximum and explain why checking only $x=0$ would fail.
What prevents the inverse-information G criterion from being defined when $I$ is singular?

Solutions. (1) The maximum is $0.35$ response-units squared, attained at the second setting. (2) Residual variance describes variation of an observation around the mean; it is distinct from uncertainty in the estimated mean. Add it only when the target is prediction-error variance for a future observation. (3) The maximum is $q(1)=q(-1)=2$; $q(0)=1$ is smaller, so evaluating only the center misses the worst case. (4) A singular $I$ has no inverse, so $g^TI^{-1}g$ is undefined; the calculation cannot produce a finite criterion by this formula.