Proportional Residual Error and Constant Coefficient of Variation

Why Proportional Residual Error Is Useful

For concentration measurements, a fixed absolute error can be a poor description of precision across the prediction range. An assay may vary by a small amount in concentration units at low concentrations but by a larger amount at high concentrations, while its error remains similar relative to the concentration being measured. A proportional residual-error model represents this pattern: its residual standard deviation rises in proportion to the model prediction.

The key distinction is between a constant standard deviation and a constant coefficient of variation (CV). Proportional error does not mean that residuals have the same variance at every prediction. It means that, under the model, their standard deviation as a fraction of the predicted value is constant.

Model and Assumptions

Definition: Let $y_{ij}$ be the observation for subject $i$ at observation $j$, and let $f_{ij}(\phi_i)$ be the model prediction for that observation given the subject’s parameters $\phi_i$. A proportional residual-error model is $$ y_{ij}=f_{ij}(\phi_i)\bigl(1+\epsilon_{ij}\bigr), \qquad \epsilon_{ij}\mid\phi_i \sim N(0,\sigma_{\mathrm{prop}}^2). $$ Equivalently, $y_{ij}=f_{ij}(\phi_i)+e_{ij}$, where $e_{ij}=f_{ij}(\phi_i)\epsilon_{ij}$. Conditional on $\phi_i$, the observation has mean $f_{ij}(\phi_i)$ and variance $\sigma_{\mathrm{prop}}^2 f_{ij}(\phi_i)^2$. The parameter $\sigma_{\mathrm{prop}}$ is a dimensionless proportional residual standard deviation.

The model describes residual variability conditional on the individual prediction. In the usual version, residual errors are also assumed conditionally independent across observations. That assumption is separate from the variance function: a model can have proportional variance and still require a different residual correlation structure if errors at nearby times are related.

For a positive prediction, the conditional coefficient of variation is the conditional standard deviation divided by the conditional mean. Thus $ \sqrt{\operatorname{Var}(y_{ij}\mid\phi_i)}/E(y_{ij}\mid\phi_i)=\sigma_{\mathrm{prop}} $. The CV is often reported as a percentage, $100\sigma_{\mathrm{prop}}\%$. For instance, $\sigma_{\mathrm{prop}}=0.12$ corresponds to a 12% conditional CV, not 0.12%.

This interpretation requires a positive, nonzero prediction. At a zero prediction the model assigns zero residual variance; the CV is undefined because the mean is zero. For negative predictions, the variance formula still uses the squared prediction, but the usual positive-concentration interpretation of CV does not apply. In concentration models, predictions are generally nonnegative, and the model’s behavior near zero should still be considered.

Why the Variance Scales With the Squared Prediction

The variance function follows directly from the observation model, rather than being an additional assumption to memorize. Conditional on $\phi_i$, treat $f_{ij}(\phi_i)$ as fixed. Since $\epsilon_{ij}$ has conditional mean zero and variance $\sigma_{\mathrm{prop}}^2$, multiplying it by the prediction multiplies its variance by the prediction squared.

\begin{aligned} E(y_{ij}\mid\phi_i) &=f_{ij}(\phi_i)\{1+E(\epsilon_{ij}\mid\phi_i)\} =f_{ij}(\phi_i),\\ \operatorname{Var}(y_{ij}\mid\phi_i) &=\operatorname{Var}\{f_{ij}(\phi_i)\epsilon_{ij}\mid\phi_i\}\\ &=f_{ij}(\phi_i)^2\operatorname{Var}(\epsilon_{ij}\mid\phi_i) =\sigma_{\mathrm{prop}}^2 f_{ij}(\phi_i)^2. \end{aligned}

Taking the square root gives a conditional residual standard deviation of $\sigma_{\mathrm{prop}}|f_{ij}(\phi_i)|$. For positive predictions this is simply $\sigma_{\mathrm{prop}}f_{ij}(\phi_i)$. Doubling the prediction doubles the residual standard deviation and multiplies the residual variance by four. This is the central scaling consequence of the model.

Worked Example: Calculating Residual Variance at One Prediction

Suppose a hypothetical concentration prediction is $8.0\ \mathrm{mg/L}$, and the fitted proportional residual SD is $\sigma_{\mathrm{prop}}=0.12$. Under the model, calculate the conditional residual SD and variance.

The conditional residual SD is $0.12$ times the prediction: $0.12(8.0\ \mathrm{mg/L})=0.96\ \mathrm{mg/L}$. The corresponding variance is $0.96^2=0.9216\ (\mathrm{mg/L})^2$. Equivalently, the variance formula gives $0.12^2(8.0)^2=0.0144(64)=0.9216\ (\mathrm{mg/L})^2$.

The implied conditional CV is $0.96/8.0=0.12$, or 12%. This is a model-based description of residual spread around the prediction; it does not say that an individual observation must be within 12% of the prediction.

Worked Example: Comparing Spread at Low and High Predictions

For two hypothetical observations, let the predictions be $4.0\ \mathrm{mg/L}$ and $16.0\ \mathrm{mg/L}$, with $\sigma_{\mathrm{prop}}=0.10$. Assume the model’s conditional normality and residual independence are adequate for these calculations. Find the residual SD at each prediction, then compare two hypothetical observed residuals: $0.30\ \mathrm{mg/L}$ at the lower prediction and $-1.20\ \mathrm{mg/L}$ at the higher prediction.

At $4.0\ \mathrm{mg/L}$, the residual SD is $0.10(4.0)=0.40\ \mathrm{mg/L}$. At $16.0\ \mathrm{mg/L}$, it is $0.10(16.0)=1.60\ \mathrm{mg/L}$. The higher prediction is four times as large, so the residual SD is four times as large and the variance is $16$ times as large.

Standardizing each residual by its model-based residual SD gives $0.30/0.40=0.75$ at the lower prediction and $-1.20/1.60=-0.75$ at the higher prediction. These residuals have equal magnitude on the proportional model’s SD scale, despite having different absolute sizes. Both correspond to 10% CV under the model.

When Constant CV Represents Assay Precision

A proportional residual model can represent constant relative assay precision when the residual error is plausibly dominated by measurement error, and the assay’s random error is approximately proportional to the concentration across the relevant range. In that setting, a constant CV means that the typical measurement spread, expressed relative to concentration, does not systematically increase or decrease with concentration.

The model alone does not establish that interpretation. The residual term may also absorb unexplained biological variation, timing error, or structural-model deficiencies. Moreover, proportional error is a model for variation conditional on the prediction. It should not automatically be described as an assay’s validated precision unless the data and design support attributing the residual variation to the assay.

The distinction between individual-level and population-level predictions matters as well. Here, the variance formula is conditional on $f_{ij}(\phi_i)$, the prediction for the subject’s parameters. If subject parameters vary, the marginal distribution across subjects also reflects that between-subject variation; it is not described by the residual variance formula alone.

Worked Example: Estimating a Relative Spread From Replicates

In a hypothetical assay check, five replicate measurements at a common target concentration are $44,\ 48,\ 50,\ 52,$ and $56\ \mathrm{mg/L}$. Suppose they are independent measurements under the same conditions, and suppose the target is adequately represented by a common prediction of $50\ \mathrm{mg/L}$. Calculate the sample CV as a descriptive estimate of proportional residual spread.

The sample mean is $(44+48+50+52+56)/5=250/5=50\ \mathrm{mg/L}$. The deviations from the mean are $-6,-2,0,2,6\ \mathrm{mg/L}$, and their squared sum is $36+4+0+4+36=80\ (\mathrm{mg/L})^2$. Thus the sample variance is $80/(5-1)=20\ (\mathrm{mg/L})^2$, and the sample SD is $\sqrt{20}=4.472\ \mathrm{mg/L}$.

The sample CV is $4.472/50=0.08944$, or approximately 8.944%. Under the proportional model, this would correspond to an estimated $\sigma_{\mathrm{prop}}$ of about $0.08944$. This is a descriptive estimate from five replicates, not proof that the CV is constant at other concentrations. Replicate independence, comparable assay conditions, and the assumption that other sources of variation are negligible are important for interpreting the estimate as assay precision.

Diagnostics, Alternatives, and Limitations

A proportional model is useful only if it describes the observed residual pattern reasonably well. Examine residuals against predictions across the range of concentrations. Raw residuals may fan out as predictions increase; that pattern is compatible with proportional error, but by itself is not sufficient evidence for it. Also examine residuals standardized by their modeled SD, such as $e_{ij}/(\sigma_{\mathrm{prop}}f_{ij})$ for positive predictions. A suitable variance model should not leave a systematic trend in their spread with prediction.

Potential warning signs include standardized residual spread that still increases or decreases with prediction, a concentration range with many zero or near-zero predictions, and unusually large residuals concentrated in one part of the range. A few extreme observations can also distort a visual assessment. Such patterns may reflect a variance function that is inadequate, a structural prediction that is systematically wrong, or data features not represented by the independent normal residual assumption. Residual plots help identify a mismatch; they do not, on their own, determine its cause.

The main contrast is with additive residual error. Additive error assumes a constant residual SD in the measurement units, whereas proportional error assumes that the SD changes in proportion to the prediction. The appropriate choice depends on the observed error pattern and scientific context. Proportional error can be unsuitable when absolute measurement noise remains important at low concentrations, since its variance shrinks toward zero as the prediction approaches zero. It also imposes a particular quadratic variance function; it is not a universal description of concentration-dependent precision.

A common mistake is to say, “The variance is proportional to the prediction, so doubling the prediction doubles the variance.” That confuses SD scaling with variance scaling. Under this model the SD doubles, but the variance becomes four times as large: $(2f)^2\sigma_{\mathrm{prop}}^2=4f^2\sigma_{\mathrm{prop}}^2$. Another mistake is to interpret a fitted proportional SD as an assay CV without qualification. It is a conditional CV for the residual model; attributing it specifically to assay precision requires supporting design and scientific assumptions.

Key takeaway: Under proportional residual error, the conditional residual SD is $\sigma_{\mathrm{prop}}f_{ij}(\phi_i)$ for positive predictions, and the conditional variance is $\sigma_{\mathrm{prop}}^2f_{ij}(\phi_i)^2$. The proportional SD is therefore a constant conditional CV, but its interpretation as assay precision depends on what contributes to the residuals.

Check Your Understanding

Try each question, then compare your reasoning with the solutions.

A positive prediction is $12\ \mathrm{mg/L}$, with $\sigma_{\mathrm{prop}}=0.08$. What are the conditional residual SD and variance?
If a positive prediction is multiplied by three, by what factor does the model’s residual SD change? By what factor does its variance change?
At a positive prediction of $10\ \mathrm{mg/L}$, an observed residual is $-1.5\ \mathrm{mg/L}$, and $\sigma_{\mathrm{prop}}=0.10$. What is the standardized residual?
Why does proportional residual error not, by itself, show that the assay has constant relative precision?

Solutions. (1) The SD is $0.08(12)=0.96\ \mathrm{mg/L}$. The variance is $0.96^2=0.9216\ (\mathrm{mg/L})^2$. (2) The SD changes by a factor of three; the variance changes by $3^2=9$, because it scales with the squared prediction. (3) The model-based SD is $0.10(10)=1.0\ \mathrm{mg/L}$, so the standardized residual is $-1.5/1.0=-1.5$. (4) Residual variation can include sources besides assay measurement error, and the model’s constant conditional CV is an assumption about residual spread. Supporting assay-specific design and evidence are needed to interpret that CV as assay precision.