Posterior Dependence and Weak Identifiability

Why Joint Posterior Structure Matters

A posterior interval for clearance or volume summarizes uncertainty about one parameter at a time. It does not show whether the posterior supports many combinations of clearance and volume, or whether high values of one parameter tend to occur with high or low values of another. That joint structure matters in sparse concentration-time data: distinct parameter combinations can produce similar predictions at the observed times. A model may therefore return plausible marginal summaries while leaving important combinations weakly determined.

A joint posterior plot makes this structure visible. An elongated cloud of draws indicates that some combinations occur more often than others; its orientation and shape can suggest confounding. The plot is a diagnostic, not a verdict: dependence can be expected from the model, can be induced or strengthened by the prior, and does not by itself establish that a parameter is unidentifiable.

Model and Definitions

Consider a hypothetical one-compartment model after an intravenous bolus of known dose \(D\). For subject \(i\), let \(CL_i\) denote clearance and \(V_i\) volume of distribution. The predicted concentration at time \(t_{ij}\) is \(C_{ij}\). A simple model for positive observed concentrations \(Y_{ij}\) is

Here \(D\) and \(V_i\) use compatible mass and volume units, \(CL_i\) has units of volume per time, and \(t_{ij}\) has units of time. The residual standard deviation \(\sigma\) is on the log-concentration scale and is dimensionless. For between-subject variation, suppose

The population parameters \(CL\) and \(V\) are typical values on the original scale; \(\Omega\) is the covariance matrix of the log-scale random effects. Its diagonal elements are the squared random-effect scales, such as \(\omega_{CL}^2\). Assume independent subjects, conditionally independent residual errors, residual errors independent of the random effects, and a correctly specified concentration model. These assumptions make the roles of between-subject variation and within-subject residual error distinct in the model, but sparse observations may still make their scales difficult to separate.

Why Clearance and Volume Can Form a Ridge

Taking the logarithm of the predicted concentration gives

The coefficient of time is the negative elimination rate constant, \(k_i=CL_i/V_i\). If observations are mainly in the terminal phase, their decline over time can inform \(k_i\), while sparse or noisy information about the concentration level can leave \(CL_i\) and \(V_i\) less separately determined. Along a line where \(CL_i/V_i\) stays constant, increasing both parameters together preserves the slope. It does not preserve the intercept, because \(\log(D/V_i)\) changes. Thus the ridge is generally approximate, not an exact invariance; early observations or precise information about the level can help distinguish the parameters.

This argument applies to individual parameters and helps explain patterns in the population parameters as well. For positive \(CL\) and \(V\), a joint posterior cloud that rises from lower-left to upper-right can indicate that larger clearance values tend to accompany larger volume values. In another design or model, the shape may differ. The orientation must be interpreted through the concentration model and the sampling times, not treated as a universal rule.

Residual Error and Random-Effect Scales

A second source of posterior dependence involves the residual scale \(\sigma\) and the random-effect scales in \(\Omega\). Residual error describes variation among observations conditional on a subject’s parameters. Random effects describe variation between subjects. With many observations per subject and many subjects, the repeated pattern within and across subjects can help distinguish these sources. With few subjects or few observations per subject, however, a model may have limited information to allocate observed variability between them.

This possible dependence does not mean the two components are interchangeable. They act at different levels of the model. It means that, for a particular sparse design, several combinations of scales may produce similar distributions of observed data. A negative posterior association between a random-effect scale and residual error, for example, can signal that draws assigning more variability to one component tend to assign less to the other.

Reading Joint Plots and Correlations

For each important pair of parameters, inspect a scatterplot of posterior draws alongside the marginal summaries. Plot positive parameters on their natural or log scales as appropriate: a curved relationship on the original scale can look more nearly linear on a log scale. Look for elongated, curved, or multiple clusters, and note whether the observed trend is consistent across chains. A correlation near zero rules out only strong linear association; it does not rule out a nonlinear relationship or other forms of dependence.

The sample correlation from \(M\) posterior draws \((x_m,y_m)\) is

It describes linear association in the sampled posterior, not a causal relationship and not a universal measure of identifiability. MCMC draws are typically autocorrelated, so a long list of draws is not equivalent to the same number of independent draws. Interpret a correlation in light of effective sample size and chain diagnostics, and do not treat a precise-looking number from a small or poorly mixing sample as reliable. Rank-based summaries or the shape of the scatterplot can help when the relationship is nonlinear, but they also remain summaries of the posterior rather than direct tests of identifiability.

A useful interpretation proceeds in three steps:

Distinguishing Dependence From Other Problems

Posterior dependence is a feature of the joint distribution; poor chain mixing is a feature of how well the sampler explores that distribution. A ridge can be a genuine property of a well-sampled posterior. Conversely, a chain that explores only part of the posterior can create a misleading apparent pattern. Use trace plots, rank-normalized \(Rhat\), effective sample size, and Monte Carlo standard errors as complementary checks, following the earlier material on those subjects. Stable chain summaries do not establish that the model is scientifically adequate, and a well-mixed posterior can still be weakly informative about a parameter.

Marginal intervals alone are an incomplete alternative: they summarize uncertainty parameter by parameter but hide which combinations are plausible. Joint plots reveal that geometry directly. Likelihood-based approaches can provide a complementary view of what the data support, while posterior dependence reflects both likelihood and prior. Prior sensitivity analysis is especially relevant when a ridge is broad: if reasonable prior choices materially change the joint shape or marginal conclusions, report that dependence rather than presenting the posterior as if it were determined by the data alone.

Limitations and Common Pitfalls

• Calling every correlation confounding. The wrong reasoning is that any large posterior correlation proves the model cannot distinguish the parameters. The correction is to check whether the relationship is expected from the model, whether the design supplies information about the parameters separately, and whether the pattern persists under reasonable prior choices.
• Treating zero correlation as independence. A curved or otherwise nonlinear joint distribution can have little linear correlation. Inspect the joint plot rather than relying on one number.
• Using a ridge as proof of sampler failure. A ridge can be a genuine posterior feature. Check chain exploration separately; do not conflate weak identifiability with an algorithmic problem.
• Interpreting a parameter pair without units or scale. State whether the plotted values are, for example, clearance in L/h and volume in L, or their logarithms. The same posterior can look different under different parameter transformations.
• Overlooking the prior. A concentrated posterior may reflect substantial prior information as well as the data. A posterior plot alone cannot apportion how much information came from each.

Posterior dependence matters because decisions or predictions may depend on parameter combinations, not isolated parameter estimates. When a derived quantity such as \(CL/V\) is more stable than either \(CL\) or \(V\), report that distinction clearly. Do not imply that a precise combination makes each component precise, or that a visually narrow ridge establishes reliable prediction outside the observed design.