Why Parameter Units Matter
An eigenvalue diagnostic summarizes the information matrix in a chosen coordinate system. That qualification matters: the same scientific model can be parameterized in milligrams or grams, or with a clearance value or its logarithm, and its Fisher matrix will change. Consequently, raw eigenvalues and condition numbers can be dominated by parameter units rather than by the design’s ability to distinguish scientifically meaningful changes.
The purpose of scaling is not to manufacture information. It is to express parameter changes on stated, comparable scales before inspecting eigenvalues, condition numbers, or weak directions. The earlier material on eigenvalue diagnostics for population information motivates these summaries; here the essential issue is how they depend on parameter coordinates.
The scale \(s_j\) has the same units as \(\theta_j\), so \(z_j\) is dimensionless. The reference value \(\theta_0\) sets the point about which changes are described; for a linear transformation it does not affect the information matrix. The choice of \(s_j\), however, does affect the numerical eigenvalues and condition number. A scale should therefore be stated, not left implicit.
Why the Information Matrix Transforms This Way
Write the log-likelihood or log-likelihood approximation as \(\ell(\theta)\), and let its score be the column vector \(U_\theta=\partial\ell/\partial\theta\). With \(\theta=\theta_0+Sz\), the chain rule gives \(U_z=S^T U_\theta\). Under the usual regularity conditions for Fisher information, its expected score-product form is \(I_\theta=E[U_\theta U_\theta^T]\). Thus:
For a diagonal scale matrix, \(S^T=S\), so the \(j,k\) entry becomes \(s_js_k(I_\theta)_{jk}\). This transformation is a congruence transformation, not a similarity transformation. Similarity transformations preserve eigenvalues; congruence transformations generally do not. The transformed matrix describes the information for the chosen scaled changes, not an invariant set of eigenvalues.
The same result holds locally for a smooth, possibly nonlinear, change of coordinates \(\theta=g(\phi)\). If \(J=\partial\theta/\partial\phi\) is the Jacobian evaluated at the nominal point, then \(I_\phi=J^T I_\theta J\). This local formula assumes a differentiable, locally one-to-one transformation. A nonlinear transformation can also change how the parameter space is represented away from that point, so the local interpretation should not be mistaken for a global linear rescaling.
Worked Examples
Worked Example: Removing a Pure Unit Disparity
Suppose a hypothetical two-parameter population Fisher matrix, in the parameters’ current units, is \(I_\theta=\begin{pmatrix}1{,}000{,}000&0\\0&1\end{pmatrix}\). The first parameter is considered in increments of \(0.001\) of its current unit and the second in increments of \(1\) of its current unit. Use \(S=\operatorname{diag}(0.001,1)\).
Transform the matrix. Since \(I_z=S^T I_\theta S\),
The unscaled eigenvalues are \(1{,}000{,}000\) and \(1\), with condition number \(1{,}000{,}000\). The scaled eigenvalues are both \(1\), with condition number \(1\). In this hypothetical example the apparent disparity arose entirely from the chosen coordinate increments: a \(0.001\)-unit change in the first parameter and a \(1\)-unit change in the second carry equal information. This conclusion depends on the stated scales; it is not evidence that the parameters have equal precision under every scientifically relevant definition of change.
Worked Example: Diagnosing a Weak Standardized Direction
Consider the hypothetical information matrix \(I_\theta=\begin{pmatrix}100&18\\18&4\end{pmatrix}\). Its diagonal entries differ substantially. To inspect dependence after normalizing each coordinate’s diagonal information to one, choose \(s_j=1/\sqrt{(I_\theta)_{jj}}\), giving \(S=\operatorname{diag}(0.1,0.5)\).
Apply the scales. The diagonal entries become \(0.1^2(100)=1\) and \(0.5^2(4)=1\); the off-diagonal entry becomes \(0.1(0.5)(18)=0.9\). Therefore:
Its eigenvectors are proportional to \((1,1)^T\) and \((1,-1)^T\), with eigenvalues \(1.9\) and \(0.1\). The condition number is \(1.9/0.1=19\). The small eigenvalue indicates that, on these diagonal-normalized scales, the contrast between the two parameter changes is much less informed than their joint increase.
For comparison, the raw matrix has trace \(104\) and determinant \(76\). Its eigenvalues are \((104\pm\sqrt{104^2-4(76)})/2\), approximately \(103.264\) and \(0.7362\), giving a condition number of about \(140.3\). Scaling changes the numerical eigenvalues and condition number; it does not change the rank. The normalized result highlights the near-dependence of the two information directions after accounting for their different diagonal scales.
Worked Example: Using a Logarithmic Coordinate for Clearance
Suppose a hypothetical one-parameter model uses clearance \(CL>0\), measured in L/h, and has local information \(I_{CL}=0.04\;(\text{h}/\text{L})^2\) at \(CL_0=10\) L/h. Define \(\phi=\log(CL/10)\), so \(CL=10e^\phi\). At the nominal value \(\phi=0\), the Jacobian is \(dCL/d\phi=10\) L/h.
For this local change of coordinates, the information is multiplied by the square of the Jacobian:
Thus information for a unit change in \(\phi\) is dimensionless and equals \(4\) at the specified nominal value. Near \(\phi=0\), a change \(\Delta\phi\) corresponds approximately to a relative change in clearance: \(CL/CL_0=e^{\Delta\phi}\), so a small \(\Delta\phi\) is approximately a fractional change of the same size. This is a local interpretation; the Jacobian, and therefore the transformed information, varies with \(CL\) if the nominal value changes.
Choosing and Reporting Scales
A defensible scale expresses a parameter change that matters for the question at hand. Possible choices include a clinically meaningful difference, a plausible range increment, a nominal parameter magnitude, or a fixed unit conversion. The appropriate choice depends on the parameter and purpose of the diagnostic. If a parameter is near zero, scaling by its nominal value can be unstable or meaningless; use a scientifically justified absolute increment instead.
Diagonal normalization, as in the second example, uses \(s_j=1/\sqrt{(I_\theta)_{jj}}\). It makes the diagonal entries of \(I_z\) equal to one and leaves the off-diagonal entries as normalized information associations. It is useful for examining relative dependence, but it is design-dependent: the scales themselves change when the diagonal information changes. It can therefore conceal the fact that a design has little absolute information in every parameter.
For design comparisons, use the same parameterization and the same fixed scale matrix for every candidate design. Otherwise, apparent differences in eigenvalues or condition numbers may reflect different scaling choices rather than different designs. If information is accumulated over subjects, preserve the same scales when comparing cohort sizes as well. Report the parameter units, the values of \(s_j\), and whether they were fixed from scientific considerations or chosen from the matrix.
What Scaling Can and Cannot Tell You
For a symmetric positive semidefinite Fisher matrix, the scaled matrix remains positive semidefinite. If \(S\) is invertible, then \(\operatorname{rank}(S^T I_\theta S)=\operatorname{rank}(I_\theta)\). Therefore, an exactly singular matrix remains singular: invertible scaling cannot resolve exact nonidentifiability. A very small eigenvalue after a clearly stated, scientifically meaningful scaling can flag a weakly informed combination on that scale, but the degree of weakness is necessarily scale-relative.
The condition number \(\kappa_2(I_z)=\lambda_{\max}(I_z)/\lambda_{\min}(I_z)\) is meaningful only when \(I_z\) is positive definite; if the smallest eigenvalue is zero, the condition number is infinite. A smaller condition number after rescaling does not by itself mean the study became more informative. Scaling changes coordinates, not observations, and cannot improve the likelihood or add information.
These diagnostics also presume that the matrix is a suitable information matrix for the stated model, design, and nominal parameters. In population design, an approximate Fisher matrix can be affected by approximation quality as well as numerical scaling. Scaling addresses coordinate comparability; it does not validate the approximation. Check symmetry, positive semidefiniteness up to numerical error, and sensitivity to reasonable scale choices. If a result changes drastically under modest, scientifically plausible choices, report that sensitivity rather than presenting one condition number as definitive.
Common Pitfalls and Alternatives
| Approach | What it is useful for | Important caution |
|---|---|---|
| Convert units | Ensuring consistent units, such as expressing a concentration in one fixed unit throughout. | Unit conversion alone does not guarantee comparable scientifically meaningful changes. |
| Use fixed diagonal scales | Comparing information on chosen absolute or relative parameter increments. | The scientific interpretation depends on the increments selected. |
| Normalize to unit diagonal | Highlighting dependence among parameters after removing diagonal magnitude differences. | Can hide low absolute information and uses scales that vary by design. |
| Whiten using the full matrix | Constructing coordinates with identity information when the matrix is positive definite. | Because this removes the eigenvalue differences being diagnosed, it is not a way to assess their severity. |
A frequent mistake is to say, “The raw condition number is enormous, so the design is nearly nonidentifiable,” without checking parameter units. The reverse mistake is to rescale until the condition number is small and then claim the design is well identified. Both conclusions overstate what the calculation shows. The correction is to state the scale, inspect the scaled eigenvalues and weak directions, and retain absolute information or uncertainty measures when judging practical precision.
Check Your Understanding
Answer each question using the stated transformations. Solutions follow the questions.
- If \(\theta=S z+\theta_0\), what is the Fisher information in \(z\), and why?
- A one-parameter information value is \(I_\theta=25\) and the chosen scale is \(s=0.2\). Calculate \(I_z\).
- Does an invertible diagonal rescaling change the exact rank of a Fisher matrix? Explain.
- Why should candidate designs use the same scaling matrix when their condition numbers are compared?
Solutions. (1) \(I_z=S^T I_\theta S\), because the chain rule gives \(U_z=S^TU_\theta\), and the information is the expected score outer product. (2) \(I_z=s^2I_\theta=(0.2)^2(25)=1\). (3) No. Multiplication on either side by an invertible matrix preserves rank, so \(\operatorname{rank}(S^TI_\theta S)=\operatorname{rank}(I_\theta)\). (4) Different scales change the transformed eigenvalues and condition number. Fixing \(S\) ensures that differences reflect candidate information matrices in the same coordinates, rather than different parameter increments.