Eigenvectors as Influential Perturbation Patterns

Why an Eigenvector Is a Perturbation Pattern

A local influence analysis can identify a direction in perturbation space along which the fitted selection-model likelihood is especially sensitive. The leading eigenvector of the normal-curvature matrix gives that direction: its entries specify the relative changes made to the perturbation coordinates. When those coordinates are subject-level weights, the entries describe a coordinated pattern of subject-weight changes. The eigenvector is not simply a ranking of subjects, and its entries are not individual influence scores.

This interpretation is useful because sensitivity may arise from a combination of subjects rather than from one conspicuous record. A leading vector dominated by one coordinate suggests a concentrated pattern; one with similarly sized entries across many coordinates suggests a distributed pattern. Neither conclusion is meaningful without checking the perturbation scale and the fitted model’s local curvature.

Definition and Assumptions

Let \(\ell(\theta;\omega)\) be the perturbed observed-data log-likelihood for a selection model, where \(\theta\) is the model parameter vector and \(\omega\) is an \(m\)-dimensional perturbation vector. Let \(\omega_0\) denote the baseline perturbation, and let \(\hat{\theta}\) be the fitted maximum-likelihood estimate at that baseline. For subject-level weight perturbations, the \(m\) coordinates of \(\omega\) correspond to subjects.

Define the observed-information Hessian \(H\) and the parameter–perturbation derivative \(\Delta\), both evaluated at \((\hat{\theta},\omega_0)\). Here \(H\) is the matrix of second derivatives with respect to \(\theta\), and column \(j\) of \(\Delta\) is the derivative of the score with respect to perturbation coordinate \(j\).

The usual interpretation requires a smooth log-likelihood near the fitted point, a score equal to zero at \(\hat{\theta}\), and a nonsingular negative-definite \(H\), so the fitted point is a strict local maximum. The perturbation must also be differentiable near \(\omega_0\). These conditions make the local calculation well-defined; they do not guarantee that the quadratic approximation remains accurate for large changes.

Because \(H\) is negative definite, \(H^{-1}\) is negative definite. Therefore, for any vector \(a\), \(a^\mathsf{T}Ca=-2(\Delta a)^\mathsf{T}H^{-1}(\Delta a)\geq 0\). Thus \(C\) is positive semidefinite, and its eigenvalues are nonnegative apart from possible small negative values caused by numerical error. This property is a useful calculation check.

Why the Leading Eigenvector Gives the Most Curved Pattern

For a small movement from the baseline along a unit direction \(d\), the local likelihood-displacement approximation has the form \(\tfrac{1}{2}t^2d^\mathsf{T}Cd\), where \(t\) is the perturbation distance. Consequently, the direction with the largest value of \(d^\mathsf{T}Cd\) is locally the direction of greatest curvature. The factor one-half is part of the quadratic approximation; the curvature itself is \(d^\mathsf{T}Cd\).

To see why the leading eigenvector solves this problem, write the orthonormal eigenvectors of \(C\) as \(v_1,\ldots,v_m\), with eigenvalues \(\lambda_1\geq\cdots\geq\lambda_m\geq 0\). Any unit vector \(d\) can be written as \(d=\sum_j a_jv_j\), where \(\sum_j a_j^2=1\). Then

Equality holds when \(d\) lies in the eigenspace for \(\lambda_1\). If that eigenvalue is distinct, its unit eigenvector is unique up to sign. If the largest eigenvalue is repeated, there is no unique leading pattern: every unit vector in the corresponding eigenspace achieves the same maximum curvature.

Reading Signs, Magnitudes, and Mixed Patterns

An eigenvector’s entries are relative perturbation components. A positive entry and a negative entry indicate movement in opposite directions relative to the chosen baseline, if signed perturbations are allowed. If the perturbation scheme permits only nonnegative changes, a vector containing both signs is still a valid mathematical direction for local curvature but may not itself be an allowable one-sided change. In that case, its interpretation is about the local quadratic geometry, not a literal permitted intervention.

The sign of an eigenvector is arbitrary: \(v\) and \(-v\) have the same eigenvalue and describe the same axis of curvature. A sign reversal does not change which subjects move together or in opposition. The entries’ magnitudes can be compared as relative components only when the perturbation coordinates have comparable units and scaling.

Diagnostics, Alternatives, and Limitations

Before interpreting a leading eigenvector, check that the calculation and perturbation definition support that interpretation. Useful diagnostics include:

• Matrix properties: verify that \(C\) is numerically symmetric and that its eigenvalues are nonnegative up to rounding error. Large asymmetry or materially negative eigenvalues can indicate an implementation error, an ill-conditioned Hessian, or a failed local-maximum assumption.
• Coordinate scale: confirm that a unit change in each perturbation coordinate represents a comparable change. Rescaling coordinates changes the matrix representation and can change which pattern appears leading.
• Eigenvalue separation: compare the leading eigenvalue with the next eigenvalue. If they are close, small numerical or modelling changes can rotate the leading vector substantially. With a repeated leading eigenvalue, report the leading eigenspace rather than claiming one unique pattern.
• Locality: treat the quadratic approximation as local. A large curvature identifies rapid local change, but does not certify the size or direction of effects under a substantial perturbation.

Eigenvector analysis complements, rather than replaces, other influence assessments. Casewise contributions can help identify individual records, while the leading eigenvector can reveal a joint pattern that no single casewise ranking captures. A prespecified subject-change analysis asks about a particular direction chosen in advance; an eigenvector instead finds the direction of greatest curvature over all unit directions in the specified coordinate system. These answer different questions.

The eigenvector depends on the perturbation scheme, its scaling, and the local fitted likelihood. It does not establish that the model is wrong, that a subject caused the observed sensitivity, or that changing the corresponding records would produce the same effect in a new study. Nor does the leading vector summarize uncertainty in a treatment contrast or isolate sensitivity in one target parameter. It is a local diagnostic of the fitted likelihood’s response to the defined perturbations.

A frequent mistake is to read the largest absolute entry of the leading eigenvector as a complete subject influence ranking. That reasoning ignores the other entries: curvature may be greatest along a coordinated pattern, and a large component can be important only in combination with the rest. The correction is to report the eigenvalue, the normalized vector (up to its arbitrary overall sign), the perturbation-coordinate definition and scaling, and whether the pattern is concentrated or distributed. Then compare it with casewise diagnostics before drawing conclusions about individual records.