Several Competing Directions of High Curvature

When One Influential Direction Is Not Enough

Local influence is often summarized by the eigenvector associated with the largest eigenvalue of a curvature matrix. That summary is most informative when the leading eigenvalue is clearly separated from the others. If several eigenvalues are similarly large, however, the largest eigenvector may be only one of several directions with nearly the same curvature. Reporting it alone can suggest a unique pattern of influential perturbations that the calculation does not support.

This issue arises in selection-model analyses just as it does for other likelihood-based local-influence calculations. The curvature matrix describes the local, second-order change in likelihood displacement under a specified perturbation scheme at the fitted model. Its eigenvectors describe perturbation directions, and its eigenvalues give the corresponding directional curvatures. The interpretation is conditional on the fitted model, the perturbation scheme, and the scaling of its coordinates; it is not a model-free ranking of subjects.

Definition: Let \(C\) be the symmetric curvature matrix for a specified local perturbation scheme. For a unit direction \(d\), so that \(d'd=1\), its directional curvature is \(K(d)=d'Cd\). If \(C v_j=\lambda_j v_j\), with orthonormal eigenvectors \(v_j\) and eigenvalues ordered \(\lambda_1\geq\lambda_2\geq\cdots\), then \(v_j\) is a principal direction and \(\lambda_j\) is its curvature. A set of leading directions is competing when multiple leading eigenvalues are close enough, relative to the scale and uncertainty of the analysis, that no single direction is clearly dominant.

The word “close” is a diagnostic judgment, not a universal numerical cutoff. A relative gap such as \((\lambda_1-\lambda_2)/\lambda_1\) is useful to report, but its size should be considered alongside the perturbation scheme and the substantive consequences of alternative directions. In particular, do not infer that there is one uniquely influential subject pattern just because software returns one first eigenvector.

Why Nearly Tied Eigenvalues Produce Competing Directions

The key fact follows by expressing a unit direction in an orthonormal eigenbasis. For a direction in the span of the two leading eigenvectors, write \(d=a v_1+b v_2\), where \(a^2+b^2=1\). Since \(Cv_j=\lambda_jv_j\) and \(v_1'v_2=0\), its curvature is a weighted average of the two eigenvalues:

K(d)=d'Cd =(a v_1+b v_2)'(a\lambda_1v_1+b\lambda_2v_2) =\lambda_1a^2+\lambda_2b^2.

Because \(a^2=1-b^2\), the loss in curvature relative to the leading direction is

\lambda_1-K(d)=(\lambda_1-\lambda_2)b^2.

Thus, when \(\lambda_1-\lambda_2\) is small, even directions with substantial weight on \(v_2\) can remain highly curved. This is the main justification for examining a leading eigenspace rather than treating \(v_1\) as the whole result. The eigenvectors themselves can also be sensitive to small changes in the fitted quantities or in the curvature calculation when the eigenvalues are nearly tied. In that case, the span of the leading vectors can be more stable and interpretable than either vector considered alone.

This argument uses the symmetric curvature matrix from the specified local quadratic approximation and unit-length perturbation directions. It does not say that every direction in the full perturbation space is highly curved: directions outside the leading eigenspace may have much smaller curvature. Nor does it establish that the underlying model is correct or that a large local curvature implies a large change under a finite perturbation.

Worked Example: Two Nearly Equal Curvatures

Suppose a hypothetical two-coordinate perturbation has curvature matrix

C= \begin{pmatrix} 9.9&0.1\\ 0.1&9.9 \end{pmatrix}.

This matrix has eigenvector \(v_1=(1,1)'/\sqrt{2}\) with eigenvalue \(\lambda_1=10.0\), and eigenvector \(v_2=(1,-1)'/\sqrt{2}\) with eigenvalue \(\lambda_2=9.8\). The relative gap is \((10.0-9.8)/10.0=0.0200\), or 2%. The two directions are therefore similarly curved on this scale.

For the opposite-sign direction \(d=v_2\), the calculation is \(Cd=(9.8/\sqrt{2},-9.8/\sqrt{2})'\). Its coordinate contributions are \(d_1(Cd)_1=9.8/2=4.9\) and \(d_2(Cd)_2=(-1/\sqrt{2})(-9.8/\sqrt{2})=9.8/2=4.9\). They sum to \(d'Cd=9.8\). The two negative signs in the second product matter: that contribution is positive, not negative.

A mixed direction \(d=(v_1+v_2)/\sqrt{2}\) has curvature \((10.0+9.8)/2=9.9\), also close to the maximum of 10.0. The evidence supports a highly curved two-dimensional span, not a uniquely determined perturbation pattern.

Subject Contributions Across the Leading Directions

When perturbation coordinates correspond to subjects, a useful decomposition of the curvature in a particular direction is

K(d)=d'Cd=\sum_i d_i(Cd)_i.

The term \(c_i(d)=d_i(Cd)_i\) is the coordinate contribution for subject \(i\) in direction \(d\). If the coordinates instead represent other perturbation units, interpret the terms at that level. The contributions always sum to the directional curvature, but individual contributions need not be nonnegative. A contribution is not an independent subject-deletion effect: it depends on the complete direction and on the off-diagonal relationships in \(C\). For that reason, contributions should be compared across several high-curvature directions, not detached from the direction that produced them.

Worked Example: Different Subjects in Two Competing Directions

Consider a hypothetical three-subject curvature matrix

C= \begin{pmatrix} 8&1&0\\ 1&8&0\\ 0&0&8.8 \end{pmatrix}.

Its leading eigenvalue is 9, with \(v_1=(1,1,0)'/\sqrt{2}\). The second eigenvalue is 8.8, with \(v_2=(0,0,1)'\). The relative gap is \((9-8.8)/9=0.0222\), so both directions warrant examination. For \(v_1\), \(Cv_1=(9/\sqrt{2},9/\sqrt{2},0)'\). The subject contributions are \(4.5,\ 4.5,\ 0\), which sum to 9. This direction implicates subjects 1 and 2 jointly.

For \(v_2\), \(Cv_2=(0,0,8.8)'\), so the contributions are \(0,\ 0,\ 8.8\). This competing direction is concentrated on subject 3. The two patterns are substantively different, even though their curvatures are close. A report of only \(v_1\) would omit the high-curvature direction concentrated on subject 3.

For a unit mixture \(d=\cos(\phi)v_1+\sin(\phi)v_2\), its curvature is \(9\cos^2(\phi)+8.8\sin^2(\phi)\). At \(\phi=\pi/4\), this is \(9(0.5)+8.8(0.5)=8.9\). Its contributions are \(2.25,\ 2.25,\ 4.4\), which sum to 8.9. This mixture remains highly curved and spreads the subject contributions across both patterns. The example shows why a contribution profile should be tied to a stated direction.

Directional Plots for a Leading Subspace

For two leading eigenvectors, parameterize unit directions in their span by \(d(\phi)=\cos(\phi)v_1+\sin(\phi)v_2\), for \(0\leq\phi<2\pi\). A directional-curvature plot graphs \(K\{d(\phi)\}\) against \(\phi\). By the derivation above, it follows the simple form

K\{d(\phi)\}=\lambda_1\cos^2(\phi)+\lambda_2\sin^2(\phi).

When the eigenvalues are close, this plot is nearly flat. A flatter curve indicates that many directions in the displayed span have similar curvature; it does not imply that all perturbations outside that span are similarly influential. A companion plot can show \(c_i\{d(\phi)\}\) for selected subjects or the full subject contribution profile as \(\phi\) varies. This reveals whether the same subjects contribute throughout the high-curvature region or whether the dominant contributors change with direction.

Worked Example: Reading a Two-Direction Curvature Plot

For the matrix in the first example, suppose the plotted span uses its orthonormal eigenvectors, with \(\lambda_1=10.0\) and \(\lambda_2=9.8\). At \(\phi=0\), the curvature is \(10.0\). At \(\phi=\pi/4\), it is \(10.0(0.5)+9.8(0.5)=9.9\). At \(\phi=\pi/2\), it is \(9.8\). The entire plotted range is only \(10.0-9.8=0.2\), so the plot would show little variation over the leading span.

The eigenvectors in this example are expressed in the original coordinates as \(v_1=(1,1)'/\sqrt{2}\) and \(v_2=(1,-1)'/\sqrt{2}\). At \(\phi=\pi/4\), \(d=(v_1+v_2)/\sqrt{2}=(1,0)'\): the mixture is not a halfway point between the two original coordinate axes, but a specific direction in the eigenvector basis. A plot’s horizontal axis must therefore be labelled as the angle within the stated eigenvector span, not as a subject index or an arbitrary perturbation magnitude.

Eigenvector signs are arbitrary: \(v_j\) and \(-v_j\) describe the same curvature direction. Reversing \(d\) leaves \(d'Cd\) unchanged, and also leaves every product \(d_i(Cd)_i\) unchanged. For a plot, fix and state a consistent eigenvector orientation so that changing signs does not create an artificial visual jump. For more than two competing directions, pairwise plots can be informative, but they do not display the whole higher-dimensional subspace at once. State which directions are included and avoid implying that one two-dimensional view exhausts the possibilities.

Assumptions, Diagnostics, and Alternatives

The comparison is meaningful only under the same fitted model, perturbation definition, and coordinate scaling for all directions being compared. The curvature calculation must provide a symmetric matrix representing the local quadratic approximation, and the directions being compared must be normalized in the same way. If one coordinate has been rescaled, eigenvalues and eigenvectors can change; comparisons across differently scaled perturbation schemes do not isolate a difference in influence.

Useful diagnostics include the leading eigenvalues and their relative gaps; the subject contribution profiles for each leading eigenvector; and directional-curvature plots over the leading span. Check whether distinct directions identify different subjects or perturbation patterns, and whether mixtures remain highly curved. If small changes to the fitted calculation substantially rotate individual eigenvectors while leaving their leading span similar, interpret the span rather than assigning a unique meaning to either eigenvector. Numerical verification of the curvature calculation is a separate step; these plots do not establish that it is correct.

The main alternatives answer different questions. Reporting only the top eigenvector is concise but can hide competing directions. Evaluating curvature for prespecified directions is directly interpretable when there are substantive perturbations of interest, but it may miss influential patterns not anticipated in advance. Subject-deletion or case-weight analyses examine particular finite changes, rather than summarizing local behavior across directions. These approaches can complement the eigenspace analysis; none makes the others unnecessary.

A local quadratic analysis is a neighborhood diagnostic. A large curvature indicates rapid local change in the displacement measure for small perturbations in that direction, under the chosen scheme. It does not by itself quantify the impact of a large perturbation, establish a causal effect, or show that the fitted selection model is adequate. If curvature changes rapidly beyond the local neighborhood, the quadratic approximation may not represent finite perturbations well. Conclusions should therefore be framed as local and conditional on the stated model and scaling.

Common pitfall: “The leading eigenvector is the influential pattern because it has the largest eigenvalue.” This conclusion is too strong when the next eigenvalue is nearly as large. Compare the gap, inspect the other leading directions and their subject contributions, and show curvature across the leading span. The appropriate conclusion may be that several distinct perturbation patterns have similarly high local curvature.

Check Your Understanding

Answer each question, then compare with the solutions.

If \(\lambda_1=12\) and \(\lambda_2=11.4\), what is the relative gap, and why might the second direction deserve attention?
For \(C=\begin{pmatrix}5&0\\0&4.8\end{pmatrix}\) and \(d=(1,-1)'/\sqrt{2}\), calculate the directional curvature and both coordinate contributions.
In the three-subject worked example, what does the contribution profile for \(v_2\) indicate, and why would a report of \(v_1\) alone be incomplete?
What does a nearly flat directional-curvature plot across a leading two-dimensional span tell you, and what does it not tell you?
Why should subject contributions not be interpreted as independent subject-deletion effects?

Solutions. (1) The relative gap is \((12-11.4)/12=0.0500\), or 5%. The second direction may deserve attention because its curvature is close to the maximum; the gap is a prompt for inspection, not a universal decision threshold. (2) \(Cd=(5/\sqrt{2},-4.8/\sqrt{2})'\). Thus \(d'Cd=5(1/2)+4.8(1/2)=4.9\). The contributions are \(5/2=2.5\) and \((-1/\sqrt{2})(-4.8/\sqrt{2})=2.4\), which sum to 4.9. (3) The profile for \(v_2\) assigns 8.8 to subject 3 and zero to subjects 1 and 2, so it describes a distinct, nearly as highly curved pattern. Reporting only \(v_1\) omits it. (4) It indicates that many directions within that displayed span have similar curvature. It does not show that directions outside the span are highly curved or that the fitted model is correct. (5) Each contribution depends on the complete direction and on the matrix’s off-diagonal terms; it is a decomposition of \(d'Cd\), not the result of deleting that subject and refitting the model.