Identifiability of Covariance Under Incomplete Patterns

Why observed patterns determine what can be learned

With incomplete multivariate data, a variable may be observed for some individuals and absent for others. A covariance between two variables presents a stricter requirement: their values must be observed together for the corresponding bivariate marginal distribution to appear in the observed-data likelihood. This creates a basic question about the model before estimation begins: which parts of the mean vector and covariance matrix are determined by the observed distributions?

Here the target is structural identifiability under a Gaussian model and an ignorable missingness mechanism. Identifiability is a property of the model and the patterns that can occur, not a claim that a particular finite sample will estimate every identified quantity accurately. The criterion below makes the role of co-observation precise.

Definition: Let \(Y=(Y_1,\ldots,Y_p)^\mathsf{T}\sim N_p(\mu,\Sigma)\), where \(\Sigma\) is positive definite. An observed-variable pattern is a subset \(S\subseteq\{1,\ldots,p\}\); for that pattern, the observed vector is \(Y_S\). A pattern is in the support if it occurs with positive probability, or is included with positive frequency in the design. A parameter feature is identifiable from the observed-data model if any two parameter values giving the same observed-data distributions for all supported patterns have the same value of that feature.

Assume the pattern mechanism is ignorable for likelihood inference: its parameters are distinct from \((\mu,\Sigma)\), and its likelihood factor does not involve \((\mu,\Sigma)\). Then the part of the observed-data likelihood involving the Gaussian parameters is

L(\mu,\Sigma)=\prod_{i=1}^n \phi_{|S_i|}(y_{i,S_i};\mu_{S_i},\Sigma_{S_iS_i}),

where \(\phi_{|S_i|}\) is the multivariate normal density for the observed coordinates. The same reasoning applies to pattern-specific marginal distributions in the population model. The criterion concerns which patterns are supported; it does not require that each pattern happen to occur many times in a particular sample.

The co-observation criterion

Theorem: In the Gaussian model with positive-definite \(\Sigma\) and an ignorable pattern mechanism, \(\mu_j\) is identifiable if and only if variable \(j\) is observed in at least one supported pattern. The covariance entry \(\sigma_{jk}\) is identifiable if and only if some supported pattern contains both \(j\) and \(k\). For \(j=k\), this says that the variance \(\sigma_{jj}\) is identifiable if and only if variable \(j\) is observed in a supported pattern.

Proof. For each supported pattern \(S\), the observed marginal distribution is multivariate normal with mean \(\mu_S\) and covariance \(\Sigma_{SS}\). Two multivariate normal distributions are equal if and only if their mean vectors and covariance matrices are equal. Thus, equality of all supported observed distributions forces equality of \(\mu_j\) whenever \(j\) occurs in at least one supported \(S\). It forces equality of \(\sigma_{jk}\) whenever \(j\) and \(k\) occur together in at least one supported \(S\).

For the converse, suppose \(j\) is absent from every supported pattern. Changing \(\mu_j\) while leaving all other entries fixed changes none of the supported marginal distributions, so \(\mu_j\) is not identifiable. If \(j\) and \(k\) never occur together, then no supported covariance submatrix contains \(\sigma_{jk}\). Starting at any positive-definite \(\Sigma\), change only the symmetric entries \(\sigma_{jk}\) and \(\sigma_{kj}\) by a sufficiently small nonzero amount. Positive definiteness is preserved for sufficiently small perturbations because the positive-definite matrices form an open set. Every supported covariance submatrix is unchanged, but \(\sigma_{jk}\) differs. Therefore that entry is not identifiable. This proves both directions. \(\square\)

The criterion can be recorded with a co-observation graph: each variable is a vertex, and an edge joins two distinct variables if some supported pattern contains both. A vertex’s presence in a supported pattern identifies its mean and variance; an edge identifies the corresponding off-diagonal covariance. A pattern containing three or more variables contributes all the pairwise edges among its members.

Worked Example: Two Overlapping Patterns in Three Variables

Suppose \(Y=(Y_1,Y_2,Y_3)^\mathsf{T}\) is Gaussian, and the supported patterns are \(S_1=\{1,2\}\) and \(S_2=\{2,3\}\). Determine which means, variances, and covariances are identified.

Variable 1 appears in \(S_1\), variable 2 appears in both patterns, and variable 3 appears in \(S_2\). Therefore \(\mu_1,\mu_2,\mu_3\) are all identified, as are the three variances \(\sigma_{11},\sigma_{22},\sigma_{33}\). The pair \((1,2)\) occurs together in \(S_1\), so \(\sigma_{12}\) is identified. The pair \((2,3)\) occurs together in \(S_2\), so \(\sigma_{23}\) is identified. No supported pattern contains both 1 and 3, so \(\sigma_{13}\) is not identified.

The observed covariance submatrices are

\Sigma_{S_1S_1}= \begin{pmatrix} \sigma_{11}&\sigma_{12}\\ \sigma_{12}&\sigma_{22} \end{pmatrix}, \qquad \Sigma_{S_2S_2}= \begin{pmatrix} \sigma_{22}&\sigma_{23}\\ \sigma_{23}&\sigma_{33} \end{pmatrix}.

Neither contains \(\sigma_{13}\). For a numerical illustration, consider the two correlation matrices

R_0= \begin{pmatrix}1&0.3&0\\0.3&1&0.4\\0&0.4&1\end{pmatrix}, \qquad R_1= \begin{pmatrix}1&0.3&0.5\\0.3&1&0.4\\0.5&0.4&1\end{pmatrix}.

They agree on both observed submatrices. Their leading principal minors are positive: for \(R_0\), the determinant is \(1-0.3^2-0.4^2=0.75\); for \(R_1\), it is \(1+2(0.3)(0.5)(0.4)-0.3^2-0.5^2-0.4^2=0.62\). In each matrix the first two leading principal minors are \(1\) and \(1-0.3^2=0.91\), also positive. Both are therefore positive definite, despite having different values of the unobserved-together correlation.

What the likelihood can and cannot distinguish

The theorem is also a likelihood statement. Each factor in the observed-data likelihood involves only the mean coordinates and covariance submatrix for that individual’s observed pattern. A parameter entry absent from every supported marginal cannot affect any of those factors. This is a structural flat direction: changing the entry leaves the model’s observed-data distributions unchanged, not merely the numerical value of the likelihood at one particular dataset.

Result: Under the theorem’s assumptions, two Gaussian parameter values give the same distributions for every supported observed pattern exactly when their means agree on every variable appearing in a supported pattern and their covariance matrices agree on every pair appearing together in a supported pattern.

Justification. If the stated entries agree, then for each supported \(S\), the two parameter values give the same \(\mu_S\) and \(\Sigma_{SS}\). Their Gaussian marginal distributions, and hence their observed-data likelihood factors, agree. Conversely, equality of the Gaussian marginal distributions for every supported \(S\) implies equality of each such marginal’s mean and covariance. Taking the union of those means and covariance entries gives precisely the stated agreement. \(\square\)

Worked Example: A Connected Co-Observation Graph Is Not Enough

Consider four variables with supported patterns \(\{1,2\}\), \(\{2,3\}\), and \(\{3,4\}\). The co-observation graph is the connected path \(1-2-3-4\). Which covariance entries are identified?

All four variables appear, so all four means and all four variances are identified. The supported pairs identify \(\sigma_{12}\), \(\sigma_{23}\), and \(\sigma_{34}\). The patterns never contain \((1,3)\), \((1,4)\), or \((2,4)\), so those three covariances are not identified. Connectivity does not change this conclusion: a path of observed covariances is not a joint observation of its endpoints, nor does it impose a Gaussian conditional-independence model that would determine their covariance.

For example, the two positive-definite covariance matrices below have unit variances and the same covariances on the three supported pairs:

\Sigma_0=I_4,\qquad \Sigma_1= \begin{pmatrix} 1&0&0.2&0\\ 0&1&0&0\\ 0.2&0&1&0\\ 0&0&0&1 \end{pmatrix}.

The eigenvalues of \(\Sigma_0\) are all 1. The eigenvalues of \(\Sigma_1\) are \(1.2,0.8,1,1\), so both matrices are positive definite. Their observed submatrices for all three patterns are identical, but \(\sigma_{13}\) is 0 in \(\Sigma_0\) and 0.2 in \(\Sigma_1\). Thus neither connectedness nor positive definiteness supplies the missing covariance.

Worked Example: A Missing Variable and Its Mean

Suppose a study measures \(Y_1\) and \(Y_2\) for some individuals, but \(Y_3\) is absent from every supported pattern. The supported patterns are \(\{1,2\}\) and \(\{1\}\). Determine the identified features.

Variables 1 and 2 appear, so \(\mu_1,\mu_2,\sigma_{11},\sigma_{22}\), and \(\sigma_{12}\) are identified from the supported marginal distributions. Variable 3 never appears, so \(\mu_3\) and \(\sigma_{33}\) are not identified. Nor are \(\sigma_{13}\) or \(\sigma_{23}\), since neither pair is co-observed.

In the likelihood, the pattern \(\{1,2\}\) contributes a density depending on \((\mu_1,\mu_2)\) and the \(2\)-by-\(2\) covariance submatrix for variables 1 and 2. The pattern \(\{1\}\) contributes a density depending only on \(\mu_1\) and \(\sigma_{11}\). Neither factor contains a parameter for variable 3. Even infinitely many observations with these patterns cannot identify those parameters from this observed-data likelihood.

Structural identification is not finite-sample precision

A positive-probability pattern establishes structural identification, but does not guarantee a useful estimate in a realized sample. If two variables are jointly observed only a few times, the likelihood may contain little information about their covariance. If they are never jointly observed in the sample but their pair has positive probability under the design, their covariance is still structurally identifiable in the model; that particular sample may nevertheless provide no direct empirical information about it.

Conversely, a large number of observations cannot resolve a structural nonidentifiability when the pair never occurs together. This distinction matters when interpreting likelihood optimization: a numerical optimizer may return a covariance matrix even when some entries are not identified. The returned values can reflect starting values, constraints, or numerical conventions rather than information in the observed-data model.

Common mistake: inferring absent covariances from positive definiteness

A tempting argument is that once enough variances and neighboring covariances are known, the requirement that \(\Sigma\) be positive definite must determine the remaining entries. Positive definiteness constrains possible values, but, at an interior positive-definite covariance matrix, a sufficiently small perturbation of a never-co-observed covariance remains positive definite. The constraint therefore does not generally identify that entry. Only an additional model restriction, such as a specified conditional-independence structure, could supply such information; that restriction is not part of the unrestricted Gaussian covariance model here.

Another related error is to treat a connected co-observation graph as if it identified every covariance. The graph’s edges identify the covariances for those pairs only. A path connecting two vertices does not itself reveal the covariance between the endpoints.

Key takeaway: Inspect the supported patterns entry by entry. A mean and variance require the variable to appear; an off-diagonal covariance requires its two variables to appear together. Identification says what the observed-data model determines, not how precisely a finite sample estimates it.

Check Your Understanding

Use the supported patterns to answer each question under an unrestricted Gaussian model with an ignorable pattern mechanism.

With patterns \(\{1,2\}\) and \(\{2,3,4\}\), which means and variances are identified, and which off-diagonal covariances are identified?
If variables 1 and 2 each appear in supported patterns but never in the same pattern, is \(\sigma_{12}\) identifiable? Explain using the observed marginal distributions.
Why does a connected co-observation graph not, by itself, identify every covariance entry?
Can a covariance be structurally identifiable even if its pair is not jointly observed in a particular finite sample? Distinguish the design support from the realized data.
What additional kind of assumption could constrain a never-co-observed covariance, and why would that be different from identification under the unrestricted covariance model?