Vec and Vech Operators

Why Turn a Matrix into a Vector?

A matrix can be treated as a single object or as a collection of scalar entries arranged by rows and columns. In multivariate methods, it is often useful to record those entries in one vector, so that matrix-valued quantities can be handled as coordinates. The order matters: the same matrix gives different vectors under different ordering conventions, so a formula involving a vectorized matrix is meaningful only when the convention is stated.

The operator $\operatorname{vec}$ applies to any rectangular matrix and stacks its columns. For a symmetric matrix, the lower triangle contains all the information: every entry above the diagonal repeats an entry below it. The operator $\operatorname{vech}$ records only that lower triangle, again column by column. This tutorial fixes these conventions and works through how to calculate each vector and recover the matrix from it.

Column-Wise Vectorization

Definition: Let $A=(a_{ij})$ be an $m\times n$ matrix. Its column-wise vectorization, denoted $\operatorname{vec}(A)$, is the $mn$-vector formed by placing the first column of $A$ first, followed by its second column, and continuing through its $n$th column: $$ \operatorname{vec}(A) = \begin{pmatrix} a_{11}\\ \vdots\\ a_{m1}\\ a_{12}\\ \vdots\\ a_{m2}\\ \vdots\\ a_{1n}\\ \vdots\\ a_{mn} \end{pmatrix}. $$ Equivalently, the entry $a_{ij}$ appears in position $i+(j-1)m$ of $\operatorname{vec}(A)$.

The dimensions are part of the definition: an $m\times n$ matrix becomes an $mn\times 1$ vector. Column-wise order is not the same as reading across the first row, then across the second row, and so on. For instance, in a $2\times 3$ matrix, $\operatorname{vec}$ lists entries in the order $a_{11},a_{21},a_{12},a_{22},a_{13},a_{23}$.

Worked Example: Vectorizing a Rectangular Matrix

Let $$ A= \begin{pmatrix} 2&-1\\ 0&3\\ 5&4 \end{pmatrix}. $$ This matrix has $m=3$ rows and $n=2$ columns, so its vectorization has $mn=6$ entries.

The first column is $(2,0,5)^\mathsf{T}$, and the second column is $(-1,3,4)^\mathsf{T}$. Stacking the columns in that order gives $$ \operatorname{vec}(A) = \begin{pmatrix} 2\\ 0\\ 5\\ -1\\ 3\\ 4 \end{pmatrix}. $$

The position formula checks the ordering. For example, $a_{32}=4$ has $i=3$, $j=2$, and $m=3$, so its position is $3+(2-1)3=6$, as shown. The entry $a_{12}=-1$ is in position $1+(2-1)3=4$.

Half-Vectorization of a Symmetric Matrix

For a symmetric $p\times p$ matrix $S$, the equality $s_{ij}=s_{ji}$ means that the upper-triangle entries can be read from the lower triangle. We use the lower triangle, including the diagonal, and list it column by column. In particular, the first column contributes $s_{11},s_{21},\ldots,s_{p1}$; the second contributes $s_{22},s_{32},\ldots,s_{p2}$; and the last contributes $s_{pp}$.

Definition: For a symmetric $p\times p$ matrix $S=(s_{ij})$, its half-vectorization, denoted $\operatorname{vech}(S)$, is the vector of its lower-triangle entries, listed column by column: $$ \operatorname{vech}(S) = \begin{pmatrix} s_{11}\\ s_{21}\\ \vdots\\ s_{p1}\\ s_{22}\\ s_{32}\\ \vdots\\ s_{p2}\\ \vdots\\ s_{pp} \end{pmatrix}. $$ It has $p(p+1)/2$ entries. More precisely, $s_{ij}$, for $i\geq j$, appears in position $\sum_{k=1}^{j-1}(p-k+1)+(i-j+1)$.

The count follows by adding the number of entries in each retained column: $p+(p-1)+\cdots+1=p(p+1)/2$. The definition requires a square symmetric matrix; applying this lower-triangle convention to a nonsymmetric matrix would discard entries that cannot be recovered from the retained ones.

Worked Example: Half-Vectorizing a Symmetric Matrix

Consider the symmetric matrix $$ S= \begin{pmatrix} 4&-2&1\\ -2&6&3\\ 1&3&5 \end{pmatrix}. $$ The first lower-triangle column contributes $s_{11}=4$, $s_{21}=-2$, and $s_{31}=1$. The second contributes $s_{22}=6$ and $s_{32}=3$. The third contributes only $s_{33}=5$.

Therefore, $$ \operatorname{vech}(S) = \begin{pmatrix} 4\\ -2\\ 1\\ 6\\ 3\\ 5 \end{pmatrix}. $$ There are $3(3+1)/2=6$ entries, matching the displayed vector. Notice that the off-diagonal value $-2$ is included once, from the lower triangle, rather than twice.

For comparison, applying ordinary vectorization to the same matrix would give $$ \operatorname{vec}(S) = \begin{pmatrix} 4\\ -2\\ 1\\ -2\\ 6\\ 3\\ 1\\ 3\\ 5 \end{pmatrix}, $$ which includes all nine entries, including both copies of each off-diagonal value.

Recovering the Matrix from Its Coordinates

These operators provide coordinate representations with no ambiguity when their matrix dimensions and ordering are known. For $\operatorname{vec}$, every entry is retained. For $\operatorname{vech}$, symmetry supplies the omitted upper-triangle entries.

Theorem: For fixed positive integers $m$ and $n$, the map $\operatorname{vec}$ gives a one-to-one correspondence between $m\times n$ matrices and vectors in $\mathbb{R}^{mn}$. For fixed positive integer $p$, the map $\operatorname{vech}$ gives a one-to-one correspondence between symmetric $p\times p$ matrices and vectors in $\mathbb{R}^{p(p+1)/2}$.

Proof. Let $x=\operatorname{vec}(A)$, where $A$ is $m\times n$. By the definition, $a_{ij}$ is the entry of $x$ in position $i+(j-1)m$. The map from a pair $(i,j)$, with $1\leq i\leq m$ and $1\leq j\leq n$, to this position lists the entries of the first column, then the second, and so forth. It uses each position from $1$ to $mn$ exactly once. Thus every entry of $A$ can be recovered from $x$, so two matrices with the same vectorization must be equal. Conversely, for any $x\in\mathbb{R}^{mn}$, place entries $x_{i+(j-1)m}$ in position $(i,j)$. This constructs an $m\times n$ matrix whose vectorization is $x$.

Now let $y=\operatorname{vech}(S)$, where $S$ is symmetric. The listed coordinates specify every entry $s_{ij}$ with $i\geq j$, and symmetry specifies the remaining entries by $s_{ij}=s_{ji}$. Thus $y$ determines all of $S$, so two symmetric matrices with the same half-vectorization must be equal. Conversely, given any vector of length $p(p+1)/2$, assign its entries in order to the lower triangle and copy each off-diagonal entry to the corresponding position above the diagonal. This produces a symmetric matrix with the given half-vectorization. Both correspondences are therefore one-to-one and onto. $\square$

The theorem also explains the different dimensions. An arbitrary $m\times n$ matrix has $mn$ freely specified entries. A symmetric $p\times p$ matrix has $p$ diagonal entries and $p(p-1)/2$ freely specified off-diagonal pairs, for a total of $p+p(p-1)/2=p(p+1)/2$ independent entries.

Worked Example: Reconstructing Matrices from Vectors

First take $x=(7,1,-2,4,0,3)^\mathsf{T}$ as the vectorization of a $3\times2$ matrix. The first three coordinates form its first column and the final three form its second column. Hence $$ \operatorname{vec}^{-1}(x) = \begin{pmatrix} 7&4\\ 1&0\\ -2&3 \end{pmatrix}. $$ Checking by stacking the columns returns $(7,1,-2,4,0,3)^\mathsf{T}$.

Now take $z=(2,-1,5,3,0,8)^\mathsf{T}$ as the half-vectorization of a symmetric $3\times3$ matrix. The first three coordinates fill the first lower-triangle column; the next two fill the rest of the second column; the last fills the final diagonal entry. Copying the off-diagonal entries across the diagonal gives $$ \operatorname{vech}^{-1}(z) = \begin{pmatrix} 2&-1&5\\ -1&3&0\\ 5&0&8 \end{pmatrix}. $$ Its lower-triangle entries in the specified order are exactly $2,-1,5,3,0,8$, so the reconstruction checks.

Common Mistake: Reading Across Rows

A frequent error is to read a matrix from left to right across its first row when calculating $\operatorname{vec}$. That produces a row-wise listing, not the column-wise convention defined here. For the rectangular matrix in the first example, row-wise reading would give $2,-1,0,3,5,4$, whereas the correct $\operatorname{vec}$ is $2,0,5,-1,3,4$. A reliable check is to write the columns vertically and copy them into the vector one at a time.

A related error is to use $\operatorname{vech}$ as if it meant “take half of $\operatorname{vec}$” in an informal sense. It is a specific operation on a symmetric matrix: list its lower-triangle entries, including the diagonal, in column-wise order. It does not average paired entries, and it does not retain an arbitrary half of the matrix. The symmetry condition is what makes the omitted entries recoverable.

Key takeaway: $\operatorname{vec}$ stacks every column of a rectangular matrix. $\operatorname{vech}$ stacks only the lower-triangle portions of successive columns of a symmetric matrix. Always state or verify the dimensions and ordering before using either coordinate vector.

Check Your Understanding

Use the column-wise conventions from this tutorial to answer the following questions.

For a $4\times2$ matrix, how many entries does its vectorization have, and which original entry is in position $7$?
Write $\operatorname{vec}(B)$ when $B=\begin{pmatrix}1&2\\3&4\end{pmatrix}$.
How many coordinates does the half-vectorization of a symmetric $5\times5$ matrix contain? Explain the count.
Write $\operatorname{vech}(C)$ for $C=\begin{pmatrix}2&-1\\-1&6\end{pmatrix}$.
Explain why a vector of length $p(p+1)/2$ determines a unique symmetric $p\times p$ matrix under the stated vech convention.

What You'll Learn