Mode Products and Kronecker Identities

Why Mode Products Require Careful Indexing

A mode product applies a matrix to one index of a tensor while leaving the other indices in place. It is a basic way to express transformations of multiway data, but the corresponding matrix formulas depend on how each unfolding orders its columns and how vectorization orders tensor entries. A correct formula with the factors in the wrong order is still incorrect.

We use column-major vectorization: the first tensor index varies fastest. For a three-way tensor, entries are therefore listed by increasing first index, then second index, then third index. We also fix the unfolding convention explicitly. In $X_{(1)}$, rows are indexed by $i_1$ and columns by $(i_2,i_3)$, with $i_2$ varying fastest. In $X_{(2)}$, rows are indexed by $i_2$ and columns by $(i_1,i_3)$, with $i_1$ varying fastest. In $X_{(3)}$, rows are indexed by $i_3$ and columns by $(i_1,i_2)$, with $i_1$ varying fastest.

Definition: Let $X$ be an $I_1\times I_2\times I_3$ tensor, and let $A$ be a $J_n\times I_n$ matrix. The mode-$n$ product $Y=X\times_n A$ is the tensor obtained by multiplying along index $n$. For example, $$ Y_{j_1i_2i_3}=\sum_{i_1=1}^{I_1}A_{j_1i_1}X_{i_1i_2i_3} $$ when $n=1$; the corresponding formula for another mode replaces the $n$th index of $X$ by the output index $j_n$. The resulting dimension in mode $n$ is $J_n$, and the other dimensions are unchanged.

Unfolding Identities for a Three-Way Tensor

The mode product has a direct expression in the unfolding whose rows correspond to the transformed mode. Transforming a different mode changes the columns of that unfolding, which is where the Kronecker product enters.

Theorem: For $Y=X\times_1 A$, $A$ of size $J_1\times I_1$, the mode-1 unfolding satisfies $Y_{(1)}=A X_{(1)}$. More generally, if $Y=X\times_1 A_1\times_2 A_2\times_3 A_3$, where $A_n$ is $J_n\times I_n$, then $$ Y_{(1)}=A_1X_{(1)}(A_3\otimes A_2)^T,\qquad Y_{(2)}=A_2X_{(2)}(A_3\otimes A_1)^T,\qquad Y_{(3)}=A_3X_{(3)}(A_2\otimes A_1)^T. $$ Each identity has dimensions $J_n\times\prod_{m\ne n}J_m$.

To verify the first identity, the entry of $Y_{(1)}$ in row $j_1$ and column $(j_2,j_3)$ is $Y_{j_1j_2j_3}$. Expanding the three mode products gives $$ Y_{j_1j_2j_3}=\sum_{i_1=1}^{I_1}\sum_{i_2=1}^{I_2}\sum_{i_3=1}^{I_3} (A_1)_{j_1i_1}(A_2)_{j_2i_2}(A_3)_{j_3i_3}X_{i_1i_2i_3}. $$ On the right-hand side, the entry of $A_1X_{(1)}(A_3\otimes A_2)^T$ at that same row and column is $$ \sum_{i_1=1}^{I_1}\sum_{i_2=1}^{I_2}\sum_{i_3=1}^{I_3} (A_1)_{j_1i_1}X_{i_1i_2i_3} (A_3\otimes A_2)_{(j_3,j_2),(i_3,i_2)}. $$ By the definition of the Kronecker product, the last factor is $(A_3)_{j_3i_3}(A_2)_{j_2i_2}$. This is exactly the expanded entry above, proving the mode-1 formula. The same entrywise argument for mode 2 uses the ordered column pairs $(i_1,i_3)$, giving $A_3\otimes A_1$; for mode 3 it uses $(i_1,i_2)$, giving $A_2\otimes A_1$. Thus all three identities hold. $\square$

Worked Example: Transforming Mode 1 and Checking the Unfolding

Let $X$ have dimensions $2\times2\times2$, with its two frontal slices $$ X(:,:,1)=\begin{pmatrix}1&2\\3&4\end{pmatrix},\qquad X(:,:,2)=\begin{pmatrix}5&6\\7&8\end{pmatrix}. $$ Take $A_1=\begin{pmatrix}1&1\\1&0\end{pmatrix}$, $A_2=I_2$, and $A_3=\begin{pmatrix}1&0\\0&-1\end{pmatrix}$.

Under the stated unfolding convention, the mode-1 unfolding is $$ X_{(1)}=\begin{pmatrix}1&2&5&6\\3&4&7&8\end{pmatrix}. $$ The identity gives $$ Y_{(1)}=A_1X_{(1)}(A_3\otimes A_2)^T. $$ Since $A_3\otimes A_2$ is diagonal with diagonal $1,1,-1,-1$, the right multiplication changes the signs of the last two columns: $$ X_{(1)}(A_3\otimes A_2)^T =\begin{pmatrix}1&2&-5&-6\\3&4&-7&-8\end{pmatrix}. $$ Multiplying on the left gives $$ Y_{(1)} =\begin{pmatrix}1&1\\1&0\end{pmatrix} \begin{pmatrix}1&2&-5&-6\\3&4&-7&-8\end{pmatrix} =\begin{pmatrix}4&6&-12&-14\\1&2&-5&-6\end{pmatrix}. $$ For instance, the first transformed entry is $Y_{111}=X_{111}+X_{211}=1+3=4$, as the first entry of this unfolding requires.

Vectorization and the Full Kronecker Identity

For a three-way tensor, column-major vectorization lists $X_{i_1i_2i_3}$ with $i_1$ varying fastest, then $i_2$, then $i_3$. With this ordering, the mode-wise matrices appear in reverse order in the Kronecker product: the factor for mode 3 is leftmost and the factor for mode 1 is rightmost.

Theorem: If $Y=X\times_1 A_1\times_2 A_2\times_3 A_3$, then, under column-major vectorization, $$ \operatorname{vec}(Y)=(A_3\otimes A_2\otimes A_1)\operatorname{vec}(X). $$ The matrix on the right has $(J_1J_2J_3)$ rows and $(I_1I_2I_3)$ columns.

To prove the identity, index a row of $A_3\otimes A_2\otimes A_1$ by $(j_3,j_2,j_1)$ and a column by $(i_3,i_2,i_1)$, consistent with the first index varying fastest in each vectorization. The corresponding Kronecker entry is $$ (A_3)_{j_3i_3}(A_2)_{j_2i_2}(A_1)_{j_1i_1}. $$ The $(j_1,j_2,j_3)$ entry of $Y$, which is the corresponding entry of $\operatorname{vec}(Y)$, is the sum over all $i_1,i_2,i_3$ of this product times $X_{i_1i_2i_3}$. That is precisely the indicated row of the Kronecker matrix multiplied by $\operatorname{vec}(X)$. The entries agree at every index, proving the identity. $\square$

The unfolding and vectorization formulas are compatible. In particular, for a mode-1 transformation alone, $\operatorname{vec}(Y)=\operatorname{vec}(A_1X_{(1)})$, and column-major matrix vectorization gives $(I_{I_2I_3}\otimes A_1)\operatorname{vec}(X_{(1)})$. Under our convention $\operatorname{vec}(X_{(1)})=\operatorname{vec}(X)$, and the identity factor corresponds to the untransformed modes. This is the same action as $I_{I_3}\otimes I_{I_2}\otimes A_1$.

Worked Example: Verifying the Full Vectorization Identity

Use the tensor and matrices from the previous example. Its vectorization is $$ \operatorname{vec}(X)=(1,3,2,4,5,7,6,8)^T. $$ The full Kronecker identity specializes to $$ \operatorname{vec}(Y)=(A_3\otimes I_2\otimes A_1)\operatorname{vec}(X). $$ Rather than treating this as a dot product with one row, group the entries by the mode-1 unfolding. The unfolding calculation already gives $$ Y_{(1)}=\begin{pmatrix}4&6&-12&-14\\1&2&-5&-6\end{pmatrix}. $$ Because the columns of $Y_{(1)}$ are ordered by $(j_2,j_3)$, vectorizing this matrix column by column yields $$ \operatorname{vec}(Y) =(4,1,6,2,-12,-5,-14,-6)^T. $$ This is the result of the full Kronecker action. The grouping matters: each two-entry block corresponds to one column of the unfolding, not to an arbitrary consecutive block in a differently ordered tensor convention.

Worked Example: Transforming Mode 2

Keep $X$ as above, set $A_1=I_2$, $A_3=I_2$, and take $A_2=\begin{pmatrix}1&1\\0&1\end{pmatrix}$. The mode-2 unfolding has rows indexed by $i_2$, and columns ordered by $(i_1,i_3)$: $$ X_{(2)}=\begin{pmatrix}1&3&5&7\\2&4&6&8\end{pmatrix}. $$ Since the other two factors are identities, the mode-2 unfolding formula reduces to $Y_{(2)}=A_2X_{(2)}$. Direct multiplication gives $$ Y_{(2)} =\begin{pmatrix}1&1\\0&1\end{pmatrix} \begin{pmatrix}1&3&5&7\\2&4&6&8\end{pmatrix} =\begin{pmatrix}3&7&11&15\\2&4&6&8\end{pmatrix}. $$ For a direct entry check, $Y_{1,1,1}=X_{1,1,1}+X_{1,2,1}=1+2=3$, the first entry of the first row. Also, $Y_{2,2,2}=X_{2,2,2}=8$, the last entry of the second row. These checks confirm both the transformation and the stipulated column ordering.

Composition, Commutation, and Dimension Checks

Two useful consequences follow from the entrywise definition. Applying $B$ and then $A$ along the same mode is equivalent to one mode product by $AB$, when the dimensions match: $(X\times_n B)\times_n A=X\times_n(AB)$. Indeed, the two sums over the original and intermediate mode indices combine to the matrix product $AB$. Products in distinct modes commute because their finite sums act on different indices; interchanging the order does not change the summand or its range.

These facts explain why the all-modes identity can be written without specifying an order for products across distinct modes. They also provide a practical check: if $A_n$ has $J_n$ rows and $I_n$ columns, then the transformed tensor must have dimension $J_n$ in mode $n$. Accordingly, each unfolding formula must have $J_n$ rows, while its right Kronecker factor must have $\prod_{m\ne n}J_m$ rows and $\prod_{m\ne n}I_m$ columns before transposition.

Common Pitfall: Reversing Factors or Mixing Index Orders

A frequent error is to write $A_2\otimes A_3$ in the mode-1 unfolding formula without checking which index varies fastest among the columns. With the convention used here, those columns are ordered by $(i_2,i_3)$, so the factor is $A_3\otimes A_2$. Reversing the factors generally permutes the column coordinates and does not give the same transformation.

A related mistake is to multiply a row of a mode matrix by the entire vectorization and present the result as though it were a vector. The dimensions must agree, and the vectorized entries must be grouped according to the relevant unfolding before using its identity. The safe procedure is to state the unfolding convention, write the unfolding with its dimensions, apply the matrix formula, and only then vectorize.

Key takeaway: With first-index-fastest vectorization and the stated three-way unfolding conventions, $Y_{(n)}=A_nX_{(n)}(\text{Kronecker product of the other mode matrices in reverse index order})^T$, while $\operatorname{vec}(Y)=(A_3\otimes A_2\otimes A_1)\operatorname{vec}(X)$. Index order determines factor order.

Check Your Understanding

Use the conventions in this tutorial when answering.

If $X$ has dimensions $I_1\times I_2\times I_3$ and $A_2$ is $J_2\times I_2$, what are the dimensions of $X\times_2 A_2$?
Write the mode-3 unfolding identity when all three modes are transformed, and explain why its right Kronecker factor is ordered as shown.
For column-major vectorization, which factor is rightmost in $A_3\otimes A_2\otimes A_1$, and what indexing choice determines that position?
Explain why reversing the factors in $A_3\otimes A_2$ generally changes the mode-1 unfolding calculation.
For the mode-2 example, verify one additional entry of $Y_{(2)}$ from the defining mode-product sum.