Kronecker Products and Their Algebra

Why Kronecker Products Appear in Matrix Work

Many multivariate calculations involve matrices that describe several variables or several measurements at once. The Kronecker product provides a structured way to combine two matrices: each entry of the first matrix is replaced by a scaled copy of the second. The result is a larger block matrix whose dimensions and algebra follow from those of its factors.

The operation is especially useful when matrix expressions involve repeated, parallel structure. For now, the essential skills are to construct the blocks correctly, track dimensions, and use the multiplication, transpose, and inverse rules only under their stated conditions.

Definition and Dimensions

Definition: Let \(A=(a_{ij})\) be an \(m\times n\) matrix and \(B\) a \(p\times q\) matrix. Their Kronecker product, written \(A\otimes B\), is the \(mp\times nq\) block matrix whose \((i,j)\) block is \(a_{ij}B\):

A\otimes B= \begin{bmatrix} a_{11}B & \cdots & a_{1n}B\\ \vdots & \ddots & \vdots\\ a_{m1}B & \cdots & a_{mn}B \end{bmatrix}.

Each block has \(p\) rows and \(q\) columns. There are \(m\) block rows and \(n\) block columns, so the full matrix has \(mp\) rows and \(nq\) columns.

The order matters: \(A\otimes B\) and \(B\otimes A\) are generally different matrices. Even when they have the same dimensions, their entries occupy different positions. The definition also applies when either factor is a row vector, column vector, or scalar matrix; dimensions always multiply in the same way.

Worked Example: Constructing the Blocks and Counting Dimensions

Let \(A=\begin{bmatrix}1&-2&0\\3&1&4\end{bmatrix}\) and \(B=\begin{bmatrix}1&2\\0&-1\end{bmatrix}\). The first factor is \(2\times3\), and the second is \(2\times2\), so \(A\otimes B\) must be \(4\times6\). Each entry of \(A\) scales the whole matrix \(B\):

A\otimes B= \begin{bmatrix} B&-2B&0B\\ 3B&B&4B \end{bmatrix} = \begin{bmatrix} 1&2&-2&-4&0&0\\ 0&-1&0&2&0&0\\ 3&6&1&2&4&8\\ 0&-3&0&-1&0&-4 \end{bmatrix}.

The displayed matrix has four rows and six columns, as required. For instance, its upper-right \(2\times2\) block is \(0B\), while its lower-left block is \(3B\). The block sizes come from \(B\); the number of blocks comes from \(A\).

Multiplication Rule

The multiplication rule requires compatible factor dimensions in both positions. Suppose \(A\) is \(m\times n\), \(C\) is \(n\times r\), \(B\) is \(p\times q\), and \(D\) is \(q\times s\). Then the two Kronecker products on the left can be multiplied, and the result has dimensions \(mp\times rs\), matching the dimensions on the right.

Theorem: For matrices with the compatible dimensions just stated,

(A\otimes B)(C\otimes D)=(AC)\otimes(BD).

To prove the rule, view \(A\otimes B\) as an \(m\times n\) array of \(p\times q\) blocks and \(C\otimes D\) as an \(n\times r\) array of \(q\times s\) blocks. The \((i,k)\) block of their product is the sum of the products of corresponding blocks:

\sum_{j=1}^{n}(a_{ij}B)(c_{jk}D) = \sum_{j=1}^{n}a_{ij}c_{jk}(BD) = \left(\sum_{j=1}^{n}a_{ij}c_{jk}\right)(BD) = (AC)_{ik}(BD).

This is exactly the \((i,k)\) block in \((AC)\otimes(BD)\). Since this holds for every block position, the matrices are equal. The block products are defined because \(B\) has \(q\) columns and \(D\) has \(q\) rows.

Worked Example: Multiplying Two Kronecker Products

Take \(A=\begin{bmatrix}1&2\\0&1\end{bmatrix}\), \(B=\begin{bmatrix}1&1\\0&1\end{bmatrix}\), \(C=\begin{bmatrix}2&0\\1&1\end{bmatrix}\), and \(D=\begin{bmatrix}1&0\\1&2\end{bmatrix}\). All four matrices are \(2\times2\), so both Kronecker products are \(4\times4\) and their product is defined. First calculate the two smaller products:

AC= \begin{bmatrix}1&2\\0&1\end{bmatrix} \begin{bmatrix}2&0\\1&1\end{bmatrix} = \begin{bmatrix}4&2\\1&1\end{bmatrix}, \qquad BD= \begin{bmatrix}1&1\\0&1\end{bmatrix} \begin{bmatrix}1&0\\1&2\end{bmatrix} = \begin{bmatrix}2&2\\1&2\end{bmatrix}.

The rule says to form \((AC)\otimes(BD)\), replacing each entry of \(AC\) by that entry times \(BD\):

(A\otimes B)(C\otimes D) = \begin{bmatrix} 4(BD)&2(BD)\\ BD&BD \end{bmatrix} = \begin{bmatrix} 8&8&4&4\\ 4&8&2&4\\ 2&2&2&2\\ 1&2&1&2 \end{bmatrix}.

The result is \(4\times4\), consistent with the product dimensions. The calculation uses ordinary multiplication within each factor and then constructs a new block matrix; it does not multiply the entries of \(A\) and \(B\) pairwise.

Transpose Rule

Theorem: For any conformable matrices \(A\) and \(B\),

(A\otimes B)^{\mathsf T}=A^{\mathsf T}\otimes B^{\mathsf T}.

To see why, the \((i,j)\) block of \(A\otimes B\) is \(a_{ij}B\). Transposing the full block matrix transposes each block and swaps its block-row and block-column positions. The transposed block is \((a_{ij}B)^{\mathsf T}=a_{ij}B^{\mathsf T}\), and it moves to position \((j,i)\). These are precisely the blocks of \(A^{\mathsf T}\otimes B^{\mathsf T}\). The dimensions also agree: both sides are \(nq\times mp\).

Worked Example: Transposing a Kronecker Product

Let \(A=\begin{bmatrix}1&2\\3&4\end{bmatrix}\) and \(B=\begin{bmatrix}0&1\\-1&2\end{bmatrix}\). The block construction gives

A\otimes B= \begin{bmatrix} 0&1&0&2\\ -1&2&-2&4\\ 0&3&0&4\\ -3&6&-4&8 \end{bmatrix}.

Transposing this \(4\times4\) matrix, or applying the rule to its factors, gives

(A\otimes B)^{\mathsf T} = \begin{bmatrix} 0&-1&0&-3\\ 1&2&3&6\\ 0&-2&0&-4\\ 2&4&4&8 \end{bmatrix} = \begin{bmatrix}1&3\\2&4\end{bmatrix} \otimes \begin{bmatrix}0&-1\\1&2\end{bmatrix} = A^{\mathsf T}\otimes B^{\mathsf T}.

The transpose changes both the arrangement of blocks and the entries inside each block. Transposing only the blocks, without swapping their positions, would not transpose the full matrix.

Inverse Rule

Theorem: If \(A\) and \(B\) are square and nonsingular, then

(A\otimes B)^{-1}=A^{-1}\otimes B^{-1}.

The conditions matter: \(A\) and \(B\) must each be square and have inverses. To prove the formula, apply the multiplication rule to the proposed inverse:

(A\otimes B)(A^{-1}\otimes B^{-1}) =(AA^{-1})\otimes(BB^{-1}) =I\otimes I =I.

The identity on the right has the required size because if \(A\) is \(m\times m\) and \(B\) is \(p\times p\), then \(I_m\otimes I_p=I_{mp}\). Reversing the order also gives \((A^{-1}A)\otimes(B^{-1}B)=I_{mp}\), so the proposed matrix is both a left and right inverse. This proves the rule.

Worked Example: Inverting a Kronecker Product

Let \(A=\begin{bmatrix}2&1\\1&1\end{bmatrix}\) and \(B=\begin{bmatrix}1&1\\0&1\end{bmatrix}\). Their inverses are \(A^{-1}=\begin{bmatrix}1&-1\\-1&2\end{bmatrix}\) and \(B^{-1}=\begin{bmatrix}1&-1\\0&1\end{bmatrix}\). For example, \(AA^{-1}=\begin{bmatrix}1&0\\0&1\end{bmatrix}\) and \(BB^{-1}=\begin{bmatrix}1&0\\0&1\end{bmatrix}\), so both factors are nonsingular. The product and its claimed inverse are

A\otimes B= \begin{bmatrix} 2&2&1&1\\ 0&2&0&1\\ 1&1&1&1\\ 0&1&0&1 \end{bmatrix}, \qquad A^{-1}\otimes B^{-1} = \begin{bmatrix} 1&-1&-1&1\\ 0&1&0&-1\\ -1&1&2&-2\\ 0&-1&0&2 \end{bmatrix}.

Rather than perform a long \(4\times4\) inverse calculation, verify the proposed inverse using the multiplication rule:

(A\otimes B)(A^{-1}\otimes B^{-1}) =(AA^{-1})\otimes(BB^{-1}) =I_2\otimes I_2 =I_4.

The matrices are \(4\times4\), as they must be: each factor in the Kronecker product is \(2\times2\). The inverse rule is efficient because it reduces the calculation to inverses of the smaller factors.

Common Pitfalls and Practical Checks

Kronecker products are straightforward once the block dimensions are kept visible. Most errors come from treating the operation as ordinary multiplication or applying a valid identity without checking its hypotheses.

Confusing multiplication with entrywise multiplication. In \(A\otimes B\), each entry of \(A\) scales a full copy of \(B\); it is not a pairwise product of entries.
Forgetting that dimensions multiply. An \(m\times n\) matrix combined with a \(p\times q\) matrix produces an \(mp\times nq\) matrix.
Using the multiplication rule with incompatible factors. The dimensions must allow both \(AC\) and \(BD\) to be formed. Compatibility of the two Kronecker products alone is not enough if the proposed factor products do not exist.
Transposing only the entries inside each block. A full transpose also swaps the block-row and block-column positions.
Inverting factors that are not invertible. The inverse rule requires both factors to be square and nonsingular; it cannot be used to manufacture an inverse when those conditions fail.

In multivariate statistics, matrices often encode transformations across variables and across repeated coordinates. Kronecker products let such structures be represented compactly, and the algebraic rules let calculations be reduced to smaller matrices. A reliable check is to write down the dimensions first, state which factor products are defined, and verify that both sides of an identity have the same size.

Check Your Understanding

Use the block definition and state dimensions as part of each answer.

If \(A\) is \(3\times2\) and \(B\) is \(4\times5\), what are the dimensions of \(A\otimes B\)?
Suppose \(A\) is \(m\times n\), \(C\) is \(n\times r\), \(B\) is \(p\times q\), and \(D\) is \(q\times s\). State the Kronecker multiplication rule and the dimensions of its result.
Explain why transposing \(A\otimes B\) requires both transposing the individual blocks and rearranging their positions.
What conditions on \(A\) and \(B\) permit the inverse rule, and what is the resulting expression?
For \(A=\begin{bmatrix}1&0\\2&-1\end{bmatrix}\) and \(B=\begin{bmatrix}3&1\end{bmatrix}\), determine the dimensions of \(A\otimes B\) and describe its four blocks.