Multinomial Means and Variances from Indicators

Why Represent Counts by Indicators?

A multinomial experiment records how many of $n$ trials fall in each of $k$ categories. A cell count is a sum over trials, but it is often easier to study that sum one trial at a time. An indicator records whether a particular trial landed in a particular category; its expectation and variance are simple, and summing the indicators recovers the count.

This representation also makes cross-moments transparent. For two different categories, one trial cannot contribute to both counts, while two different trials can contribute to the two counts separately. Keeping those two cases distinct is the key to the formulas.

Trial-Level Indicators

Suppose $n$ independent trials have the same categorical distribution on $k$ categories. Write $p_i$ for the probability of category $i$, where $p_i\geq 0$ and $\sum_{i=1}^k p_i=1$. Let $N_i$ be the number of trials in category $i$. For trial $r$ and category $i$, define the indicator $I_{ri}$ to equal 1 if trial $r$ falls in category $i$, and 0 otherwise.

Definition: The category indicator for trial $r$ and category $i$ is $I_{ri}=\mathbf{1}\{\text{trial }r\text{ is in category }i\}$. It satisfies $P(I_{ri}=1)=p_i$, $P(I_{ri}=0)=1-p_i$, and $$ N_i=\sum_{r=1}^n I_{ri}. $$ For a fixed trial $r$, the indicators $I_{r1},\ldots,I_{rk}$ are mutually exclusive: at most one can equal 1. Indicators from different trials are independent.

Each $I_{ri}$ is a Bernoulli random variable with success probability $p_i$. The indicators across trials for a fixed category are independent because the trials are independent. Indicators for different categories on the same trial are not independent: if one equals 1, the others equal 0. This distinction will determine the variance and cross-moment calculations.

Worked Example: Writing Three Cell Counts as Indicator Sums

A trial has outcomes A, B, or C with probabilities $0.2$, $0.3$, and $0.5$. In five independent trials, define $I_{rA}$ to indicate outcome A on trial $r$. Then the A count is $N_A=I_{1A}+I_{2A}+I_{3A}+I_{4A}+I_{5A}$. Each term is 1 with probability $0.2$ and 0 with probability $0.8$, so $E[I_{rA}]=0.2$.

On any one trial, $I_{rA}I_{rB}=0$, since the trial cannot have both outcomes A and B. On two distinct trials, the indicators are independent; for example, $E[I_{1A}I_{2B}]=(0.2)(0.3)=0.06$. A count product $N_A N_B$ sums contributions of precisely this latter kind: the same-trial terms are zero, and the different-trial terms can be nonzero.

Means and Variances of Individual Counts

For a Bernoulli indicator, $I_{ri}^2=I_{ri}$, since its only possible values are 0 and 1. In particular, $E[I_{ri}]=p_i$ and $\operatorname{Var}(I_{ri})=p_i(1-p_i)$. Linearity of expectation gives the count mean directly; independence across trials gives its variance.

Theorem: For $n$ independent, identically distributed categorical trials with category probabilities $p_1,\ldots,p_k$, each cell count satisfies $$ E[N_i]=np_i,\qquad \operatorname{Var}(N_i)=np_i(1-p_i). $$ These formulas hold when some $p_i$ is 0 or 1.

Proof. From $N_i=\sum_{r=1}^n I_{ri}$ and linearity,

E[N_i]=\sum_{r=1}^n E[I_{ri}]=\sum_{r=1}^n p_i=np_i.

For the variance, the indicators $I_{1i},\ldots,I_{ni}$ are independent, so the variance of their sum is the sum of their variances. Each has variance $p_i(1-p_i)$, hence

\operatorname{Var}(N_i) =\sum_{r=1}^n\operatorname{Var}(I_{ri}) =np_i(1-p_i).

No division by $p_i$ or $1-p_i$ was used, so the calculation includes the boundary cases. $\square$

Worked Example: Mean and Variance for Each Cell

Suppose $n=10$ trials have category probabilities $p_1=0.2$, $p_2=0.3$, and $p_3=0.5$. For category 1,

E[N_1]=10(0.2)=2,\qquad \operatorname{Var}(N_1)=10(0.2)(0.8)=1.6.

For category 2,

E[N_2]=10(0.3)=3,\qquad \operatorname{Var}(N_2)=10(0.3)(0.7)=2.1.

For category 3,

E[N_3]=10(0.5)=5,\qquad \operatorname{Var}(N_3)=10(0.5)(0.5)=2.5.

The expected counts add to $2+3+5=10$, as they must because the counts always sum to 10. The variances do not add to a variance of 10: the counts are dependent, and their covariances matter when taking the variance of their sum.

Second Moments and Cross-Moments

A cross-moment here means the raw product expectation $E[N_iN_j]$, not the covariance. To calculate it, expand both counts into indicators. If $i\neq j$, terms from the same trial vanish because the categories are mutually exclusive. Terms from different trials have expectation $p_i p_j$ by independence.

Theorem: Under the same multinomial-trial assumptions, the raw second and cross-moments are $$ E[N_i^2]=np_i+n(n-1)p_i^2 $$ and, for $i\neq j$, $$ E[N_iN_j]=n(n-1)p_i p_j. $$ Consequently, $\operatorname{Cov}(N_i,N_j)=-np_i p_j$ for $i\neq j$.

Proof. For one count, expand its square:

N_i^2 =\left(\sum_{r=1}^n I_{ri}\right)^2 =\sum_{r=1}^n I_{ri}^2+\sum_{\substack{r,s=1\\r\neq s}}^n I_{ri}I_{si}.

The second sum is over ordered pairs $(r,s)$: it contains $n(n-1)$ terms. Since $I_{ri}^2=I_{ri}$, the expected value of the first sum is $np_i$. For each term in the second sum, $r\neq s$, so independence across trials gives $E[I_{ri}I_{si}]=p_i^2$. Therefore $E[N_i^2]=np_i+n(n-1)p_i^2$.

For distinct categories $i\neq j$, expand the product:

N_iN_j=\sum_{r=1}^n\sum_{s=1}^n I_{ri}I_{sj}.

The $n$ terms with $r=s$ are zero, because one trial cannot be in both categories. There are $n(n-1)$ terms with $r\neq s$; each has expectation $p_i p_j$ by independence across trials. Thus $E[N_iN_j]=n(n-1)p_i p_j$. Subtracting $E[N_i]E[N_j]=(np_i)(np_j)$ yields

\operatorname{Cov}(N_i,N_j) =n(n-1)p_i p_j-n^2p_i p_j =-np_i p_j.

This also recovers the covariance formula for distinct multinomial counts. $\square$

Worked Example: From a Cross-Moment to a Covariance

Let $n=10$, $p_1=0.2$, and $p_2=0.3$. The cross-moment of the first two counts is

$$ E[N_1N_2]=10(9)(0.2)(0.3)=5.4. $$

Their means are $E[N_1]=2$ and $E[N_2]=3$, so their product is $E[N_1]E[N_2]=6$. Therefore

\operatorname{Cov}(N_1,N_2)=E[N_1N_2]-E[N_1]E[N_2]=5.4-6=-0.6.

The direct covariance formula gives the same result: $-np_1p_2=-10(0.2)(0.3)=-0.6$. The raw cross-moment is positive, while the covariance is negative; these quantities answer different questions and should not be interchanged.

Worked Example: The Same-Cell Second Moment

For $n=10$ and $p_1=0.2$, the first cell count has mean 2 and variance 1.6. Its second moment can be computed from the indicator expansion:

$$ E[N_1^2] =10(0.2)+10(9)(0.2)^2 =2+3.6 =5.6. $$

As a check, the identity $E[N_1^2]=\operatorname{Var}(N_1)+(E[N_1])^2$ gives $1.6+2^2=5.6$, the same answer. This check is useful because a variance is a centered second moment, whereas $E[N_1^2]$ is not centered.

Boundary Cases and Interpretation

The indicator derivation remains valid at the edges of the probability range. If $p_i=0$, then every $I_{ri}=0$ with probability 1, so $N_i=0$ with probability 1; the formulas give mean and variance both equal to 0. If $p_i=1$, all trials fall in category $i$, so $N_i=n$ with probability 1; the formulas give mean $n$ and variance 0. No special limiting argument is needed.

The negative covariance between different cell counts reflects the fixed total $N_1+\cdots+N_k=n$. A trial assigned to one category is unavailable to the others. This is not independence: even though trials are independent of one another, counts formed from the same set of trials are generally dependent.

Common Mistake

Wrong reasoning: “The trials are independent, so the different category counts are independent.” Independence applies to distinct trials, not to different category indicators on the same trial or to the resulting counts. In fact, for $i\neq j$, $\operatorname{Cov}(N_i,N_j)=-np_i p_j$, which is negative when both probabilities are positive.
Correction: When expanding a product of counts, separate terms from the same trial from terms from different trials. Same-trial cross-category terms are zero; different-trial terms factor by independence.
Another frequent error: Counting only the $n(n-1)/2$ unordered pairs of distinct trials in the cross-moment. The expansion of $N_iN_j$ has ordered pairs: a trial contributing to category $i$ and a different trial contributing to category $j$ is a distinct term from the reversed assignment. There are $n(n-1)$ such terms.

Key takeaway: Write $N_i=\sum_{r=1}^n I_{ri}$. Linearity yields $E[N_i]=np_i$; independence across trials yields $\operatorname{Var}(N_i)=np_i(1-p_i)$; and separating same-trial from different-trial terms yields the second and cross-moments.

Check Your Understanding

Try these questions using the indicator representation.

For $n=8$ and $p_i=0.25$, calculate $E[N_i]$ and $\operatorname{Var}(N_i)$.
For distinct categories with $n=6$, $p_i=0.2$, and $p_j=0.5$, calculate $E[N_iN_j]$.
Why are the same-trial terms in the expansion of $N_iN_j$ zero when $i\neq j$?
Explain why the cross-moment $E[N_iN_j]$ is not the same quantity as $\operatorname{Cov}(N_i,N_j)$.
What do the indicator formulas imply about $N_i$ when $p_i=1$?