Categorical Trials and Multinomial Counts

From Two Outcomes to Several Categories

A Bernoulli trial records one of two possibilities, often called success and failure. Many experiments have more than two possible outcomes: a patient may have no response, a partial response, or a complete response; a manufactured item may fall into one of several quality grades. In each case, one trial produces exactly one category. Repeating the trial produces a sequence of outcomes, while counting how often each category occurs gives a compact summary of that sequence.

The distinction between a sequence and its counts is important. The sequence retains the order in which outcomes occurred. The count vector records how many times each category occurred, but not when. This tutorial defines that count representation and the repeated-trial model that produces it.

A Categorical Trial and Its Count Vector

Definition: A categorical trial has one outcome in a finite set of \(k\) mutually exclusive and exhaustive categories, labelled \(1,\ldots,k\). Write \(Y\) for the category observed on one trial, and let \(p_j=P(Y=j)\). The category probabilities satisfy \(p_j\geq 0\) and \(\sum_{j=1}^{k}p_j=1\).

“Mutually exclusive” means that a single trial cannot produce two categories at once. “Exhaustive” means that every possible outcome belongs to one of the listed categories. The labels are just identifiers: category 2 need not be larger, better, or more likely than category 1.

Definition: For \(n\) independent categorical trials with the same category probabilities \(p_1,\ldots,p_k\), write \(Y_i\) for the category observed on trial \(i\). The count in category \(j\) is \(X_j=\sum_{i=1}^{n} I(Y_i=j)\), where \(I(A)\) equals 1 when \(A\) occurs and 0 otherwise. The vector \(X=(X_1,\ldots,X_k)\) is the vector of category counts, or the multinomial count vector.

The assumptions in this definition are separate. Independence concerns whether one trial’s outcome gives information about another. Having the same category probabilities means that each trial uses the same probability vector \((p_1,\ldots,p_k)\). Neither assumption requires all categories to be equally likely. The model is called the multinomial model with sample size \(n\) and category probabilities \(p_1,\ldots,p_k\).

What the Count Vector Records

A count vector has one entry for each category, in the same order as the category labels. For example, if the labels are “no response,” “partial response,” and “complete response,” then \(X=(3,2,1)\) means that three trials produced no response, two produced a partial response, and one produced a complete response. It does not say which patients had each outcome or the order in which the outcomes were recorded.

The count vector is formed from the ordered trial outcomes by adding one indicator per trial to the entry for that trial’s category. This representation gives an immediate accounting identity.

Theorem: For \(n\) categorical trials, the category counts satisfy \(\sum_{j=1}^{k}X_j=n\) for every possible sequence of outcomes.

Proof. For each trial \(i\), exactly one of the events \(Y_i=1,\ldots,Y_i=k\) occurs, because the categories are mutually exclusive and exhaustive. Therefore exactly one of the indicators \(I(Y_i=j)\), as \(j\) ranges from 1 to \(k\), is 1; the others are 0. Hence \(\sum_{j=1}^{k}I(Y_i=j)=1\). Summing this identity over all trials and rearranging finite sums gives

\sum_{j=1}^{k}X_j =\sum_{j=1}^{k}\sum_{i=1}^{n}I(Y_i=j) =\sum_{i=1}^{n}\sum_{j=1}^{k}I(Y_i=j) =\sum_{i=1}^{n}1 =n.

This is an exact identity, not an expected-value statement: whatever the observed outcomes, the counts account for all \(n\) trials. \(\square\)

Worked Examples

Worked Example: Counting Clinical Response Categories

A study records one response category for each of six patients: no response, partial response, or complete response. The recorded sequence is no response, partial response, complete response, no response, no response, partial response. Find the count vector in that category order.

There are six trials and three categories. “No response” occurs in positions 1, 4, and 5, so its count is 3. “Partial response” occurs in positions 2 and 6, so its count is 2. “Complete response” occurs in position 3, so its count is 1. Thus \(X=(3,2,1)\). The total is \(3+2+1=6\), as required.

If each patient’s outcome is independent of the others and the category probabilities are \(0.5, 0.3,\) and \(0.2\), the probability of this particular ordered sequence is \(0.5(0.3)(0.2)(0.5)(0.5)(0.3)=0.00225\). This calculation concerns the displayed order of six outcomes; it is not, by itself, the probability of observing the count vector \((3,2,1)\).

Worked Example: Unequal Category Probabilities

A quality check assigns each item exactly one label: pass, minor defect, major defect, or reject. Suppose five inspected items have labels minor defect, pass, pass, reject, pass. List the count vector in the stated category order and find the probability of this ordered sequence if the independent trials have probabilities \(0.82, 0.10, 0.06,\) and \(0.02\), respectively.

The pass count is 3, the minor-defect count is 1, the major-defect count is 0, and the reject count is 1. In the order pass, minor defect, major defect, reject, the count vector is \(X=(3,1,0,1)\), and its entries sum to 5.

For the ordered sequence shown, independence gives the product of the probability for each observed label: \(0.10(0.82)(0.82)(0.02)(0.82)=0.10(0.82)^3(0.02)=0.001102736\). The zero in the major-defect entry means that no inspected item received that label; it does not mean that the probability of that label is zero. In this model, its probability is \(0.06\).

Worked Example: Counts Do Not Preserve Order

A survey records a respondent’s preferred contact method as web, phone, or in person. For seven respondents, suppose the sequence is phone, web, web, in person, phone, web, in person. Give the count vector in the order web, phone, in person, and compare it with the sequence web, phone, in person, web, phone, web, in person.

In the first sequence, web occurs 3 times, phone occurs 2 times, and in person occurs 2 times, so \(X=(3,2,2)\). In the second sequence, those same categories occur 3, 2, and 2 times, respectively. Its count vector is also \((3,2,2)\), even though the order differs.

Thus the count vector is a summary of category frequencies, not a complete record of the trial outcomes. If the order itself matters—for example, if outcomes are recorded over time—the count vector alone discards that information.

Probability of a Specified Ordered Sequence

Independence makes the probability of a specified sequence especially direct. This result concerns the ordered outcomes \(Y_1,\ldots,Y_n\), rather than the probability of a count vector.

Theorem: Suppose \(Y_1,\ldots,Y_n\) are independent categorical trials with common category probabilities \(p_1,\ldots,p_k\). For any specified labels \(a_1,\ldots,a_n\) in \(\{1,\ldots,k\}\), \(P(Y_1=a_1,\ldots,Y_n=a_n)=\prod_{i=1}^{n}p_{a_i}\).

Proof. By independence, the events \(\{Y_i=a_i\}\), for \(i=1,\ldots,n\), have joint probability equal to the product of their individual probabilities. Since every trial has the same category probabilities, \(P(Y_i=a_i)=p_{a_i}\) for each \(i\). Substituting these marginal probabilities into the independence product yields

P(Y_1=a_1,\ldots,Y_n=a_n) =\prod_{i=1}^{n}P(Y_i=a_i) =\prod_{i=1}^{n}p_{a_i}.

If a specified label appears more than once, its probability appears as a factor for each occurrence. For example, a sequence with three pass outcomes contributes three factors of the pass probability. This calculation does not yet combine the probabilities of all ordered sequences that share a count vector. \(\square\)

Checking the Model and Avoiding Common Mistakes

Before using a multinomial count model, check the experiment at the level of individual trials. Each trial must yield exactly one category; the trials must be independent; and the category probabilities must remain the same from trial to trial. These conditions determine whether the model applies. A categorical outcome with many possible labels is not automatically a repeated independent-trial model.

Mistake: treating a count vector as the observed sequence. The vector records frequencies only. Different orders can produce exactly the same vector.
Mistake: assuming categories must be equally likely. A categorical trial may have unequal probabilities, provided the probabilities are nonnegative and sum to 1.
Mistake: allowing a trial to enter multiple categories. The categories must describe mutually exclusive outcomes. If one patient can be counted in both “partial response” and “adverse event,” those labels are not mutually exclusive categories for a single categorical outcome.
Mistake: overlooking the repeated-trial assumptions. If the probability of an outcome changes between trials, or if outcomes are dependent, the independent, common-probability model defined here does not describe the experiment as stated.

The two-category case connects to the binomial model: labeling one category “success” and the other “failure” makes the count in the success category a binomial count under the corresponding independent, common-probability assumptions. With more categories, the count vector keeps track of all category totals at once. In either case, first define what one trial is and confirm the assumptions before interpreting the counts.

Key takeaway: A multinomial count vector summarizes \(n\) independent trials with the same categorical probability vector. Its entries count the occurrences of each category, and their sum is exactly \(n\); the vector does not preserve the order of outcomes.

Check Your Understanding

Use the category order stated in each question, and distinguish ordered sequences from their count summaries.

A trial has categories A, B, and C. In five independent trials the outcomes are C, A, B, C, A. What is the count vector, and what must the sum of its entries equal?
For one categorical trial with four categories, what conditions must the category probabilities satisfy?
Two sequences have the same numbers of each category but a different order. Must their count vectors differ? Explain.
Suppose the probability of a specified label on trial \(i\) is \(p_{a_i}\), and the trials are independent. What is the probability of the specified ordered sequence \(a_1,\ldots,a_n\)?
A patient can be recorded in both of two proposed labels on a single trial. What condition on categories does this violate?