Hidden Markov Models as Joint Probability Models

States That Are Hidden and Observations That Are Seen

In many sequential-data problems, the quantity that evolves over time is not observed directly. A clinical condition may change between visits while only test results are recorded; a physical system may switch operating modes while producing noisy measurements. A hidden Markov model (HMM) gives a probability model for both sequences: a Markov chain describes the hidden states, and an emission distribution describes each observation conditional on its current state.

The central object here is the joint probability of a particular hidden-state sequence and a particular observation sequence. It is built from three ingredients: the probability of the initial hidden state, the transition probabilities between successive hidden states, and the probabilities of the observations under their corresponding states. Keeping those factors distinct is essential: transitions govern states, while emissions govern observations.

Definition and Factorization

Work on a probability space $(\Omega,\mathcal{F},\mathbb{P})$. For a sequence of length $T\ge1$, let $X_1,\ldots,X_T$ be hidden states in a finite set $S$, and let $Y_1,\ldots,Y_T$ be observations in a finite alphabet $\mathcal{Y}$. Write $\mathcal{F}_t=\sigma(X_1,\ldots,X_t,Y_1,\ldots,Y_t)$ for the information generated by the state and observation history through time $t$, with $\mathcal{F}_0$ the trivial sigma-field. The HMM is specified by an initial-state distribution $\pi=(\pi_i)_{i\in S}$, a row-stochastic transition matrix $P=(p_{ij})_{i,j\in S}$, and emission probabilities $b_i(y)$, for $i\in S$ and $y\in\mathcal{Y}$.

Definition: A finite-state HMM has $\pi_i=\mathbb{P}(X_1=i)$, $p_{ij}=\mathbb{P}(X_{t+1}=j\mid X_t=i)$, and $b_i(y)=\mathbb{P}(Y_t=y\mid X_t=i)$. The state process is Markov: its next-state distribution given the past depends only on its current state. Given the hidden-state sequence, the observations are conditionally independent, and $Y_t$ has distribution $b_{X_t}$. Thus, when the relevant conditioning event has positive probability, the conditional law of the next state and observation given the past is $p_{X_t,j}b_j(y)$ for the pair $(X_{t+1},Y_{t+1})=(j,y)$. At time 1 the corresponding joint probability is $\pi_i b_i(y)$.

Each distribution must be valid: $\sum_{i\in S}\pi_i=1$, every row of $P$ sums to 1, and $\sum_{y\in\mathcal{Y}}b_i(y)=1$ for every state $i$. All entries are nonnegative. Conditioning on an event of probability zero does not define a unique conditional probability; the HMM’s joint probabilities are specified by its initial distribution and conditional probability rules on events where those rules are meaningful.

Theorem (Joint path-observation probability): For $x_1,\ldots,x_T\in S$ and $y_1,\ldots,y_T\in\mathcal{Y}$, the HMM joint probability is $$ \mathbb{P}(X_1=x_1,\ldots,X_T=x_T,\;Y_1=y_1,\ldots,Y_T=y_T) = \pi_{x_1}b_{x_1}(y_1)\prod_{t=2}^{T}p_{x_{t-1}x_t}b_{x_t}(y_t). $$ For $T=1$, the product over $t=2,\ldots,T$ is an empty product, which by convention equals 1.

Proof. At time 1, the model specifies $\mathbb{P}(X_1=x_1,Y_1=y_1)=\pi_{x_1}b_{x_1}(y_1)$. Suppose the joint probability through time $t$ has the displayed product form. Given a history through $t$ with positive probability, the HMM rule assigns conditional probability $p_{x_t x_{t+1}}b_{x_{t+1}}(y_{t+1})$ to the next state-observation pair. The chain rule for probabilities therefore multiplies the probability through $t$ by this factor to obtain the joint probability through $t+1$. If the history has probability zero, every extension of that history also has probability zero, and the product through $t$ already contains a zero factor. Induction from $t=1$ to $T$ proves the formula, including paths with zero probability. $\square$

The formula can also be read in two parts. The hidden path has probability $\pi_{x_1}\prod_{t=2}^{T}p_{x_{t-1}x_t}$; conditional on that path, the observation sequence has probability $\prod_{t=1}^{T}b_{x_t}(y_t)$, when the path has positive probability. Multiplying gives the joint probability. If a hidden path has probability zero, its joint probability with any observation sequence is zero as well. This interpretation follows directly from the model’s Markov and conditional-independence assumptions.

Worked Joint-Probability Calculations

Worked Example: A Two-State Path and Three Binary Observations

Let $S=\{A,B\}$, with initial distribution $\pi=(\tfrac23,\tfrac13)$ and transition matrix $$ P=\begin{bmatrix} \tfrac34&\tfrac14\\ \tfrac12&\tfrac12 \end{bmatrix}. $$ The row sums are $\tfrac34+\tfrac14=1$ and $\tfrac12+\tfrac12=1$; the initial probabilities sum to $\tfrac23+\tfrac13=1$. Let $\mathcal{Y}=\{0,1\}$, with emissions $b_A(0)=\tfrac34,\ b_A(1)=\tfrac14,\ b_B(0)=\tfrac13,\ b_B(1)=\tfrac23$. The emission sums are $\tfrac34+\tfrac14=1$ and $\tfrac13+\tfrac23=1$.

Find the joint probability of hidden path $(A,B,B)$ and observation sequence $(0,1,0)$. The factors, in time order, are the initial probability of $A$, the emission of 0 from $A$, the transition $A$ to $B$, the emission of 1 from $B$, the transition $B$ to $B$, and the emission of 0 from $B$:

\begin{aligned} \mathbb{P}(A,B,B;\,0,1,0) &=\tfrac23\cdot\tfrac34\cdot\tfrac14\cdot\tfrac23\cdot\tfrac12\cdot\tfrac13\\ &=\tfrac12\cdot\tfrac14\cdot\tfrac23\cdot\tfrac12\cdot\tfrac13\\ &=\tfrac18\cdot\tfrac23\cdot\tfrac12\cdot\tfrac13\\ &=\tfrac1{12}\cdot\tfrac12\cdot\tfrac13\\ &=\tfrac1{24}\cdot\tfrac13 =\tfrac1{72}. \end{aligned}

Check by grouping factors: the hidden-path probability is $\tfrac23\cdot\tfrac14\cdot\tfrac12=\tfrac1{12}$. The conditional observation probability for that path is $\tfrac34\cdot\tfrac23\cdot\tfrac13=\tfrac16$. Their product is $\tfrac1{12}\cdot\tfrac16=\tfrac1{72}$, as required.

Worked Example: A Three-State Model

Let $S=\{1,2,3\}$, $\pi=(\tfrac12,\tfrac13,\tfrac16)$, and $$ P=\begin{bmatrix} \tfrac12&\tfrac13&\tfrac16\\ \tfrac14&\tfrac12&\tfrac14\\ \tfrac13&\tfrac13&\tfrac13 \end{bmatrix}. $$ The transition row sums are $\tfrac12+\tfrac13+\tfrac16=1$, $\tfrac14+\tfrac12+\tfrac14=1$, and $\tfrac13+\tfrac13+\tfrac13=1$; the entries of $\pi$ sum to $\tfrac12+\tfrac13+\tfrac16=1$. For alphabet $\{a,b\}$, take emission probabilities $(b_1(a),b_1(b))=(\tfrac23,\tfrac13)$, $(b_2(a),b_2(b))=(\tfrac14,\tfrac34)$, and $(b_3(a),b_3(b))=(\tfrac12,\tfrac12)$. Each emission pair sums to 1.

For hidden path $(2,3,1)$ and observations $(b,a,b)$, the initial factor is $\pi_2=\tfrac13$; the two transition factors are $p_{23}=\tfrac14$ and $p_{31}=\tfrac13$; and the emission factors are $b_2(b)=\tfrac34$, $b_3(a)=\tfrac12$, and $b_1(b)=\tfrac13$. Hence

\begin{aligned} \mathbb{P}(2,3,1;\,b,a,b) &=\tfrac13\cdot\tfrac34\cdot\tfrac14\cdot\tfrac12\cdot\tfrac13\cdot\tfrac13\\ &=\tfrac14\cdot\tfrac14\cdot\tfrac12\cdot\tfrac13\cdot\tfrac13\\ &=\tfrac1{16}\cdot\tfrac12\cdot\tfrac13\cdot\tfrac13\\ &=\tfrac1{32}\cdot\tfrac13\cdot\tfrac13 =\tfrac1{96}\cdot\tfrac13 =\tfrac1{288}. \end{aligned}

Check by grouping factors: the path probability is $\tfrac13\cdot\tfrac14\cdot\tfrac13=\tfrac1{36}$. The emission product is $\tfrac34\cdot\tfrac12\cdot\tfrac13=\tfrac18$. Thus the joint probability is $\tfrac1{36}\cdot\tfrac18=\tfrac1{288}$.

Worked Example: A Zero Emission Probability

Let $S=\{C,W\}$, $\pi=(\tfrac13,\tfrac23)$, and $$ P=\begin{bmatrix} \tfrac23&\tfrac13\\ \tfrac14&\tfrac34 \end{bmatrix}. $$ The transition rows sum to $\tfrac23+\tfrac13=1$ and $\tfrac14+\tfrac34=1$, and the initial probabilities sum to $\tfrac13+\tfrac23=1$. Let $\mathcal{Y}=\{\text{red},\text{blue}\}$, with $b_C(\text{red})=1,\ b_C(\text{blue})=0$, and $b_W(\text{red})=\tfrac13,\ b_W(\text{blue})=\tfrac23$. Both emission distributions sum to 1.

Compute the joint probability for path $(W,C,C)$ and observations $(\text{blue},\text{red},\text{red})$:

\begin{aligned} \mathbb{P}(W,C,C;\,\text{blue},\text{red},\text{red}) &=\tfrac23\cdot\tfrac23\cdot\tfrac14\cdot1\cdot\tfrac23\cdot1\\ &=\tfrac49\cdot\tfrac14\cdot\tfrac23\\ &=\tfrac19\cdot\tfrac23 =\tfrac2{27}. \end{aligned}

Check by grouping factors: the path probability is $\tfrac23\cdot\tfrac14\cdot\tfrac23=\tfrac19$. The conditional emission probability is $\tfrac23\cdot1\cdot1=\tfrac23$. Their product is $\tfrac19\cdot\tfrac23=\tfrac2{27}$. The zero emission $b_C(\text{blue})=0$ does not occur on this path, so it does not make this joint probability zero. If the observation at time 2 were blue instead, the factor $b_C(\text{blue})=0$ would make that path-observation joint probability zero.

What the Factorization Does—and Does Not—Say

The product gives a probability for a joint event: one specified state path together with one specified observation sequence. It does not, by itself, give the probability of the observations alone. That event does not specify the hidden states, so it requires accounting for the possible state paths; the calculation of sequence likelihoods is a separate step. Likewise, the product of emissions describes observations conditional on a specified state path, not necessarily independent observations without that conditioning.

Dependence can remain after the hidden states are unobserved. For example, if a state tends to persist, an observation that is informative about the current state can also provide information about nearby states. Consequently, two observations can be dependent even though the model makes them conditionally independent given the full hidden-state sequence. The conditioning clause is part of the HMM definition, not an optional simplification.

Common mistake: using the reverse transition. In the first example, the transition from the first hidden state $A$ to the second hidden state $B$ is $p_{AB}=\tfrac14$, not $p_{BA}=\tfrac12$. Replacing it with $p_{BA}$ gives the incorrect calculation $$ \tfrac23\cdot\tfrac34\cdot\tfrac12\cdot\tfrac23\cdot\tfrac12\cdot\tfrac13 =\tfrac1{36}. $$ The path is ordered $A,B,B$, so the relevant transition entries are $p_{AB}=\tfrac14$ and $p_{BB}=\tfrac12$. The corrected product is $$ \tfrac23\cdot\tfrac34\cdot\tfrac14\cdot\tfrac23\cdot\tfrac12\cdot\tfrac13 =\tfrac1{72}. $$ The error is not a matter of convention: $p_{ij}$ is the probability of moving from row state $i$ to column state $j$.

Before multiplying, check the model and the path: the initial probabilities, transition rows and emission distributions must each sum to 1; the path’s states must be in the stated order; and each observation factor must use the state at the same time. A zero initial, transition or emission factor makes the entire joint probability zero. If observations take values in a general measurable space rather than a finite alphabet, the same structure holds with emission probability distributions (or densities when densities exist) in place of the mass functions $b_i(y)$.

Check Your Understanding

For each question, identify the initial, transition and emission factors before simplifying.

For a path of length one, what does the joint probability formula reduce to, and why is there no transition factor?
In the first worked example, which transition probability corresponds to the move from the second state to the third state?
If $b_i(y)=0$ at one time on a specified path, what is the joint probability of that path and observation sequence?
Explain why conditional independence of observations given the hidden path does not assert their unconditional independence.
For a proposed HMM, what sums should be checked for the initial distribution, transition matrix and emission distributions?

Hidden Markov Models as Joint Probability Models

What You'll Learn