Diagnosing Baum-Welch Fits with Restarts and Likelihood Traces

Why a Single Baum-Welch Fit Is Not Enough

Baum-Welch uses the expectation-maximization algorithm to improve an HMM parameter estimate, but it does not guarantee that different initial parameters lead to the same fit. The observed-data likelihood can have multiple local maxima, and some runs can leave a state with almost no posterior support. A useful diagnosis therefore compares several initializations, inspects each likelihood trace, and checks whether the fitted parameters still define valid probability distributions.

Let \((\Omega,\mathcal{F},\mathbb{P})\) be a probability space. Consider a discrete-time hidden Markov model with hidden process \((X_t)_{1\le t\le T}\) on a finite state space \(S=\{1,\ldots,K\}\), and observations \(Y_1,\ldots,Y_T\) in a finite alphabet \(\mathcal{Y}\). Its parameters are an initial row distribution \(\pi\), a row-stochastic transition matrix \(P=(p_{ij})\), and emission probabilities \(b_j(y)=\mathbb{P}(Y_t=y\mid X_t=j)\). The observed-data likelihood for a fixed sequence \(y_{1:T}\) is \(L(\theta)=\mathbb{P}_{\theta}(Y_1=y_1,\ldots,Y_T=y_T)\), where \(\theta=(\pi,P,b)\). When \(L(\theta)>0\), the log-likelihood is \(\ell(\theta)=\log L(\theta)\).

For categorical emissions, an admissible parameter must satisfy \(\pi_j\ge0\), \(\sum_j\pi_j=1\), \(p_{ij}\ge0\), \(\sum_jp_{ij}=1\) for every state \(i\), and \(b_j(y)\ge0\), \(\sum_{y\in\mathcal{Y}}b_j(y)=1\) for every state \(j\). In a Gaussian-emission HMM, the emission parameters instead need to satisfy the conditions of the chosen Gaussian model, including positive variance or a positive-definite covariance matrix. These checks are separate from asking whether the likelihood is large.

What a Likelihood Trace Can and Cannot Say

For an exact Baum-Welch update with a well-defined E-step and M-step, the observed-data likelihood does not decrease. This monotonicity result was established in the earlier material on why Baum-Welch does not decrease likelihood. Consequently, a substantial decrease in a computed trace is a warning to inspect the implementation, the likelihood calculation, and the update being used. Small decreases can also arise from rounding or approximate updates, so the numerical tolerance and implementation should be checked before concluding that the model is wrong.

A nondecreasing trace does not show that the run has found a global maximum. Nor does a flat trace alone prove convergence to a useful fit: parameters can still be changing in directions that barely affect the likelihood, or the model can have redundant states. The fitted likelihoods across restarts provide evidence about sensitivity to initialization, not a proof of global optimality. If two runs have different likelihoods, prefer the larger likelihood only after confirming both runs used the same data, model, and likelihood convention.

Proof. Write the likelihood as a sum over all hidden paths \(x_{1:T}\in S^T\): each summand is \(\pi_{x_1}b_{x_1}(y_1)\prod_{t=2}^T p_{x_{t-1}x_t}b_{x_t}(y_t)\). A permutation of \(S\) is a bijection from the set of paths \(S^T\) to itself. Under consistent relabeling, the summand for a path equals the original summand for its inverse-permuted path, since every initial, transition, and emission factor is unchanged apart from its state index. Reindexing the finite sum by this bijection gives the same likelihood. This also covers \(T=1\), where the transition product is empty, and \(K=1\), where the only permutation is the identity. \(\square\)

This label symmetry matters when interpreting restarts. Two fits can describe the same statistical model while assigning different names to the states. Compare likelihoods and state characteristics after accounting for relabeling; do not treat a mere swap of state labels as evidence of instability.

Checking Expected State Occupancy

The forward-backward smoothing probabilities give a direct diagnostic for weakly used states. At each time \(t\), the probabilities \(\gamma_t(j)\) sum to one, provided the observation sequence has positive likelihood. Summing over time gives \(N_j\), the expected number of visits to state \(j\) conditional on the observed data. The occupancy fraction \(N_j/T\) lies between zero and one. A very small fraction can indicate that a state is effectively unused in this fitted run, but it does not by itself show that the state is unnecessary for the scientific problem or that another model size is preferable.

Proof. Positive likelihood makes the conditional probabilities defining \(\gamma_t(j)\) meaningful. For each fixed \(t\), the events \(\{X_t=j\}\), \(j\in S\), are disjoint and their union is the whole hidden-state space. Therefore \(\sum_{j\in S}\gamma_t(j)=1\). Summing this equality over \(t=1,\ldots,T\) and interchanging two finite sums gives \(\sum_{j\in S}N_j=\sum_{t=1}^T\sum_{j\in S}\gamma_t(j)=\sum_{t=1}^T1=T\). Nonnegativity follows because conditional probabilities are nonnegative; hence \(N_j\le T\) as well. \(\square\)

Probability Constraints and Restart Comparisons

Constraint checks should be performed on the actual fitted parameters, not inferred from the likelihood trace. For a categorical HMM, inspect the initial vector, every transition row, and every emission distribution. Negative entries, a row sum different from one, or a non-finite value mean the fitted object is not an admissible probability model, even if a software routine returned a score. If parameters are rounded for display, check the unrounded values when possible; displayed row sums can differ slightly because of rounding.

For multiple restarts, record at least the final log-likelihood, the trace behavior, constraint-check results, and state occupancies. Compare all runs on the same data and under the same likelihood convention. If several runs reach comparable high likelihoods but their parameters differ substantially after matching state labels, report that sensitivity rather than selecting a single run without qualification. If one run has clearly lower likelihood, verify that its trace and constraints are sound before treating it as an inferior local solution.

Common mistake. A frequent error is to regard a nondecreasing trace as proof that the fit is valid and globally optimal. For example, the sequence of reported likelihoods \(\tfrac12,\tfrac34,\tfrac89\) increases, but the invalid transition matrix in the previous example still fails the row-sum condition. Monotonicity checks the direction of likelihood change, not whether the parameters define a probability model, whether the computation is correct, or whether another initialization reaches a better solution. Check these issues separately.

A practical diagnosis therefore asks three different questions: did the likelihood behave as expected; do the fitted parameters satisfy their probability constraints; and are all states meaningfully represented in the smoothed posterior occupancy? Agreement across these checks and across restarts supports a stable fit, but it does not establish that the model is scientifically appropriate. Interpretation still depends on the application and the assumptions built into the HMM.