Baum-Welch many possible observations

Question

I have implemented the baum-welch algorithm in python but I am now encountering a problem when attempting to train HMM (hidden markov model) parameters A , B , and pi . The problem is that I have many observation sequences Y = (Y_1=y_1, Y_2=y_2,...,Y_t=y_t) . And each observation variable Y_t can take on K possible values, K=4096 in my case. Luckily I only have two states N=2 , but my emission matrix B is N by K so 2 rows by 4096 columns.

Now when you initialize B, each row must sum to 1. Since there are 4096 values in each of the two rows, the numbers are very small. So small that when I go to compute alpha and beta their rows eventually approach 0 as t increases. This is a problem because you cannot compute gamma as it tries to compute x/0 or 0/0 . How can I run the algorithm without it crashing and without permanently altering my values?

Answer 1

This sounds like the standard HMM scaling problem. Have a look at "A Tutorial on Hidden Markov Models ..." (Rabiner, 1989), section VA "Scaling".

Briefly, you can rescale alpha at each time to sum to 1, and rescale beta using the same factor as the corresponding alpha , and everything should work.