马尔可夫链：寻找终端状态计算

Question

I'm trying to figure out this problem. Hopefully someone can tell me how to complete this. I consulted the following pages, but I was unable to write a code in java/python that produces the correct output and passes all test cases. I'd appreciate any and all help.

Markov chain probability calculation - Python

Calculating Markov chain probabilities with values too large to exponentiate

Write a function answer(m) that takes an array of array of nonnegative ints representing how many times that state has gone to the next state and return an array of ints for each terminal state giving the exact probabilities of each terminal state, represented as the numerator for each state, then the denominator for all of them at the end and in simplest form. The matrix is at most 10 by 10. It is guaranteed that no matter which state the ore is in, there is a path from that state to a terminal state. That is, the processing will always eventually end in a stable state. The ore starts in state 0. The denominator will fit within a signed 32-bit integer during the calculation, as long as the fraction is simplified regularly. For example, consider the matrix m:

[
[0,1,0,0,0,1],  # s0, the initial state, goes to s1 and s5 with equal    probability
[4,0,0,3,2,0],  # s1 can become s0, s3, or s4, but with different probabilities
[0,0,0,0,0,0],  # s2 is terminal, and unreachable (never observed in practice)
[0,0,0,0,0,0],  # s3 is terminal
[0,0,0,0,0,0],  # s4 is terminal
[0,0,0,0,0,0],  # s5 is terminal
]

    So, we can consider different paths to terminal states, such as:

    s0 -> s1 -> s3

    s0 -> s1 -> s0 -> s1 -> s0 -> s1 -> s4

    s0 -> s1 -> s0 -> s5

    Tracing the probabilities of each, we find that

    s2 has probability 0

    s3 has probability 3/14

    s4 has probability 1/7

    s5 has probability 9/14

Answer 1

I'm not sure what the results for the edge cases should be, but what I did for this problem is:

1. Created a second matrix that held all of the denominators for each probability by adding up all of the numerators in each row.
2. Find the first terminal state in the matrix to use as the bound of the non-terminal states.
3. Subtract the matrix bounded by the first terminal from the identity matrix of the same size.
4. Find the inverse of the difference. There's a couple ways to do this, I decided to augment the matching identity matrix with the difference.
5. Multiply the inverse by the matrix bounded from the first terminal to the end of the matrix.
6. Then, find the resulting denominator and return the first row of numerators of the matrix bounded from the first terminal to the end of the matrix.

Side notes:

• You'll need to write a simplify function that simplifies fractions (you may also need to write gcf and lcm functions to help you simplify).
• You may also need to sort the matrix so the terminals are at the end of the matrix so it is in proper form.
• edge cases: 1x1 matrix, matrix that only has 1 nonterminal state, 10x10 matrix, matrix that only has 1 terminal state

Answer 2

I know it's a bit old topic but maybe someone will be interested.

In my case, this PDF helped me a lot: https://math.dartmouth.edu/archive/m20x06/public_html/Lecture14.pdf

The algorithm is easy to implement.

As Ana said you need to sort matrix, remember to sort rows and columns at the same time to get proper results.

Regarding edge cases:

• 1x1 is always 100% if you start from the only one state and it must terminal as there is no other state.

• if there is only one nonterminal state, then the result will be the same as this row. No calculation needed.

The last two edge cases from Ana's answer (which should be accepted in my opinion) are not the edge cases, to be frank, they are regular cases so you need to calculate the answer normally.

Answer 3

As an alternative approach, one may consider Engel's algorithm for absorbing Markov chains to compute absorption probabilities. This requires no matrix inversion / linear system solution, and hence there is no need in rational number arithmetic.