Markov Chain Probability
Two players, A and B, play a game in which the winner gives 1 dollar to the other player. Player A has 1 dollar and player B has 2 dollars. Player A is better in this game and wins 2/3 of the games. They play until one of them is bankrupt. What is the probability that player A wins?


Take a look at the lesson about Markov Chain Probability (Probability).


This is a pretty straightforwards Markov chain problem. There are 4 states. The transition graph is given in Figure 1.

Figure 1 - Transition graph for this problem.

The problem starts at state 1. As has been explained in the lessons of this course, we use the following equation:



Furthermore, and . Then we have



Solving these equation gives us and . So, starting with 1 dollar, player A has a 4/7 chance of winning.

If we substitute Equation 4 in Equation 3, we have