Modified Engel Algorithm and Applications in Absorbing/Non-Absorbing Markov Chains and Monopoly Game

1. Introduction

The chip-firing game (CFG), also known as a probabilistic abacus, is a dynamic process described in [8,9]. It involves a graph with a set of finite vertices and edges, on which multiple tokens, or $``$chips", are placed. In this game, if the number of chips on a vertex is greater than or equal to the degree of that vertex, it can be $``$fired" by moving one chip along each incident edge and placing it on the adjacent vertex. When a vertex is fired, its chip count is reduced by its degree, while its neighbors’ chip counts are increased by one. The CFG is a directed weighted multigraph with a set of vertices and multiple edges sharing the same vertex. Firing a vertex means sending one chip along each outgoing edge from that vertex. It can be formulated as the transition digraph for an absorbing Markov chain with transition probabilities. Engel provided another method for solving chip-firing games based on the canonical decomposition of absorbing Markov chains in [21]. To start the game, a pile of pre-assigned chips is given. An indicated number of chips is set at each transient state, and zero chips are set at each absorbing state. Then, a vertex k is considered a critical position, and the number of chips $g_k$ is placed on each adjacent vertex neighborhood of that vertex k based on the transition probabilities between adjacent vertices and the vertex k. This is referred to as $``$critical loading" in Kaushal et al.’s work [13,14]. If the number of chips at the critical position (vertex k) is not enough to fire, an additional chip (or additional chips) will taken from the pile and placed on the vertex k, and then fired to each adjacent vertex. This process continues as long as the critical position does not reappear in the transient states in the transition diagram. Once the number of chips in all the transient states match the initial number of chips in those states, then the game stops [15]. Chips placed in absorbing states can have a value of either nonzero or zero. By using a normalization procedure, we can determine the stationary distribution of the Markov chains from the transient states to the absorbing states based on the number of chips in each absorbing state. Engel’s method involves representing each absorbing Markov chain as a directed graph in order to find a solution. This method is applicable to any Markov chains with at least one absorbing state.

The generalized Engel algorithm, as proposed by Kaushal et al. [13,14], is designed to solve the CFG by focusing on the transition states and absorbing states of a Markov chain. This algorithm is known for its higher accuracy compared to other methods but does depend on several factors. Kaushal et al. [13] implemented the Engel algorithm into a computer program to expand its application to more complex absorbing Markov chains, which they termed the generalized Engel algorithm. They achieved this by splitting the transition matrix into two separate matrices: the numerator matrix containing the numerators of entries in the transition matrix, and the denominator matrix containing the denominators of entries in the transition matrix as weights or for scaling. The numerator matrix represents the proportional distribution values of each transition state, while the denominator matrix represents the capacity of each transition state. They used the numerator matrix to simulate $``$firing" and the denominator matrix to reduce the divided chips. Our modified algorithm has a broader application and can provide process values. For example, in a directed graph, we can construct the set of configurations obtained from the CFG by a sequence of firings. Through process values, we can track the movement of every chip, determine how many chips pass through states, and record each configuration with the updated chips.

Our paper introduces a modified version of the generalized Engel algorithm, which we call the mEngel algorithm. This modified algorithm is designed to solve both absorbing and non-absorbing Markov chain problems. The key difference between the generalized Engel algorithm and our modified version is how it handles non-absorbing Markov chains. We accomplish this by reformulating the transition matrix decomposition into two separate matrices, similar to the approach in [13,14]. However, we further split the numerator matrix into nine sub-block matrices, representing a transformation from non-absorbing Markov chains to absorbing Markov chains and back to non-absorbing Markov chains. The mEngel algorithm works by determining the number of chips at each vertex and then using the algorithm to find all probability distribution integral values and store them in each corresponding vertex. The process continues until it stops when the tolerance is satisfied. Essentially, the mEngel algorithm provides approximate solutions for non-absorbing Markov chains. The study of Monopoly as a Markov process and its solution solver using the power method and the canonical decomposition of absorbing Markov chains has been investigated by several researchers such as [1,2,3,10,11,18,19,20,23,26], among others. Various strategies related to Monopoly’s Jail were explored by analyzing the size of the transition matrix of non-absorbing Markov chains [4,5,7,17], as well as the long and short jail scenarios [24]. The mEngel algorithm was utilized to compare the ranking results of the long and short jail strategies.

The rest of the paper is organized as follows. Section 2 introduces the transition diagram for the non-absorbing Markov chain model using a nested iterative setting. Section 3 first introduces the formulation of the transition probability matrix for absorbing/non-absorbing Markov chains and then provides a proof of the mEngel algorithm for solving the non-absorbing Markov chain problem using a nested iterative approach. This method is associated with the direct approaches of the power method and the canonical decomposition of absorbing Markov chains. The flow of the mEngel algorithm is provided, along with detailed descriptions of Algorithms 1 - 3. Section 4 presents the numerical results of the mEngel algorithm in solving for the absorbing probabilities in Torrence’s problem [22,25]. Finding the stationary probabilities of the non-absorbing Markov chains based on the rules of the Monopoly game regarding Jail are presented, i.e., the probability of landing on each state is found. Emphasis is placed upon the Monopoly player regarding $``$Go to Jail" for three reasons, 1. Landing on $``$Go to Jail", 2. Drawing a $``$Go to Jail" card, 3. Rolling doubles three times, using the long jail and short jail strategies [4,24], which can be formulated as the $43\times 43$ model and the $41\times 41$ model, respectively, where the number of the player’s turns to profit is studied and ranking based on the properties of the same colour group are examined. Another model, known as the $123\times 123$ model [5,17], is also presented in this paper. It involves getting out of Jail by rolling consecutive doubles three times. Section 5 draws conclusions and identifies opportunities for further research.

2. From the Complete Graph to the Modified Graph

The CFG, a discrete dynamic model, can be played on undirected and directed graphs. In this paper, we focus on directed graphs and their relationship with Markov chains, which can be viewed as an absorbing Markov chain. A directed graph $G(V,E)$ is defined with vertices $V\left(G\right)$ and edges $E\left(G\right)$. Each vertex $v\in V\left(G\right)$ has a capacity d. An arbitrary number of chips $g_v$ are placed on each vertex. If the number of chips $g_v$ exceeds the capacity d, i.e., $g_v\ge d$, some chips will be passed to each neighboring vertex along the outgoing edges, called firing [13,14,15].

Suppose that a CFG starts with critical loading and ends with critical loading reoccurring. In that case, the chip distribution of this CFG is equal to the stationary distribution of the absorbing Markov chain. The critical loading occurs when each vertex $v_i$ has one less chip than it needs in order to fire. In other words, $g_{v_i}=d_i-1$. The critical loading of each CFG will reoccur, which has been proven by [15]. Hence, the Engel algorithm is complete, which means it always provides the solution for the absorbing Markov chain.

Let us have the directed graph $G(V,E)$ of a Markov Chain, represented as a transition diagram as shown in Figure 1. Assume that every jump passes through this directed graph. We take the vertex and edge of it to create a new directed graph, as shown in Figure 2.

Let the new graph be $\widehat{G}$. We have the vertex set $V\left(\widehat{G}\right)$ and the edge set $E\left(\widehat{G}\right)$, where $V\left(\widehat{G}\right)=\left\{s\right\}\cup V\left(G\right)\cup V\left(G^{\prime }\right)$, and $V\left(G^{\prime }\right)$ is a set with the same size as $V\left(G\right)$, for ${\widehat{e}}_{ij}\in E\left(\widehat{G}\right)$, $e_{ij}\in E\left(G\right)$, ${\widehat{e}}_{ij}=e_{ij}$ if $v_i\in V\left(G\right),v_j\in V\left(G^{\prime }\right)$, ${\widehat{e}}_{ij}=0$ if $v_i,v_j\in V\left(G\right)$ or $v_i,v_j\in V\left(G^{\prime }\right)$.

The mEngel algorithm proposed for solving the absorbing Markov chain as well as the non-absorbing Markov chain consists of three steps:

Step 1 

We begin with the state space of a non-absorbing Markov chain denoted by S, then create another state space of an absorbing Markov chain with the same number of states as in the original chain of the transition diagram, denoted by $S_1$. In this new absorbing chain, the state space $S_1$ is not fully connected. Then, we introduce the additional state space denoted by $S_2$, which is also not fully connected. All states in $S_2$ are recurrent, meaning they eventually return to themselves. In general, a state is considered recurrent if, whenever we leave that state, we will return to it in the future with probability one. $S_2$ cannot go back to $S_1$.

Step 2 

We introduce an artificial state s in S as a starting state. This state is not part of the original Markov chain but is added to the algorithm. The transition probabilities from S to each state in $S_1$ are set to probability distribution values (i.e., certain transitions). In contrast, the transition probabilities from S to each state in $S_2$ are all set to zeros (i.e., impossible transitions).

Step 3 

The stationary distribution of this modified Markov chain (including S, $S_1$, and $S_2$) is equivalent to the original non-absorbing Markov chain. The proof of this equivalence is provided in Section 3.

In Figure 2, we pick a one-time jump (the player moves one time (one turn of the game)) of the Markov chain. With this time limit (a chain with only one jump), the Markov chain becomes absorbing. Then, we can iterate this absorbing Markov chain to get the stationary distribution of the whole Markov chain. The mEngel algorithm allows us to get the probability of the evolution process through an iterative process, i.e.,

$$\left\{S\to S_1\to S_2\right\}\to \left\{S\to S_1\to S_2\right\}\to \dots .$$

For example, one observes how often chips pass through one state before the game ends by a sequence of configurations.

3. Structure of the mEngel Algorithm

The transition probability matrix P of the Markov chain for the mEngel algorithm can then be represented in the following canonical form, assuming time homogeneity, that is

$$P=\left[{\displaystyle \frac{n_{ij}}{d_{ij}}}\right],i,j=1,\dots ,n$$

where $N=\left[n_{ij}\right]$ is a state (nominator) matrix, $D=\left[d_{ij}\right]$ is a scaling (denominator) matrix, and n is the total number of states. Note that $P=N/D$ is a non-absorbing matrix and $\widehat{P}=\widehat{N}/\widehat{D}$ is an absorbing matrix. The distinction between P and $\widehat{P}$ arises from the operation of Algorithms 1 - 2. Specifically, when we input P into Algorithm 2, it generates N and D as outputs. Similarly, when we input $\widehat{P}$, the algorithm produces $\widehat{N}$ and $\widehat{D}$. It is important to note that in our computations, we only utilize $\widehat{P}$.

The mEngel algorithm’s transition diagram is composed of three state processes: 1) $S=\left\{s\right\}$ is a source state; 2) $S_1=\{1,\dots ,n\}$ is a set of original states from the given problem; 3) $S_2=\{1^{\prime },\dots ,n^{\prime }\}$ is a set of fictitious states for an mEngel algorithm setting. We use labels (e.g., s) instead of full names for vertices $V_s$, i.e., $S=\left\{s\right\}=\left\{V_s\right\}$. The transition diagram is shown in Figure 2.

Define

$$\begin{array}{c}\hfill \widehat{N}=\begin{array}{cc}& \begin{array}{ccc}s& \{1,\dots ,n\}& \{1^{\prime },\dots ,n^{\prime }\}\end{array}\\ \begin{array}{c}s\\ \{1,\dots ,n\}\\ \{1^{\prime },\dots ,n^{\prime }\}\end{array}& \left[\begin{array}{ccc}{\mathbf{0}}_{1\times 1}& {\mathit{d}}_{1\times n}^{\left(k\right)}& {\mathbf{0}}_{1\times n}\\ {\mathbf{0}}_{n\times 1}& {\mathbf{0}}_{n\times n}& \mathrm{ceil}\left(c·P\right)\\ {\mathbf{0}}_{n\times 1}& {\mathbf{0}}_{n\times n}& I_n\end{array}\right]\end{array}\end{array}$$

(1)

and

$$\begin{array}{c}\hfill \widehat{D}=\begin{array}{cc}& \begin{array}{ccc}s& \{1,\dots ,n\}& \{1^{\prime },\dots ,n^{\prime }\}\end{array}\\ \begin{array}{c}s\\ \{1,\dots ,n\}\\ \{1^{\prime },\dots ,n^{\prime }\}\end{array}& \left[\begin{array}{ccc}{m}_{1\times 1}^{\left(k\right)}& {\mathit{m}}_{1\times n}^{\left(k\right)}& {\mathit{m}}_{1\times n}^{\left(k\right)}\\ c·{\mathbf{1}}_{n\times 1}& c·{\mathbf{1}}_{n\times n}& c·{\mathbf{1}}_{n\times n}\\ {\mathbf{1}}_{n\times 1}& {\mathbf{1}}_{n\times n}& {\mathbf{1}}_{n\times n}\end{array}\right]\end{array}=\left(\begin{array}{c}{\mathit{m}}_{1\times 2n+1}^{\left(k\right)}\\ c·{\mathbf{1}}_{n\times 2n+1}\\ {\mathbf{1}}_{n\times 2n+1}\end{array}\right)\end{array}$$

(2)

where ${\displaystyle m_{1\times 1}^{\left(k\right)}=\sum _{i=1}^n{d}_i^{\left(k\right)}}$, $\mathit{m}={\left[\begin{array}{cccc}{\displaystyle \sum _{i=1}^n{d}_i^{\left(k\right)}}& {\displaystyle \sum _{i=1}^n{d}_i^{\left(k\right)}}& \dots & {\displaystyle \sum _{i=1}^n{d}_i^{\left(k\right)}}\end{array}\right]}_{1\times n}$ and c is a suitably sized scaling number.

• The ceil operation is used here to ensure that every element of the matrix $\widehat{N}$ is an integer since the distributing chips are in an integer process. For example, if an element of the transition matrix P for a non-absorbing Markov chain/absorbing chain is 0.949 and $c=100,$ then the unrounded value will be $94.9$. Thus, the rounding-up value is 95. One can choose c as 1000, and the rounding-up procedure is avoided. Similarly, each corresponding entry of the ratio of $\widehat{N}$ and $\widehat{D}$ is equal to each position of the transition matrix P. Hence, $\widehat{D}$ is a matrix scaled by c.
• From S to $\{1,\dots ,n\}$, each arrow means a directed movement, where the number of chips being distributed is reshuffled and added. At the end of the iteration process, the number of configurations (chips) between the initial and final transient states remains the same.
• From $\{1,\dots ,n\}$ to $\{1^{\prime },\dots ,n^{\prime }\}$, the original transition matrix P is scaled by c. As a result of this rounding-up process, the chips in a vertex are distributed to its neighboring vertices based on the probabilities obtained from rounding up the values of $c·P$ to the nearest integers.
• From $\{1^{\prime },\dots ,n^{\prime }\}$ to $\{1^{\prime },\dots ,n^{\prime }\}$, a set of transient states will be forced into a set of recurrent states, e.g., a state j is called an absorbing state if, with probability 1, the process will eventually return to j after it leaves $j.$ Hence, this is crucial for transforming the transient state into the absorbing state. Typically, a self-loop at each state means that a state of a Markov chain is called absorbing if, once entered, it cannot be left. Chips is stored at each vertex.
• At the end of the iterative process, within the pre-assigned tolerance and the number of iterations, the stationary distribution for non-absorbing Markov chains is obtained by summing all chips from all vertices, i.e., ${\displaystyle m_{1\times 1}^{\left(k\right)}=\sum _{i=1}^n{d}_i^{\left(k\right)}}$, using the normalization procedure, i.e., $d_i^{\left(k\right)}/{\sum }_{i=1}^n{d}_i^{\left(k\right)}$ for all i.

Given the initial probability distribution ${\mathit{d}}^{\left(0\right)}$. Our aim here is to show that

$$\mathrm{mEngel}\left({\mathit{d}}^{\left(0\right)}\right)={\mathit{d}}^{\left(1\right)}={\mathit{d}}^{\left(0\right)}P\to \mathrm{mEngel}\left({\mathit{d}}^{\left(1\right)}\right)={\mathit{d}}^{\left(2\right)}={\mathit{d}}^{\left(1\right)}P\to$$

$$\mathrm{mEngel}\left({\mathit{d}}^{\left(2\right)}\right)={\mathit{d}}^{\left(3\right)}={\mathit{d}}^{\left(2\right)}P\to \dots \to \mathrm{mEngel}\left({\mathit{d}}^{\left(k\right)}\right)={\mathit{d}}^{(k+1)}={\mathit{d}}^{\left(k\right)}P.$$

For each iteration in the mEngel algorithm, we will get the same probability distribution for a non-absorbing/absorbing Markov chain using the power method, i.e.,

$${\mathit{d}}^{\left(1\right)}={\mathit{d}}^{\left(0\right)}P\to {\mathit{d}}^{\left(2\right)}={\mathit{d}}^{\left(1\right)}P\to {\mathit{d}}^{\left(3\right)}={\mathit{d}}^{\left(2\right)}P\to \dots \to {\mathit{d}}^{(k+1)}={\mathit{d}}^{\left(k\right)}P.$$

What needs to shown is that

$$\mathrm{mEngel}\left({\mathit{d}}^{\left(1\right)}\right)={\mathit{d}}^{\left(2\right)}={\mathit{d}}^{\left(1\right)}P$$

is true using the first step transition.

We aim to prove $\mathrm{mEngel}\left({\mathit{d}}^{\left(1\right)}\right)={\mathit{d}}^{\left(1\right)}P$. By the power method, we have ${\displaystyle {\left({\mathit{d}}^{\left(1\right)}P\right)}_i=\sum _{j=1}^n{d}_j^{\left(1\right)}{p}_{ji}}$. This means that we need to prove ${\displaystyle {\left(\mathrm{mEngel}\left({\mathit{d}}^{\left(1\right)}\right)\right)}_i=\sum _{j=1}^n{d}_j^{\left(1\right)}{p}_{ji}}$.

The canonical decomposition of P is defined as

$$P=\begin{array}{cc}& \mathrm{TR.}\mathrm{ABS.}\\ \begin{array}{c}\mathrm{TR.}\\ \mathrm{ABS.}\end{array}& \left[\begin{array}{cc}Q& R\\ \mathbf{0}& I\end{array}\right]\end{array}$$

where TR. states are transient states, while ABS. states are absorbing states (e.g., see Figure 3). $\mathbf{0}$ represents a zero matrix and I represents an identity matrix with appropriate dimensions.

To do so, let us recall

$$\widehat{P}=\begin{array}{cc}& \begin{array}{ccccccc}s& 1& \dots & n& 1^{\prime }& \dots & n^{\prime }\end{array}\\ \begin{array}{c}s\\ 1\\ ⋮\\ n\\ 1^{\prime }\\ ⋮\\ n^{\prime }\end{array}& \left[\begin{array}{ccccccc}0& d_1^{\left(1\right)}& \dots & d_n^{\left(1\right)}& 0& \dots & 0\\ 0& 0& \dots & 0& p_{11}& \dots & p_{1n}\\ 0& 0& \ddots & 0& ⋮& \ddots & ⋮\\ 0& 0& \dots & 0& p_{n1}& \dots & p_{nn}\\ 0& 0& \dots & 0& 1& \dots & 0\\ 0& 0& \ddots & 0⋮& \ddots & ⋮\\ 0& 0& \dots & 0& 0& \dots & 1\end{array}\right]\end{array}$$

Using the absorbing Markov chain formulation, we have

$$Q=\begin{array}{c}& \begin{array}{cccc}s& 1& \dots & n\end{array}\\ \begin{array}{c}s\\ 1\\ ⋮\\ n\end{array}& \left[\begin{array}{cccc}0& d_1^{\left(1\right)}& \dots & d_n^{\left(1\right)}\\ 0& 0& \dots & 0\\ 0& 0& \ddots & 0\\ 0& 0& \dots & 0\end{array}\right]\end{array}$$

$$R=\begin{array}{cc}& 1^{\prime }\dots n^{\prime }\\ \begin{array}{c}s\\ 1\\ ⋮\\ n\end{array}& \left[\begin{array}{ccc}0& \dots & 0\\ p_{11}& \dots & p_{1n}\\ ⋮& \ddots & ⋮\\ p_{n1}& \dots & p_{nn}\end{array}\right]\end{array}$$

The fundamental matrix for $\widehat{P}$ is

$$J={({\mathbf{1}}_{n+1\times n+1}-Q)}^{-1}=\begin{array}{cc}& \begin{array}{cccc}s& 1& \dots & n\end{array}\\ \begin{array}{c}s\\ 1\\ ⋮\\ n\end{array}& \left[\begin{array}{cccc}1& d_1^{\left(1\right)}& \dots & d_n^{\left(1\right)}\\ 0& 1& \dots & 0\\ 0& 0& \ddots & 0\\ 0& 0& \dots & 1\end{array}\right]\end{array}$$

and the time-to-absorption matrix is

$$\begin{array}{cc}\hfill B& =JR=\begin{array}{cc}& \begin{array}{c}s1\dots n\end{array}\\ \begin{array}{c}s\\ 1\\ ⋮\\ n\end{array}& \left[\begin{array}{cccc}1& d_1^{\left(1\right)}& \dots & d_n^{\left(1\right)}\\ 0& 1& \dots & 0\\ 0& 0& \ddots & 0\\ 0& 0& \dots & 1\end{array}\right]\end{array}\begin{array}{cc}& \begin{array}{ccc}{1}^{\prime }& \dots & n^{\prime }\end{array}\\ \begin{array}{c}s\\ 1\\ ⋮\\ n\end{array}& \left[\begin{array}{ccc}0& \dots & 0\\ p_{11}& \dots & p_{1n}\\ ⋮& \ddots & ⋮\\ p_{n1}& \dots & p_{nn}\end{array}\right]\end{array}\hfill \\ \hfill & =\begin{array}{cc}& \begin{array}{ccc}{1}^{\prime }& \dots & n^{\prime }\end{array}\\ \begin{array}{c}s\\ 1\\ ⋮\\ n\end{array}& \left[\begin{array}{ccc}\sum _{j=1}^n{d}_j^{\left(1\right)}{p}_{j1}& \dots & \sum _{j=1}^n{d}_j^{\left(1\right)}{p}_{jn}\\ p_{11}& \dots & p_{1n}\\ ⋮& \ddots & ⋮\\ p_{n1}& \dots & p_{nn}\end{array}\right]\end{array}=\begin{array}{cc}& \begin{array}{ccc}{1}^{\prime }& \dots & n^{\prime }\end{array}\\ \begin{array}{c}S\\ 1\\ ⋮\\ n\end{array}& \left[\begin{array}{ccc}{d}_1^{\left(2\right)}& \dots & d_n^{\left(2\right)}\\ p_{11}& \dots & p_{1n}\\ ⋮& \ddots & ⋮\\ p_{n1}& \dots & p_{nn}\end{array}\right]\end{array}\hfill \end{array}$$

So the ${\displaystyle {\left(\mathrm{mEngel}\left({\mathit{d}}^{\left(1\right)}\right)\right)}_i}$ is $B_{1i}$, which is ${\displaystyle \sum _{j=1}^n{d}_j^{\left(1\right)}{p}_{ji}}$. This can also be expressed as ${\left(d^{\left(1\right)}P\right)}_i$. From s to $\{1^{\prime },\dots ,n^{\prime }\}$ in B, the distribution of $\{1^{\prime },\dots ,n^{\prime }\}$ is ${\mathit{d}}^{\left(2\right)}$, i.e.,

$$\mathrm{mEngel}\left({\mathit{d}}^{\left(1\right)}\right)={\mathit{d}}^{\left(2\right)}={\mathit{d}}^{\left(1\right)}P.$$

The rest of the iteration results follow, i.e.,

$$\mathrm{mEngel}\left({\mathit{d}}^{\left(k\right)}\right)={\mathit{d}}^{(k+1)}={\mathit{d}}^{\left(k\right)}P,$$

where P can be a transition matrix or an absorbing transition matrix. From $\{1,\dots ,n\}$ to $\{1^{\prime },\dots ,n^{\prime }\}$ in B, we check to see that P remains unchanged at each iteration. The $k+1$-step transition will follow.

3.1. Implementation of the mEngel Algorithm

Algorithm 1 presents the pseudo-code structure, whereas Algorithm 2 provides the matrix formulations of $\widehat{N}$ and $\widehat{D}$ for (1) and (2), respectively. Algorithm 3 demonstrates the nested mEngel algorithm with an accuracy tolerance as the stopping criterion. We use N and D for illustrative purposes in these three algorithms to avoid confusion.

Let

$${\mathit{d}}^{\left(0\right)}=\left[\begin{array}{cccc}1& 0& \dots & 0\end{array}\right]$$

be the initial probability distribution vector. The details of Algorithm 1 are described as follows:

• In Lines 3 - 5, $MD$ represents the capacity array of the states. An element in $MD$ is equal to 0 when it is a recurrent state and not equal to zero when it is a transient state. This part is also called critical loading.
• In Lines 15 - 39, the while loop ends when the initial transient states’ chips are the same as the end states’ chips.
• In Lines 20 - 30, chips are firing if available. Otherwise, it simply counts the number of unavailable states.

  -

  In Lines 21 - 22, if the available chips are more than the capacity for chips, chips get stored in $new1$.

  -

  In Lines 24 - 28, $j\ne i$, j is the order of states that the chips move to, i is the order of states that the chips start from, and j is not equal to i means that in this round of loop, the current state is not the starting state.

• In Lines 31 - 32, if the number of states that are not available is equal to the transient states, the while loop ends.
• In vector $temp$, the total of d is needed, i.e., $sum\left(d\right)$, because the starting state needs $sum\left(d\right)$ chips to be available to be fired. We set $counteq=0$, which is the number of unavailable transient states. The loop will stop when $counteq$ is equal to the number of transient states n.
• In Lines 37 - 38, check whether the number of current transient state’s chips is the same as the chips of the initial one, i.e., critical loading. If yes, the while loop ends.
• In Lines 40 - 41, the probability distribution vector is updated.
• In Lines 42 - 43, the normalized probability distribution vector is calculated.

The details of Algorithm 2 are described as follows:

• Matrix N of (1): In Lines 1 - 12, matrix N is constructed with the following dimensions: $(2n+1)\times (2n+1)$ where $S=\left\{s\right\},$$\{1,\dots ,n\}$, and $\{1^{\prime },\dots ,n^{\prime }\}$ represent the rows and columns:

  -

  $N[1,1]=0$

  -

  $N[1,2:n+1]=$ initial distribution ${\mathit{d}}^{\left(0\right)}=\left[\begin{array}{cccc}1& 0& \dots & 0\end{array}\right]$

  -

  $N[1,n+2:2n+1]=0$

  -

  The first column is defined as 0.

  -

  The 2nd through $n+1$ columns are defined as 0.

  -

  The $n+2$ through $2n+1$ columns are defined as 0.

  -

  $N[2:n+1,1]=0$

  -

  The $n\times n$ submatrix $N[n+2:2n+1,n+2:2n+1]$ is $I_n$, which is an identity matrix of order n.

  -

  The remaining elements of N are defined as $\mathrm{ceil}\left(c·P\right)$, where P is a transition matrix and c is a constant.

• Matrix D of (2): In Lines 13 - 16, matrix D is constructed with the same dimensions as matrix N.

  -

  $D[1,1]=m_{1\times 1}^{\left(k\right)}$, which is a given scalar value.

  -

  $D[1,2:n+1]={\mathit{m}}_{1\times n}^{\left(k\right)}$, which is a given scalar vector.

  -

  $D[1,n+2:2n+1]={\mathit{m}}_{1\times n}^{\left(k\right)}$, which is a given scalar vector.

  -

  The first column is defined as $c·{\mathbf{1}}_{n\times 1}$, where c is a constant.

  -

  The 2nd through $n+1$ columns are defined as $c·{\mathbf{1}}_{n\times 1}$.

  -

  The $n+2$ through $2n+1$ columns are defined as $c·{\mathbf{1}}_{n\times 1}.$

  -

  The remaining elements of D are defined as ${\mathbf{1}}_{n\times 1}$ and ${\mathbf{1}}_{n\times n}$, respectively.

Algorithm 1:Structure of the mEngel algorithm

Algorithm 2:Matrix formulations of N and D

Input: P: the transition matrix; n: the # of states; $c=10000$   Output: N and D  1 $S1\leftarrow round(c*P)$  2 $d(0:n)\leftarrow 0$  3 $d\left(1\right)\leftarrow 1$  4 $z\leftarrow 0$  5 $z1(0:n)\leftarrow 0$  6 $z2(0:2n)\leftarrow 0$  7 $z3\leftarrow matrix\left(\right(n,n),0)$  8 $z4\leftarrow diag\left(n\right)$  9 $p1\leftarrow [z;z2]$ 10 $p2\leftarrow [d;z3;z3]$ 11 $p3\leftarrow [z1;S1;z4]$ 12 $N\leftarrow [p1,p2,p3]$ 13 $q1\leftarrow matrix\left(\right(n,2*n+1),1)$ 14 $q2\leftarrow matrix\left(\right(n,2*n+1),c)$ 15 $q3\leftarrow matrix\left(\right(2,n+1),1)$ 16 $D\leftarrow [q3;q2;q1]$

Algorithm 3:Illustration of the nested mEngel algorithm

4. Numerical Results

In this section, we apply the mEngel algorithm to the absorbing problem, which is Torrence’s problem, and non-absorbing problems such as Monopoly. We use our homemade R code to generate all numerical results and graphical plots. All the transition matrices defined below are input in Algorithms 1 - 3.

Let $X_n$ represent the state at time n, and $P(X_n=j|X_{n-1}=i)$ denote the conditional probability that the state at time $n-1$ is i and at time n is j. Then, we have $P(X_n=j|X_{n-1}=i_{n-1},\dots ,X_0=i_0)=P(X_n=j|X_{n-1}=i_{n-1})$, which is known as the Markov property. It is important to note that $P\left(j\right|i)=P(X_n=j|X_{n-1}=i)$. For instance, in the Markov matrix M, we have $M_{ij}=P\left(j\right|i)$, and it has the property that ${\displaystyle \sum _{j\in S}{M}_{ij}=1}$ for all $i\in S$, where S is the state set.

4.1. Torrence’s problem

Torrence’s problem is a random walk problem given as a weekly Riddler puzzle on the FiveThirtyEight website [25]. The transition diagram for Torrence’s problem shown in Figure 3 is the one from [22].

The transition matrix of Torrence’s problem is given by

$$P=\begin{array}{cc}& \begin{array}{cccccccccc}1& 2& 3& 4& 5& 6& 7& 8& 9& 10\end{array}\\ \begin{array}{c}1\\ 2\\ 3\\ 4\\ 5\\ 6\\ 7\\ 8\\ 9\\ 10\end{array}& \left[\begin{array}{cccccccccc}0& \mathrm{1/3}& 0& 0& \mathrm{1/3}& \mathrm{1/3}& 0& 0& 0& 0\\ \mathrm{1/3}& 0& \mathrm{1/3}& 0& 0& 0& \mathrm{1/3}& 0& 0& 0\\ 0& \mathrm{1/3}& 0& \mathrm{1/3}& 0& 0& 0& \mathrm{1/3}& 0& 0\\ 0& 0& \mathrm{1/3}& 0& \mathrm{1/3}& 0& 0& 0& \mathrm{1/3}& 0\\ \mathrm{1/3}& 0& 0& \mathrm{1/3}& 0& 0& 0& 0& 0& \mathrm{1/3}\\ 0& 0& 0& 0& 0& 1& 0& 0& 0& 0\\ 0& 0& 0& 0& 0& 0& 1& 0& 0& 0\\ 0& 0& 0& 0& 0& 0& 0& 1& 0& 0\\ 0& 0& 0& 0& 0& 0& 0& 0& 1& 0\\ 0& 0& 0& 0& 0& 0& 0& 0& 0& 1\end{array}\right]\end{array}.$$

A person begins in the dark inner circle state and aims to reach the other colored states. The probability of each state is shown in Figure 4. The probability of each recurrent state is 5/11 for the closest state, 2/11 for the second closest two states, and 1/11 for the furthest state. These distribution values are consistent with the existing ones. The mEngel’s algorithm has 22 configurations, resulting in a stationary distribution mirroring the final round’s chip distribution. All transient states eventually hold an equal number of chips to the initial configuration, as shown in Table 1. These findings are consistent with [22].

4.2. Monopoly

Monopoly, a widely enjoyed board game, can be modelled as a non-absorbing Markov chain problem. The game’s intricate rules result in a complex transition matrix. We plan to construct this transition matrix by multiplying four separate matrices (e.g, [4]), which we will detail subsequently. We dissect the entire Markov chain into segments, each of which can be considered a sub-chain. Each sub-chain corresponds to a specific rule, and their combination reconstitutes the original chain.

In Monopoly, a player’s decision is primarily determined by the ownership of land on the board. In decision-making, the player only needs to consider the probability of landing on each piece of land and the benefits of landing on each piece of land. In this section, we calculated the probability of landing on each piece of land and the benefits of purchasing each piece of land in the two jail strategies.

Using the long and short jail strategies [24], we aim to find the probability of landing in Jail for three scenarios: 1. Landing on $``$Go to Jail", 2. Drawing a $``$Go to Jail" card, 3. Rolling doubles three times. Within the non-absorbing Monopoly Markov chains consisting of 41 states, where the first 40 states are regular Monopoly squares as listed, for a short jail strategy, at state 41, if a player starts their turn in Jail, they have the option to play a $``$Get Out of Jail Free" card (Community Chest or Chance) or pay $50 to roll the dice and exit Jail normally. For a long jail strategy, assuming the player is incarcerated, the non-absorbing Monopoly Markov chains consist of 43 states, where the first 40 states are regular Monopoly squares as listed, state 41 represents the initial turn in Jail, state 42 the second, and state 43 the third. In Monopoly, the player moves around the 40 squares by rolling a pair of fair dice. If the player rolls doubles, they get an extra turn. If they roll doubles again on this extra turn, they get yet another additional turn. However, if the player rolls doubles for a third time, they are sent to Jail instead of proceeding as usual. This is called the $120\times 120$ model [5]. In what follows, we also use long and short jail strategies for obtaining the stationary distribution of the Monopoly Markov chains formulated by the $123\times 123$ model.

4.2.1. Construction of Monopoly Matrices

Firstly, players use two dice to move, with the number of steps corresponding to the sum of the two dice values, as shown in Table 2.

To end up in Jail, a player must roll 3 doubles to end up in jail, land on the square, or get the card that sends them there. Once in Jail, the player can either pay to leave on their next turn or try to roll doubles. If neither of these options happens, the player must stay in Jail for three turns. These rules are important for understanding the jail matrix and the movement matrix. For example, the movement matrix involves a short jail strategy with an extra state and a long jail strategy with three extra states, in the case of three extra states, 1. If the player chooses not to use a card or pay the fee, they still need to roll the dice. 2. If the player rolls doubles, they move out of Jail the designated number of squares without earning an additional turn, even though doubles are rolled. 3. Failing to roll doubles on the third attempt requires paying $50 to leave and proceed with the indicated number of squares. Landing on a Chance or Community Chest square prompts players to draw a card that could lead them to other squares (the player takes the top card from the draw pile and immediately places it on the discard pile). The probability of drawing a card from the Chance or Community Chest squares is shown in Table 3 and Table 4. The $``$Stay" card groups all non-moving square cases. In Table 3, there are 16 cards available, where 14 of them do not involve moving to another square (they involve either gaining or losing money in some way). Of the 14 $``$Stay" cards, 9 reward the player with cash, 4 make the player pay money to the bank or other players, and 1 is a $``$Get Out of Jail Free" card. One card sends the player directly to Jail ($``$Go to Jail"), and another card sends the player to the start position ($``$Advance to Go"). Table 4 shows that one card sends the player directly to Jail ($``$Go to Jail"), three of the six $``$Stay" cards make the player pay money to the bank or other players, two reward the player with cash, and one is a $``$Get Out of Jail Free" card. The remaining seven direct the player to a property, railroad, or utility, highlighted in bold.

We aim to categorize these rules into four groups, each represented by a matrix. These matrices are named according to the rules they represent: the movement matrix $\mathit{M}=\left[M_{ij}\right]$, jail matrix $\mathit{J}=\left[J_{ij}\right]$, chance matrix $\mathit{Ch}=\left[C{h}_{ij}\right]$, and community chest matrix $\mathit{Cc}=\left[C{c}_{ij}\right]$. All four matrices adhere to an order that mirrors the priority of the rules in Monopoly. For instance, a player must first move and then act based on the new square they land on, making the movement matrix the first. The jail matrix is second as we consider the possibility of jail upon arriving at a new square. The order of the chance and community chest matrices is also fixed. If a player lands on a Chance square, they may be sent to a Community Chest square. This is due to one of the 16 chance cards instructing the player to $``$Go back 3 spaces", and the Chance square is three ahead of the Community Chest square.

This approach differs from [4], where we attempted to change the order of the chance and community chest matrices, resulting in a different outcome. This paper considers the scenario where a player can move from a Chance square to a Community Chest square. Upon researching the commutative law of matrix multiplication, we found that these two matrices do not satisfy the conditions of the commutative law.

Upon integrating these matrices, we derive the transition matrix, defined as follows:

$$P=\mathit{M}\mathit{J}\mathit{Ch}\mathit{Cc}.$$

Two strategies emerge: long jail formulated by the $43\times 43$ model and short jail formulated by the $41\times 41$ model. The long jail strategy involves players remaining in Jail for as long as possible, implying they will not pay to leave. Conversely, the short jail strategy permits players to leave immediately on their next turn. The numerical distinction between these two strategies lies in the last three rows of the movement matrix, while the other matrices remain unchanged.

Let us define the movement matrix in the long jail strategy as ${\mathit{M}}^L$ and the movement matrix in the short jail strategy as ${\mathit{M}}^S$. The transition matrix in the long jail strategy is given by $P={\mathit{M}}^L\mathit{J}\mathit{Ch}\mathit{Cc}$, and the transition matrix in the short jail strategy is given by $P={\mathit{M}}^S\mathit{J}\mathit{Ch}\mathit{Cc}$, as shown in Table A1 and Table A2 in Appendix A. Next, we will construct the Markov matrix for ${\mathit{M}}^L,{\mathit{M}}^S,\mathit{J},\mathit{Ch}$, and $\mathit{Cc}$ as shown in Table A3, Table A4, Table A5, Table A6, and Table A7, respectively, in Appendix A.

The movement matrix in the long jail strategy is given by:

$$M_{ij}^L=\left\{\begin{array}{cc}1/36\hfill & \mathrm{if}j=i+2,i-38,i+12,i-28\mathrm{and}1\le i,j\le 40\mathrm{or}j=13,23,i=43\hfill \\ \hfill & \mathrm{or}j=13,15,17,19,21,23,i=41,42\hfill \\ 2/36\hfill & \mathrm{if}j=i+3,i-37,i+11,i-29\mathrm{and}1\le i,j\le 40\mathrm{or}j=14,22,i=43\hfill \\ 3/36\hfill & \mathrm{if}j=i+4,i-36,i+10,i-30\mathrm{and}1\le i,j\le 40\mathrm{or}j=15,21,i=43\hfill \\ 4/36\hfill & \mathrm{if}j=i+5,i-35,i+9,i-31\mathrm{and}1\le i,j\le 40\mathrm{or}j=16,20,i=43\hfill \\ 5/36\hfill & \mathrm{if}j=i+6,i-34,i+8,i-32\mathrm{and}1\le i,j\le 40\mathrm{or}j=17,19,i=43\hfill \\ 6/36\hfill & \mathrm{if}j=i+7,i-33\mathrm{and}1\le i,j\le 40\mathrm{or}j=18,i=43\hfill \\ 30/36\hfill & \mathrm{if}j=42,i=41\mathrm{or}j=43,i=42\hfill \end{array}\right.$$

For the 1st-40th rows: Take the first row as an example. This means that the first row starts from $``$Go" (1st state) and goes to other states based on the probability of the outcome of the roll of the two dice.

The probability of moving from the ith state to the jth state is $P\left(j\right|i)=P(j-i)$. Apply it, and we can get the movement matrix $M_{ij}^L=P\left(j\right|i)$. The last three rows have 2 steps:

Step 1 

The probability of leaving from Jail is $1/6,$ while the probability of staying in Jail is $1-1/6=5/6$.

Step 2 

They have the same formation as the 1st-40th rows if `leaving Jail’. But in states 41 and 42, the entries $M_{ij}^L$, (where $i=41,42,j<41$), is $M_{ij}^L=(5/6)=P\left(j\right|13)$ ( $5/6$ because `if leaving’, from Step 1). And $M_{ij}^L$, (where $i=43,j<41$), is $M_{ij}^L=P\left(j\right|13)$.

In Monopoly, the last three states show how to leave Jail. In Jail, in states 41 and 42, players have a $1/36$ chance to roll doubles. If they do, they leave Jail and move forward that many spaces. Because we added extra states for being in Jail, but rolling doubles means they move from the Jail square, if they do roll doubles, they move to state 11, $``$Just Visiting Jail".

$$\begin{array}{cccccccccccccc}\mathrm{Squares}\mathrm{from}\mathrm{Jail}& 11& 12& 13& 14& 15& 16& 17& 18& 19& 20& 21& 22& 23\\ & 11& 11+1& 11+2& 11+3& 11+4& 11+5& 11+6& 11+7& 11+8& 11+9& 11+10& 11+11& 11+12\\ \mathrm{Probability}\mathrm{of}& & & & & & & & & & & & & \\ \mathrm{Leaving}\mathrm{after}\mathrm{the}& 0& 0& 1/36& 0& 1/36& 0& 1/36& 0& 1/36& 0& 1/36& 0& 1/36\\ \mathrm{First}\mathrm{Turn}\mathrm{in}\mathrm{Jail}& & & & & & & & & & & & & \\ \mathrm{Probability}\mathrm{of}& & & & & & & & & & & & & \\ \mathrm{Leaving}\mathrm{after}\mathrm{the}& 0& 0& 1/36& 0& 1/36& 0& 1/36& 0& 1/36& 0& 1/36& 0& 1/36\\ \mathrm{Second}\mathrm{Turn}\mathrm{in}\mathrm{Jail}& & & & & & & & & & & & & \\ \end{array}$$

A player has a 5/6 chance of staying in Jail, meaning the chance of moving from state 41 to 42 is 5/6. This is because state 41 is when they are first put in Jail, and if they do not roll doubles, they move to their first turn in Jail, state 42. The same 5/6 chance applies to moving from state 42 to 43. After the third turn in Jail, they must pay to leave. So, they leave from state 43 with chances based on the rolling of two dice, like the rest of the matrix. The chances of moving from state 43 to other states on the board are outlined below:

$$\begin{array}{ccccccccccccccc}\mathrm{Squares}& 1\dots 11& 12& 13& 14& 15& 16& 17& 18& 19& 20& 21& 22& 23& 24\dots 43\\ \mathrm{Probability}\mathrm{of}& & & & & & & & & & & & & & \\ \mathrm{Leaving}\mathrm{after}\mathrm{the}& 0\dots 0& 0& 1/36& 2/36& 3/36& 4/36& 5/36& 6/36& 5/36& 4/36& 3/36& 2/36& 1/36& 0\dots 0\\ \mathrm{Third}\mathrm{Turn}\mathrm{in}\mathrm{Jail}& & & & & & & & & & & & & & \\ \end{array}$$

The movement matrix in the short jail strategy is given by:

$$M_{ij}^S=\left\{\begin{array}{cc}1/36\hfill & \mathrm{if}j=i+2,i-38,i+12,i-28\mathrm{and}1\le i,j\le 40\hfill \\ \hfill & \mathrm{or}j=13,23,i=41,42,43\hfill \\ 2/36\hfill & \mathrm{if}j=i+3,i-37,i+11,i-29\mathrm{and}1\le i,j\le 40\hfill \\ \hfill & \mathrm{or}j=14,22,i=41,42,43\hfill \\ 3/36\hfill & \mathrm{if}j=i+4,i-36,i+10,i-30\mathrm{and}1\le i,j\le 40\hfill \\ \hfill & \mathrm{or}j=15,21,i=41,42,43\hfill \\ 4/36\hfill & \mathrm{if}j=i+5,i-35,i+9,i-31\mathrm{and}1\le i,j\le 40\hfill \\ \hfill & \mathrm{or}j=16,20,i=41,42,43\hfill \\ 5/36\hfill & \mathrm{if}j=i+6,i-34,i+8,i-32\mathrm{and}1\le i,j\le 40\hfill \\ \hfill & \mathrm{or}j=17,19,i=41,42,43\hfill \\ 6/36\hfill & \mathrm{if}j=i+7,i-33\mathrm{and}1\le i,j\le 40\hfill \\ \hfill & \mathrm{or}j=18,i=41,42,43\hfill \end{array}\right.$$

For the first row and any rows of the 2nd to 40th rows of $M_{ij}^S$, the probability of moving from the ith state to the jth state is $P\left(j\right|i)=P(j-i)$, which is the same as the ones in $M_{ij}^L$. The last three rows are computed as follows:

• For short jail, the probability of staying in Jail for one more turn is 0. (in comparison with long jail)
• They have the same formation as the 1st-40th rows if `leaving Jail’. (like $M_{ij}^L$)
• Starting with the 11st state (share with $``$Just Visiting Jail"), the entries $M_{ij}^S=P\left(j\right|i)$ are the probabilities from rolling two dice, where $j=13,\dots ,23$, and $i=41,42,43$.

The jail matrix is given by

$$J_{ij}=\left\{\begin{array}{cc}215/216\hfill & \mathrm{if}i=j\mathrm{and}1\le i\le 40\mathrm{with}i\ne 31\hfill \\ 1/216\hfill & \mathrm{if}j\mathrm{and}1\le i\le 40\mathrm{with}i\ne 31\hfill \\ 1\hfill & \mathrm{if}i=31,j=41,\mathrm{or}41\le i,j\le 43\hfill \\ 0\hfill & \mathrm{if}\mathrm{otherwise}\hfill \end{array}\right.$$

where it describes the rule for rolling doubles three times, the probability of which is ${(1/6)}^3=1/216$, so the probability of staying out of Jail is $1-1/216=215/216$, and $J_{ij}=P\left(j\right|i)=215/216$, when $i=j,j\ne 41$, and $J_{ij}=P\left(j\right|i)=0$, when $i\ne j,j\ne 41$, $J_{ij}=P\left(j\right|i)=1/216$ when $j=41$.

Special cases: 31st, 41st, 42nd, and 43rd.

• For the 31st, when players land in this state, they will go to Jail. So $J_{ij}=P\left(j\right|i)=0$ when $j\ne 41$, $J_{ij}=1$ when $j=41$.
• The 41st, 42nd, and 43rd players who are already in Jail will stay in Jail. So $J_{ij}=P\left(j\right|i)=0$ when $i\ne j$, $J_{ij}=1$ when $i=j$.

We will now examine the jail matrix. Adding two states and the doubles rule makes it more complex. The chance of rolling a double is ${(6/36)}^2=(1/6)(1/6)=6/36$, so the chance of rolling three doubles in a row is ${(6/36)}^3$. Each roll is independent, meaning the chance of rolling a double does not change, no matter the previous rolls. So, the chance of getting three doubles in a row is ${(6/36)}^3=(1/6)(1/6)(1/6)=(1/216)$. In the $43\times 43$ jail matrix, all the diagonal entries are ones, except for state 31, the Policeman square. Here, the chance to stay in each state is $215/216$, and to go to Jail in state 41 is $1/216$. Players can be sent to Jail anytime by rolling three doubles in a row. The exception is state 31, where going to Jail is certain. Players in states 41, 42, or 43 stay put. So, we made a jail matrix that accounts for the chance of rolling doubles three times.

The chance matrix is given by

$$C{h}_{ij}=\left\{\begin{array}{cc}1/16\hfill & \mathrm{if}j=1,6,12,25,40,41,i=8,23,37,\hfill \\ & [8,6],[8,5],[8,13],[23,20],[23,29],[37,29],[37,34]\hfill \\ 2/16\hfill & \mathrm{if}[23,26],[37,36]\hfill \\ 3/16\hfill & \mathrm{if}j=6\mathrm{and}i=8\hfill \\ 6/16\hfill & \mathrm{if}j=i\mathrm{and}i=8,23,37\hfill \\ 1\hfill & \mathrm{if}1\le i,j\le 43\mathrm{with}8,23,37\hfill \\ 0\hfill & \mathrm{otherwise}\hfill \end{array}\right.$$

The rules for calculating probabilities are as follows:

• For the first and last rows, as well as for all other rows except the 8th, 23rd, and 37th, there is no change. Therefore, $C{h}_{ij}=P\left(j\right|i)=0$ for $i\ne j$ when $i\ne 8,23,37$.
• Additionally, $C{h}_{ij}=P\left(j\right|i)=1$ when $i=j$ and $i\ne 8,23,37$.
• In the special case of the 8th, 23rd, and 37th rows, the probability depends on randomly drawn cards. Therefore, there are no specific formulations, and the probability is equal to the probability of the drawn cards as shown in Table 4.

The community chest matrix is given by

$$C{c}_{ij}=\left\{\begin{array}{cc}1/16\hfill & \mathrm{if}i=3,18,34\mathrm{and}1\le j\le 41\hfill \\ 4/16\hfill & \mathrm{if}i=3,18,34\mathrm{and}i=j\hfill \\ 1\hfill & \mathrm{if}i\ne 3,18,34\mathrm{and}i=j\hfill \\ 0\hfill & \mathrm{otherwise}\hfill \end{array}\right.$$

For the 1st and last row (in fact for all rows except the 3rd, 18th, and 34th), there is no change. So, $C{c}_{ij}=P\left(j\right|i)=0$ for $i\ne j$, when $i\ne 3,18,34$. For $i=j$, when $i\ne 3,18,34$, $C{c}_{ij}=P\left(j\right|i)=1$. For the 3rd, 18th, and 34th rows, the probability depends on randomly drawn cards, as shown in Table 3.

This matrix has ones down the diagonal except in the 3rd, 18th, and 34th states, where the diagonal is $14/16$. This is because 14 cards in the Community Chest will not move the player, and there are two that do-one being $``$Advance to Go" and the other being $``$Go to Jail". For the 3rd, 18th, and 34th rows, these cards will send a player to state 1 or state 41, with the probability of each being $1/16$.

4.2.2. Numerical Results for the $43\times 43$ Model (Long Jail) and for the $41\times 41$ Model (Short Jail)

Solutions for the stationary distribution of the Monopoly Markov chains using the mEngel algorithm, the power method, and the canonical decomposition of absorbing Markov chains are shown in Table 5. There is no sizable difference in the results for all the distribution values when they are judged to three decimal places. A comparison of the convergence of the Monopoly Markov chains using the mEngel algorithm and the power method is shown in Table 6. These results indicated that by reducing the tolerance, adjusting the parameters such as c, and increasing the number of iterations, the differences between the two results get smaller and smaller, which implies that the solutions for the mEngel algorithm are close to the results for the power method. Ranking based on the stationary distribution of the Monopoly Markov chains using the long jail strategy and the short jail strategy is shown in Table 7. The ranking order of our mEngel results is the same as [1,6] and [4]. The probabilities of Jail using the $43\times 43$ model and the $41\times 41$ model as shown in Table 8 are 11.70% and 6.302%, respectively. The most landed on the square in both strategies is Jail. Following $``$In Jail/Just Visiting", the second most frequently visited square on the board is Illinois Avenue. Note that there exists an Advance to Illinois Avenue card in the Chance deck. In terms of the short jail strategy, the subsequent most commonly landed-on squares are St. James Place, Tennessee Avenue and New York Avenue, situated 6, 8, and 9 squares beyond Jail, respectively. Based on the results of these two strategies, the player should aim to get out of Jail as quickly as possible by paying the fine before all properties have been purchased.

4.3. The $123\times 123$ Model

Let us define $\widehat{S}$ as the order set of 123 states and S as the order set of 43 states. Two sets $\widehat{S}$ and S have the same order as their matrices. And we have divided each state $s_i$ (except Jail) into 3 states $s_i^1,s_i^2,s_i^3$. For

$$S=\{s_1,s_2,\dots ,s_{40},j_1,j_2,j_3\},$$

then

$$\widehat{S}=\{s_1^1,s_2^1,\dots ,s_{40}^1,s_1^2,s_2^2,\dots ,s_{40}^2,s_1^3,s_2^3,\dots ,s_{40}^3,j_1,j_2,j_3\}.$$

Let us construct the transition matrix $\widehat{P}$ for the $123\times 123$ model by making use of the first 40 squares of the regular transition matrix

$$P={\mathit{M}}_{43\times 43}{\widehat{\mathit{J}}}_{43\times 43}{\mathit{Ch}}_{43\times 43}{\mathit{Cc}}_{43\times 43}$$

and the last three states in the long jail strategy, where $\mathit{M}$, $\mathit{Ch}$, and $\mathit{Cc}$ are same as for the $43\times 43$ model, and $\widehat{\mathit{J}}$ is different from $\mathit{J}$. $\widehat{\mathit{J}}$ is a diagonal matrix, but in the 31st row ${\widehat{\mathit{J}}}_{31,31}=0$ and ${\widehat{\mathit{J}}}_{31,41}=1$, which is "Go to Jail":

$$\widehat{P}=\begin{array}{cc}& \begin{array}{cccc}1-40& 41-80& 81-120& 121-123\end{array}\\ \begin{array}{c}1-40\\ 41-80\\ 81-120\\ 121-123\end{array}& \left[\begin{array}{cccc}\left({\displaystyle \frac{5}{6}}\right)P& \left({\displaystyle \frac{1}{6}}\right)\left({\displaystyle \frac{5}{6}}\right)P& \left({\displaystyle \frac{1}{6}}\right)\left({\displaystyle \frac{1}{6}}\right)P& J1\\ \left({\displaystyle \frac{5}{6}}\right)P& \left({\displaystyle \frac{1}{6}}\right)\left({\displaystyle \frac{5}{6}}\right)P& \left({\displaystyle \frac{1}{6}}\right)\left({\displaystyle \frac{1}{6}}\right)P& J1\\ \left({\displaystyle \frac{5}{6}}\right)P& \left({\displaystyle \frac{1}{6}}\right)\left({\displaystyle \frac{5}{6}}\right)P& \left({\displaystyle \frac{1}{6}}\right)\left({\displaystyle \frac{1}{6}}\right)P& J1\\ J3& 0& 0& J2\end{array}\right]\end{array},$$

where $J{1}_{ij}=\left\{\begin{array}{cc}{P}_{\left(imod40\right)41}\hfill & j=121\hfill \\ 0\hfill & j=122,123\hfill \end{array}\right.$, $J{2}_{ij}=P_{(i-80)(j-80)}$, and $J{3}_{ij}=P_{(i-80)j}$.

To clarify, $J1$ represents the transition probability from 40 normal states to 3 Jail states. Each state is equivalent to the state with an index 40 greater and 80 greater when used as the starting state for this round. Therefore, the probability $\widehat{P}$ in $[1-40,121-123]$, which denotes the probability of moving from the `1st to the 40th’ state to the `121st to the 123rd’ state,, should be the same as $\widehat{P}$ in $[41-80,121-123]$ and $[81-120,121-123]$. As a result, $[1-40,121-123]$, $[41-80,121-123]$, and $[81-120,121-123]$ are all denoted as $J1$. Moreover, the transition from $[121-123]$ to $[1-40]$ represents starting from 3 Jail states and transitioning to 40 normal states, which is equivalent to $[41-43,1-40]$ in P. Thus, this part should be labelled as $J3$. The transition from $[121-123]$ to $[41-120]$ represents starting from 3 Jail states and transitioning to 40 normal states with some doubles. However, since players cannot roll doubles to go to Jail when `leaving Jail’, the probability for this transition is 0. The transition from 3 Jail states to 3 Jail states is labelled as $J2$ because it is equivalent to $[41-43,41-43]$ in P.

If we use the transition matrix in the long jail strategy to calculate P, we will obtain the transition matrix for the long jail strategy with 120 states. Similarly, if we use the transition matrix in the short jail strategy to calculate P, we will obtain the transition matrix for the short jail strategy with 120 states.

It is important to note that both ${\widehat{P}}_{ij}$ and $P_{ij}$ represent the probability of going from state i to state j. The difference lies in the order of states, and $\widehat{P}$ has more states than P, as explained above (see S and $\widehat{S}$). Understanding the $123\times 123$ model is crucial for determining the relationship between the i-th state in P and $\widehat{P}$. In ${\widehat{P}}_{ij}$, i is the starting state, and j is the ending state, using the symbols in S and $\widehat{S}$. When $s_i$ is the starting state, it is the same as $s_i^1$, $s_i^2$, and $s_i^3$, and this implies $P_{x|s_i}={\widehat{P}}_{x|s_i^1}={\widehat{P}}_{x|s_i^2}={\widehat{P}}_{x|s_i^3}$. However, when $s_i$ is the ending state, it is not the same as $s_i^1$, $s_i^2$, and $s_i^3$, but follows the relationship $P_{si|x}=\left({\displaystyle \frac{5}{6}}\right){\widehat{P}}_{s_i^1|x}=\left({\displaystyle \frac{1}{6}}\right)\left({\displaystyle \frac{5}{6}}\right){\widehat{P}}_{s_i^1|x}=\left({\displaystyle \frac{1}{6}}\right)\left({\displaystyle \frac{1}{6}}\right){\widehat{P}}_{s_i^1|x}$, where $x\in \widehat{S}$.

The structure consists of 43 states, including 40 regular states and 3 Jail states, and 123 states including 120 regular states and 3 Jail states. No additional jail states have been added.

Ranking based on the stationary distribution of the Monopoly Markov chains using the long jail strategy and the short jail strategy for the $123\times 123$ model is shown in Table 9. These two ranking results are in similar order. The probabilities of landing on Jail using the $123\times 123$ model as shown in Table 8 are 10.64% and 5.89%, respectively. The most landed on the square in both strategies is Jail. Following $``$In Jail/Just Visiting", the subsequent frequently visited square on the board is Illinois Avenue for long jail, but Illinois Avenue is the third place for short jail.

4.4. The Return of Monopoly

We know that some properties (states) in Monopoly have the same colours. If we collect all states with the same colour, we will profit from it. Return is the ratio of expected money and cost. It shows how much money a player can take back at every turn. Players should buy all the properties of one colour and choose the colour based on which will provide the highest returns.

The formula for determining the return based on the number of turns is given by [16]:

$$\mathrm{Turn}={\displaystyle \frac{\mathrm{Cost}}{p·R·E\left(x\right)}}$$

where $\mathrm{Cost}$ is the development cost for a house or a hotel and is different for each property, p is the probability of landing on the property, R is the rent earnings, and $E\left(x\right)$ is the expectation of rolling x times in one turn. If a player rolls doubles three times, this player will go to Jail. So, a player can roll doubles at most three times before the next player rolls. Let x be the number of rolls in one turn, $x\in \{1,2,3\}$. Then $p\left(x\right)$ is the probability of a player rolling x times in one turn. Then $p\left(1\right)={\displaystyle \frac{30}{36}}={\displaystyle \frac{5}{6}}$, where the player rolls no doubles. $p\left(2\right)={\displaystyle \frac{6}{36}}\left({\displaystyle \frac{30}{36}}\right)={\displaystyle \frac{1}{6}}\left({\displaystyle \frac{5}{6}}\right)$, where the player rolls doubles once and non-doubles once. And $p\left(3\right)={\displaystyle \frac{6}{36}}\left({\displaystyle \frac{6}{36}}\right)\left(1\right)={\displaystyle \frac{1}{6}}\left({\displaystyle \frac{1}{6}}\right)$, where the player rolls two doubles and then, regardless of what the player rolls next, it will end his turn. So the expectation of rolling x times in one turn is:

$$E\left(x\right)=1·p\left(1\right)+2·p\left(2\right)+3·p\left(3\right)=1.19\overline{4}.$$

The rent and cost values are sourced from [12,24]. Let us consider Mediterranean Avenue as an example, using the long jail strategy shown in Table A8 (See Appendix B). If the player owns Mediterranean Avenue, it takes approximately 1234 turns to recoup the cost of the purchase price. Similarly, if the player builds a house, generating revenue will take approximately 699 turns. Here are the rest of the cases:

• For $p=0.020355$, $\mathrm{Cost}=60$, $R=2$, the number of the turns is calculated as $\mathrm{Turn}=60·{\left(0.02\right)}^{-1}·2^{-1}·{\left(1.19\overline{4}\right)}^{-1}=1233.914$.
• For 1 house, with $R=10$ and $\mathrm{Cost}=170$, $\#\mathrm{of}\mathrm{Turns}=170·{\left(0.020355\right)}^{-1}·{10}^{-1}·{\left(1.19\overline{4}\right)}^{-1}=699.2179$.
• For 2 houses, with $R=30$ and $\mathrm{Cost}=220$, $\#\mathrm{of}\mathrm{Turns}=220·{\left(0.020355\right)}^{-1}·{30}^{-1}·{\left(1.19\overline{4}\right)}^{-1}=301.6234$.
• For 3 houses, with $R=90$ and $\mathrm{Cost}=270$, $\#\mathrm{of}\mathrm{Turns}=270·{\left(0.020355\right)}^{-1}·{90}^{-1}·{\left(1.19\overline{4}\right)}^{-1}=123.3914$.
• For 4 houses, with $R=160$ and $\mathrm{Cost}=320$, $\#\mathrm{of}\mathrm{Turns}=320·{\left(0.020355\right)}^{-1}·{160}^{-1}·{\left(1.19\overline{4}\right)}^{-1}=82.2609$.
• For a hotel, with $R=250$ and $\mathrm{Cost}=370$, $\#\mathrm{of}\mathrm{Turns}=370·{\left(0.020355\right)}^{-1}·{250}^{-1}·{\left(1.19\overline{4}\right)}^{-1}=60.8731$.

Now, let us examine the example of Mediterranean Avenue using the long jail strategy for the $123\times 123$ model, as shown in Table A9 (see Appendix B). An inspection of Table A9 reveals that the number of turns required to recoup the purchase price of Mediterranean Avenue, including the player’s properties, houses, and a hotel, is nearly identical to that of the $43\times 43$ model.

Similar calculations were performed for all the states to calculate the number of turns in Monopoly needed to recoup the cost of the purchase price for each property. This was done using the long jail strategy for both the $43\times 43$ model and the $123\times 123$ model, and using the short jail strategy for both the $41\times 41$ model and the $123\times 123$ model. The results are summarized in Table A8, Table A9, Table A10, and Table A11 in Appendix B, respectively, where NA stands for not applicable. Although the stationary probabilities varied slightly for each property, the number of turns remained the same. In Table A8, Table A9, Table A10, and Table A11 in Appendix B, we also observe how fast a player can start to make a profit from each state and each colour group. For the state, Boardwalk and New York Avenue require the least number of turns before they start making money.

The rule for building on properties of the same colour in Monopoly is that a player must own all the sites of the same colour before building on them. We take Brown (Mediterranean Avenue and Baltic Avenue) as an example. To build one house on Mediterranean Avenue, they must first buy all the brown land. So, a player must pay for all the land before building a house. The land cost is 60 for each. Then the house cost is 50 for each. So, the cost of land on brown is $2\left(60\right)=120$, the cost to build 1 house on brown is $2(60+50)=220$, and the cost to build 2 houses on brown is $2(60+(50\left)2\right)=320$. Therefore the cost of building 1 house on Mediterranean Avenue is $2\left(60\right)+50=170$. We want to calculate how much we need to pay to build h houses in some states. Let $\mathrm{baseCost}$ be the cost of buying all lands of the same colour, and $\mathrm{houseCost}$ be the cost of building one house in a state. We first need to buy all the land of the same colour to build a house. So we need to pay $\mathrm{baseCost}$ first, and then we pay $h\times \mathrm{houseCost}$ on building h houses. Hence we cost $\mathrm{baseCost}+h\times \mathrm{houseCost}$ to build h houses on the state, which is noted as $\mathrm{Cost}=\mathrm{baseCost}+h\times \mathrm{houseCost}$. Let n be the number of states in some colour. If we want to calculate the cost of building h houses on each site in the same colour group, which is noted as ${\mathrm{Cost}}_{\mathrm{color}}$, we need to pay $\mathrm{baseCost}$ to buy all lands and pay $h\times n\times \mathrm{houseCost}$ on building houses. So ${\mathrm{Cost}}_{\mathrm{color}}=\mathrm{baseCost}+h\times n\times \mathrm{houseCost}$. We have $h\times \mathrm{houseCost}=\mathrm{Cost}-\mathrm{baseCost}$ then ${\mathrm{Cost}}_{\mathrm{color}}=\mathrm{baseCost}+n\times (\mathrm{Cost}-\mathrm{baseCost})$. For properties of the same colour group, the formula is given as:

$$\mathrm{Turn}={\displaystyle \frac{{\mathrm{Cost}}_{\mathrm{color}}}{\sum p·\mathrm{Average}\mathrm{of}\mathrm{Rent}·E\left(x\right)}}$$

where ${\mathrm{Cost}}_{\mathrm{color}}=\mathrm{Cost}·n-\mathrm{baseCost}·(n-1).$

For instance, let us consider the Brown property using the short jail strategy. Below is the number of turns required for the Brown property group to generate income, including the player’s properties and 2 houses. Similar calculations apply to other houses and a hotel:

• $p=0.02125+0.02158=0.04282$, baseCost = 60 + 60 = 120, $R={\displaystyle \frac{2+4}{2}}=3$, the number of the turns is calculated as $\#\mathrm{of}\mathrm{Turns}=120·{\left(0.04282\right)}^{-1}·3^{-1}·{\left(1.19\overline{4}\right)}^{-1}=782.0731$.
• For 2 houses, with $R={\displaystyle \frac{30+60}{2}}=45$, $\mathrm{cost}=220+220-\mathrm{baseCost}·(2-1)=440$, $\#\mathrm{of}\mathrm{Turns}=320·{\left(0.04282\right)}^{-1}·{45}^{-1}·{\left(1.19\overline{4}\right)}^{-1}=307.0004$.

Table 10 summarises similar calculations for all color groups using the long jail strategy for both the $43\times 43$ model and $123\times 123$ model, and using the short jail strategy for both the $41\times 41$ model and the $123\times 123$ model, respectively. The table shows the number of turns it takes for a player to recoup the cost of the purchase price. We compared the ranking based on the number of turns in Monopoly required to recoup the costs of owning hotels for the same color groups. Our observations indicate that the Orange and Light Blue properties are the quickest to start making money of all the colors.

The hotel returns ranking findings for both strategies are in the same order and are consistent with [16] and [7].

Table 11. Ranking the number of turns in Monopoly for properties of the same color with a hotel, using the long jail strategy for both the $43\times 43$ model and the $123\times 123$ model, and using the short jail strategy for both the $41\times 41$ model and the $123\times 123$ model.

Rank of Hotel Return	States	Long jail	States	Short jail
		$43\times 43$ model		$41\times 41$ model
1	Brown	36.10167 ≈ 36	Brown	33.97609 ≈ 34
2	Green	34.25238 ≈ 34	Green	32.25762 ≈ 32
3	Yellow	29.61302 ≈ 30	Yellow	27.82833 ≈ 28
4	Dark Blue	28.88921 ≈ 29	Dark Blue	27.26097 ≈ 27
5	Red	28.57838 ≈ 29	Red	26.66321 ≈ 27
6	Light Purple	28.09622 ≈ 28	Light Purple	26.66097 ≈ 27
7	Light Blue	24.13126 ≈ 24	Light Blue	22.7328 ≈ 23
8	Orange	19.44593 ≈ 19	Orange	18.33277 ≈ 18
Rank of Hotel Return	States	Long jail	States	Short jail
		$123\times 123$ model		$123\times 123$ model
1	Brown	35.82262 ≈ 36	Brown	33.43983 ≈ 33
2	Green	35.23033 ≈ 35	Green	32.79666 ≈ 33
3	Yellow	29.96575 ≈ 30	Yellow	28.13598 ≈ 28
4	Dark Blue	29.31405 ≈ 29	Dark Blue	27.10931 ≈ 27
5	Red	28.15502 ≈ 28	Red	26.79369 ≈ 27
6	Light Purple	26.72414 ≈ 27	Light Purple	26.14594 ≈ 26
7	Light Blue	23.01746 ≈ 23	Light Blue	22.10345 ≈ 22
8	Orange	18.90063 ≈ 19	Orange	18.38298 ≈ 18

Table 11. Ranking the number of turns in Monopoly for properties of the same color with a hotel, using the long jail strategy for both the $43\times 43$ model and the $123\times 123$ model, and using the short jail strategy for both the $41\times 41$ model and the $123\times 123$ model.

Rank of Hotel Return	States	Long jail	States	Short jail
		$43\times 43$ model		$41\times 41$ model
1	Brown	36.10167 ≈ 36	Brown	33.97609 ≈ 34
2	Green	34.25238 ≈ 34	Green	32.25762 ≈ 32
3	Yellow	29.61302 ≈ 30	Yellow	27.82833 ≈ 28
4	Dark Blue	28.88921 ≈ 29	Dark Blue	27.26097 ≈ 27
5	Red	28.57838 ≈ 29	Red	26.66321 ≈ 27
6	Light Purple	28.09622 ≈ 28	Light Purple	26.66097 ≈ 27
7	Light Blue	24.13126 ≈ 24	Light Blue	22.7328 ≈ 23
8	Orange	19.44593 ≈ 19	Orange	18.33277 ≈ 18
Rank of Hotel Return	States	Long jail	States	Short jail
		$123\times 123$ model		$123\times 123$ model
1	Brown	35.82262 ≈ 36	Brown	33.43983 ≈ 33
2	Green	35.23033 ≈ 35	Green	32.79666 ≈ 33
3	Yellow	29.96575 ≈ 30	Yellow	28.13598 ≈ 28
4	Dark Blue	29.31405 ≈ 29	Dark Blue	27.10931 ≈ 27
5	Red	28.15502 ≈ 28	Red	26.79369 ≈ 27
6	Light Purple	26.72414 ≈ 27	Light Purple	26.14594 ≈ 26
7	Light Blue	23.01746 ≈ 23	Light Blue	22.10345 ≈ 22
8	Orange	18.90063 ≈ 19	Orange	18.38298 ≈ 18

5. Conclusions

In the present study, we have unveiled the mEngel algorithm, which addresses the computation of absorbing and non-absorbing Markov chains through a nested iterative process. We have also furnished proof to demonstrate the efficacy of the mEngel algorithm in resolving non-absorbing Markov chain issues, correlating it with the power method’s procedures and the canonical decomposition of absorbing Markov chains. The mEngel algorithm was implemented in a sequential manner using R, Algorithms 1 - 3 being elucidated in detail. Our proposed algorithm has two key features that set it apart from the original Engel algorithm [8,9,21] and the approach in [13]:

• It can be applied to non-absorbing and absorbing Markov chains, whereas the original Engel algorithm is limited to absorbing cases.
• It provides process values, such as the passing frequency of intermediate states, which traditional methods cannot provide. This capability allowed us to identify which state had at firing, referred to as a firing state, and record this information for each configuration.

We explored the Engel algorithm’s capability to compute the absorbing probabilities for Torrence’s problem, with findings that align with established scholarly works. Determining the steady-state probabilities for non-absorbing states in Monopoly, particularly concerning the Jail rules, was also investigated. Using the long jail strategy, the short jail strategy, and the strategy of getting out of Jail by rolling consecutive doubles three times, the $43\times 43$ model, the $41\times 41$ model, and the $123\times 123$ model, respectively, were formulated and tested, and their results are consistent with existing literature. Our findings show that using the short jail strategy is a common practice. Early in the game, being in Jail reduces the player’s ability to purchase property, but during the endgame, it protects the player from paying the player’s opponent’s rent. The player should get out of Jail immediately if the player can pay $50 or use a $``$Get Out of Jail Free" card at any time from the Community Chest or the Chance card pile, moving the player’s token to $``$Just Visiting". And we gave the rewards of each state and color group under different jail strategies, and players can get their best strategy based on the results. Further research is necessary when considering a case-by-case card evaluation function [27] to assess the value of properties using the mEngel algorithm. The main drawback of the mEngel algorithm is that it takes longer computational time to achieve high accuracy compared to other methods in the same problem setting. To decrease the computational time, it is essential to reorganize the architecture of Algorithms 1 - 3 by, for example, adjusting the number of iterations, nested loops, and recursive calls. These results will be published elsewhere.