Equilibrium Strategies for Overtaking-Free Queueing Networks under Partial Information

1. Introduction

This work aims to find the equilibrium strategy adopted by customers who arrive at an overtaking-free queueing network, under the assumption that, on their arrival, they are informed only about the total number of customers already in the system but not their distribution among the queues. The analysis of optimal strategies for queueing systems can be traced back since 1969, when Naor found the individual’s optimizing strategy that gives a rule for join or balk in the observable $M/M/1$ model [1]. This work was later extended by Edelson and Hildebrand in 1975, by looking at the unobservable $M/M/1$ model [2]. A nice recollection of results on queueing systems under the economical perspective, can be found in the recent monograph by Hassin and Haviv [3].

Since the 2000s, studies on partially observable models have intensified. In particular, regarding single service systems, this topic was discussed in 2007 by Guo and Zipkin, who looked at the effects on the model with: no information, partial information and full information [4], and, in 2008, by Economou and Kanta, who explored the single server Markovian queue with compartmented waiting space [5]. As for batch service queueing systems, in 2017, Bountali and Economou studied the strategic customer behavior for the observable and unobservable $M/M/1$ queue with single arrivals and batch services [6], and, in 2019, they extended this model, by considering the partially observable case, where the arriving customers observe only the number of complete batches that wait to be served [7].

While most of the above research on optimal strategies in queueing systems, refers to single queues, recently few results have been obtained about queueing networks. A first attempt has been done in D’Auria and Kanta [8], who analyzed the two-node tandem network and showed that a pure threshold equilibrium policy exists, assuming that arriving customers are informed only about the total number of users in the system. Later, Bara Kim and Jeongsim Kim generalized this result by considering multi-node tandem networks [9]. This work extends these results by focusing on the general overtaking-free queueing networks under partial information.

The class of overtaking-free networks appeared in the queueing context already in 1979, when Walrand and Varaiya proved that in an open multiclass Jackson network the sojourn times of a customer along an overtaking-free path are independent [10]. Kelly and Pollett obtained a similar result in 1983 analyzing closed Jackson network [11]. The overtaking-free condition ensures that no customer can influence the sojourn times of any other customer arrived earlier in the system. In this study, we characterize the topology of the FIFO overtaking-free networks with exponential service times, and show how to compute and optimize the profit function of partially informed arriving customers.

In the sequel, Section 2 contains the characterization of the overtaking-free networks, under the mentioned assumptions. In Section 3, we introduce the model and the notation. Then, in Section 4, we get the stationary distribution for the number of customers in the system in the unobservable and the observable case and compute the expected sojourn time at any queue given the total number of customers in the system. In Section 5, we derive the equilibrium threshold strategies. In Section 6, we provide some numerical computations by comparing two different queueing networks and finally in Section 7 we draw some conclusions.

2. Overtaking-Free Networks

In this section, we show that, under a FIFO discipline and with exponential service times, a necessary and sufficient condition for a queueing network to be overtaking-free is that the its underlying graph is an out-tree.

We define by $G(V,E_0)$ the underlying finite directed graph of a queueing network. V represents the set of vertices, that correspond to the nodes where customers can queue up and receive their service. $E_0$ is the set of edges, that correspond to the allowed movements between queues and towards to the outside. We assume that $V\subset \mathbb{N}$ and $E_0\subset {\mathbb{N}}_0\times {\mathbb{N}}_0$, where ${\mathbb{N}}_0=\mathbb{N}\cup \left\{0\right\}$. We denote by $0\notin V$ the outside vertex, and we let $V_0=V\cup \left\{0\right\}$ and $E=E_0\cap (\mathbb{N}\times \mathbb{N})$. Sometimes we refer to the graph $G(V_0,E_0)$ with the intention to include the outside node inside the set of vertices . $D\left(i\right)=\{j\in V_0:(i,j)\in E_0\}$ denotes the set of vertices, in $V_0$, that are reachable from a vertex $i\in V_0$. After completing the service at queue $i\in V$, a customer either joins a queue among the nodes in $D\left(i\right)\cap V$ or leaves the system, whenever $0\in D\left(i\right)$. We let $d\left(i\right)=\left|D\right(i\left)\right|$ be the out-degree of vertex $i\in V_0$.

We assume that in the graph $G(V_0,E_0)$, the outside node can access and can be accessed by any node via a given path of edges. We also assume that the directed graph $G(V,E)$ is connected, that is, for any non trivial subset $A\subset V$, there exists an edge $(i,j)\in E$, such that either $i\in A$ and $j\in V\setminus A$ or $i\in V\setminus A$ and $j\in A$. This implies that the network cannot be split into two separated sub-networks. If $d\left(0\right)=1$, we say that the graph $G(V,E_0)$ (as well as $G(V,E)$) is rooted. In this case, we assume that the root node is the vertex $1\in V$. The directed graph $G=(V,E)$, with root vertex $1\in V$, is an out-tree if $\left|E\right|=\left|V\right|-1$ and, for any $v\in V\setminus \left\{1\right\}$, there exists a directed path from 1 to v. It also holds that such a path is unique.

Now, we give a formal definition of overtaking-free network [10,11]. We have in mind that the outside node 0 is a source of customers. We can think of it as being a single server queue with infinite number of customers in its line. In general, we are going to assume that this source of arriving customers is stochastically modeled as a Poisson process. We then define a tagged customer as a customer that just left the outside node 0 to reach any node in the set $D\left(0\right)\subset V$, and that we are going to analyze.

Definition 1 

(overtaking-free network). A queueing network is overtaking-free if and only if the sojourn times of a tagged customer at any queue along her path do not depend on the behavior of the customers arriving at the network at later times. That is, her sojourn times stay the same if no more customers are allowed to enter the network.

We define the following standing assumptions:

Assumptions 1. 

• the inter-arrival times, the service times and the routing probabilities are independent random variables, whose distribution may only be node dependent;
• each node is a single server queue;

The above assumptions are implicitly assumed in the statement of the following theorem as well as for the rest of the paper.

Theorem 1 

(Characterization of overtaking-free networks). Under the FIFO discipline and exponential service times at each queue, a necessary and sufficient condition for a queueing network to be overtaking-free is that its underlying graph $G(V,E)$ is an out-tree.

Proof. 

It is clear that if the graph $G(V,E)$ is an out-tree and the service discipline is FIFO, then the network is overtaking-free. Indeed, each node would be reachable by a unique path, and along it, customers are served according to a FIFO discipline. On the other hand, assume that the network structure is not an out-tree, and consider two alternative paths joining two vertices $v_1,v_2\in V$. Then, there is a positive probability that two consecutive arriving customers go along the two alternative paths $0\to v_1\to v_2\to 0$, where the arrows denote a sub-path between two nodes. Since the support of the exponential distribution is $(0,\infty )$, there is a positive probability that the second customer, the one arrived later, will reach the node $v_2$ before the first customer, and will queue there in front of her. This would contradict the overtaking-free network hypothesis that the presence of the second user in the network should not alter the sojourn times of the first one. □

The previous proof would hold even if we substitute the exponential service distribution with any distribution whose support is unbounded or dense close to 0. However, in general, if we change the service discipline or the distribution of the service times, Theorem 1 may no longer be valid. One can easily construct counterexamples by using degenerate deterministic service times.

We conclude that, under a FIFO discipline and with exponential service times, in an overtaking-free network, node 1 serves as the root of the tree, and all other nodes have only one entering edge.

3. The Model

Under the assumption of exponentially distributed service times, an overtaking-free network is characterized by having a branching structure, described in Section 2, represented by its underlying graph that is an out-tree. A tandem network is a special subclass of overtaking-free network, and it has been studied in [8] for the case of two queues and [9] for a general finite number of queues. Here we extend those analyses to the whole class of overtaking-free networks.

The methodology consists in exploiting the product form expression of the stationary distribution of the network and comparing it with a closed one having the same number of customers inside as the one communicated to the tagged customer at the arrival epoch. We are allowed to make this comparison because all customers behind the tagged one can be overlooked under the overtaking-free assumption since they do not interfere in the calculation of her expected sojourn time.

Using this methodology, we compute a common equilibrium threshold strategy that all customers are going to implement in deciding whether to join or balk the system. This decision is only based on the partial information received at the arrival epoch, consisting in the total number of customers already inside the network.

The optimal strategy is computed by building an expected profit function P, that only depends on the number of users in the system and the common strategy followed by the rest of customers. The sign of the profit function corresponds to the decision of the tagged customes, she decides to join if the sign is positive, otherwise she decides to balk. Once joined the system, she is committed to her decision and cannot renege, and, if she chooses not to join, she will not have another opportunity to come back.

Finally, we verify that this optimal response leads to an equilibrium.

Consider an overtaking-free network and denote with $G=(V,E)$ the associated out-tree graph. Let $V=\{1,2,\dots ,N\}$ and $E=\{e_2,e_3,\dots ,e_N\}$, so that N is the total number of queues, and $e_i$, for $i=2,\dots ,N$, denotes the edge entering in node i, which we know to be unique.

Customers arrive at the network through node 1, according to a Poisson process with intensity $\lambda$ and are served based according to a FIFO discipline. A customer completing her service at node $i\in V$, is routed to another queue at node $j\in D\left(i\right)$ with probability $p_{ij}>0$. Remember that 0, the outside node, may belong to $D\left(i\right)$ allowing the customer to leave the network when served at node $i\in V$. The service times of customers at each queue $i\in V$ are independent and exponentially distributed with mean $1/{\mu }_i$.

Figure 1 illustrates an example of overtaking-free network.

4. Stationary Distribution

The only information that is provided to the customers at the moment of their arrival consists in the total number of customers in the system. Using this information they may decide to join the network or balk, and they may do it according to a strategy potentially random. It follows that we may represent a strategy, say s, as an infinite vector, that is $s=(s_0,s_1,\dots )$,where $s_k$, is the probability that an arriving customer will join the system after observing a total number of k customers in it. In particular, for $K\in \mathbb{N}$, we denote by ${\sigma }_K$, the K-threshold strategy, with the property that $s_k={\mathbb{1}}_{\{k<K\}}$.

Remark 1. 

We say that a strategy s is a pure strategy whenever $s_k\in \{0,1\}$ for all $k\in \mathbb{N}$. A K-threshold strategy is a pure strategy. We say that a strategy s is a mixed x-threshold strategy, where $x=n+p$ with $n\in \mathbb{N}$ and $p\in (0,1)$, if $s_k=1$ for $k=0,\dots ,n-1$, $s_n=p$, and $s_k=0$ for $k>n$. This means that an arriving customer certainly joins if there are less than n customers in the system, joins with probability p if there are exactly n customers, and otherwise she balks. A mixed threshold strategy is a convex combination of two pure threshold strategies, indeed ${\sigma }_x=(1-p){\sigma }_n+p{\sigma }_{n+1}$.

We say that a strategy is admissible if whenever $s_K=0$, for a given $K\in \mathbb{N}$, then $s_k=0$ for any $k>K$. In this case the number of customers in the network can be at most K and the network is called semi-open. If the strategy s is positive, that is $s_k>0$ for all $k\in {\mathbb{N}}_0$, the network is said open and we set $K=\infty$. To simplify the exposition, in the following when we say strategy we implicitly mean admissible strategy, and we denote by $\left|s\right|=K\in \mathbb{N}\cup \{\infty \}$, the maximum number of customers allowed in the network given the strategy s.

Under the assumption that everyone uses the common strategy s, we denote by $P\left(k\right|s)$ the expected potential profit function for an arriving customer that observes k users in the network, that is

$$P\left(k\right|s)=R-C\left(k\right|s)=R-\sum _{i\in V}{C}_i{T}_i\left(k\right|s),$$

(1)

where R is the positive reward for entering the system and $C\left(k\right|s)={\sum }_{i\in V}{C}_i{T}_i\left(k\right|s)$ is the expected cost for sojourn time. $C_i$ is the positive cost for unit sojourn time at queue i, also said cost rate, and $T_i\left(k\right|s)={\mathbb{E}}_s\left[W_i\right|Q_a=k]$ is the expected sojourn time that a joining customer will spend at queue i, under the common strategy s and having received the information $Q_a=k$. $W_i$ is the corresponding sojourn time and $Q_a$ denotes the total number of customers at her arrival time. The profit for balking is assumed to be 0.

In Section 5, we give an algorithm to compute $P\left(k\right|s)$, and find the equilibrium strategy by maximizing it under the assumption that all customers use the same strategy.

Thanks to the assumptions of independence and exponential service times, we can describe the dynamics of the network only by keeping trace of the number of customers at each queue, that is the vector $\overrightarrow{Q}\left(t\right)=(Q_i\left(t\right),i\in V)$. Indeed, the system can be described as a continuous time Markov chain with values in ${\mathbb{N}}_0^{\left|V\right|}$. In particular, this kind of network can be seen as a generalized Jackson network, whose special property consists in having a state-dependent arrival rate due to the joining strategy s; according to the nomenclature in [12], this network belongs to the class of Whittle networks. For a background on Jackson network, consult the Chapter 2 of [13], and for more general Whittle networks the book [12]. In our case the arrival rate is equal to $\lambda \left(t\right)=s_{Q\left(t\right)}\lambda$, where $Q\left(t\right)=|\overrightarrow{Q}\left(t\right)|$ is the total number of customers in the system, and $\lambda$ is the maximum arrival rate, the one achieved if no customers would decide to balk.

A nice property of Jackson and Whittle networks is that the stationary distribution, when it exists, has the so-called product form, that is the queues behave like they were almost independent.

We denote by $\overrightarrow{\nu }=({\nu }_i,i\in V_0)$ the solution of the normalized traffic equations, satisfying the normalizing condition ${\nu }_0=1$ and the following relation

$${\nu }_i=\sum _{j\in V_0}{\nu }_j{p}_{ji},i\in V.$$

(2)

We set $\overrightarrow{\eta }=({\eta }_i,i\in V_0)$ with ${\eta }_i={\nu }_i/{\mu }_i$.

When the network is overtaking-free, and its graph is an out-tree, the solution of the traffic equations (2) are readily written. Indeed, denoting by $E_i\subset E$ the set of edges that belong to the unique path from node 1 to node $i\in V$, then ${\nu }_1=1$ and, for $i\ne 1$, ${\nu }_i={\prod }_{e\in E_i}p\left(e\right)$, where $p\left(e\right)=p_{ij}$ for $e=(i,j)$.

Let ${\overrightarrow{Q}}^{*}=(Q_i^{*},i\in V)$ be a vector distributed as the stationary number of customers in the network, and let ${\overrightarrow{\pi }}_s=\left({\pi }_s\left(\overrightarrow{n}\right),\overrightarrow{n}\in {\mathbb{N}}_0^{\left|V\right|}\right)$ be the stationary distribution, so that ${\pi }_s\left(\overrightarrow{n}\right)=\mathbb{P}({\overrightarrow{Q}}^{*}=\overrightarrow{n})$ , with $\overrightarrow{n}=(n_i,i\in V)\in {\mathbb{N}}_0^{\left|V\right|}$ denoting a general state.

It is known that in a Whittle network, with any topological structure, at most one transition by time may occur, and that the time between two transitions is exponentially distributed with state dependent rate. We define by ${\mathcal{T}}_{ij}$ the operator that moves one customer from node $i\in V_0$ to node $j\in V_0$, that is ${\mathcal{T}}_{ij}\overrightarrow{n}=\overrightarrow{n}-{\overrightarrow{e}}_i+{\overrightarrow{e}}_j$, where ${\overrightarrow{e}}_i$, for $i\in V$, is the $\left|V\right|$-dimensional unit vector with the i-th coordinate equal to 1 whereas ${\overrightarrow{e}}_0=\overrightarrow{0}$. The transition rate from $\overrightarrow{n}\in {\mathbb{N}}_0^{\left|V\right|}$ to ${\mathcal{T}}_{ij}\overrightarrow{n}\in {\mathbb{N}}_0^{\left|V\right|}$, whenever $j\in D\left(i\right)$, is

$$q_s(\overrightarrow{n},{\mathcal{T}}_{ij}\overrightarrow{n})=\left\{\begin{array}{cc}{s}_{|\overrightarrow{n}|}\lambda p_{0j},\hfill & i=0\hfill \\ {\mu }_i{p}_{ij},\hfill & i\in V\hfill \end{array}\right.$$

In an overtaking-free network, $p_{01}=1$, however we keep the notation general as the stationary regime does hold for a more general topology.

Assuming that the stationary distribution exists, that occurs when the network is stable, it has the following form

$${\pi }_s\left(\overrightarrow{n}\right)\propto S\left(\right|\overrightarrow{n}\left|\right){\lambda }^{|\overrightarrow{n}|}\prod _{i\in V}{\eta }_i^{n_i},|\overrightarrow{n}|\le |s|.$$

(3)

In the sequel, we will not check the stability assumption as it always holds when $\left|s\right|<\infty$. In (3), $S\left(n\right)={\prod }_{k=0}^{n-1}{s}_k$, and the missing normalizing constant is

$${\kappa }_s^{-1}=\sum _{|\overrightarrow{n}|\le |s|}S\left(\right|\overrightarrow{n}\left|\right){\lambda }^{|\overrightarrow{n}|}\prod _{i\in V}{\eta }^{n_i}.$$

To check that (3) indeed holds, one can verify that the global balance conditions are satisfied, see also Example 1.49 in [12].

In the following, we are also going to deal with networks whose number of customers inside is kept constant, say $K\in \mathbb{N}$. In this case the network is called closed, and in it, transitions from/to node $0\in V_0$ are not allowed, therefore the strategy employed by all customers is irrelevant. When the network is closed, the normalizing condition in (2) is substituted with ${\nu }_1=1$. A closed network is always stable, and its stationary distribution has the following product form, compare with (3),

$${\pi }^{\mid K}\left(\overrightarrow{n}\right)\propto \prod _{i\in V}{\eta }_i^{n_i},\left|\overrightarrow{n}\right|=K.$$

(4)

Denoting by ${\overrightarrow{Q}}^{\mid K}$ a random vector distributed according to this stationary distribution, it has the same joint distribution as $\left|V\right|$ independent Geometric random variables with corresponding parameters ${\left({\eta }_i\right)}_{i\in V}$ conditioned to have a total sum equal to K.

Jackson and Whittle networks enjoy the MUSTA property (Moving Unit Sees a Time Average) at Simple Network Transitions, see Example 4.38 in [12], according to which, any customer moving, among the queues or from/to the outside sees, before joining the destination node, a stationary network having one less maximum number of customers.

To better formalize this property, we need to introduce the Palm probabilities. We consider the stationary network process $({\overrightarrow{Q}}^{*}\left(t\right),t\in \mathbb{R})$, whose trajectories belong to the space $\mathbb{D}$ of right-continuous functions with left limits with values in ${\mathbb{N}}_0^{\left|V\right|}$, and for fixed $i,j\in V_0$ with $i\ne j$, we define the set $\mathcal{T}$ of simple network transitions from i to j as

$$\mathcal{T}=\{({\overrightarrow{Q}}^{*}\left(t\right),t\in \mathbb{R})\in \mathbb{D}:{\overrightarrow{Q}}^{*}\left(0\right)={\mathcal{T}}_{ij}{\overrightarrow{Q}}^{*}(0-)\}.$$

(5)

Assuming the common strategy s, we denote by ${\overrightarrow{Q}}^{\mid \mathcal{T}}=({\overrightarrow{Q}}^{*}\left(0\right)-{\overrightarrow{e}}_j\mid \mathcal{T})$ the state of the network at time 0 just before a customer, making a $\mathcal{T}$-transition, has joined her destination node, and we define ${\overrightarrow{\pi }}_s^{\mid \mathcal{T}}$ its distribution. This distribution takes the name of Palm distribution, it has the special property to be conditioned to the occurrence of a $\mathcal{T}$-transition that is a null event, and it was first introduced by Palm in [14].

We give a heuristic proof of the MUSTA property in the following proposition, for a more formal proof see [12, Theorem4.37 & Example4.38].

Proposition 1 

(MUSTA property). Fix $i,j\in V_0$, $i\ne j$, and consider $\mathcal{T}$ as in (5).

For a (semi-)open network employing the strategy s, and with $s^{\prime }$ denoting the shifted strategy, that is $s_k^{\prime }=s_{k+{\mathbb{1}}_{\{i\ne 0\}}}$, then it follows that

$${\overrightarrow{\pi }}_s^{\mid \mathcal{T}}={\overrightarrow{\pi }}_{s^{\prime }},$$

(6)

where ${\overrightarrow{\pi }}_{s^{\prime }}$ denotes the distribution of a similar network in which all customers employ the strategy $s^{\prime }$.

If the stationary distribution ${\overrightarrow{\pi }}_s$ refers to a closed network with a number of customers equal to K, in which the strategy s is irrelevant, the distribution ${\overrightarrow{\pi }}_{s^{\prime }}$ refers to a closed network with $K-1$ number of customers.

Proof. 

To simplify notation, we set ${\eta }_0=1$ and $n+{\mathbb{1}}_{\{i\ne 0\}}=|\overrightarrow{n}+{\overrightarrow{e}}_i|$, for $\overrightarrow{n},\overrightarrow{n}+{\overrightarrow{e}}_i\in {\mathbb{N}}^{\left|V\right|}$ and $|\overrightarrow{n}+{\overrightarrow{e}}_i|\le |s|$. To compute ${\pi }_s^{\mid \mathcal{T}}\left(\overrightarrow{n}\right)$, we do the following calculations,

$$\begin{array}{cc}\hfill {\pi }_s^{\mid \mathcal{T}}\left(\overrightarrow{n}\right)& =\underset{t\downarrow 0}{lim}{\displaystyle \frac{\mathbb{P}({\overrightarrow{Q}}^{*}\left(t\right)=\overrightarrow{n}+{\overrightarrow{e}}_j,{\overrightarrow{Q}}^{*}(0-)=\overrightarrow{n}+{\overrightarrow{e}}_i)}{{\displaystyle \sum _{|\overrightarrow{m}+{\overrightarrow{e}}_i|\le |s|}\mathbb{P}({\overrightarrow{Q}}^{*}\left(t\right)=\overrightarrow{m}+{\overrightarrow{e}}_j,{\overrightarrow{Q}}^{*}(0-)=\overrightarrow{m}+{\overrightarrow{e}}_i)}}}\hfill \\ \hfill & =\underset{t\downarrow 0}{lim}{\displaystyle \frac{\mathbb{P}({\overrightarrow{Q}}^{*}(0-)=\overrightarrow{n}+{\overrightarrow{e}}_i)(q_{ij}t+o\left(t\right))}{{\displaystyle \sum _{|\overrightarrow{m}+{\overrightarrow{e}}_i|\le |s|}\mathbb{P}({\overrightarrow{Q}}^{*}(0-)=\overrightarrow{m}+{\overrightarrow{e}}_i)(q_{ij}t+o\left(t\right))}}}\hfill \\ \hfill & ={\displaystyle \frac{{\overrightarrow{\pi }}_s(\overrightarrow{n}+{\overrightarrow{e}}_i)}{{\displaystyle \sum _{|\overrightarrow{m}+{\overrightarrow{e}}_i|\le |s|}{\overrightarrow{\pi }}_s(\overrightarrow{m}+{\overrightarrow{e}}_i)}}}={\displaystyle \frac{S(n+{\mathbb{1}}_{\{i\ne 0\}}){\lambda }^{n+{\mathbb{1}}_{\{i\ne 0\}}}{\prod }_{k\in V}{\eta }_k^{n_k}{\eta }_i}{{\displaystyle \sum _{|\overrightarrow{m}+{\overrightarrow{e}}_i|\le |s|}S(m+{\mathbb{1}}_{\{i\ne 0\}}){\lambda }^{m+{\mathbb{1}}_{\{i\ne 0\}}}\prod _{k\in V}{\eta }_k^{m_k}{\eta }_i}}}\hfill \\ \hfill & ={\displaystyle \frac{S^{\prime }\left(n\right){\lambda }^n{\prod }_{k\in V}{\eta }_k^{n_k}}{{\displaystyle \sum _{|\overrightarrow{m}|\le |s^{\prime }|}{S}^{\prime }\left(m\right){\lambda }^m\prod _{k\in V}{\eta }_k^{m_k}}}}={\overrightarrow{\pi }}_{s^{\prime }}\left(\overrightarrow{n}\right)\hfill \end{array}$$

where $S^{\prime }\left(n\right)={\prod }_{k=0}^{n-1}{s}_k^{\prime }$. To get the result for a closed network with K customers, it is enough to repeat the same reasoning above, after setting $S\left(n\right)={\lambda }^{-n}$, considering $|\overrightarrow{n}+{\overrightarrow{e}}_i|=K$ and summing over $|{\mathcal{T}}_{ij}^{-1}\overrightarrow{m}|=K$. □

When $i=0$, the $\mathcal{T}$-transitions correspond to the Poisson arrivals. This is the most recurrent case, and the property takes the well known name of PASTA (Poisson Arrivals See Time Averages), see [15]. We collect this fundamental result in the following corollary.

Corollary 1 

(PASTA property). In stationary regime, an arrival customer sees an open network in a state distributed according to the stationary distribution, that is

$${\overrightarrow{Q}}_a\sim {\overrightarrow{Q}}^{*}$$

(7)

If the network is closed (semi-open) with K (maximum) number of customers, $Q^{*}$ in (7) refers to the stationary distribution of a similar network with $K-1$ (maximum) number of customers.

Thanks to the PASTA property, $T_i\left(k\right|s)={\mathbb{E}}_s\left[W_i\right|Q^{*}=k]$, where $Q^{*}=\left|{\overrightarrow{Q}}^{*}\right|$. In applying this formula, it is important to interpret the random variable $W_i$ as the sojourn time that a new customer, joining the queue $j\in V$ with probability $s_k{p}_{0j}$ (in an overtaking-free network $j=1$), is going to spend in queue $i\in V$ along her path throw the network. The total number of customers in the network, in case and after the new one has joined, is then $k+1$.

Another property of a Jackson network with state-dependent arrival rate consists in the fact that, under the partial information $Q^{*}=k$, its stationary distribution coincides with the one of a similar closed network with k circulating customers. We collect this well-known result in the following proposition, noticing that its proof is an immediate consequence of the product form (3) and the definition of the conditional probability.

Proposition 2. 

With ${\overrightarrow{Q}}^{\mid k}$ being a random vector distributed as the stationary number of customers in a closed network with $k\in {\mathbb{N}}_0$ total customers, the following equality in distribution holds

$$\left({\overrightarrow{Q}}^{*}|Q^{*}=k\right)\sim {\overrightarrow{Q}}^{\mid k}.$$

(8)

Next proposition gives a known efficient algorithm to compute the expected number of customers, $L_i\left(k\right)=\mathbb{E}\left[Q_i^{\mid k}\right]$, at each node $i\in V$, see Section 2.5.5 in [13] for a proof.

Proposition 3 

(Mean value analysis). The mean number of customers at node $i\in V$ in a closed queueing network with $K\in \mathbb{N}$ customers, can be computed recursively according to the following algorithm

$$L_i\left(K\right)={\displaystyle \frac{K{\eta }_i(L_i(K-1)+1)}{{\sum }_{j\in V}{\eta }_j(L_j(K-1)+1)}},$$

(9)

with $L_i\left(0\right)=0$.

Corollary 1 and Proposition 2 imply that an arriving customer that sees k customers in the system behaves in the same way as an arriving customer belonging to a closed network with $k+1$ customers. That is, before the arriving customer joins, the customers in the network are distributed as ${\overrightarrow{Q}}^{\mid k}$ and just after joining they are distributed as ${\overrightarrow{Q}}^{\mid k+1}$.

Since the tagged customer is indistinguishable from all others, we can use the information about the distribution of the customers in the system to finally estimate her expected sojourn times at each queue.

Theorem 2. 

In an overtaking-free network, assuming that the tagged customer only knows the total number of customers in the network at her arrival epoch, say k, independently of the common strategy s, her expected sojourn time at node $i\in V$ is equal to

$$T_i\left(k\right)={\eta }_i(1+L_i\left(k\right)),$$

(10)

and it can be computed recursively by the formula $T_i\left(0\right)={\eta }_i$ and, for $k\in \mathbb{N}$,

$$T_i\left(k\right)={\eta }_i\left(1+{\displaystyle \frac{k{T}_i(k-1)}{{\sum }_{j\in V}{T}_j(k-1)}}\right).$$

(11)

Proof. 

At the arrival epoch of the tagged customer, by the PASTA property (7), she encounters in the network a stationary number of users. Once she gets the information that the total number of users in the network is equal to k, the distribution of the customers in the network reduces to the one of a stationary closed network with k customers, according to Proposition 2.

After that moment, and assuming that she decides to join, by the overtaking-free condition, the future arrivals will not interfere with her future sojourn times she will spend at each queue $i\in V$ along her random path throw the network. Therefore, we are allowed to change the future arrivals, so that we may assume that the network evolves as a closed network with $k+1$ customers in it. The tagged customer is then indistinguishable with respect to any other customer of this new closed network.

Looking now at the tagged customer as a typical customer in a closed network with $k+1$ customers, an application of the MUSTA property (6) implies that at any transition she will meet a network whose customers are distributed as a closed network with k customers inside. This allows to compute her expected sojourn time. Indeed, in a stationary closed network with k customers, these customers are distributed according to the distribution given in (4).

The tagged customer visits the node $i\in V$ with probability ${\nu }_i$, and it spends there a sojourn time equal to $(1+Q_i)/{\mu }_i$. We get that $W_i=B{\sum }_{k=1}^{1+Q_i}{E}_k$, with B an independent Bernoulli random variable with parameter ${\nu }_i$ and ${\left\{E_k\right\}}_{k>0}$ independent exponentially distributed random variables with parameter ${\mu }_i$. By taking the expectation, (10) readily follows. Then using this formula and (9), we get (11). □

Remark 2. 

A short observation about the use of the overtaking-free property in the proof of Theorem 2 is due, as it is subtle. Since the MUSTA property (6) holds for any Jackson and Whittle network, one would be tempted to believe that the overtaking-free condition is not required, and that also in a general network in which overtaking is allowed, one could easily compute the distribution or the expectation of the waiting times at each queue. The observation that should be made is that, in this case, the tagged customer would not be any more an indistinguishable customer belonging to her network. Indeed, at the moment of her entrance, she had an information more with respect to the others, that is, she knew that in the network there were exactly k customers circulating. However, she is not even belonging to a $k+1$ closed network because, due to the permitted overtaking, the number of customers ahead of her may change by some later arrival. Therefore, we are not allowed to apply the MUSTA property in the way we have done in the proof of the theorem.

Remark 3. 

The result in Theorem 2 extends the one in [9], limited to tandem queues, to the more general class of overtaking-free Jackson networks with rate dependent arrivals. In [9], the strategy to compute the expected sojourn time in each queue, consists in using the Little’s law, that allows to get the value of ${\mathbb{E}}_{{\sigma }_K}\left[W_i\right]$ for any $K\ge 0$. Then, by using the relation,

$$T_i\left(K\right|s_{K+1})={\displaystyle \frac{{\mathbb{E}}_{{\sigma }_{K+1}}\left[W_i\right]}{{\mathbb{P}}_{{\sigma }_{K+1}}(Q^{*}=K)}}-{\displaystyle \frac{{\mathbb{P}}_{{\sigma }_{K+1}}(Q^{*}<K)}{{\mathbb{P}}_{{\sigma }_{K+1}}(Q^{*}=K)}}{\mathbb{E}}_{{\sigma }_{K+1}}\left[W_i\right|Q^{*}<K],$$

(12)

(Equation (4) in [9]), one can finally get the result given in (10). However, we point out that the Little’s law is very robust, meaning that it works in almost any queueing context, practically regardless of any distributional condition. This may misleadingly lead to believe that a similar result holds for more general tandem networks, or even for overtaking-free networks. However, this is not the case, see next Remark 4. Indeed, in order to make computable the expression in (12), one still needs that ${\mathbb{E}}_{{\sigma }_{K+1}}\left[W_i\right|Q^{*}<K]={\mathbb{E}}_{{\sigma }_K}\left[W_i\right]$. This equality only holds for very specific models, such as the Jackson and Whittle networks that are very restrictive in terms of the distributions of their structural random variables. As a byproduct of the special properties of these networks, one also gets that ${\mathbb{P}}_{{\sigma }_{K+1}}(Q^{*}=K)/{\mathbb{P}}_{{\sigma }_{K+1}}(Q^{*}<K)={\mathbb{P}}_{{\sigma }_K}(Q^{*}=K)$.

Remark 4. 

To be convinced that Theorem 2 makes strong use of the assumption of exponentially distributed service times, we recall here some counter examples. In particular, a single queue may always be seen as a tandem network with a single node, and consequently also as an overtaking-free network. In the literature, the single queue has been extensively studied, and in particular [16] has shown an example of M/G/1 queue that does not admit optimal equilibrium strategies. Moreover, the M/G/1 queue has been analyzed in full generality in [17], where it is shown that, depending on the service distribution, the equilibrium is not necessarily unique, and either the avoid the crowd phenomenon or the follow the crowd phenomenon may occur.

5. Equilibrium Strategies

Under an economic perspective, customers will join or balk the network according to an evaluation of their expected profit. Therefore, to look for the existence of an optimum equilibrium strategy, we need to study the expected profit function, $P\left(k\right)$ in (1), of a tagged customer observing, at the arrival epoch, that the network already hosts k customers. We have omitted in the notation the explicit reference to the common strategy s, as it is actually independent of it, as shown more clearly below. The customer decides to join the system if $P\left(k\right)>0$ and balks if $P\left(k\right)<0$, while she may take indifferently both actions when $P\left(k\right)=0$. This suggests that an optimal strategy would be a threshold one, that is ${\sigma }_K$, where K is the turning point of the sign of the function $P\left(k\right)$. In this section we are going to show that, indeed, this threshold strategy leads to an equilibrium.

We start by studying monotonicity properties of the cost functions $T_i\left(k\right|s)$ appearing in (1) and computed in Theorem 2, by exploiting the ordering relations satisfied by the random variables $Q_i^{\mid k}$ as functions of k.

Lemma 1 

(Stochastic ordering). The vectors ${\left\{{\overrightarrow{Q}}^{\mid k}\right\}}_{k\in {\mathbb{N}}_0}$ are stochastic ordered in $k\in {\mathbb{N}}_0$. That is, for any function $f:{\mathbb{R}}^{\left|V\right|}\to \mathbb{R}$ non-decreasing in each coordinate,

$$\mathbb{E}\left[f\left({\overrightarrow{Q}}^{\mid k+1}\right)\right]\ge \mathbb{E}\left[f\left({\overrightarrow{Q}}^{\mid k}\right)\right].$$

Proof. 

The proof relies on a coupling analysis, that is we construct on the same probability space two closed networks with respectively $k\in {\mathbb{N}}_0$ and $k+1$ customers, and show that the number of customers in each queue in the latter network is at least equal to the corresponding one in the former. The coupling is done by considering a closed network with $k+1$ customers inside, k of white color and one of red color. We also assume that the order of service is FIFO for the white customers, and preemptive resuming with low priority for the red customer. That is, the red customer is always pushed back at the end of the queue when there are some white customers at the same node, also when the red customer is the one in service, being alone, and a white customer joins the same node. It follows that the dynamics of the white customers are not affected by the presence of the red customer, in other words if we count only them in the network, they behave exactly as a closed network with k customers inside. On the other side, if we consider all customers, regardless of their colour, since the service times are exponentially distributed, they behave exactly as a closed queueing network of $k+1$ customers. Since the number of customers in each queue of the latter network is at least equal to the one in the former, counting only the white customers, the result follows. □

The monotonicity property proved in Lemma 1 immediately implies the following result on the expected sojourn times. It extends and corrects the results given in [8], see Appendix B.

Theorem 3 

(Monotonicity of the expected sojourn time). For $i\in V$, the expected sojourn time at queue i, $T_i\left(k\right|s)$, is strictly increasing and $T_i\left(k\right|s)\to +\infty$ as $k\to \infty$.

Proof. 

The result follows by showing that, for $i\in V$, the variables ${\left\{\left(W_i\right|Q_a=k)\right\}}_{k\ge 0}$ are strictly stochastically ordered in k. We have that $\left(W_i\right|Q_a=k)\sim B{\sum }_{k=1}^{1+Q_i^{\mid k}}{E}_k$, with B an independent Bernoulli random variable with parameter ${\nu }_i$ and ${\left\{E_k\right\}}_{k>0}$ independent exponentially distributed random variables with parameter ${\mu }_i$. Since $W_i$ is an increasing function of $Q_i^{\mid k}$, the result follows by the stochastic order of the random variables ${\left\{Q_i^{\mid k}\right\}}_{k\ge 0}$ proved in Lemma 1. The order is strict as $P(Q_i=k+1|Q_a=k+1)>0$ while $P(Q_i=k+1|Q_a=k)=0$ for any $k\ge 0$. The last result mentioned in the statement, follows by applying the L’Hopital’s Rule to the limit of the expression (10) as $k\to \infty$. □

By Theorem 2 and (1), we can compute the expected profit function $P\left(k\right|s)$ in the following way

$$P\left(k\right)=R-\sum _{i\in V}{C}_i{T}_i\left(k\right)k\in {\mathbb{N}}_0.$$

(13)

As anticipated,the profit function $P\left(k\right)$ does not depend on the strategy s and of the arrival rate $\lambda$. Furthermore, by Theorem 3, $P\left(k\right)$ is strictly decreasing in k and ${lim}_{k\to +\infty }P\left(k\right)=-\infty$, which means that the benefit to join the network decreases as a function of the customers that are already inside.

It follows that at some point the function $P\left(k\right)$ becomes negative, and we use this fact to show that there exists a common strategy, in particular in the class of threshold ones, that is optimal for all customers to implement.

Theorem 4 

(Equilibrium strategies). Let $K=min\{k\in {\mathbb{N}}_0:P\left(k\right)<0\}$ be the threshold at which the profit function becomes negative. If $P(K-1)>0$, ${\sigma }_K$ is the only pure equilibrium threshold strategy. If $P(K-1)=0$, the only equilibrium strategies are all the mixed threshold strategies ${\sigma }_{K-1+p}$, with $p\in [0,1]$.

Proof. 

Assume that everyone else is using the strategy s, and consider a tagged customer that follows the strategy $\tilde{s}$. Then her payoff, as a function of the customers she finds in the network is given by

$$\tilde{P}\left(k\right)={\tilde{s}}_k\times P\left(k\right|s)={\tilde{s}}_k\times P\left(k\right),$$

and does not depend on the common strategy s. Therefore, by optimizing her profit, we get that the optimal strategy ${\tilde{s}}^{*}$ satisfies ${\tilde{s}}_k^{*}={\mathbb{1}}_{\{k<K\}}$ for $k\ne K-1$ and, assuming $K>0$,

$$\begin{array}{ccc}{\tilde{s}}_{K-1}^{*}\hfill & =1\hfill & \mathrm{if}P(K-1)>0,\hfill \\ {\tilde{s}}_{K-1}^{*}\hfill & \in [0,1]\hfill & \mathrm{if}P(K-1)=0.\hfill \end{array}$$

(14)

That is, $\tilde{s}={\sigma }_K$ is an optimal strategy and since it is optimal independently of the common strategy s its also optimal against itself. In addition, if $K>0$ and $P(K-1)=0$, then also the strategy ${\sigma }_{K-1}$ is optimal and an equilibrium strategy, as well as any convex combination of ${\sigma }_{K-1}$ and ${\sigma }_K$. □

6. Numerical Computations

The easiest example of an overtaking-free network that is not a tandem network has an underlying out-tree $G(V,E)$ with $V=\{1,2,3\}$ and $E=\left\{\right(1,2),(1,3\left)\right\}$. We compare this network with a two-node tandem network by looking at the equilibrium threshold strategies for different values of waiting costs per unit of time, as the routing probabilities vary. We refer to the first network with model A and to the two-node tandem network with model B.

As the Figure 2 shows, we choose for model A the routing probabilities $p_{12}=p_{13}=1/2$, which imply ${\nu }_1=1$ and ${\nu }_2={\nu }_3=1/2$, and all service rates equal to $\mu$. For model B (see [8]), we choose ${\mu }_1=\mu$ and ${\mu }_2=2\mu$, ${\nu }_1={\nu }_2=1$.

Both models have the first node equal, however in model A, after completing service at queue 1, customers are routed to queue 2 or queue 3 with equal probability and served there at rate $\mu$ while the customers in model B are all served at queue 2 but with twice the service rate, that is $2\mu$. In both systems, customers pay a cost C per unit of sojourn time at any queue.

We expect the equilibrium strategies to be similar to each other, due to the fact that, even if in model A queues 2 and 3 work at a speed that is half than the one of queue 2 in model B, they should handle, in a long run, half of the customers each.

Since both models have a simple topology we can get explicit formulas for the profit function as expressed by the following theorem.

Theorem 5. 

The profit functions for the models A and B are respectively equal to

$$\begin{array}{cc}\hfill P_A\left(k\right)& =R-{\displaystyle \frac{(k+1)C}{\mu }}·{\displaystyle \frac{2^{k+3}-(k+4)}{2^{k+3}-2(k+3)}}\hfill \end{array}$$

(15)

$$\begin{array}{cc}\hfill P_B\left(k\right)& =R-{\displaystyle \frac{(k+1)C}{\mu }}·{\displaystyle \frac{2^{k+2}-1}{2^{k+2}-2}}\hfill \end{array}$$

(16)

Proof. 

For model A, we have that the vector $(Q_1^{\mid k},Q_2^{\mid k}+Q_3^{\mid k})\sim {\overline{(G_1,E_2)}}^k$ is given by the truncation of a bivariate vector made of two independent random variables with distributions, respectively $\mathrm{Geo}\left({\eta }_1\right)$, and $\mathrm{dErlang}(2,{\eta }_2)$. Since ${\eta }_2/{\eta }_1=1/2$, by Lemma A2 we have $Q_2^{\mid k}+Q_3^{\mid k}\sim {\overline{\mathrm{dErlang}(2,1/2)}}^k$, and by (A5)

$$m_E\left(k\right)\mathbb{E}[Q_2^{\mid k}+Q_3^{\mid k}]=2{\displaystyle \frac{(k+1)(k+4)-2^{k+3}+4}{2(k+3)-2^{k+3}}}.$$

Then, having that $T_A\left(k\right)=(k+2-m_E\left(k\right)/2)/\mu$, the relation in (15) holds.

For model B, we have that the vector $(Q_1^{\mid k},Q_2^{\mid k})\sim {\overline{(G_1,G_2)}}^k$ is given by the truncation of a bivariate vector made of two independent Geometric random variables with parameters, respectively ${\eta }_1$, and ${\eta }_2$. By Lemma A1, $Q_1^{\mid k}\sim {\overline{\mathrm{Geo}\left(2\right)}}^k$, since ${\eta }_1/{\eta }_2=2$, and, by (A1),

$$m_G\left(k\right)\mathbb{E}\left[Q_1^{\mid k}\right]={\displaystyle \frac{k+1}{p^{k+1}-1}}+k+{\displaystyle \frac{1}{1-p}}.$$

Then, having that $T_B\left(k\right)=(k+3-m_G\left(k\right))/\left(2\mu \right)$, the relation in (15) holds. □

The functions $P_A\left(k\right)$ and $P_B\left(k\right)$ are graphically represented in Figure 3a, where we fixed $R=9$, $C=2$ and $\mu =1$. Figure 3b shows the equilibrium thresholds as functions of $\mu$. We observe that $P_A\left(k\right)$ is slightly lower than $P_B\left(k\right)$ and therefore the equilibrium thresholds in model A are generally lower than the ones in model B. This means that the customers in model A are more intimidated by the presence of customers in the network and decide to stop to enter for a less crowded network than the ones in model B. This is due to a well known fact in queueing theory, indeed model A is a less efficient network in that it wastes service capacity when one queue, among the queues 2 and 3, is idle while the other is working.

Now, we take a closer look at a variant of model A, by changing the routing probabilities and the costs for unit of sojourn time, in order to see how the equilibrium threshold are affected. We fix ${\mu }_i=i$ for $i=1,\dots ,3$, $R=20$, $C_1=1$, and we relax the assumption that $p_{12}=p_{13}$. We assume that $C_1+C_2+C_3=13$ (i.e. $C_3=12-C_2$), with $C_2\in [0,12]$, and that $p_{12}+p_{13}=1$ (i.e. $p_{13}=1-p_{12}$) with $p_{12}\in [0,1]$. The values of the equilibrium threshold are represented in Figure 4. We observe that the thresholds are higher for low $C_2$ cost values and high $p_{12}$ probability values as well as for high $C_2$ cost values and low $p_{12}$ probability values. On the other side, they are lower when both $C_2$ and $p_{12}$ values are both high or low, forming a sort of saddle point in the middle.

7. Conclusions

The overtaking-free networks are special networks with the property that no customers entering behind a tagged one may influence her sojourn time in the network. Under the assumption of single server queues and exponentially distributed inter-arrival and service times, these networks are characterized by having an out-tree structure of the underlying graph.

For these networks, under the assumption that a tagged customer is informed about how many customers are already in the network at her arrival epoch, we compute the equilibrium strategies, that are shown to be of threshold type. Indeed, this follows by the fact that, under the partial information received by the arriving tagged customer, her sojourn times, as well as her profit function, do not depend on the common strategy used by the other customers.

Finally, we remark that the results heavy rely on the special property of the exponential distribution as there are known and easy to construct counter examples by using the simple M/G/1 queue.