Definition of Optimal Time Intervals in the Queues’ Analysis: The Use of Epsilon-Entropy and Epsilon-Capacity

1. Introduction

The use of queue models implies the knowledge of the arrival and departure rates, which in their turn require the well-defined time intervals [3].

For example, consider a clerk serving clients of some office during the day. Then, the arrival rate $\lambda$ and the departure rate $\mu$ per day completely describe the state of the system at the end of the day but do not provide any information about the system during the day. On other hand, specification of the rates $\lambda$ and $\mu$, for example, per minute is also useless since both the clients and the clerk do not act with such rates.

The incorrect specification of the time intervals leads to incorrect consideration of the processes with unsteady arrivals or departures. In many cases, such situations are resolved using the queues with time dependent rates [5,8], but even in such considerations certain time intervals per which the rates are defined have to be specified.

Similar problem appears in statistics while plotting histograms of the data sample $X$ and it is required to define the lengths $\delta$ of the bins. Since there is no strictly proven formula, which defines the bin length with respect to the number $n$ of data counts or distribution over the sample, the heuristic formulas are used.

For example, the simplest heuristic defines the bin length as

$${\delta }_1={\displaystyle \frac{\max \left(X\right)-\min \left(X\right)}{\sqrt{n}}}.$$

(1)

The Sturges rule [15] defines the number of bins as $⌈{\log }_{2}n⌉+1$. Then the bin length is

$${\delta }_2={\displaystyle \frac{\max \left(X\right)-\min \left(X\right)}{⌈{\log }_{2}n⌉+1}}.$$

(2)

The Scott rule [12] defines the bin length

$${\delta }_3={\displaystyle \frac{3.49s}{\sqrt[3]{n}}}$$

(3)

with respect to the standard deviation $s$ of the sample. Finally, the Freedman-Diaconis formula [2] uses the interquartile range instead of the standard deviation and defines the bin length

$${\delta }_4={\displaystyle \frac{2\left(Q_3-Q_1\right)}{\sqrt[3]{n}}},$$

(4)

where $Q_3$ is the third quartile and $Q_1$ is the first quartile of the sample.

In general, this problem can be considered in the terms of discretization of stochastic processes [4], where it is required to build a discretization scheme, which is a sequence $t_i$, $i=0, 1, 2, \dots$, of stopping times such that $\mathsf{\Delta }=t_{i+1}-t_i$ and $t_i=i\mathsf{\Delta }$. But if $t_i$ is a random variable, then the length $\mathsf{\Delta }$ of the time intervals is also random and depend on the distribution of the considered process. Similarly, if the discretization scheme is regular with constant interval lengths $\mathsf{\Delta }$, then the increments of the process at the times $t_i$ are random.

For example, let $W_t$ be a Wiener process at the time interval $T=\left[0,t_m\right]$ starting with $W_0=0$. In such a process, the increments $d{W}_t$ are independent and for any $t_i$ and $t_j>t_i$ the differences $W_{t_j}-W_{t_i}$ have normal distribution $\mathcal{N}\left(0,{\sigma }_t^2\right)$ with the variance ${\sigma }_t^2=t_j-t_i$. Assume that the interval $T$ is divided to $n$ sub-intervals with the length $\mathsf{\Delta }=t_m/n$. Then, the stopping times are $t_i=i\mathsf{\Delta }$, $i=\mathrm{0,1}, 2,\dots ,$ and the increments $d{W}_t=W_{t_{i+1}}-W_{t_i}$ are normally distributed with ${\sigma }_t^2=\mathsf{\Delta }$.

In this paper, we seek the answer to the following question formulated by Yaakov Reis [9]. Given a total period, a sequence of clients arriving at the times $t_0, t_1, t_2,\dots ,t_m$ to a service point, what is an optimal length $\mathsf{\Delta }$ of the time interval, on which the arrival rate $\lambda$ and the service rate $\mu$ (which is a departure rate) have to be defined?

An immediate answer to this question follows the heuristics used for definition of the bin length $\delta$ in histogram. However, such heuristics cannot be considered as the best method and their result is not strictly proved approximation.

To find an optimal length $\mathsf{\Delta }$ we follow the line of the Schwarz information criterion [11] and apply well-known concepts of $\epsilon$-entropy and $\epsilon$-capacity, which were introduced by Kolmogorov and Tikhomirov [6]. The calculations of the optimal interval are also based on the concept of the entropy of partition introduced by Rokhlin [10].

Initially $\epsilon$-entropy and $\epsilon$-capacity were used for analysis of functions and functional spaces and then, as well as the entropy of partition, were applied to the studies of dynamical systems. For many examples of application of these concepts and their relationship with the Shannon entropy [13] see the paper by Dinaburg [1] and the books by Vitushkin [16] and by Sinai [14].

2. Problem Formulation

Let $T=\left[t_0, t_m\right]$ be a time interval of the length $t_m-t_0>0$ and assume that during this interval sequentially occur $m+1$ events $a_0,a_1,a_2,\dots ,a_m$. The times of occurrences of these events are $t_0\le t_1\le t_2\le \dots \le t_m$, respectively.

The problem is to define a length $\mathsf{\Delta }$ of the time interval or, that is the same, the stopping times $t_i=i\mathsf{\Delta }$, $i=0, 1, 2, \dots ,n$, such that $n$ intervals $T_i=\left[t_i,t_{i+1}\right]$ cover the interval $T$ and such that they as better as possible represent the times $t_j$, $j=0, 1, 2,\dots ,m$, when the considered events occurred.

To illustrate the problem, let us consider a simple example of a non-steady supply process. Assume that the mentioned above clerk serves the clients with the rate $\mu =5$ clients/hour. During the workday of $8$ hours arrive $24$ clients. Then, the arrival rate of the clients defined over a workday is $\lambda =24/8=3$ clients/hour and the transition rate $\rho =\lambda /\mu =3/5<1$ that should guarantee that at the end of the workday all clients will be served.

Additionally, assume that in the morning, during the first two hours of the day, arrive $12$ clients. Then during the next four hours the clients do not arrive and then, in the evening, during the last two hours of the day, arrive the last $12$ clients. Thus, in the morning and in the evening the arrival rate is $\lambda =12/2=6$ clients/hour, that means that the first $12$ clients will wait in the queue and the last $12$ clients will not be served until the end of the workday.

Certainly, such phenomena are well-known; in the queue theory they are solved using the state-dependent and time-dependent arrival rates [3,5,8], and in practice are overcome by adding the clerks in the morning and in the evening and by stopping the service in the midday. However, a prior definition of the appropriate time intervals can simplify further analysis and even decrease the expected number of varying rates.

Finally, note that the considered problem is essentially discrete problem, where it is required to split the discrete dataset. Together with that, since it is closely related to the discretization problems dealing with the continuous functions, below we will make some remarks on such problems as well.

3. Methods

The suggested solution of the problem is based on the concepts of $\epsilon$-entropy and $\epsilon$-capacity, which were introduced by Kolmogorov and Tikhomirov in the middle of 1950-s and presented in detail in their paper [6]. In addition, it uses the multiplication of partitions as it was implemented by Rokhlin [10] and by Sinai [14] in the studies of dynamical systems.

3.1. $\mathit{\epsilon }$ -Entropy and $\mathit{\epsilon }$-Capacity

Let $U\subset R$ be a non-empty bounded set of a metric space $R$ and let $\epsilon >0$ be a real number.

The set $\alpha =\left\{A:A\subset R\right\}$ is called $\epsilon$-covering of the set $U$, if $U\subseteq \bigcup _{A\in \alpha }A$ and the diameter of any $A\in \alpha$ is not greater than $2\epsilon$.

The set $U$ is said to be $\epsilon$-distinguishable, if any two of its distinct points are located at distance greater than $\epsilon$.

Given a bounded set $U\subset R$, for any $\epsilon >0$ there exists a finite $\epsilon$-covering of $U$, and for any $\epsilon >0$ any $\epsilon$-distinguishable set $U\subset R$ is finite.

Denote by ${\mathcal{N}}_{\epsilon }\left(U\right)$ the minimal number of the sets in $\epsilon$-covering $\alpha$ of the set $U$, and by ${\mathcal{M}}_{\epsilon }\left(U\right)$ the maximal number of points in an $\epsilon$-distinguishable subset of the set $U$.

The value

$${\mathcal{H}}_{\epsilon }\left(U\right)={\log }_2{\mathcal{N}}_{\epsilon }\left(U\right)$$

(5)

is called the $\epsilon$-entropy of the set $U$, and the value

$${\mathcal{E}}_{\epsilon }\left(U\right)={\log }_2{\mathcal{M}}_{\epsilon }\left(U\right)$$

(6)

is called the $\epsilon$-capacity of the set $U$.

These values are interpreted as follows: $\epsilon$-entropy ${\mathcal{H}}_{\epsilon }\left(U\right)$ is a minimal number of bits required to transmit the set $U$ with the precision $\epsilon$, and $\epsilon$-capacity ${\mathcal{E}}_{\epsilon }\left(U\right)$ is a maximal number of bits, which can be memorized by $U$ with the precision $\epsilon$.

Among the properties of $\epsilon$-entropy ${\mathcal{H}}_{\epsilon }\left(U\right)$ and $\epsilon$-capacity ${\mathcal{E}}_{\epsilon }\left(U\right)$ we will use the following fact [6]: given the bounded set $U$, both $\epsilon$-entropy and $\epsilon$-capacity as functions of $\epsilon$ are non-increasing with increasing $\epsilon$.

Examples of calculation of the $\epsilon$-entropy and $\epsilon$-capacity of the sets in different metric spaces can be found in the paper by Kolmogorov and Tikhomirov [6] and in the book by Vitushkin [16].

3.2. ɛ-Entropy of Partition

Let $\beta =\left\{B:B\subset U\right\}$ be a partition of the set $U\subset R$ that is $U=\bigcup _{B\in \beta }B$ and for any two sets $B^{\prime },B^{\prime \prime }\in \beta$ holds $B^{\prime }\cap B^{\prime \prime }=\varnothing$.

The entropy of partition is defined as follows [10,14]. Let $\mu$ be a non-negative measure on the set $U$ such that $\mu \left(\varnothing \right)=0$ and $\mu \left(U\right)=1$. Then, $\mu \left(B\right)\in \left[0, 1\right]$ for any $B\in \beta$. The value

$${\mathcal{H}}_{\mu }\left(\beta \right)=-{\sum }_{B\in \beta }\mu \left(B\right){\log }_2\mu \left(B\right)$$

(7)

is called the entropy of partition. If $\mu$ is the probability measure on $U$, then the sets $B\in \beta$ can be interpreted as events and the entropy ${\mathcal{H}}_{\mu }$ is equivalent to the Shannon entropy [13].

Assume that the partition $\beta$ is finite and the number of the sets in $\beta$ is $N$. Define the measure $\mu$ on the set $U$ as follows:

$$\mu \left(B\right)=\left\{\begin{array}{l}0, B=\varnothing \\ {\displaystyle \frac{1}{N}}, B\ne \varnothing , B\ne U\\ 1, B=U\end{array}\right.$$

(8)

Then, the entropy of partition is reduced to the value

$${\mathcal{H}}_{\mu }\left(\beta \right)={\log }_{2}N$$

(9)

Finally, if the diameter of any set $B\in \beta$ is not greater than $2\epsilon$, then the partition $\beta$ is an $\epsilon$-covering and called $\epsilon$-partition. Then, the entropy ${\mathcal{H}}_{\mu }\left(\beta \right)$ of partition is equivalent to the $\epsilon$-entropy ${\mathcal{H}}_{\epsilon }\left(U\right)$ of the set $U=\bigcup _{B\in \beta }B$ defined by equation (5).

Let $\beta =\left\{B:B\subset U\right\}$ be $\epsilon$-partition of the set $U$ with $\epsilon ={\epsilon }_B$ and $\gamma =\left\{C:C\subset U\right\}$ be another $\epsilon$-partition of the set $U$ with $\epsilon ={\epsilon }_C$. Multiplication of the partitions $\beta$ and $\gamma$ is the partition

$$\beta \vee \gamma =\left\{D=B\cap C: B\in \beta ,C\in \gamma \right\}.$$

(10)

Each set $D\in \beta \vee \gamma$ is a subset of some set $B\in \beta$ and of some set $C\in \gamma$. Then it is said that $\beta \vee \gamma$ is a refinement of both $\beta$ and $\gamma$; this fact is denoted by $\beta \preccurlyeq \beta \vee \gamma$ and $\gamma \preccurlyeq \beta \vee \gamma$. Hence, following the properties of the entropy of partition,

$${\mathcal{H}}_{\mu }\left(\beta \right)\le {\mathcal{H}}_{\mu }\left(\beta \vee \gamma \right)\mathrm{and}{\mathcal{H}}_{\mu }\left(\gamma \right)\le {\mathcal{H}}_{\mu }\left(\beta \vee \gamma \right)$$

(11)

and

Moreover, the entropy ${\mathcal{H}}_{\mu }\left(\beta \vee \gamma \right)$ of the multiplication $\beta \vee \gamma$ of the partition $\beta$ and $\gamma$ is the $\epsilon$-entropy ${\mathcal{H}}_{\epsilon }\left(U\right)$ of the set $U$ with any $\epsilon \in \left[\min \left\{{\epsilon }_B,{\epsilon }_C\right\}, \max \left\{{\epsilon }_B,{\epsilon }_C\right\}\right]$.

For the other properties of the entropy ${\mathcal{H}}_{\mu }$ and its application for analysis of dynamical systems see the paper [10] and the book [14].

4. Suggested Solution

Let $T=\left[t_0, t_m\right]$, $t_m>t_0$, be a time interval and let $\mathcal{T}=\left\{t_0,t_1, t_2,\dots ,t_m\right\}$ be the set of moments in which certain events occur. We assume that the moments $t_j$ have an increasing order such that $t_j<t_{j+1}$, $j=0, 1, 2,\dots ,m-1$.

As it follows from the formulation of the problem, in the consideration below the interval $T$ plays a role of the set $U$ and the intervals $\left[t_j,t_{j+1}\right]$, $j=0, 1, 2,\dots ,m-1$, are considered as the elements of $\epsilon$-partitions of the interval $T$.

Given $\epsilon >0$, the minimal number of sets in the $\epsilon$-covering $\alpha$ of the set $\mathcal{T}$

$${\mathcal{N}}_{\epsilon }\left(\mathcal{T}\right)=⌈{\displaystyle \frac{t_m-t_0}{2\epsilon }}⌉.$$

(12)

Then, the $\epsilon$-entropy of the set $\tau$ is

$${\mathcal{H}}_{\epsilon }\left(\mathcal{T}\right)={\log }_2{\mathcal{N}}_{\epsilon }\left(\mathcal{T}\right)={\log }_2⌈{\displaystyle \frac{t_m-t_0}{2\epsilon }}⌉.$$

(13)

Let

$${\epsilon }_{min}={\displaystyle \frac{1}{2}}{\displaystyle \frac{t_m-t_0}{m^2}}$$

(14)

be a minimal value of $\epsilon$ for the set $\mathcal{T}$. Then, the value

$${\mathcal{H}}_{{\epsilon }_{min}}\left(\mathcal{T}\right)={\log }_2⌈{\displaystyle \frac{t_m-t_0}{2{\epsilon }_{min}}}⌉=2{\log }_{2}m$$

(15)

is maximal $\epsilon$-entropy of the set $\mathcal{T}$.

Finally, assume that on the interval $T$ two sets ${\mathcal{T}}_1=\left\{t_{\mathrm{1,0}},t_{\mathrm{1,1}},t_{\mathrm{1,2}},\dots ,t_{1{,m}_1}\right\}$ and ${\mathcal{T}}_2=\left\{t_{\mathrm{2,0}},t_{\mathrm{2,1}},t_{\mathrm{2,2}},\dots ,t_{2,m_2}\right\}$, $t_{\mathrm{1,0}}=t_{\mathrm{2,0}}$ and $t_{1,m_1}=t_{2,m_2}$, of moments are defined. Denote by ${\tau }_1=\left\{\left[t_{\mathrm{1,0}},t_{\mathrm{1,1}}\right],\left[t_{\mathrm{1,1}},t_{\mathrm{1,2}}\right],\dots ,\left[t_{1,m_1-1},t_{1,m_1}\right]\right\}$ partition of the interval $T$ corresponding to the set ${\mathcal{T}}_1$ and by ${\tau }_2=\left\{\left[t_{\mathrm{2,0}},t_{\mathrm{2,1}}\right],\left[t_{\mathrm{2,1}},t_{\mathrm{2,2}}\right],\dots ,\left[t_{2,m_2-1},t_{2,m_2}\right]\right\}$ partition of the interval corresponding to the set ${\mathcal{T}}_2$. The number of intervals in the partition ${\tau }_1$ is $m_1$ and the number of intervals in the partition ${\tau }_2$ is $m_2$.

Then, since the multiplication ${\tau }_1\vee {\tau }_2$ is a refinement of each of the partitions ${\tau }_1$ and ${\tau }_2$ of the sets, the size $m_{1\vee 2}$ of the partition ${\tau }_1\vee {\tau }_2$ is $m_{1\vee 2}\ge \max \left\{m_1, m_2\right\}$, and the entropy ${\mathcal{H}}_{\mu }\left({\tau }_1\vee {\tau }_2\right)$ of the multiplication ${\tau }_1\vee {\tau }_2$ is not smaller than the entropies ${\mathcal{H}}_{\mu }\left({\tau }_1\right)$ and ${\mathcal{H}}_{\mu }\left({\tau }_2\right)$ of the partitions ${\tau }_1$ and ${\tau }_2$.

Hence, if

$$2\epsilon \ge \underset{j=\mathrm{0,1},2,\dots ,m_1-1}{\max }\left(t_{1,j+1}-t_{1,j}\right) \mathrm{and} 2\epsilon \ge \underset{j=\mathrm{0,1},2,\dots ,m_2-1}{\max }\left(t_{2,j+1}-t_{2,j}\right),$$

(16)

then, following equation (11),

$${\mathcal{H}}_{\epsilon }\left({\mathcal{T}}_1\right)\le {\mathcal{H}}_{\epsilon }\left({\mathcal{T}}_1\vee {\mathcal{T}}_2\right)\mathrm{and}{\mathcal{H}}_{\epsilon }\left({\mathcal{T}}_2\right)\le {\mathcal{H}}_{\epsilon }\left({\mathcal{T}}_1\vee {\mathcal{T}}_2\right).$$

(17)

Following the line of the Schwarz information criterion [11], let us define $\epsilon$-information of the set $\mathcal{T}$.

Let $\tau$ be a partition corresponding to the set $\mathcal{T}$ and let ${\tau }_{\epsilon }$ be a partition corresponding to the set ${\mathcal{T}}_{\epsilon }=\left\{t_{\epsilon ,0},t_{\epsilon ,1},t_{\epsilon ,2},\dots ,t_{\epsilon {,m}_{ϵ}}\right\}$, $t_{\epsilon ,0}=t_0$ and $t_{\epsilon ,m_{\epsilon }}=t_m$, in which $t_{\epsilon ,j+1}-t_{\epsilon ,j}=2\epsilon$, $j=0,1,2,\dots ,m-2$. In the partition ${\tau }_{\epsilon }$ all intervals except the last are of the length $2\epsilon$.

Denote by $\mathcal{T}\vee {\mathcal{T}}_{\epsilon }$ the set of moments corresponding to the multiplication $\tau \vee {\tau }_{\epsilon }$ of the partitions $\tau$ and ${\tau }_{\epsilon }$. Then, $\epsilon$-information of the set $\mathcal{T}$is defined as follows

$${\mathcal{I}}_{\epsilon }\left(\mathcal{T}\right)={\mathcal{H}}_{{\epsilon }_{min}}\left(\mathcal{T}\right)-{\mathcal{H}}_{\epsilon }\left(\mathcal{T}\right)-{\mathcal{H}}_{\epsilon }\left(\mathcal{T}\vee {\mathcal{T}}_{\epsilon }\right).$$

(18)

In this formula, the first term represents the number of bits required to transmit the set $\mathcal{T}$ with maximal precision, the second term represents the number of bits required to transmit the set $\mathcal{T}$ with precision $\epsilon$, and the last term represents the number of bits required to transmit the set $\mathcal{T}$ with precision $\epsilon$ using additional set ${\mathcal{T}}_{\epsilon }$ generated with precision $\epsilon$. Thus, the value ${\mathcal{I}}_{\epsilon }\left(\mathcal{T}\right)$ is the number of bits remained after the transmission of the set $\mathcal{T}$ with the precision $\epsilon$. In the other words, $\epsilon$-information of the set $\mathcal{T}$ characterizes the part of the set, which cannot be transmitted with the precision $\epsilon$.

Using equations (13) and (15), formula (18) of $\epsilon$-information can be simplified and written in the form

$${\mathcal{I}}_{\epsilon }\left(\mathcal{T}\right)=2{\log }_{2}m-{\log }_2⌈{\displaystyle \frac{t_m-t_0}{2\epsilon }}⌉-{\mathcal{H}}_{\epsilon }\left(\mathcal{T}\vee {\mathcal{T}}_{\epsilon }\right).$$

(19)

The value of the entropy ${\mathcal{H}}_{\epsilon }\left(\mathcal{T}\vee {\mathcal{T}}_{\epsilon }\right)$ depends on the distribution of time moments $t_j\in \mathcal{T}$, $j=0, 1, 2,\dots , m$, over the interval $T$. If the moments $t_j$ are distributed evenly, then $\mathcal{T}\vee {\mathcal{T}}_{\epsilon }=\mathcal{T}$ and

$${\mathcal{H}}_{\epsilon }\left(\mathcal{T}\vee {\mathcal{T}}_{\epsilon }\right)={\mathcal{H}}_{\epsilon }\left(\mathcal{T}\right)={\log }_2⌈{\displaystyle \frac{t_m-t_0}{2\epsilon }}⌉.$$

(20)

Note that in general case equation (20) does not hold and calculation of the entropy of multiplication of partitions is processed according to the algorithm presented by Function 1 (see section 5).

Similarly, the value of $\epsilon$-capacity ${\mathcal{E}}_{\epsilon }\left(\mathcal{T}\right)$ depends on the distribution of the time moments $t_j$ over the interval $T$. If the moments $t_j$ are distributed evenly such that $t_{j+1}-t_j=t_{j+2}-t_{j+1}$ and $t_{j+1}-t_j>\epsilon$ for any $j=0, 1, 2,\dots ,m-2$, then

$${\mathcal{M}}_{\epsilon }\left(\mathcal{T}\right)=⌈{\displaystyle \frac{t_m-t_0}{\epsilon }}⌉$$

(21)

and

$${\mathcal{E}}_{\epsilon }\left(\mathcal{T}\right)={\log }_2{\mathcal{M}}_{\epsilon }\left(\mathcal{T}\right)={\log }_2⌈{\displaystyle \frac{t_m-t_0}{\epsilon }}⌉.$$

(22)

If the distribution of the moments $t_i$ is such that $t_{m-1}-t_0\le \epsilon$, which means that all the moments except $t_m$ are located between $t_0$ and $t_{m-1}$, and $t_m-t_0>\epsilon$, then

$${\mathcal{M}}_{\epsilon }\left(\mathcal{T}\right)=2$$

(23)

and

$${\mathcal{E}}_{\epsilon }\left(\mathcal{T}\right)={\log }_{2}2=1.$$

(24)

Finally, if $t_m-t_0\le \epsilon$, then the set $T$ does not contain $\epsilon$-distinguishable subset, and we assume that

$${\mathcal{M}}_{\epsilon }\left(\mathcal{T}\right)=1$$

(25)

and

$${\mathcal{E}}_{\epsilon }\left(\mathcal{T}\right)={\log }_{2}1=0.$$

(26)

Calculation of $\epsilon$-capacity in general case follows the algorithm of Function 2 (see section 5).

The length $\mathsf{\Delta }$ of the time interval, which defines the stopping times $t_i=i\mathsf{\Delta }$, $i=0, 1, 2, \dots ,n$, is defined as

$$\mathsf{\Delta }=2\epsilon ,$$

(27)

where $\epsilon$ is such a value for which $\epsilon$-information ${\mathcal{I}}_{\epsilon }\left(\mathcal{T}\right)$ of the set $\mathcal{T}$ is as close as possible to $\epsilon$-capacity ${\mathcal{E}}_{\epsilon }\left(\mathcal{T}\right)$ of this set.

Note that given the set $\mathcal{T}$, the entropy ${\mathcal{H}}_{{\epsilon }_{min}}\left(\mathcal{T}\right)$ is constant and both entropies ${\mathcal{H}}_{\epsilon }\left(\mathcal{T}\right)$ and ${\mathcal{H}}_{\epsilon }\left(\mathcal{T}\vee {\mathcal{T}}_{\epsilon }\right)$ as functions of $\epsilon$ are decreasing. Thus, $\epsilon$-information ${\mathcal{I}}_{\epsilon }\left(\mathcal{T}\right)$ increases with $\epsilon$. Along with that, $\epsilon$-capacity ${\mathcal{E}}_{\epsilon }\left(\mathcal{T}\right)$ as function of $\epsilon$ decreases.

Hence, the problem of finding the length $\mathsf{\Delta }$ is formulated as follows: given the set $\mathcal{T}$, find the value of $\epsilon$ such that

$$\left|{\mathcal{I}}_{\epsilon }\left(\mathcal{T}\right)-{\mathcal{E}}_{\epsilon }\left(\mathcal{T}\right)\right|\to min.$$

(28)

To illustrate the calculation of the length $\mathsf{\Delta }$, let us consider a simple example. Assume that the considered time interval is $T=\left[t_0, t_m\right]$ and the set $\mathcal{T}=\left\{t_0,t_1, t_2,\dots ,t_m\right\}$ consists of the evenly distributed moments $t_j$ such that $t_{j+1}-t_j=t_{j+2}-t_{j+1}$ for any $j=0, 1, 2,\dots ,m-2$. Then (here for simplicity we omit the notion of ceiling),

$${\mathcal{I}}_{\epsilon }\left(\mathcal{T}\right)-{\mathcal{E}}_{\epsilon }\left(\mathcal{T}\right)=2{\log }_{2}m-2{\log }_2{\displaystyle \frac{t_m-t_0}{2\epsilon }}-{\log }_2{\displaystyle \frac{t_m-t_0}{\epsilon }}={\log }_2{m}^2-{\log }_2{\displaystyle \frac{{\left(t_m-t_0\right)}^3}{4{\epsilon }^3}}.$$

(29)

Hence, according to the criterion (28), it is required to specify the value $\epsilon$ such that

$${\displaystyle \frac{{\left(t_m-t_0\right)}^3}{4{\epsilon }^3}}=m^2,$$

(30)

which is

$$\epsilon ={\displaystyle \frac{t_m-t_0}{\sqrt[3]{4m^2}}}.$$

(31)

and finally

$$\mathsf{\Delta }=2{\displaystyle \frac{t_m-t_0}{\sqrt[3]{4m^2}}}.$$

(32)

For example, if $\mathcal{T}=\left[0, 10\right]$ and $A=\left\{0, 1, 2,\dots ,10\right\}$, then

$$\mathsf{\Delta }=2{\displaystyle \frac{10-0}{\sqrt[3]{4{\times 11}^2}}}=2.55.$$

For comparison, the indicated above methods (1)-(4) of specifying the bin length in the histograms result in the following values:

-

the simplest rule:         ${\delta }_1={\displaystyle \frac{\max A-\min A}{\sqrt{m}}}={\displaystyle \frac{10-0}{\sqrt{11}}}=3.01$,

-

the Sturges rule [15]:        ${\delta }_2={\displaystyle \frac{\max A-\min A}{{\log }_{2}m+1}}={\displaystyle \frac{10-0}{{\log }_{2}11+1}}=2.24$,

-

the Scott rule [12]:         ${\delta }_3={\displaystyle \frac{3.49s}{\sqrt[3]{m}}}={\displaystyle \frac{3.49\times 3.32}{\sqrt[3]{11}}}=5.20$,

-

the Freedman-Diaconis formula [2]:   ${\delta }_4={\displaystyle \frac{2\left(Q_3-Q_1\right)}{\sqrt[3]{m}}}={\displaystyle \frac{2\times \left(8-3\right)}{\sqrt[3]{11}}}=4.94$.

In the considered example, the interval length calculated using the suggested method is compatible with the lengths obtained using the methods of calculating the bin lengths, but for the other distributions the interval lengths can be strongly different.

Note again that in general case the interval lengths cannot be calculated using close formulas. In the next section we summarize the suggested methods in the form of an algorithm which is applicable to arbitrary data.

5. Algorithmic Implementation

We summarize the suggested solution in the form of an algorithm which can be directly implemented in any high-level programming language. In our trials we used the MATLAB^® environment.

Algorithm 1. Computing an optimal interval length

Input:  Set $\mathcal{T}=\left\{t_0,t_1, t_2,\dots ,t_m\right\}$ of time moments, $t_j<t_{j+1}$, $j=0, 1, 2,\dots ,m-1$; step $s>0$. Output:  Optimal interval length $\mathsf{\Delta }$.

1.  Calculate ${\epsilon }_{min}=\left(t_m-t_0\right)/\left(2m^2\right)$ {minimal value of $\epsilon$, equation (14)}. 2.  Calculate ${\mathcal{H}}_{{\epsilon }_{min}}\left(\mathcal{T}\right)=2{\log }_{2}m$ {maximal $\epsilon$-entropy, equation (15)}. 3.  For $\epsilon ={\epsilon }_{min}$ to $\left(t_m-t_0\right)/2$ with step $s$ do: 4.    Calculate ${\mathcal{H}}_{\epsilon }\left(A\right)={\log }_2⌈\left(t_m-t_0\right)/2\epsilon ⌉$ {$\epsilon$-entropy, equation (13)}. 5.    Create set ${\mathcal{T}}_{\epsilon }=\left\{t_{\epsilon ,0},t_{\epsilon ,1}, t_{\epsilon ,2},\dots ,t_{\epsilon ,m_{\epsilon }}\right\}$ such that $t_{\epsilon ,j}<t_{\epsilon ,j+1}$, $j=0, 1, 2,\dots ,m_{\epsilon }-1$, and $t_{\epsilon ,j+1}-t_{\epsilon ,j}=2\epsilon$, $j=0, 1, 2,\dots ,m_{\epsilon }-2$. 6.    Compute ${\mathcal{H}}_{\epsilon }\left(\mathcal{T}\vee {\mathcal{T}}_{\epsilon }\right)=eps\_entropy\left(\mathcal{T},{\mathcal{T}}_{\epsilon }\right)$ {entropy of $\mathcal{T}\vee {\mathcal{T}}_{\epsilon }$, Function 1} 7.    Calculate ${\mathcal{I}}_{\epsilon }\left(\mathcal{T}\right)={\mathcal{H}}_{{\epsilon }_{min}}\left(\mathcal{T}\right)-{\mathcal{H}}_{\epsilon }\left(\mathcal{T}\right)-{\mathcal{H}}_{\epsilon }\left(\mathcal{T}\vee {\mathcal{T}}_{\epsilon }\right)$ {$\epsilon$-information, equation (18)}. 8.    Compute ${\mathcal{E}}_{\epsilon }\left(\mathcal{T}\right)=eps\_capacity\left(\mathcal{T},\epsilon \right)$ {$\epsilon$-capacity, Function 2}. 9.    If ${\mathcal{I}}_{\epsilon }\left(\mathcal{T}\right)>{\mathcal{E}}_{\epsilon }\left(\mathcal{T}\right)$ then 10.     Break 11.   End if. 12. End for. 13. Return $\mathsf{\Delta }=2\epsilon$.

The algorithm includes two functions, $eps\_entropy\left(\mathcal{T},{\mathcal{T}}_{\epsilon }\right)$ and $eps\_capacity\left(\mathcal{T},\epsilon \right)$ which are defined as follows.

Function 1. $eps\_entropy\left(\mathcal{T},{\mathcal{T}}_{\epsilon }\right)$

Input:  Set $\mathcal{T}=\left\{t_0,t_1, t_2,\dots ,t_m\right\}$ of time moments, $t_j<t_{j+1}$, $j=0, 1, 2,\dots ,m-1$; set ${\mathcal{T}}_{\epsilon }=\left\{t_{\epsilon ,0},t_{\epsilon ,1}, t_{\epsilon ,2},\dots ,t_{\epsilon ,m_{\epsilon }}\right\}$ of time moments, $t_{\epsilon ,j}<t_{\epsilon ,j+1}$, $j=0, 1, 2,\dots ,m_{\epsilon }-1$. Output:  $\epsilon$-entropy ${\mathcal{H}}_{\epsilon }\left(\mathcal{T}\vee {\mathcal{T}}_{\epsilon }\right)$ of the set $\mathcal{T}\vee {\mathcal{T}}_{\epsilon }$.

Join the sets $\mathcal{T}$ and ${\mathcal{T}}_{\epsilon }$: ${\mathcal{T}}_{joint}=\mathcal{T}\cup {\mathcal{T}}_{\epsilon }$. Find the number $N\left({\mathcal{T}}_{joint}\right)$ of elements in the set ${\mathcal{T}}_{joint}$. Set ${\mathcal{N}}_{\epsilon }\left({\mathcal{T}}_{joint}\right)=N\left({\mathcal{T}}_{joint}\right)-1$. Set ${\mathcal{H}}_{\epsilon }\left(\mathcal{T}\vee {\mathcal{T}}_{\epsilon }\right)={\log }_2{\mathcal{N}}_{\epsilon }\left({\mathcal{T}}_{joint}\right).$ Return ${\mathcal{H}}_{\epsilon }\left(\mathcal{T}\vee {\mathcal{T}}_{\epsilon }\right)$.

The function $eps\_entropy$ was implemented in MATLAB^® by concatenation of the sets $\mathcal{T}$ and ${\mathcal{T}}_{\epsilon }$ using the function cat with further removing of the doubling elements by the function unique.

Function 2. $eps\_capacity\left(\mathcal{T},\epsilon \right)$

Input:  Set $\mathcal{T}=\left\{t_0,t_1, t_2,\dots ,t_m\right\}$ of time moments, $t_j<t_{j+1}$, $j=0, 1, 2,\dots ,m-1$; radius $\epsilon >0$. Output:  $\epsilon$-capacity ${\mathcal{E}}_{\epsilon }\left(\mathcal{T}\right)$ of the set $\mathcal{T}$.

1.  If $\left(t_m-t_0\right)\le \epsilon$ then 2.    Set ${\mathcal{M}}_{\epsilon }\left(\mathcal{T}\right)=1$. 3.  Else 4.    Set ${\mathcal{M}}_{\epsilon }\left(\mathcal{T}\right)=2$. 5.    Set $j=0$. 6.    For $i=1$ to $m-1$ do: 7.      If $\left(t_i-t_j\right)\le \epsilon$ or $\left(t_m-t_i\right)\le \epsilon$ then 8.        Continue. 9.      Else 10.       Set ${\mathcal{M}}_{\epsilon }\left(\mathcal{T}\right)={\mathcal{M}}_{\epsilon }\left(\mathcal{T}\right)+1$. 11.       Set $j=i$. 12.     End if. 13.   End for. 14. End if. 15. Set ${\mathcal{E}}_{\epsilon }\left(\mathcal{T}\right)={\log }_2{\mathcal{M}}_{\epsilon }\left(\mathcal{T}\right)$. 16. Return ${\mathcal{E}}_{\epsilon }\left(\mathcal{T}\right)$.

The function $eps\_capacity$ computes the number ${\mathcal{M}}_{\epsilon }\left(\mathcal{T}\right)$ of $\epsilon$-distinguishable elements in the set $\mathcal{T}$ for given $\epsilon$ and then computes ${\mathcal{E}}_{\epsilon }\left(\mathcal{T}\right)$ as ${\mathrm{l}\mathrm{o}\mathrm{g}}_2$ of this number.

Time complexity $\mathcal{C}$ of Algorithm 1 includes the following terms: $\mathcal{O}\left(1\right)$ – complexity of the lines 1-4; $\mathcal{O}\left(m\right)$ – complexity of the line 5; $\mathcal{O}\left(m\log m\right)$ – complexity of the line 6; $\mathcal{O}\left(1\right)$ – complexity of the line 7; $\mathcal{O}\left(m\right)$ – complexity of the line 8 and $\mathcal{O}\left(1\right)$ – complexity of the lines 9-13. Then, time complexity of each iteration of the algorithm is $\mathcal{O}\left(m\log m\right)$. The maximal number of iterations is $n=\left(t_m-t_0\right)/2s$; hence complexity of Algorithm 1 is

$$\mathcal{C}=n\times \mathcal{O}\left(m\log m\right).$$

(33)

Convergence of Algorithm 1 is guaranteed by the indicated above fact that $\epsilon$-information ${\mathcal{I}}_{\epsilon }\left(\mathcal{T}\right)$ increases with increasing $\epsilon$ while $\epsilon$-capacity ${\mathcal{E}}_{\epsilon }\left(\mathcal{T}\right)$ decreases with increasing $\epsilon$. Since the interval $T=\left[0, t_m\right]$ is bounded, the difference between increasing $\epsilon$-information ${\mathcal{I}}_{\epsilon }\left(\mathcal{T}\right)$ and decreasing $\epsilon$-capacity ${\mathcal{E}}_{\epsilon }\left(\mathcal{T}\right)$ has its minimum in $T$, which is a terminating point of the algorithm.

Dependence of the functions ${\mathcal{I}}_{\epsilon }\left(\mathcal{T}\right)$ and ${\mathcal{E}}_{\epsilon }\left(\mathcal{T}\right)$ on the interval length $\mathsf{\Delta }=2\epsilon$ is illustrated in Figure 1.

The computed interval is $\mathsf{\Delta }=14.21$. For this interval and $\epsilon =\mathsf{\Delta }/2=7.10$, the values of $\epsilon$-information and $\epsilon$-capacity are ${\mathcal{I}}_{\epsilon }\left(\mathcal{T}\right)={\mathcal{E}}_{\epsilon }\left(\mathcal{T}\right)\approx 3.7$ bit. Note that the accuracy of computing the interval $\mathsf{\Delta }$ increases with decreasing the step $s$.

6. Examples

First, let us consider the examples of computing the interval lengths for different distributions of time intervals. In all considered cases we assume that the length of time interval $T=\left[0, t_m\right]$ is $t_m=100$ and $m=100$.

The data were generated by the MATLAB^® function random with respect to the distribution created by the MATLAB^® function makedist. In the examples, we used uniform distribution with $a=0$ and $b=t_m$, normal distribution with $\mu =t_m/2$ and $\sigma =t_m/6$, and exponential distribution with $\mu =2$.

The obtained interval lengths $\mathsf{\Delta }$ were used as bin lengths $\delta$ in the histograms. For comparison, we present the histograms plotted with the bin lengths $\delta$ calculated using the Scott rule (see equation (3)), which is also used as a basis for a default method in MATLAB^®. The resulting histograms are shown in Figure 2.

The values of the interval lengths $\mathsf{\Delta }$ are:

-

evenly distributed data:

Δ = 14.21, δ₁ = 9.90, δ₂ = 12.95, δ₃ = 21.81 and δ₄ = 21.54,

-

uniform distribution with $a=0$ and $b=t_m$:

Δ = 14.11, δ₁ = 9.83, δ₂ = 12.87, δ₃ = 22.69 and δ₄ = 22.65,

-

normal distribution with $\mu =t_m/2$ and $\sigma =t_m/6$:

Δ = 14.91, δ₁ = 8.27, δ₂ = 10.83, δ₃ = 13.11 and δ₄ = 10.38,

-

exponential distribution with $\mu =2$:

Δ = 1.20, δ₁ = 1.07, δ₂ = 1.41, δ₃ = 1.35 and δ₄ = 0.91.

The suggested method results in the interval lengths $\mathsf{\Delta }$ that are close to the interval lengths $\delta$ provided by the conventional methods with respect to the distribution of the data. In fact, for evenly and uniformly distributed data interval length $\mathsf{\Delta }$ is close to the interval length ${\delta }_2$ resulted by the Sturges method, for normal distribution ${\delta }_2<\mathsf{\Delta }<{\delta }_3$ and for exponential distribution ${\delta }_1<\mathsf{\Delta }<{\delta }_2$.

Now let us consider the use of the suggested algorithm for specification of the arrival rates $\lambda$ and corresponding service rates $\mu$. Assume that the office, where the mentioned above clerk works, serves $480$ clients $8$ hours during the day that is $T=8\times 60=480$ minutes. Also, assume that the clients arrive by three “waves” – in the morning, in the midday and in the evening. The histogram of the number of clients during the day is shown in Figure 3.a. In this histogram the bin length is computed by the Scott rule (value ${\delta }_3$ below).

The values of the interval lengths $\mathsf{\Delta }$ for this distribution are:

Δ = 22.0, δ₁ = 21.91, δ₂ = 48.45, δ₃ = 67.99 and δ₄ = 66.41.

Histogram of the number of clients during the day with the bin length $\delta =\mathsf{\Delta }$ computed by the suggested algorithm is shown in Figure 3.b. Dependence of the functions ${\mathcal{I}}_{\epsilon }\left(A\right)$ and ${\mathcal{E}}_{\epsilon }\left(A\right)$ on $\epsilon$ for this distribution is shown in Figure 3.c.

From the results of computations of the interval length $\mathsf{\Delta }$ and the bin lengths $\delta$ it follows that by the suggested algorithm the arrival rates during a day should be calculated each $22$ minutes, while by the Scott they should be calculated each $68$ minutes. Thus, for multimodal distribution the suggested algorithm results in shorter intervals that provides more exact representation of the data.

7. Conclusion

In the paper, we suggested the method of calculating optimal time intervals required for definition of arrival and departure rates. The method is useful for specification of the bin lengths in histograms, especially for the data with multimodal distributions.

The method utilizes the Kolmogorov and Tikhomirov $\epsilon$-entropy and $\epsilon$-capacity and the Rokhlin entropy of partition. Optimality of the partition is defined basing on the $\epsilon$-information.

The procedure is presented in the form of a ready-to-use algorithm, which was compared with the known methods used for calculation of the interval lengths in histograms and demonstrated its robustness and correct sensitivity to the data.