Energy-Saving Adaptive Routing for High-Speed Railway Monitoring Network Based on Improved Q Learning

1. Introduction

With the rapid development of the high-speed railway (HSR) network, higher requirements have been placed on the safety and stability of high-speed trains, and the train operating environment is directly related to the safety of high-speed trains in transit. How to quickly find and solve basic faults is the focus of high-speed railway (HSR) systems around the world, and high-speed train operation environment monitoring is an important guarantee for railway safety operations[1]. The safety of high-speed railway systems can be significantly improved through real-time monitoring and inspection. The high-speed railway operation environment monitoring system includes the railway infrastructure monitoring system, the high-speed railway natural disaster and foreign body intrusion monitoring system, the EMU on-board dynamic monitoring system, and the EMU operation and maintenance management system . At present, the wired communication network adopted by these monitoring systems has been verified to be stable and reliable. However, the high-speed railway operating environment is complex, and the high cost of wired network is not conducive to large-scale deployment of the global high-speed railway monitoring system, and the complex terrain is inconvenient for wired network maintenance[2,3].

With the development and maturity of wireless sensor network technology, it provides a more efficient and reliable, low-cost, easy-to-implement and maintain, high-tech means for the field of high-speed train operation environment monitoring. The use of wireless systems to monitor the operating environment of high-speed railways can realize large-scale deployment along the railway, ensure the breadth and accuracy of data collection, and can monitor the condition of slopes, tunnels, roadbeds, bridges, and other facilities, which can effectively meet the needs of high-speed trains. The need for comprehensive monitoring of the operating environment reduces the cost of train operating environment testing and is suitable for large-scale deployment and long-term online monitoring in key areas and remote areas along the high-speed railway. However, the energy resources of wireless devices are strictly limited. Wireless sensor networks’ biggest problem is saving energy, and the length of time the network lasts is a good measure of how well it works[4,5].

In order to ensure the reliability and real-time performance of high-speed railway operating environment monitoring information and the energy utilization efficiency of the network system, a protocol that adapts to the characteristics of the high-speed railway operating environment monitoring network is needed to efficiently utilize the limited energy resources of the network system and provide more long-term energy efficiency[6]. Serve. The increase in communication distance will increase the energy consumption of some nodes and shorten the network life. Each protocol chooses to increase the number of forwarding hops to achieve a balance of energy consumption, and the overall energy consumption is evenly distributed to more nodes to avoid premature death of some nodes. But the increase in hop count will inevitably increase the delay of information transmission, which is bad news for the high-speed railway monitoring network[7,8]. In fact, it is difficult to guarantee the real-time performance of the network and its maximum lifespan at the same time. Extending the lifespan and increasing the number of hops will result in a large delay and affect the service quality of the monitoring network. To this end, in the intelligent high-speed railway monitoring system , the information in the network can be divided into several parts: receiving, transmitting, processing, state evaluation and prediction, and control decision-making . Generally speaking, the transmission of data along the shortest path can minimize the energy consumption of the network, but this approach will introduce the problem of unbalanced energy consumption. The energy consumption of the sensor device closer to the sink node is faster, which is the so-called energy hole phenomenon [9,11].

This phenomenon will destroy the balance of energy consumption among nodes, affect the life of the network, and also have a negative impact on the real-time performance of data transmission, hindering the normal service of the network system. Therefore, in addition to minimizing the network energy consumption, the energy consumption balance among network nodes should also be considered when designing the routing algorithm to extend the life of the network system. Therefore, when designing a routing protocol for a high-speed railway operation environment monitoring network, it is necessary to minimize and balance network energy consumption while reducing data transmission delay, so as to improve real-time performance, improve network energy utilization, and prolong network service life. So, the goal of this paper is to look into how a high-speed railway operation environment monitoring network can use an adaptive routing method to meet the needs of different services[12,13].

Table A1 introduces some typical existing studies and evaluates them in terms of optimization goals, adoption methods, network structures, and advantages and disadvantages. Most of the research is aimed at improving the lifetime of network systems, and the strategies to maximize network lifetime can be divided into deployment optimization, data processing, and protocol design. Among them, routing protocol design optimization is a more effective and widely used strategy, and protocol design can be divided into single-hop, multi-hop, and clustering methods. However, for a typical linear network such as a high-speed railway operating environment monitoring network, it is obviously more appropriate to use a multi-hop routing protocol. In some studies, the network lifetime is extended by minimizing the total network energy consumption (MTECR). However, its shortcomings are also obvious. As mentioned above, the problem of unbalanced network energy consumption while minimizing energy consumption leads to network energy holes, resulting in the premature death of some nodes, which affects the service life of the network. In addition, some other research focuses on balancing the energy consumption of nodes in the network, such as MVECR and AUMRP, and proves that their balancing scheme based on the residual energy of nodes is more effective for prolonging the network lifetime.

To the best of our knowledge, there are few studies on the co-optimization of lifetime and delay for high-speed railway operational environmental monitoring networks. Traditional routing algorithms cannot ensure stable transmission services in dynamic environments because they cannot cope with network dynamics and voids. If maintaining global routing information, this will incur great overhead, and the complex structure will also reduce the efficiency of the current high-speed railway linear monitoring network. These issues are addressed in this article.

2. Motiviation

Power supply and grid maintenance for high-speed rail operating lines are very difficult. With the development and maturity of wireless sensor network technology, high-speed train operation provides more efficient, reliable, and low-cost in the field of environmental monitoring and detection, and then implements high-tech manual maintenance. Experts have done a lot of in-depth research on the coordination of the routes of the plane network, but less research on the coordination of the high-speed railway network. If there is a general problem with existing routing protocols, it is their lack of applicability to monitoring regional environments. There are many types of sensors in high-speed rail systems. Various sensors vary widely in terms of latency, transfer rate, data volume, etc. Inspired by the above requirements and existing work, this paper proposes a heterogeneous network data aggregation model and adaptive routing algorithm for high-speed railway monitoring network based on reinforcement learning. The overall design is shown in Figure 1.

The main contributions of this paper can be summarized as follows:

• A lossless data aggregation transmission model for HSR networks is proposed, which can effectively reduce the amount of data in the network and reduce the energy consumption of data transmission;
• A DoubleQ-values model based on data aggregation is proposed.Forn the two Q values, we consider the data aggregation degree, the remaining energy level, the link strength, the distance from the node to the sink, and the forwarding delay to consider the network lifetime and the real-time performance of data forwarding. The defined reward function can capture the dynamic changes of the network in real time and achieve dynamic control of the entire network with less overhead;
• An adaptive energy-saving routing algorithm based on DoubleQ-values is proposed to classify HSR network devices according to their real-time requirements and life cycle requirements. An adaptive control algorithm is adopted for different business priorities. It meets the real-time requirements of the HSR network and prolongs the network’s life and improves service quality.

The work arrangement of this paper is as follows. Sec.3 presents the typical structure of the high-speed railway monitoring network as well as the analysis of node requirements and introduces the data aggregation scheme. Sec.4 describes in detail the proposed demand-aware energy-saving routing algorithm based on Q-learning. Sec.5 discussed and analyzed the performance metrics of the proposed routing protocol through simulation experiments. The conclusions are summarized in Sec.6.

3. System Profile and Overall Scheme

This section may be divided by subheadings. It should provide a concise and precise description of the experimental results, their interpretation as well as the experimental conclusions that can be drawn.

The system involved in this paper consists of the following four parts:

• Structure and characteristics of communication;
• Keeping track of objects and requirements;
• Policy of aggregation and intelligent routing design;
• Algorithm for adaptive routing.

3.1. Communication structure and characteristics

The high-speed train operating environmental monitoring system based on the wireless sensor network is simply called HSR-N, which is used for the completion, replenishment, or replacement of the transmission high-speed train operating environmental monitoring system. The proposed wireless sensor network can be roughly considered as a linear network.

In HSR-N, its monitoring areas are wide and varied, and different monitoring targets have great differences. Therefore, relevant parameters and network technologies need to be designed for specific monitoring targets. With limited network energy, HSR-N needs an energy-efficient transmission protocol that meets its requirements.

3.2. Monitoring objects and requirements

Line inspection mainly includes bridge inspection, tunnel monitoring, road base monitoring, contact network monitoring, and rail monitoring. Estimate the volume of five parts.

Some monitoring objects and their corresponding characteristics are shown in Table A2. The real-time requirements and life cycle requirements of monitoring objects are different. We make a brief evaluation of their different functional requirements to provide support for the following work. The evaluation is mainly allocated based on the urgency of the data and the amount of data. For data types with high real-time requirements, we give higher real-time evaluation to reduce delay; for periodic data with low real-time requirements, we Based on how the data is set up, the aggregation processing is made to give the network a longer life cycle [22,23].

According to the different functions and requirements of various types of sensors, this paper divides all sensors into high, medium, and low grades according to their life cycle and real-time requirements[24]. In the operation of high-speed railways, their security has higher requirements for information delay. However, the energy constraints of the wireless network determine the conflict between the network lifetime and the delay. The change of most infrastructure states is a very slow process, so the information collected by the band basically changes slowly or unchanged. The collection and transmission of information should consider the life cycle of the network as much as possible. However, changes in the power supply network of high-speed railways and changes in some natural environments, especially changes in conditions such as meteorological disasters, require strong real-time performance. Under such conditions, real-time performance has a higher priority.

3.3. Aggregation policy and intelligent routing design

For most of the monitoring requirements of chronic changes, we only need to collect their data periodically and analyze it, while some monitoring requires only a large number of repeated measurements as the train passes by. For this data with low real-time requirements, we consider data aggregation on the transmission path to reduce the transmission burden of nodes to achieve the purpose of saving energy and prolonging the network life cycle. Data aggregation A lossless aggregation model is adopted in this study. This means that the original data can be reconstructed by the sink node from the received aggregated data packets without any damage or loss of data. The data aggregation model is expressed as follows:

$$\begin{array}{c}DA\left\{Q_i^t\left(n\right)\right\}=\left\{\begin{array}{cc}{U}_m^t\times {\log }_2(D{P}_i\left(n\right)+1)& i{f}_{}0<D{P}_i\left(n\right)\\ 0& i{f}_{}D{P}_i\left(n\right)=0\end{array}\right.\hfill \end{array}$$

(1)

The data aggregation model is defined in this paper. The data buffer area of each sensor is partitioned, and each sensor maintains storage areas of multiple data types. For the same type of data, when the real-time requirement is low, the sensor performs data aggregation before forwarding to optimize the data volume. When a packet of data type T is transmitted in the network, each node transmits to the next hop after the interval $S{I}^t$. Obviously, the longer the waiting time for data aggregation, the greater the delay for the data packet to reach the sink node. It can be seen from Table 2 that different types of sensors have different real-time requirements for data. Therefore, in the design of routing protocols, the real-time requirements of different nodes need to be considered when energy-saving design is carried out[25,26]. This will be explained in Sec. IV. The aggregation process design is shown in Figure 2 and model parameters are listed in Table 1.

If the sensor node $S_i$ is the next hop selected by $S_j$,$S_k$, and $S_l$, the $t1$-type aggregated data packets ${\mathrm{AD}}_{\mathrm{j}}^{t1}\left(n\right)$ from the node $S_j$ and the $t1$-aggregated data packets ${\mathrm{AD}}_{\mathrm{k}}^{t1}\left(n\right)$ of the $S_k$ node are stored together with the $t1$ data packets ${\mathrm{OD}}_{\mathrm{i}}^{t1}\left(n\right)$ observed by the $S_i$ node from the surrounding environment After aggregated into data packets ${\mathrm{AD}}_{\mathrm{i}}^{t1}\left(n\right)$ in the $t1$ data type queue ${\mathrm{Q}}_{\mathrm{i}}^{t1}\left(n\right)$ of node $S_i$, they are sent to the next hop node $S_p$ of $t1$ type. Similarly, the $t2$ type data packets ${\mathrm{AD}}_{\mathrm{k}}^{t}2\left(n\right)$ from nodes $S_k$ and $S_l$, ${\mathrm{AD}}_{\mathrm{l}}^{t2}\left(n\right)$ is in the $t2$ data of node $S_i$ The data packets ${\mathrm{AD}}_{\mathrm{i}}^{t2}\left(n\right)$ are aggregated in the type queue ${\mathrm{Q}}_{\mathrm{i}}^{t2}\left(n\right)$ and sent to the next-hop node $S_q$ of type $t2$. After this aggregation process, the data of the same type will be aggregated first and sent to the next-hop node with the same data type until The data packet is sent to the sink node to complete the aggregation and transmission of the data.

3.4. Adaptive routing algorithm

In the aggregation model described in Section 3.3, the aggregated data of each sensor node is sent to the optimal next-hop node, and the next-hop node selection is determined by the Q-learning adaptive algorithm proposed in this paper (See Section 4). The adaptive routing algorithm consists of three parts: 1)The sending node selects the node with the highest priority in the Q routing table to send the data packet; 2)The receiving node feeds back the reward value information to the sending node according to the received data packet; 3)The sending node accepts the reward value information and updates its Q routing table.

4. Model and Methodology of the Adaptive Protocol

In order to maximize the network life cycle and meet the functional requirements of various types of sensors, an adaptive protocol model based on Q-learning is proposed in this section, as shown in Figure 3. For a linear heterogeneous network such as a high-speed railway monitoring network, according to different functional requirements, ensuring the real-time of emergency or security service information while taking into account the life cycle of the entire network is crucial for monitoring the service quality of the system[27].

Q-learning is a model-free reinforcement learning algorithm whose core is Q-value and reward[28]. This technology enables nodes in the network to use the local Q-table to select the best next hop without the overhead of maintaining global routing information, which makes it applicable to routing problems in sensor networks. Based on the traditional single-objective Q-learning route, this paper proposes a double-Q-value adaptive algorithm to meet the needs of the high-speed railway monitoring system to ensure the real-time performance and life cycle of the monitoring network. The parameters and definitions used in this section are shown in Table 2.

4.1. Energy model

Nodes adopt a periodic sleep/active work mode. The main energy consumption of nodes can be divided into two parts:

• Energy consumption in active mode. We use ${\omega }_a$ denotes the energy consumption rate in this mode.
• Energy consumption when nodes send and receive data.

We adopt the typical WSN energy consumption model to calculate the energy consumption of sending, receiving, and aggregated data, which is given by (3),(4)and(5) respectively.

$$E_T\left(l\right)=\left\{\begin{array}{c}l\times E_{elec}+l\times {\epsilon }_{fs}\times d^2,d<d_0\\ l\times E_{elec}+l\times {\epsilon }_{mp}\times d^4,d\ge d_0\end{array}\right.$$

(2)

$$E_R\left(l\right)=l\times E_{elec}$$

(3)

$$E_D\left(l\right)=l\times E_{ec}$$

(4)

Among them, $E_{e}lec$ represents the node’s energy consumption when sending, and $d_0$ represents the distance between nodes. When $d<d_0$, the node energy consumption is in the normal loss mode; when $d>d_0$, the node energy consumption is gradually reduced; ${\epsilon }_{fs}$ and ${\epsilon }_{amp}$ represent the energy at different distances $E_{e}c$ is the energy expended for computation; l is the length of the data frame.

4.2. Double Q value learning model

The Q-learning algorithm is a value function-based algorithm in RL, and for any finite Markov decision process, Q-learning can find an optimal policy. Q-learning involves an agent, a set of states S, and a set of actions A. By performing actions in the environment that cause the agent to move from one state to another, the action in a particular state is rewarded. That is, $Q(s,a)$ is the expected reward for performing action a($ainA$) in state s($sinS$) at a given time[29]. The algorithm used in this study is shown in Figure 4.

4.3. Figures, Tables and Schemes

4.3.1. State and Aaction

In the proposed double-Q value learning model for high-speed railway wireless monitoring network routing, adjacent nodes exchange routing information in a cooperative way to ensure that nodes in the network can dynamically follow network changes and reduce the burden of maintaining the global routing table[30]. Define a sensor node’s node set S, action $ainA$, and action state set A as follows when it sends a specific type of data to the next hop node:

$$\begin{array}{c}S=\left\{s_1,s_2,\dots ,s_n\right\}\hfill \\ A=\left\{A_1,A_2,\dots ,A_n\right\},A_i=\left\{a_j=s_j|s_j\in F_{si}\right\}\hfill \end{array}$$

(5)

where n is node number and $F_{si}$ is the Set of forwarding nodes of node $s_i$.

4.3.2. Initialize the Double Q-values

In Q-learning, the forwarding of data between nodes uses a Q-table to find the best action, where Q-value is the expectation of nodes when forwarding[31]. In the double Q-value model designed in this paper, the action value functions are divided into life cycle functions and real-time functions $Q_L$ and $Q_T$. $Q_L$ consists of three parts: data aggregation degree, node energy status, and link strength. The first part aims to increase the aggregability of forwarded data packets and reduce the data size to reduce the energy loss caused by data transmission. The second part avoids selecting energy. Nodes with low values are forwarded, and the third part aims to reduce communication overhead and save energy.$Q_T$ consists of two parts: the number of hops reaching the sink node and the forwarding delay estimation, both of which aim to ensure the real-time performance of the data packet reaching the sink node. As shown in (7), double Q-values are initialized as a weighted sum of the probabilities of their respective parts.

$$\begin{array}{c}{Q}_L(s,a)=DA(s,a)+E_s^a+L_s^a\hfill \\ Q_T(s,a)=d(a,sink)+T_s^a\hfill \end{array}$$

(6)

Where $DA(s,a)$ denotes the degree to which node s aggregates data to the node indicated by its action a; $E_s^a$ is the remaining energy of the node pointed to by action a; The link strength between node s and the node pointed to by action a is represented by $L_{s}a$. $d(a,sink)$ is the distance between the node indicated by action a and the sink. $T_s^a$ is forwarding time from node s to the node pointed to by its action a; Before starting, the Q-values are initialized only by the initial energy and the distance to the sink, and other parameters will be updated after running.

4.3.3. Double Q-values update

In this paper,$Q_{L,T}(s,a)$ defines the possibility of state s acting a and provides various types of businesses with a Q table based on their business requirements, which is defined as follows:

$$Q_{L,T}(s,a)=\left[\begin{array}{c}{Q}_{L,T}^{t1}(s,a)\\ Q_{L,T}^{t2}(s,a)\\ ⋮\\ Q_{L,T}^{tn}(s,a)\end{array}\right]$$

(7)

Among them, $t1-tn$ is the business type, $Q_L$ is the life cycle measurement of an action, and $Q_T$ is the real-time measurement of an action.

When a node selects the optimal next hop in its Q table to send a packet, it gets a reward from the receiving node and updates its Q value accordingly. The new Q value is (8).

$$Q_{L,T}(s,a)=Q(s,a)+\alpha \left\{R_{\mathrm{L},T}-\gamma ·Q(s,a)\right\}$$

(8)

where $\alpha$ is the learning rate and $\gamma$ is the discount factor for the future reward.

4.3.4. Explore strategies

Usually, action selection relies only on the highest Q value, but this fixed selection can get stuck in a local optimum. To do this, we use an epsilon-greedy algorithm that makes it possible to escape local optima with partial probability.

$$a_{*}|\mathrm{s}=\left\{\begin{array}{cc}\arg \max Q(s,a)& withprobablity1-ϵ\\ anyactiona& withprobablityϵ\end{array}\right.$$

(9)

4.3.5. Future rewards

In this stage, rewards are given for the action performed in the previous step, which can be divided into three situations.

1) The node receiving data packets is not the sink node, and the energy level is normal. We give each component of double Q-Values its own reward scheme, calculated as follows.

$$D{A}_{nor}=\left\{\begin{array}{cc}\frac{Q_{s^{\prime }}^t\left(n\right)}{A{D}_{s^{\prime }}^t\left(n\right)}-1& if\frac{Q_{s^{\prime }}^t\left(n\right)}{A{D}_{s^{\prime }}^t\left(n\right)}-1<r_{DA}^{\max }\\ r_{DA}^{\max }& else\frac{Q_{s^{\prime }}^t\left(n\right)}{A{D}_{s^{\prime }}^t\left(n\right)}-1>r_{DA}^{\max }\end{array}\right.$$

(10)

$$E_{nor}=E_r^{s^{\prime }}/E_i^{s^{\prime }}$$

(11)

$$\begin{array}{c}{L}_s(s_i,s_j)=\frac{\mathrm{Re}{c}_{s_{i}p}}{\mathrm{Re}{c}_{s_{j}ob}}\hfill \\ \mathrm{Re}{c}_{s_{j}ob}={⌈\frac{p_r\left(d\right)}{p_r\left(d_0\right)}⌉}_{db}=-10\rho \log \left(\frac{d}{d_0}\right)+X_{db}\hfill \\ L_{nor}=l_s(s,s^{\prime })\hfill \end{array}$$

(12)

$$\begin{array}{c}radv=d(s,sink)-d(s^{\prime },sink)\hfill \\ rad{v}_{avg}={\sum }_{i=1}^n\left|rad{v}_i\right|/n\hfill \\ A_{nor}=radv/rad{v}_{avg}\hfill \end{array}$$

(13)

$$T_{nor}=T(s,s^{\prime })/T_{avg}$$

(14)

2)The receiving node is the $sink$. The reward is a constant $R_s$ when the chosen action sends the data packet to the sink node.

$$R=R_s$$

(15)

3) The receiving node is not a sink, and the energy level is too low. In order to maintain the performance and life of the network, we suggest not giving the node the function of forwarding data when the energy level is too low to ensure that the basic monitoring service of the network is normal [33]. Therefore, we give a negative reward to avoid packets from neighbor nodes.

$$R=-R_e$$

(16)

Based on the above definition, corresponding to the double Q-values, the future reward should also be divided into two parts to give different rewards for its life cycle and real-time performance. At the same time, in order to avoid increasing the probability of forwarding to nodes far away from the sink, it is necessary to provide a discount value to the reward. The reward $R_L$ and $R_T$ for the state s are calculated as follows:

$$R_L=\left\{\begin{array}{cc}{\alpha }_l\times R_{DA}+{\beta }_l\times R_E+{\gamma }_l\times R_L& s^{\prime }\ne sink\\ R_s& s^{\prime }=sink\\ -R_e& E_{s^{\prime }}islow\end{array}\right.$$

(17)

$$R_T=\left\{\begin{array}{cc}{\alpha }_t\times R_A+{\beta }_t\times R_T& s^{\prime }\ne sink\\ R_s& s^{\prime }=sink\\ -R_e& E_{s^{\prime }}islow\end{array}\right.$$

(18)

4.4. Adaptive routing protocol based on double Q-values

In order to meet different business needs, this section proposes an adaptive weighting scheme based on the proposed double Q-values model and the principles of maximizing network lifetime and adaptively meeting the functional requirements of various types of sensors, trying to consider both energy saving and delay reduction. Two objectives are used to adapt to different business objectives. The multi-objective function formula is as follows:

$$\max Q=\max ({\alpha }_w\times Q_L+(1-{\alpha }_w)\times Q_T)$$

(19)

where ${\alpha }_w$ is a business adaptive weighting factor used to adjust the weight of life cycle and real-time goals. As shown in Table 1, different services in the system have different requirements for life cycle and real-time performance. When the real-time performance requirement of the monitoring object is low, it means that the object has higher requirements for the continuity of data transmission. The adaptive weight factor will increase with the improvement of life requirements; and when the real-time requirement of the monitored object is high, the algorithm will give priority to meeting its real-time requirement. Therefore, the design of the adaptive weight factor is as follows: in this formula, low, medium, and high values are 0.2, 0.5, and 0.8.

$$\begin{gathered} {\alpha }_w=\left\{\begin{array}{cc}\frac{\log (\eta \times (D_l^s/\phantom{D_l^s{D}_t^s}{D}_t^s\left)\right)}{\log (\eta \times \max (D_l^s/\phantom{D_l^s{D}_t^s}{D}_t^s\left)\right)}& D_t^s=0.2\\ \frac{0.5\times \log (\eta \times (D_l^s/\phantom{D_l^s{D}_t^s}{D}_t^s\left)\right)}{\log \max \left(D_l^s/\phantom{D_l^s{D}_t^s}{D}_t^s\right)} \\ +\frac{0.5\times 1}{1+e^{\frac{(D_l^s-D_t^s)\times D_t^s}{(\xi -(D_l^s-D_t^s)\times D_t^s)}}}& D_t^s=0.5\\ \frac{1}{1+e^{\frac{(D_l^s-D_t^s)\times D_t^s}{(\xi -(D_l^s-D_t^s)\times D_t^s)}}}& D_t^s=0.8\end{array}\right. \end{gathered}$$

(20)

Among them, the life cycle demand amplification factor $\eta$ and the real-time demand amplification factor $\xi$ are defined as follows:

$$\begin{array}{cc}\eta =\frac{\max D_l^s}{\max D_t^s}& 0<D_l^s<1,0<D_t^s<0.5\end{array}$$

(21)

$$\begin{array}{cc}\xi =\max (D_t^s-D_l^s)& 0<D_l^s<1,0.5<D_t^s<1\end{array}$$

(22)

5. PERFORMANCE COMPARISON AND VALIDATION

In this section, we compare and analyze the performance of the proposed Double Q-values Adaptive Aggregation Routing Protocol (DQAAR) in terms of energy consumption, network lifetime, transmission delay, and data retransmission energy loss. At present, there is little research on multi-objective optimization of high-speed railway wireless sensor network delay and life cycle. This paper uses MATLAB R2018b to realize the simulation environment and compares it with the other three excellent routing protocols. They are METCR, AUMRP, and MVECR.

5.1. Parameters Configuration

In this study, the node communication energy consumption adopts the space energy loss model, and the simulation parameter configuration is shown in Table 3.

The initial energy of the sensor nodes involved in this paper is $0.5j$, and the energy of $sink$ nodes is unlimited. According to the node types and business requirements in Table 1, the network model is constructed proportionally to verify the performance of the adaptive routing model in this paper. In the comparative analysis before, we introduced a few concepts about performance indicators. 1) FND (the time at which the first node dies). 2) HND (time when half of the nodes die). 3) CP Index (Comprehensive Performance Index): The utility of a high-speed railway monitoring network is determined by its life cycle and real-time performance. In this paper, we built a complete evaluation model for life cycle and real-time performance. It looks like this:

$$\mathrm{U}=\lambda \times \frac{L}{\max \left(L\right)}+(1-\lambda )\frac{min\left(T\right)}{T}$$

(23)

Among them,U is the comprehensive performance index,L is the life cycle, and T is the delay.

5.2. Results and Discussion

In order to verify the effectiveness of the adaptive routing protocol proposed in this paper, this paper first verifies the effect of extending the network life cycle of each protocol, and then simulates environmental changes and different application scenarios by changing some system parameters. Several aspects, such as extending the comprehensive efficiency index, are compared with the three routing protocols mentioned above, which verifies the superiority of DQAAR.

5.2.1. Lifetime evaluation

Figure 5 shows the change in the number of surviving nodes for each routing protocol with the network running time. For the high-speed railway monitoring network, the main task is to collect as much on-site information as possible to ensure the safety of railway operation, so we first focus on the life cycle of the network. Figure 5 (a) shows that DQAAR has a longer running time than METCR, MVECR, and AUMRP under the same conditions. Because METCR only pays attention to the overall energy loss of the network and does not care about the energy balance of the network, its first The death time of each node occurs in about 1000s of Compared with METCR, MVECR, and AUMRP, they impose some constraints on the overall energy consumption balance of the network, but while prolonging the death time of the first node, they also cause a large area of low-energy nodes to die in the network around 1500 s. The DQAAR proposed in this paper delays the death time of the first node in the network to about 2000s, and then there is no continuous death of large-scale nodes but maintains a relatively slow trend. This is because DQAAR gives dynamic rewards based on data aggregation while paying attention to the balanced use of network energy, and each node learns the best next-hop node. Efficient data aggregation paths greatly reduce the amount of data transmitted over the network, which greatly delays node death times. Figure 5 (b) shows the FND and HND data of the four routing protocols, respectively. The FND and HND of the worst performing METCR are 985 and 2024, respectively, and the AUMRP data of FND is only 1565 and 2155, although it is effective. The node death time is delayed, but the process from the death of the first node to the death of half of the nodes is not very slow. The FND and HND of the DQAAR proposed in this paper are 1982 and 3016, respectively. It can be seen that DQAAR gives better answers in terms of balancing network energy consumption and improving network life.

5.2.2. Latency Time

Figure 6 shows the real-time performance of DQAAR and the three protocols mentioned above. METCR achieves the best real-time performance with an average latency of 1780 ms, but this is due to its advantage of reducing hop count at the expense of lifetime. In the initial stage of the network, the efficiency of DQAAR is low, and its real-time performance is greatly improved after a period of learning, which is not much different from METCR.

Compared with the other three routing protocols, the DQAAR delay in the stable operation stage has better stability performance. That is to say, the initial high delay is caused by the exploration behavior of DQAAR. In the stable operation stage, the real-time performance of DQAAR far exceeds that of MVECR and AUMRP, achieving good real-time performance.

5.2.3. scenario analysis

Case 1 verifies the impact of network size changes on its performance and the performance of each routing protocol. Among them, a single data packet on the network data = 200 bits. Figure 7 shows the performance of the life cycle, delay, and overall energy efficiency of each routing protocol when the distance from the head end to the end of the network changes from d = 100 m to d = 500 m. Figure 7 (a) shows that with the increase of the network range, the energy balance of the network system will be destroyed, which also causes the life cycle of each routing protocol to decrease significantly with the increase of d, but compared with AUMRP, MTECR, MVECR, and DQAAR have better performance in extending the network life cycle. When the network range is expanded to 500m, its lifetime is still about 30% higher than the worst performing MTECR. Figure 7 (b) shows the delay performance of each routing protocol as the network range increases, and the time for each routing protocol data packet to reach the sink node gradually increases as the communication distance increases. Among them, MVECR and AUMRP lack the constraints on the delay, which leads to the rapid increase of the delay when the distance increases, and DQAAR reduces the delay through dynamic learning so that the network can obtain good real-time performance. In order to obtain better real-time performance, MTECR reduces the number of hops of data packet forwarding in the network, which greatly increases the energy consumption and shortens the network life. Figure 7 (c) shows the comprehensive energy efficiency coefficients of each protocol.

Obviously, the DQAAR proposed in this paper is ahead of AUMRP, MTECR, and MVECR in terms of comprehensive energy efficiency. When the network range is small, the life cycle of AUMRP and MVECR is close to that of DQAAR, and their comprehensive energy efficiency is also close to that of DQAAR. The life cycle index and delay index of the network system using each protocol are shown in Figure 8 and Figure 9.

Case 2 verifies the impact of changes in the amount of data in the network on its performance and the performance of the four routing protocols. Among them, the network range is d=200 m. Figure 10 shows the network life cycle, delay and comprehensive energy efficiency level of the DQAAR proposed in this paper and the other three routing protocols (AUMRP, MVECR, and METCR) when the single data packet size of the network node changes from Data=100bit to Data=600bit. As shown in Figure 10(a), an increase in the amount of data in the network is accompanied by a rapid decrease in its life cycle, because the sending and receiving of data consumes the most energy in sensor nodes. MVECTR pays too much attention to the energy consumption balance of each node in the network, but it increases its total energy consumption, and the life cycle has a disadvantage compared with AUMRP. DQAAR’s transmission path planning based on data aggregation effectively reduces the amount of data in the network and thus prolongs the life cycle of the network, and its performance is higher than the other three routing protocols. The delay of the network system using DQAAR is smaller than AUMRP and MVECR but slightly larger than MTECR, and the delay increases with the increase of data volume, which is caused by the increase in transmission time caused by the increase in data volume. Data latency increases. Also, as the amount of data increases, the trend of delay growth for DQAAR is not as fast as it is for AUMRP and MVECR. This shows that the data transmission delay is reduced enough by the data aggregation strategy used in this paper to make up for the time it takes to aggregate the data. It can be seen from Figure 10 (c) that DQAAR can effectively reduce the amount of data when the amount of data in the network increases, while the delay does not cause a significant change. Its comprehensive energy efficiency is much greater than the other three routing protocols, and the network system life The cycle and delay maintain stable performance with the increase of data volume.The lifetime index and delay index of the network system using each protocol are shown in Figure 11 and Figure 12.

Finally, the simulation verification results can be summarized into the following three points:

1) Compared with AUMRP, the life of the network system using DQAAR has been improved to a certain extent, and both MVECR and MTECR have been improved to a certain extent, which effectively prolongs the dead time of the first node in the network system. It ensures the balance of network energy consumption and allows nodes with heavy loads in the network to survive longer when the energy level is low to ensure the monitoring quality of the network.

2) In the high-speed railway monitoring system, the life cycle of the network and the delay of data transmission are both important performance indicators, and the single-objective network optimization algorithm is difficult to meet the actual needs. The adaptive routing algorithm based on double Q-values proposed in this paper can effectively improve the network life and obtain good real-time performance. The comprehensive energy efficiency index is used to evaluate the routing protocol and verify the superiority of DQAAR in these two aspects.

3) The design of the adaptive operator and Q-value in this paper comes from the business requirements of each sensor in the high-speed railway monitoring network system. In different application scenarios, the adaptive operator and Q-value can be designed differently. This ensures the multi-scene adaptability of the adaptive model based on double Q-values established in this paper.

6. Conclusion

In this paper, we propose a DoubleQ-values-based adaptive routing algorithm (DQAAR) for the business requirements of high-speed railway monitoring network systems. The proposed method is different from most of the existing methods and has the following contributions:

First, we propose a DoubleQ-values model based on data aggregation. For the two Q values, we consider the data aggregation degree, the remaining energy level, the link strength, the distance from the node to the sink, and the forwarding delay to consider the network lifetime and the real-time performance of data forwarding. The defined reward function can track the network’s changes in real time and keep the whole thing under control with less work.

Second, an adaptive weight is proposed based on the different requirements of each service for network lifetime and real-time performance. This makes the algorithm proposed in this paper better able to adapt to different situations.

Finally, the algorithm proposed in this paper is verified in different scenarios. The results show that DQAAR is better than AUMRP and MVECR in achieving network energy balance and prolonging network life, and its real-time performance is also better than these two routing protocols. Compared with MTECR, although the routing protocol proposed in this paper is slightly insufficient in real-time performance, it is far better than it in extending network life. From the point of view of overall energy efficiency, the DQAAR that is proposed in this paper is a lot better than other routing protocols.