QOS-Aware Task Scheduling Using Reinforcement Learning in Long Rage Wide Area Network IOT Application

Research Design

Algorithm Design and Implementation

Training Phase of the Proposed Scheduling Algorithm

The Trained Proposed Scheduling Algorithm Diagram

Algorithm Implementation

Pseudocode for Task Scheduling Algorithm

• Initialization
• State Observation
• Action Selection
• Environment Interaction (OpenAI Gym Integration)
• Reward Calculation
• Q-Value Update (Learning)
• Training Loop
• Policy Improvement and Execution

• Algorithm 1 Initialization

1)

Initialize Q-network with random weights

2)

Initialize target Q-network with the same weights as Q-network

3)

Initialize Replay Memory D with capacity N

4)

Set ϵ for ϵ-greedy policy

5)

Set learning rate α, discount factor γ, and batch size

6)

Define action space A = {channel selection, task prioritization, gateway allocation}

7)

Define state space S = {channel status, signal strength, gateway congestion, task deadlines}

8)

Define reward function R(s, a) based on QoS metrics

9)

Periodically synchronize target Q-network with Q-network weights every K episodes

• Algorithm 2 State Observation

1)

FunctionObserveState()

2)

Initialize state as an empty list

3)

Normalize current channel status, signal strength (SINR), gateway congestion, and task deadlines

4)

Append normalized values to state

5)

returnstate

• Algorithm 3 Action Selection using ϵ-Greedy Policy

1)

Function SelectAction(state, ϵ)

2)

Generate a random number and ∈ [0, 1]

3)

if rand < ϵ then

4)

Choose a random action from action space A

5)

else

6)

Compute Q-values for all actions using Q-network

7)

Choose action argmax(Q-values) // Select the action with the highest Q-value

8)

end if

9)

return action

• Algorithm 4 Environment Interaction

1)

FunctionPerformAction(action)

2)

Initialize the OpenAI Gym environment

3)

if action == "channel selection" then

4)

Select the channel with the lowest interference and load

5)

else ifaction == "task prioritization" then

6)

Prioritize tasks based on deadlines

7)

else ifaction == "gateway allocation" then

8)

Assign tasks to gateways with optimal load balancing and signal quality

9)

end if

10)

Execute the selected action in the LoRaWAN environment via OpenAI Gym

11)

Observe the resulting state, reward, and whether the episode is done using GetEnvironmentFeedback() from Gym environment

12)

returnnew state, reward, done

• Algorithm 5 Reward Calculation

1)

FunctionCalculateReward(state, action)

2)

Initialize reward = 0

3)

ifQoS metrics are improved then

4)

reward += k // Positive reward for improved QoS metrics

5)

else

6)

reward -= k // Negative reward for decreased QoS metrics

7)

end if

8)

returnreward

• Algorithm 6 Q-Value Update (Learning)

1)

FunctionUpdateQNetwork()

2)

Sample a random minibatch of transitions (state, action, reward, next state) from Replay Memory D

3)

foreach transition in the minibatch do

4)

target = reward

5)

ifnot done then

6)

target += γ × max(target Q-network.predict(next state))

7)

end if

8)

Compute loss as Mean Squared Error (MSE) between target and Q- network.predict(state, action)

9)

Perform gradient descent step to minimize loss

10)

end for

11)

Periodically synchronize target Q-network with Q-network weights

• Algorithm 7 Training Loop

1)

forepisode in range(total_episodes) do

2)

state = ObserveState()

3)

done = False

4)

while not done do

5)

action = SelectAction(state, ϵ)

6)

new_state, reward, done = PerformAction(action)

7)

Store transition (state, action, reward, new_state, done) in Replay Memory D

8)

iflen(Replay Memory) > batch size then

9)

UpdateQNetwork()

10)

end if

11)

state = new_state

12)

end while

13)

if ϵ > ϵ_min then

14)

 ϵ *= epsilon_decay // Decay exploration rate

15)

end if

16)

ifepisode % evaluation_interval == 0 then

17)

EvaluatePolicyPerformance()

18)

end if

19)

end for

• Algorithm 8 Policy Improvement and Execution

1)

FunctionEvaluatePolicyPerformance()

2)

Initialize performance metrics

3)

fortest episode in range(test episodes) do

4)

state = ObserveState()

5)

done = False

6)

whilenot done do

7)

action = SelectAction(state, ϵ = 0) // Greedy action selection during evaluation

8)

new state, reward, done = PerformAction(action)

9)

Update performance metrics based on reward and QoS metrics

10)

state = new state

11)

end while

12)

end for

13)

Return metrics

Algorithm Complexity Analysis

Network Delay

Packet Delivery Ratio (PDR)

Packet Error Rate (PER)

Throughput