Machine Learning-Enhanced Cloud-Assisted Last-Mile Delivery Optimization with Priority Management Based on Queueing Theory

1. Introduction

When same-day delivery becomes the new normal, how do logistics companies handle sudden order surges and traffic congestion? The digital transformation has made last-mile delivery both the most critical and challenging component of modern supply chains. Recent industry events, such as China’s Double 11 shopping festival delivery delays, highlight the inadequacy of traditional methods in handling dynamic demand patterns [1].

Traditional approaches to delivery optimization struggle with three fundamental challenges that limit their effectiveness in modern urban environments. Centralized routing systems create computational bottlenecks that become prohibitive for large-scale urban networks while establishing single points of failure that can compromise entire delivery operations. Distributed approaches, while addressing computational scalability, lack comprehensive coordination mechanisms, leading to suboptimal resource allocation and conflicts among competing delivery requests. Most critically, existing machine learning applications treat artificial intelligence as an auxiliary optimization layer rather than integrating it with fundamental queueing theory principles that govern service operations under uncertainty.

While recent research has made substantial progress in applying machine learning to vehicle routing [2,3], these approaches typically ignore priority handling mechanisms essential for real-world delivery operations. For instance, Konovalenko and Hvattum [4] demonstrated the effectiveness of deep reinforcement learning for dynamic vehicle routing but failed to address priority orders that can preempt ongoing deliveries, a critical requirement in commercial logistics. Similarly, existing queueing theory applications in logistics assume stationary arrival and service processes, limiting their applicability to dynamic delivery environments where demand patterns and service capabilities vary significantly throughout operational periods.

This research addresses these gaps by developing the Machine Learning-Enhanced Cloud-Assisted Last-Mile Optimization (ML-CALMO) framework, which provides a novel integrated approach combining machine learning techniques with established queueing-theoretic principles. Our contributions include the development of a comprehensive system architecture with clear component interactions, rigorous mathematical formulation ensuring theoretical soundness, and extensive experimental validation using standard benchmark instances against state-of-the-art algorithms.

2. Literature Review and Research Gap Analysis

The academic literature surrounding last-mile delivery optimization has evolved considerably over the past decade, with recent comprehensive reviews emphasizing technological integration in addressing modern delivery challenges [1]. Classical approaches find their theoretical foundations in the Vehicle Routing Problem (VRP) and its numerous variants, including the Capacitated VRP and VRP with Time Windows. These deterministic optimization models assume known, fixed parameters for travel times and customer demands, assumptions that prove increasingly problematic in dynamic urban environments characterized by unpredictable traffic patterns, variable customer availability, and fluctuating demand intensity.

Deep reinforcement learning has emerged as a particularly promising approach for addressing dynamic vehicle routing challenges. Nazari et al. [2] demonstrated reinforcement learning’s potential for VRP solutions by training neural network models to find near-optimal solutions using only reward signals, eliminating the need for explicit optimization algorithms. Building upon this foundation, Kool et al. [3] introduced attention mechanisms that significantly improved both scalability and solution quality, establishing attention-based architectures as fundamental components of modern neural approaches to combinatorial optimization. Recent advances by Li et al. [5] and Pan and Liu [6] have further expanded deep reinforcement learning applications to encompass stochastic and uncertain vehicle routing scenarios, addressing real-world complexities previously ignored by deterministic methods.

However, these machine learning approaches exhibit critical limitations that restrict their practical applicability. Konovalenko and Hvattum [4] demonstrated that careful state-space design significantly impacts algorithm performance in dynamic environments, but their formulation lacks sophisticated priority handling mechanisms necessary for commercial delivery operations. Most existing machine learning methods treat routing as a pure combinatorial optimization problem, ignoring the underlying stochastic service dynamics that fundamentally characterize real delivery operations where service times, travel conditions, and customer availability vary unpredictably.

Queueing theory applications in delivery systems have traditionally modeled vehicles as servers processing customer orders according to various service disciplines, providing theoretical frameworks for understanding system performance under uncertainty. Jackson’s foundational work [7,8] established the theoretical framework for analyzing service station interactions in complex networked systems, enabling analytical performance evaluation and stability analysis. Modern applications build upon classical queueing network theory [9,10] and stochastic modeling principles [11] to provide rigorous mathematical foundations for service system analysis. Vilaplana et al. [12] successfully demonstrated queueing theory’s applicability to distributed computing systems, showing how open Jackson networks can model cloud platforms while providing Quality of Service guarantees. Advanced queueing network analysis [13] and factory physics principles [14] provide additional theoretical foundations for understanding complex service systems operating under stochastic conditions.

Nevertheless, classical queueing analysis assumes stationary arrival and service processes, limiting direct applicability to delivery systems where demand patterns vary significantly over daily, weekly, and seasonal cycles. Current literature exhibits three major research gaps that prevent effective integration of advanced computational methods with established theoretical principles. Machine learning and queueing theory are typically treated as separate analytical domains, preventing synergistic integration that could leverage predictive capabilities while maintaining theoretical rigor. Existing approaches inadequately address priority order management that can disrupt carefully optimized routes, despite advances in multi-objective optimization [15] and ensemble approaches for heterogeneous delivery problems [16]. Classical queueing models cannot effectively incorporate machine learning predictions for time-varying system parameters, even as metaheuristic approaches [17] and statistical learning theory [18,19] provide increasingly powerful computational tools for handling complex optimization scenarios.

ML-CALMO addresses these fundamental gaps by providing seamless integration of machine learning capabilities with established queueing theory principles, enabling both superior operational performance and rigorous theoretical grounding necessary for practical deployment in commercial logistics operations.

3. Materials and Methods

3.1. Integrated System Architecture

The ML-CALMO framework operates through sophisticated integration of three core machine learning components coordinated by queueing theory principles to ensure both performance optimization and system stability. Figure 1 illustrates the comprehensive system architecture, demonstrating how LSTM-based demand forecasting, CNN-based traffic analysis, and DQN-based route optimization interact dynamically through a central queueing theory controller.

The LSTM Demand Predictor serves as the primary forecasting engine, processing comprehensive historical order data to generate accurate predictions of future arrival rates ${\widehat{\lambda }}_{pi}\left(t\right)$ and ${\widehat{\lambda }}_{ci}\left(t\right)$ for both priority and regular orders, respectively. This component analyzes complex temporal patterns including time-of-day variations, day-of-week cycles, seasonal fluctuations, and external factors such as weather conditions, special events, and economic indicators that influence delivery demand. The LSTM architecture captures long-term dependencies in demand patterns while maintaining sensitivity to recent trends, enabling robust prediction even during periods of high variability.

The CNN Traffic Analyzer processes spatiotemporal traffic information to provide accurate travel time predictions ${\widehat{T}}_{ij}^{\mathrm{travel}}\left(t\right)$ essential for route optimization and service rate calculations. This component analyzes multi-dimensional traffic grid data incorporating real-time congestion information, road conditions, construction zones, and historical traffic patterns to generate precise travel time estimates between all relevant locations in the delivery network. The convolutional architecture effectively captures spatial correlations in traffic patterns while temporal layers handle time-varying congestion dynamics.

The DQN Route Optimizer makes sophisticated routing decisions based on the dynamic vehicle-customer preference matrix $H_j\left(t\right)$ while respecting system constraints derived from queueing theory analysis. This component learns complex strategies that balance immediate operational rewards against long-term system performance, incorporating considerations such as vehicle capacity utilization, customer service requirements, priority order handling, and overall system efficiency. The deep Q-learning approach enables the system to discover optimal policies that would be difficult to specify through traditional optimization methods.

The Queueing Theory Controller provides the theoretical foundation ensuring system stability and service quality guarantees. This component continuously monitors system utilization factors ${\widehat{\rho }}_i\left(t\right)$, validates machine learning predictions against analytical bounds, and triggers corrective actions when stability thresholds approach critical levels. The controller maintains mathematical rigor while enabling adaptive response to changing operational conditions, ensuring that performance optimizations do not compromise system reliability.

3.2. Mathematical Formulation and Variable Notation

To ensure clarity and consistency throughout this paper, we establish comprehensive variable notation that distinguishes between actual values, predicted values, and derived metrics. Table 1 presents the complete variable notation system used throughout our mathematical formulation.

Consider an urban delivery network comprising N heterogeneous vehicles serving customers distributed across a metropolitan region, where time evolves continuously as $t\in [0,\infty )$ to reflect real-time operational requirements. The system processes two fundamentally different order types: regular orders ${\mathcal{O}}_r$ representing standard delivery requests with flexible scheduling windows, and priority orders ${\mathcal{O}}_p$ demanding immediate attention with preemption capabilities.

Service rate calculations incorporate all relevant time components affecting delivery operations. The effective service rate for vehicle i serving order j is defined as:

$${\mu }_{ij}\left(t\right)={\displaystyle \frac{1}{T_{ij}^{\mathrm{travel}}\left(t\right)+T_j^{\mathrm{service}}+T_j^{\mathrm{loading}}}}$$

(1)

For optimization purposes, we use predicted service rates based on CNN travel time estimates:

$${\widehat{\mu }}_{ij}\left(t\right)={\displaystyle \frac{1}{{\widehat{T}}_{ij}^{\mathrm{travel}}\left(t\right)+T_j^{\mathrm{service}}+T_j^{\mathrm{loading}}}}$$

(2)

System-wide arrival rate consistency requires proper aggregation using both actual and predicted values:

$$\begin{array}{cc}\hfill \sum _{i=1}^N{\lambda }_{pi}\left(t\right)& ={\lambda }_p\left(t\right)(\mathrm{actual}\mathrm{arrival}\mathrm{rates})\hfill \end{array}$$

(3)

$$\begin{array}{cc}\hfill \sum _{i=1}^N{\lambda }_{ci}\left(t\right)& ={\lambda }_c\left(t\right)(\mathrm{actual}\mathrm{arrival}\mathrm{rates})\hfill \end{array}$$

(4)

$$\begin{array}{cc}\hfill \sum _{i=1}^N{\widehat{\lambda }}_{pi}\left(t\right)& ={\widehat{\lambda }}_p\left(t\right)(\mathrm{predicted}\mathrm{arrival}\mathrm{rates})\hfill \end{array}$$

(5)

$$\begin{array}{cc}\hfill \sum _{i=1}^N{\widehat{\lambda }}_{ci}\left(t\right)& ={\widehat{\lambda }}_c\left(t\right)(\mathrm{predicted}\mathrm{arrival}\mathrm{rates})\hfill \end{array}$$

(6)

The optimization objective minimizes total system costs using predicted travel times for decision making:

$$\underset{\mathbf{x}}{min}\mathbb{E}\left[\sum _{i=1}^N\sum _{j=1}^M{x}_{ij}\left(t\right)·{\widehat{C}}_{ij}^{\mathrm{total}}\left(t\right)+\alpha \sum _{j\in {\mathcal{O}}_p}{P}_j^{\mathrm{penalty}}\left(t\right)\right]$$

(7)

where predicted total costs incorporate CNN travel time estimates:

$$\begin{array}{cc}\hfill {\widehat{C}}_{ij}^{\mathrm{total}}\left(t\right)& ={\widehat{T}}_{ij}^{\mathrm{travel}}\left(t\right)+T_j^{\mathrm{service}}+\beta ·{\widehat{T}}_j^{\mathrm{waiting}}\left(t\right)\hfill \end{array}$$

(8)

$$\begin{array}{cc}\hfill P_j^{\mathrm{penalty}}\left(t\right)& =max(0,t-t_j^{\mathrm{deadline}})·w_j^{\mathrm{priority}}\hfill \end{array}$$

(9)

subject to:

$$\begin{array}{cc}\hfill \sum _{i=1}^N{x}_{ij}\left(t\right)& =1,\forall j\in \mathcal{O}(\mathrm{assignment}\mathrm{constraint})\hfill \end{array}$$

(10)

$$\begin{array}{cc}\hfill \sum _{j=1}^M{x}_{ij}\left(t\right)·D_j^{\mathrm{demand}}& \le C_i^{\mathrm{available}}\left(t\right),\forall i(\mathrm{capacity}\mathrm{constraint})\hfill \end{array}$$

(11)

$$\begin{array}{cc}\hfill {\widehat{\rho }}_i\left(t\right)& <1,\forall i,t(\mathrm{stability}\mathrm{constraint})\hfill \end{array}$$

(12)

3.3. Queueing Theory Integration

The theoretical foundation of ML-CALMO rests upon continuous-time Markov chain modeling that captures essential stochastic dynamics while incorporating machine learning predictions. Following Jackson’s seminal work [7,8], we model the delivery system as an interconnected network of service stations with time-varying parameters predicted by machine learning algorithms.

The system state space encompasses all possible configurations:

$$\mathcal{S}=\{(n_{p1},n_{c1},\dots ,n_{pN},n_{cN},k):n_{pi},n_{ci}\ge 0,k\in \{0,1\}\}$$

(13)

where $n_{pi}$ and $n_{ci}$ represent priority and regular orders assigned to vehicle i, and k captures preemption events.

State transition rates are governed by both arrival processes and service completions:

$$q(s\to s^{\prime })=\left\{\begin{array}{cc}{\lambda }_{pi}\left(t\right)\hfill & \mathrm{if}{s}^{\prime }=s+e_i^p(\mathrm{priority}\mathrm{arrival})\hfill \\ {\lambda }_{ci}\left(t\right)\hfill & \mathrm{if}{s}^{\prime }=s+e_i^c(\mathrm{regular}\mathrm{arrival})\hfill \\ n_{pi}{\mu }_{pi}\hfill & \mathrm{if}{s}^{\prime }=s-e_i^p(\mathrm{priority}\mathrm{completion})\hfill \\ n_{ci}{\mu }_{ci}\hfill & \mathrm{if}{s}^{\prime }=s-e_i^c(\mathrm{regular}\mathrm{completion})\hfill \\ 0\hfill & \mathrm{otherwise}\hfill \end{array}\right.$$

(14)

Steady-state analysis proceeds through global balance equations ensuring probability conservation across all system states:

$$\pi \left(s\right)\sum _{s^{\prime }\ne s}q(s\to s^{\prime })=\sum _{s^{\prime }\ne s}\pi \left(s^{\prime }\right)q(s^{\prime }\to s)$$

(15)

System utilization is calculated as:

$${\rho }_{\mathrm{system}}=\sum _{s\in \mathcal{S}}\pi \left(s\right){\displaystyle \frac{{\sum }_{i=1}^N(n_{pi}+n_{ci})}{N}}$$

(16)

3.4. Integrated Algorithm Implementation

The ML-CALMO algorithm integrates all system components through a sophisticated control loop coordinating machine learning predictions with queueing theory constraints. Algorithm 1 presents the complete integration process.

Algorithm 1 ML-CALMO Integrated Optimization Algorithm

1: Input: Current system state $S\left(t\right)$, historical data $H_{t-\tau :t-1}$, traffic data $T_{\mathrm{grid}}\left(t\right)$   2: Initialize: LSTM model ${\mathsf{\Theta }}_{\mathrm{LSTM}}$, CNN model ${\mathsf{\Theta }}_{\mathrm{CNN}}$, DQN model ${\mathsf{\Theta }}_{\mathrm{DQN}}$   3: while $t<T_{\max }$ do   4:    Generate demand predictions ${\widehat{\lambda }}_{pi}(t+1),{\widehat{\lambda }}_{ci}(t+1)$ using LSTM analysis of historical patterns $H_{t-\tau :t-1}$   5:    Compute travel time predictions ${\widehat{T}}_{ij}^{\mathrm{travel}}\left(t\right)$ through CNN processing of current traffic conditions $T_{\mathrm{grid}}\left(t\right)$   6:    Calculate predicted utilization factors ${\widehat{\rho }}_i(t+1)={\displaystyle \frac{{\widehat{\lambda }}_{pi}(t+1)}{{\mu }_{pi}}}+{\displaystyle \frac{{\widehat{\lambda }}_{ci}(t+1)}{{\mu }_{ci}}}$ for stability analysis   7:    if any vehicle approaches critical utilization ${\widehat{\rho }}_i(t+1)\ge 0.95$ then   8:      Implement load balancing by redistributing predicted arrival rates ${\widehat{\lambda }}_{pi}(t+1),{\widehat{\lambda }}_{ci}(t+1)$ across available vehicles   9:      Update arrival rate predictions to ensure ${\widehat{\rho }}_i(t+1)<0.95$ for all vehicles 10:    end if 11:    Construct dynamic preference matrix $H_j\left(t\right)$ incorporating success probabilities $P_{ij}^{\mathrm{success}}\left(t\right)$, predicted travel times ${\widehat{T}}_{ij}^{\mathrm{travel}}\left(t\right)$, available capacities $C_i^{\mathrm{available}}\left(t\right)$, and priority scores $R_j^{\mathrm{priority}}$ 12:    Execute DQN decision making to select optimal assignment actions $a^{*}=arg{max}_a{Q}_{{\mathsf{\Theta }}_{\mathrm{DQN}}}(S\left(t\right),a;H_j\left(t\right))$ 13:    Update system state $S(t+1)$ based on executed assignments, vehicle movements, order completions using actual travel times $T_{ij}^{\mathrm{travel}}\left(t\right)$, and new arrivals with actual rates ${\lambda }_{pi}\left(t\right),{\lambda }_{ci}\left(t\right)$ 14:    Advance time $t\leftarrow t+\Delta t$ 15: end while 16: Output: Optimized delivery routes and schedules with performance guarantees

4. Results

Our comprehensive experimental investigation employs both synthetic datasets and standard benchmark instances to ensure rigorous validation of the ML-CALMO framework against established optimization methods. The experimental design incorporates realistic operational scenarios based on Solomon benchmark instances [20] and Li & Lim datasets [21], adapted to include dynamic arrival processes and priority order management.

4.1. Experimental Setup

The simulation environment represents diverse metropolitan delivery districts through realistic urban networks incorporating varying customer density, geographic constraints, traffic patterns, and operational requirements. Fleet configurations range from 10 to 25 heterogeneous vehicles with capacities between 80 and 150 units, representing the diversity of commercial delivery operations.

Baseline algorithm selection focuses on state-of-the-art methods: OR-Tools VRP Solver, Attention-VRP [3], RL-VRP [2], Dynamic-VRP [22], and Queueing-VRP implementing pure queueing theory approaches without machine learning enhancement.

Performance evaluation employs comprehensive metrics distinguishing operational efficiency from service quality. Service efficiency measures the proportion of time vehicles spend on productive delivery activities, while service success rate quantifies the proportion of orders delivered within specified time windows.

4.2. Performance Comparison Results

Machine learning component evaluation uses Mean Absolute Percentage Error (MAPE) for LSTM demand predictions and CNN traffic analysis:

$$\begin{array}{cc}\hfill {\mathrm{MAPE}}_{\mathrm{LSTM}}& ={\displaystyle \frac{1}{n}}\sum _{k=1}^n{\displaystyle \frac{|{\widehat{\lambda }}_k\left(t\right)-{\lambda }_k\left(t\right)|}{|{\lambda }_k\left(t\right)|}}\hfill \end{array}$$

(17)

$$\begin{array}{cc}\hfill {\mathrm{MAPE}}_{\mathrm{CNN}}& ={\displaystyle \frac{1}{n}}\sum _{k=1}^n{\displaystyle \frac{|{\widehat{T}}_k^{\mathrm{travel}}\left(t\right)-T_k^{\mathrm{travel}}\left(t\right)|}{|T_k^{\mathrm{travel}}\left(t\right)|}}\hfill \end{array}$$

(18)

Table 2 presents comprehensive performance comparisons.

Table 3 demonstrates the effectiveness of each algorithmic element.

Table 4 provides theoretical model validation results.

5. Discussion

The experimental results provide compelling evidence for the effectiveness of integrating machine learning techniques with queueing theory principles in last-mile delivery optimization. The 28.3% delivery time improvement and 16.2% service efficiency gain demonstrate substantial operational benefits that translate directly to cost reduction, customer satisfaction enhancement, and competitive advantage.

Several key insights emerge from our comprehensive analysis. Machine learning-enhanced demand prediction reduces system uncertainty by approximately 34.7%, enabling proactive resource allocation and route planning. The integration of queueing theory constraints prevents machine learning overfitting while ensuring system stability, addressing a critical limitation of pure learning-based approaches. Most significantly, the integrated approach consistently outperforms individual components by 15-25% across all performance metrics, confirming that sophisticated system integration provides substantial benefits beyond individual algorithmic contributions.

However, several limitations constrain immediate applicability. Computational complexity increases quadratically with fleet size, following $O\left(N^{2}M\right)$ scaling for assignment decisions that becomes prohibitive for very large delivery networks exceeding 50-100 vehicles. Data requirements and processing demands may limit deployment in resource-constrained environments.

6. Conclusions

This research presents ML-CALMO, a comprehensive framework that successfully integrates machine learning techniques with established queueing theory principles to address fundamental challenges in last-mile delivery optimization. The framework achieves a 28.3% reduction in delivery time, 89.7% service success rate, and 20.9% total cost reduction compared to baseline methods, validated through rigorous experimentation and theoretical analysis.