Bayesian Modelling of Travel Times on the Example of Food Delivery: Part 1 - Spatial Data Analysis and Processing

1. Introduction

The e-commerce sector is in a constant state of growth and evolution, particularly within its subdomain of online food delivery (OFD) [1,2]. Recent market forecasts indicate a steady rise in revenue for companies offering such services. With numerous players in the market, ensuring customer satisfaction is paramount for a company’s survival. Customers increasingly demand user-friendly applications that simplify the ordering process with just a few taps, while also providing features such as delivery time estimates and communication channels with couriers [3]. However, estimating delivery times accurately without real-time data presents a significant challenge. While companies can track courier positions via GPS, they lack access to real-time information on factors such as traffic, accidents, and roadworks.

Before any model, algorithm, or computational technique can be applied, the initial step involves finding and preparing data. Effective data analysis and processing play a pivotal role in determining the performance and outcomes of models, given that many techniques rely on clean and comprehensive data. Nonetheless, real-world data seldom meet these ideal conditions and are often challenging to interpret. Even though Bayesian models, which will be explored further in part 2 of this article [4], are less affected by noisy data and outliers, they still necessitate data processing and sensible assumptions.

Visualization of data is equally significant. It offers valuable insights, particularly when combined with expertise in the relevant field. For instance plotting geospatial data on maps can unveil outliers, patterns, or anomalies. Proper visualization is also essential to draw appropriate conclusion and evaluate models output [5].

This article leverages spatial data analysis and standard data preprocessing techniques to enhance the data quality utilized in Bayesian models aimed at predicting food delivery times. By examining the spatial distribution of online food deliveries, we identify and rectify faulty data points. Drawing from existing literature, we identify the most crucial features for our models. Additionally, we investigate the relationships between variables in our dataset to uncover any patterns that could impact the behavior of our models.

The contributions of this paper can be summarized as follows: (1) By leveraging the OSRM API, we have effectively transformed raw data into meaningful predictors, which are essential for the implementation of Bayesian models. (2) Standardizing the data proved to be a crucial step for numerical stability in the models without compromising their interpretability. (3) Visualization provided valuable insights into outliers, enabling us to justifiably exclude them from the training data.

The remainder of this paper is organized as follows. In the next section, the relevant studies in the extant literature are reviewed and discussed. Section 3 provide overview of the used data and describe in details data preprocessing and visualization methods. In Section 4 we perform in-depth analysis of processing results and spatial analysis of routes. Finally in Section 5 we make a conclusion and describe the possible future work.

2. Literature Review

Typically OFD platforms acquire customers’ GPS coordinates while their open the app to perform the order [6]. Then it is used for restaurant recommendations and delivery destination. Subsequently customer GPS coordinates can be used to predict travel time in origin-destination based approach. Although it allows for estimating time without knowing the exact route and distance, a significant issue is that the crucial factor is also the time of departure. Moreover most of the research using origin-destination (OD) based methods have focused on other applications such as taxi trip duration [7].

Our problem demands that we provide customers with accurate waiting times without relying on knowledge of the precise preparation time [8] or the courier’s departure. Consequently, OD-based models are unsuitable for our needs. Furthermore, Bayesian models cannot accommodate geographic coordinates as input data. Given the above limitations, transforming GPS coordinates into some form of distance is required.

Joshi et al. have articulated a compelling argument advocating for the consideration of road distance over geometric distances like the Haversine distance for accurate delivery time estimation. They contend that relying solely on geometric distances can result in oversimplified representations, potentially leading to unrealistic data inputs into predictive models. Their approach doesn’t entail route generation, but rather involves mapping GPS coordinates onto a road network map of the analyzed area [9]. Ji et al. adopt a method for estimating travel time that utilizes GPS trajectories aligned with specific road network segments. Their research, however, is tailored towards optimizing the grouping of tasks related to efficient delivery operations [10].

Alternatively, some researchers have turned to popular routing tools like Google Maps API, Baidu Maps API, and the TOM-TOM API [11,12,13,14]. Yet, the utilization of these services incurs exorbitant expenses for OFD companies, often surpassing a staggering one million dollars annually [15]. As a cost-effective solution, exploration into open-source routing services like OSRM (Open-Source Routing Machine) has been initiated [16].

The primary objective of spatial analysis in OFD is to explore the factors influencing the utilization of these services, particularly in relation to built infrastructure. Typically, densely populated areas are examined [14,17] because online food outlets tend to be more prevalent in urban regions [18].

Spatial analysis of deliveries or journeys is not as common and typically focuses on exploring the connections between distinct regions[10,19]. In the realm of region segmentation, researchers often employ two primary methods: the grid-based approach and the road network-based approach. The grid-based method is particularly useful for visualizing smaller areas as this in Wang et al. article [19]. Road network based approach is claimed to be more informative yet, in densely populated urban areas, this approach may yield excessively small regions, necessitating a merging step prior to further analysis. [10].

3. Materials and Methods

3.1. Data

In this study, we employed the Food Delivery Dataset, which is presently accessible on Kaggle [20]. Initially, the dataset was made available by HackerEarth for their machine learning competition. It encompasses more than 45,000 deliveries spanning 21 cities across India. The data spans a concise three-month period, encompassing February through April of 2023. The dataset includes location of restaurant and delivery destination and other nonspatial information such as weather, traffic conditions, time and date the order was placed. Table 1 provides a comprehensive list of all variables under consideration, along with their respective meanings.

3.2. Routes Generation

While the exact trajectory of the courier remains uncertain and unpredictable, approximating the route is essential for our purposes. This will not only ensure an appropriate level of reality reproduction, but will also allow to detect incorrect data between which it is impossible to make a delivery. To achieve this, we’ve opted to leverage the OSRM API [21], an open-source routing engine. By constructing tailored queries containing GPS coordinates for both the starting and ending points of the route, we can utilize the API to generate optimal route suggestions [22]. Furthermore, customization options such as transportation mode (e.g. car) and route type (e.g.) allow for further refinement of our queries. After receiving the routing request, the OSRM API passes it on to the OSRM routing engine, which employs the OpenStreetMap (OSM) [23] data to produce the optimal route according to the specified parameters. The routing engine considers multiple factors, including road types, speed limits, and turn restrictions, to guarantee an accurate and efficient route. The output of the routing request contains information about the route, including the route geometry, distance, and estimated travel time. If there is an issue processing the request, API will provide an error code (e.g. "InvalidUrl"), which will inform about the reason of the failure.

3.3. Preproessing

Preparing data as model needed is essential part of delivery time estimation. Figure 1 shows preprocessing steps which are described in details in following.

Data cleaning is initial part of preprocessing. It ensures that missig values will not lead to poor results and wrong conclusions. In our approach we decided to remove rows with missing data and rows where restaurant or destination GPS coordinates are outside the India bounding box.

Routes generation is done using OSRM API which was described in Section 3.2. Created OSRM client uses asynchronous HTTP to find the shortest route between restaurant and delivery destination. If response is successful distance and route geometry are saved, otherwise unsuccessful request and its reason are logged for future analysis. Distance obtained form routing API is integrated with rest of dataset.

Data transformation techniques are used to convert data into sustainable format. Date and time are especially difficult to analyze due to various factors. To overcome this meal preparation time is calculated based on Time_Order and Time_Order_picked. This will enable the model to be utilized in cases where the restaurant furnishes an estimated preparation duration for the order, or where the statistics of preparation time can be derived from historical data [8].

Feature selection is one of the ways to cope with dimensionality. The goal is to remove irrelevant and redundant features which may include accidental correlations in models and reduce their generalization abilities. Feature selection also decrease the risk of over-fitting and reduce the search space which making learning process faster and less memory consuming [24].

Feature scaling is a critical step in constructing effective models as it helps mitigate bias stemming from variations in the ranges and magnitudes of data. Among the most widely used techniques are normalization and standardization. We have chosen to implement standardization for selected features with a continuous distribution. This decision ensures that operations and results of the model will be more straightforward to interpret.

Mapping categorical data is essential to use them as input for a model. It’s crucial to note that, unlike in machine learning models, for Bayesian models created using the Stan library [25], data must be mapped starting from 1 upwards (as vectors are indexed from 1).

3.4. Visualization

As mention previously visualization is also important part of preprocessing. Plotting the data can reveal outliers or anomalies that cannot be easily identify in other ways. In this research we will perform visual analysis of the generated routes followed by analysis of distributions of input data of the models.

The main focus of the route analysis is spatial visualization. We will explore various visualization methods, including heat maps, interactive maps, and road network graph maps. The steps involved in the visualization process are illustrated in Figure. Figure 1.

For the route analysis, each of the 21 cities is treated separately, allowing for a more detailed examination. The allocation of routes to each city was accomplished through clustering. We utilized the KMeans algorithm provided by scikit-learn [26], with clustering based on the locations of restaurants and delivery destinations.

To implement interactive maps, we used Folium [27], which is a wrapper for the Leaflet.js library. Folium allows for the creation of interactive Leaflet maps and supports a wide range of overlay formats, such as images, videos, and GeoJSON, enabling the embedding of multiple layers. For our maps, we used OSM as the base layer and added routes based on the geometry returned by the OSRM API. Additionally, routes from each city were plotted on separate layers for clarity.

Heatmaps are particularly useful for analyzing large datasets or densely located points. Our goal is to depict road usage with an intensity map similar to Navarro’s approach [28]. However, the raw route geometry data are unevenly distributed, as illustrated in the Figure 2. Points are clustered near intersections and turns, whereas on straight sections, the routes are sparsely spaced. To achieve a meaningful scale, we need to interpolate points on these straight sections. We implement linear interpolation for each route segment where the distance between consecutive points exceeds a selected threshold (e.g., 5 meters).

Another method we considered was visualizing routes on road network graphs. This required generating a graph representing the road network. We utilized the OSMnx library [29], which provides tools to model, analyze, and visualize street networks and other geospatial features from OSM. The generated graph, along with the routes obtained from the OSRM API, needed to be converted into a common format and then plotted as a high-resolution image.

In order to minimize computation time and ensure the appropriate level of map detail we recommend generating a graph only for the city area where the analyzed routes occur. In our case, the maximum and minimum values of longitude and latitude that occurred on routes in a given city were selected as the limit values.

4. Results

4.1. Preprocessing Results

Preprocessing was conducted as outlined in Section 3.3. During data transformation, we discovered instances where deliveries, from the moment the courier picked up the order to its delivery, were recorded with zero or negative times. Consequently, additional data cleansing was necessary.

To estimate delivery times, companies and researchers commonly utilize a variety of features. These include spatial features (such as the location of the restaurant and destination, and the city road map), cooking time features, order features (like the number of items ordered and the date and time), and courier features (such as workload)[8,9,30,31,32]. For our models, we have selected the following variables: the distance between the restaurant and delivery destination, meal preparation time, traffic density, the number of simultaneous deliveries, and courier rating. A detailed analysis of the input data will be presented in Section 4.1.2.

4.1.1. Routes Generation

All approximate delivery route distances were successfully generated using the OSRM API, with no incorrect routes identified between the restaurant and delivery destination. The histogram of the obtained distances is shown in the accompanying Figure 3, and the statistics of the road distance projections are presented in the Table 2. The histogram and statistics reveal that the distances in our dataset are significantly larger than those in other studies. While most deliveries occur within 6 km, as restaurants within this range are shown to customers [14], our data indicates that the 95th percentile for distance is approximately 28.5 km.

The number of deliveries significantly drops when the distance exceeds 25 km. However, there are routes with distances that are considerably longer. The histogram indicates two distinct groups: one around 65 km and another around 120 km. Both of these groups fall above the 99th percentile, classifying them as outliers. To understand the reasons behind these outliers, we conducted a thorough investigation and compared their routes with other routing engines, particularly Google Maps, which employs different mapping techniques. This comparison will help us identify the causes of these anomalies and develop solutions to prevent such unlikely values in the future.

All routes with road distances around 65 km are concentrated in the Dehradun area. Dehradun is situated in a valley at the foot of the Himalayas, resulting in a limited number of roads leading to the surrounding areas. A comparison of the selected route determined by the OSRM API and Google Maps is shown in the Figure 4. Both engines visually identified the same route; however, Google Maps indicates that the route is approximately 2 km longer. This discrepancy may stem from differences in point positioning on various maps and the distinct distance calculation algorithms used. While the exact route lengths also vary among engines utilizing only OSM data, these differences are negligible over shorter distances.

The second problematic group of routes is found in the city of Agra. Analyzing this case is particularly challenging because online routing engines display varying results depending on the day of access. Additionally, Google Maps often selects different destinations for the same location; sometimes directing to an expressway, and other times to a parallel street. The OSM data used by OSRM indicates that the given location is situated in one of the lanes on the nearby Yamuna Expressway. This suggests that the routes allowing direct U-turns on this type of road are incorrectly designated. A comparison of different routes for the same locations is illustrated in the Figure 5.

To ensure our models are trained on meaningful data, we decided to filter out the outliers. We set a 30 km upper limit for deliveries, which encompasses over 96% of the data available after preprocessing. Standardization was performed after applying this distance filter to prevent excessive data spread caused by biases in the mean and standard deviation.

4.1.2. Input Data Analysis

Following the described preprocessing and outlier filtering, we ended up with nearly 35,000 data samples. The data prepared for Bayesian models is presented in the Figure 6.

The basic distance statistics remained relatively stable after filtration. The average distance decreased to 13.33 km, while the standard deviation reduced to 7.17 km.This outcome was expected, given that the filtered values constituted only about 3% of the dataset eligible for model utilization.

The distribution of order preparation time presents an intriguing puzzle. Initially perceived as a continuous variable, the values are distributed almost evenly across 5, 10, and 15-minute intervals. This phenomenon could stem from the provision of approximate order and pick-up times, rather than precise values.

The prevalence of high ratings among couriers has led to an inflated average of 4.6 in the standardization process. Furthermore, the remarkably small standard deviation of 0.32 translates to substantially reduced standardized values for couriers with lower ratings. Notably, no courier has a rating below 2.5.

The categorical variables "traffic density" and "number of deliveries" appear to align with expectations. Couriers typically handle no more than two deliveries concurrently. Interestingly, deliveries occur with equal frequency during rush hours and periods of very low car traffic. However, deliveries in moderate traffic conditions are relatively less common.

4.2. Spatial Analysis

The aim of the spatial analysis was to analyze the frequency of use of road segments. Orders are not evenly distributed among all 21 cities. The analysis included all those located in India and a route for them could be determined (including data considered as outliers). Jaipur is the clear leader in terms of the number of orders, with over 3,400 deliveries located there. eight cities have very similar values at the level of 3,150 deliveries. These include the largest cities in India such as Bangalore and Mumbai. Cities in which the previously calculated outliers occurred have a significantly lower number of deliveries in their area. A chart showing the number of orders for each city is shown in the Figure 7.

Interactive maps created using Folium present completed routes on a road map with OSM. The intensity of a given road fragment depends on how often it occurs on routes. Additionally, for readability and easier analysis, the routes from each city have been added as a separate layer that can be activated. Sample visualization results are shown in the Figure 8.

It can be noticed that the roads in city centers are definitely the most intensive. The size of the marker is not related to the results because it is automatically adjusted to the scale. The closer the area, the thinner the drawn routes. Routes that have been traveled once/twice are barely visible, for example this is at the left edge of the Japiur image.

The visualization result using a city graph for similar cities is shown in the Figure 9. Similarly to the previous maps, the intensity of a road fragment depends on its frequency in the routes. In general in large-scale images generated using OSMnx, differences in saturation levels are not so easily noticeable. Due to the dense network of streets in the centers where deliveries are concentrated, all routes appear to be little traveled. However, after a small zoom they are visible much better than on maps generated using Folium.

The very visible difference in the intensity of bridges on the Mumabiu map made using different methods was interesting, so we decided to analyze it more closely. As it turned out, there was another road running in the immediate vicinity of the bridge, which also had a route which, at a sufficiently large distance, was displayed directly on top of itself, disturbing the results. Moreover, we were dealing with a route that made no sense because it ended in the middle of a bridge (Google maps even projected the point onto the waters of the bay) and it was filtered as outlier (it was about 40 km long)

Despite interpolation to increase the number of points in the routes, heatmap visualization did not bring the expected results. It was not possible to choose such an image scale that, on the one hand, the routes were visible on it, and on the other hand, it was not too large and had some significance. Taking into account that our routes are not concentrated in one area of the city, and there are even two separate clusters of routes (as in the case of Jaipur), our data is not suitable for analysis using a heatmap.

5. Conclusions

This research underscores the importance of data preprocessing and spatial analysis in the context of online food delivery services. By integrating routing information from the OSRM API, we have demonstrated a method for identifying and eliminating outliers in delivery data, thus enhancing the accuracy of subsequent predictive models. Our spatial analyzes indicates that outliers are predominantly deliveries to more distant locations, whereas deliveries within city limits exhibit fewer outliers. This is likely because outliers often stem from incorrect geographic coordinates, which are more apparent in longer-distance deliveries. Therefore, a comprehensive analysis of potential errors in input coordinates is required. It additionally confirms that the OFD market is concentrated in more urbanised areas.

It is important to recognize that this study’s findings may be biased due to the limited amount of data available for each city, which does not accurately reflect the actual workload of OFD companies. Future studies would benefit from obtaining a dataset from one of the leading OFD providers to ensure more comprehensive and representative results.

In the second part of this article, we will delve into the application of Bayesian models to the preprocessed dataset, examining their efficacy in predicting delivery times and exploring potential improvements to the modeling approach. Additionally, future studies could consider refinement of the preprocessing steps, such as advanced handling of missing values and more sophisticated outlier detection methods.