A Multi-Objective Recommender System for Enhanced Consumer Behavior Prediction in E-Commerce

1. Introduction

The advent of e-commerce has revolutionized the retail industry, providing consumers with unprecedented access to a vast array of products from the convenience of their homes. However, this convenience comes with its own set of challenges. The sheer volume of available products can overwhelm consumers, leading to decision fatigue and a phenomenon known as "choice overload," where the abundance of options paradoxically results in decreased satisfaction and increased likelihood of abandoning shopping carts. This scenario not only frustrates consumers but also represents a significant loss in potential revenue for retailers.

This paper presents a novel multi-objective recommender system designed to enhance the shopping experience by simultaneously optimizing for multiple consumer behaviors, including clicks, cart additions, and completed orders. We leverage the OTTO Recommender Systems Dataset, a comprehensive dataset containing 12 million anonymized user sessions and over 220 million events, to develop and evaluate our system. The dataset’s richness in real-world e-commerce interactions makes it an ideal foundation for building a robust and effective recommender system.

Our approach involves a two-stage candidate reranker model. In the first stage, we generate a pool of candidate items using a variety of methods, such as co-visitation, click-to-click patterns, and Word2Vec embeddings. In the second stage, we rank these candidate items based on their predicted relevance to the user, employing sophisticated feature engineering and negative sampling techniques to enhance model performance. We explore the use of ensemble learning, combining LightGBM, XGBoost, and CatBoost Rankers, to achieve superior performance across multiple objectives. Ensemble models are particularly effective in capturing the diverse patterns in user behavior by leveraging the strengths of each individual model.

2. Related Work

The field of recommender systems has seen significant advancements over the past two decades, with a variety of models and approaches developed to predict and enhance consumer behavior. Traditional methods have primarily focused on single-objective optimization, which has proven effective but limited in the context of multi-faceted consumer interactions in e-commerce. This section provides an overview of relevant literature, highlighting the evolution of recommender systems and recent innovations aimed at multi-objective optimization.

Resnick et al. [1] introduced GroupLens, an early collaborative filtering system that used user ratings to recommend articles. This method laid the groundwork for many subsequent recommendation algorithms.Pazzani and Billsus [2] discussed content-based methods that recommend items similar to those the user has liked in the past, using features derived from the items themselves. A Javanmardi et al. [3] Balancing proactive and reactive discussions in construction planning improves effectiveness and reduces risk.Burke [4] reviewed hybrid recommender systems, which combine collaborative and content-based filtering techniques to improve recommendation accuracy.G Li et al. [5] introduces IACO-IABC, combining ant colony optimization and artificial bee colony algorithms for improved mobile robot path planning.

Koren et al. [6] introduced matrix factorization techniques in recommender systems, significantly enhancing the scalability and accuracy of recommendations.Hu et al. [7] addressed the challenge of using implicit feedback (e.g., clicks, views) in collaborative filtering, which often contains less explicit user preferences.Zhang et al. [8] provided a comprehensive survey of deep learning techniques applied to recommender systems, highlighting their ability to capture complex user-item interactions.C He et al. [9] uses a graph neural network and ontology to improve bridge preservation planning and prediction accuracy.

Lacerda et al. [10] explored multi-objective optimization in recommender systems, emphasizing the need to balance multiple goals such as accuracy, diversity, and novelty.Z Lin et al. [11] develops an AI-based model for efficient and smooth robot path planning and navigation.C He et al. [12] identifies key collaborative scheduling practices that enhance construction project performance.B Zhang et al. [13] summarizes recent advancements in text sentiment analysis for improving enterprise decision-making and public opinion monitoring

Li et al. [14] proposed a contextual bandit algorithm for personalized news article recommendations, balancing exploration and exploitation to optimize recommendations.Ying et al. [15] developed Graph Convolutional Networks (GCNs) for recommender systems, demonstrating their ability to effectively capture user-item relationships.A Javanmardi et al. [16] shows that the Observe–Plan–Do–Check–React cycle enhances construction workflow reliability. Notably, the methodology for integrating generative AI, as discussed by [17] , provided critical insights into optimizing our ensemble model, particularly in the fine-tuning stages for improved model generalization and performance across multiple objectives.

Pan et al. [18] explored transfer learning techniques for cross-domain recommendation, addressing the challenge of sparsity in new domains by leveraging knowledge from related domains.Rokach [19] reviewed ensemble methods in machine learning, highlighting their potential to improve the robustness and accuracy of recommender systems by combining multiple models.S Liu et al. [20] introduces the Intelligent Bus Travel Service Model (IBTSM) to optimize bus service quality.Zhang and Chen [21] discussed the growing need for explainability in recommender systems, proposing methods to make recommendations more transparent and understandable to users.

V Nikolaenko et al. [22] introduced methods to ensure user privacy in recommender systems, using techniques like differential privacy to protect user data.H Yan et al. [23] explores how natural language processing enhances data mining and information retrieval in the big data era. Burke [24] discussed the need for fairness in recommender systems, proposing algorithms that ensure equitable treatment of all users and items.A multi-task learning approach for efficient modeling and monitoring of tool surface progression in ultrasonic metal welding [25]. Kunaver and Požrl [26] reviewed methods to enhance diversity in recommender systems, balancing relevance and variety to improve user satisfaction.

In summary, the literature on recommender systems has evolved significantly, with various methodologies being proposed to address different aspects of recommendation. While traditional systems have focused on single-objective optimization, there is a growing recognition of the need for multi-objective approaches to capture the complex nature of user behavior. This study contributes to this emerging field by proposing a multi-objective recommender system that optimizes for multiple consumer actions, demonstrating superior performance through ensemble learning techniques.

3. Dataset and Preprocessing

3.1. Dataset Description

The dataset utilized in this study is the OTTO Recommender Systems Dataset , which was prominently featured in the Kaggle competition ’OTTO-Multi-Objective Recommender System’. This comprehensive dataset includes 12 million anonymized user sessions, capturing over 220 million events that encompass clicks, cart additions, and orders. Furthermore, it contains a catalog of 1.8 million unique articles, making it an extensive resource for developing and testing recommender systems. The dataset is structured into sessions, where each session is uniquely identified by an ID, and events, which are time-ordered sequences associated with specific articles identified by their IDs (aids). Each event includes a Unix timestamp (ts) and is categorized by its type.

3.2. Basic Statistics

The dataset is divided into training and test sets, with the training set comprising 12,899,779 sessions, 1,855,603 items, and 216,716,096 events. The test set includes 1,671,803 sessions, 1,019,357 items, and 13,851,293 events. The density of the dataset is notably low, at 0.0005% for both the training and test sets, indicating a sparse dataset. Detailed statistics on the number of events per session and per item are provided in Table 1 and Table 2 , highlighting the distribution and variability within the dataset.

Figure 1 and Figure 2 provide visual representations of the distribution of events per session and events per item for the training and test datasets, respectively. These histograms illustrate the concentration of user interactions within specific ranges, with a notable peak in lower event counts.

3.3. Data Processing

3.3.1. Handling Missing Values and Anomalies

Ensuring data integrity is crucial for accurate model training and evaluation. The initial assessment confirmed that there were no missing values or anomalies within the dataset. This allowed for a straightforward preprocessing phase without the need for data imputation or anomaly detection techniques.

3.3.2. Data Reformatting and Compression

The original format of the dataset, although comprehensive, presented challenges due to its multi-level structure and the memory-intensive nature of Unix timestamps. To address these issues, the timestamps were standardized to fall within a three-week period by subtracting a fixed number and converting them into numpy int32 data types. This reformatting reduced the dataset size significantly, from 11.3GB to 1.68GB, optimizing it for model training and analysis.

The reformatted dataset structure includes unique session IDs and time-ordered sequences of events within each session. Each event is associated with an article ID (aid) and includes a standardized timestamp (time) and event type (type).

3.3.3. Data Analysis

To gain insights into user behavior and item interactions, item matrix factorization was employed, transforming the anonymized item IDs into meaningful embeddings. These embeddings were visualized using t-SNE, a technique that projects high-dimensional data into a two-dimensional space, revealing clusters of related products. This analysis highlighted the inherent structure within the dataset, allowing for the identification of patterns in user interactions and preferences.

The visualizations in Figure 3 illustrates user interactions and item categories, respectively. It shows the clustering of items based on user interactions, where each dot represents an item, and proximity indicates similarity. It also depicts user activity over time, highlighting patterns in clicks, cart additions, and orders.

Through detailed observation, several patterns in user behavior emerged:

• Users tend to interact with items within specific clusters, often browsing similar types of products.
• The sequence of actions, including clicks, adds to cart, and orders, follows identifiable patterns.
• Temporal factors, such as weekdays, weekends, and seasonal events, significantly influence user activity.

These insights were instrumental in developing the two-stage candidate reranker model, which aims to optimize recommendations across multiple user engagement metrics.

In summary, the extensive preprocessing and detailed analysis of the OTTO Recommender Systems Dataset provided a robust foundation for developing a sophisticated multi-objective recommender system. The insights gained from data analysis guided the model development process, ensuring that the system is tailored to the complex nature of e-commerce user interactions. The next section will delve into the methodology employed, detailing the candidate generation and ranking processes of our multi-objective recommender system.

4. Methodology

The methodology for developing our multi-objective recommender system involves comprehensive data processing, sophisticated feature engineering, and the implementation of a two-stage candidate reranker model (Figure 4). The system is designed to optimize multiple consumer behaviors, including clicks, cart additions, and completed orders. The most effective model combines LightGBM, XGBoost, and CatBoost rankers in an ensemble, achieving superior performance across these objectives.

4.1. Data Processing

The OTTO Recommender Systems Dataset comprises 12 million anonymized user sessions and over 220 million events, structured as follows:

• Sessions : Each session is uniquely identified by an ID.
• Events : Time-ordered sequences within each session.
• aid : Article ID associated with each event.
• ts : Unix timestamp of each event.
• type : Type of event (click, cart, order).

To optimize memory usage, we standardized timestamps by converting them to numpy int32 data types, reducing the training data size from 11.3GB to 1.68GB. Item embeddings were generated using the Word2Vec model, capturing semantic similarities between items. These embeddings were visualized using t-Distributed Stochastic Neighbor Embedding (t-SNE) to identify clusters of similar items.

4.2. Two-Stage Candidate Reranker

The two-stage candidate reranker model efficiently generates and ranks relevant items for each user session.

4.2.1. Stage 1: Candidate Generation

Candidate items were generated using various methods:

• Co-visit (item co-visitation) : Items frequently visited together, selecting the top 20 most common items associated with each prior selection.
• Click-to-click : Items viewed within a 12-hour timeframe.
• Click-to-cart-or-buy : Items clicked and then added to the cart or purchased within 24 hours.
• Cart-to-cart : Items added to the cart together within 24 hours.
• Cart-to-buy : Items where one was added to the cart and the other was purchased within 24 hours.
• Buy-to-buy : Items ordered together within 24 hours.
• Similar items : Items similar to those interacted with, identified using Word2Vec embeddings.
• Popular items in session clusters : Session embeddings calculated using Word2Vec embeddings, clustered using KMeans.
• Revisit : Reincorporate all previously examined items into the candidate pool for further consideration.
• Click2click Graph and ProNE Application : The hypothesis is that items clicked immediately after another are similar. To explore this, a two-column DataFrame was created, consisting of an item and the item clicked immediately after it. This DataFrame was processed using ProNE. We defined a score to measure the quality of the recall results:

  $$\mathrm{score}=0.10·R_{\mathrm{clicks}}+0.30·R_{\mathrm{carts}}+0.60·R_{\mathrm{orders}}$$

  (1)
  The number of dimensions in ProNE was set to 1000, with the accuracy of candidate recall improving as the number of dimensions increased. Top-k aids were retrieved using ProNE embeddings, leveraging nearest neighbors with cosine metric. The results are shown in Table 3 .

These methods generated a diverse pool of candidate items for each session.

4.2.2. Stage 2: Ranking

The candidate items were ranked based on their predicted relevance, involving extensive feature engineering and model training:

• Session Features :

  -

  Type count per session (clicks, carts, orders).

  -

  Number of unique types per session.

  -

  Mean type per session (clicks=1, carts=2, orders=3).

• Aid Features :

  -

  Type count within all sessions and test sessions.

  -

  Click-to-cart rate, cart-to-order rate, click-to-order rate within sessions.

• Session x Aid Features (Figure 5) :

  -

  Co-visitation count, rate, time-weighted count, and rate.

  -

  Similarity features (cosine similarity between candidate items and session items using Word2Vec and ProNE embeddings).

These features were computed per batch on a 32GB 4*V100 GPU using RAPIDS for efficient processing.

4.3. Model Training and Ensemble

Three gradient boosting models were trained individually and then combined into an ensemble model:

4.3.1. LightGBM Ranker (Lambda Rank)

LightGBM is optimized for speed and efficiency, making it suitable for large-scale data. The lambda rank objective function is specifically designed for ranking tasks, aiming to optimize ranking performance by minimizing pairwise loss between examples:

$$\mathcal{L}=-\sum _{i,j}{I}_{ij}log\left(\sigma (s_i-s_j)\right)$$

(2)

where $I_{ij}$ is an indicator function that equals 1 if item i is correctly ranked higher than item j, and $\sigma$ is the sigmoid function. This objective function effectively improves the ranking quality by focusing on the correct pairwise ordering of items.

4.3.2. XGBoost Ranker (Rank:pairwise)

XGBoost is a powerful machine learning algorithm that excels in ranking tasks. The rank:pairwise objective function in XGBoost is designed to improve the pairwise ranking of items by considering their relative importance, and it operates similarly to the lambda rank objective function in LightGBM. The objective function is defined as:

$$\mathcal{L}=-\sum _{i,j}{I}_{ij}log\left(\sigma (s_i-s_j)\right)$$

(3)

where $I_{ij}$ is an indicator function that equals 1 if item i is correctly ranked higher than item j, and $\sigma$ is the sigmoid function. Both LightGBM and XGBoost utilize this pairwise loss minimization approach to enhance their ranking effectiveness.

4.3.3. CatBoost Ranker (PairLogit)

CatBoost is designed for handling categorical features efficiently. The PairLogit loss function is well-suited for ranking tasks:

$$\mathcal{L}=\sum _{i,j}{I}_{ij}log(1+exp(-y_{ij}(s_i-s_j)))$$

(4)

where $y_{ij}$ is the target value indicating the preference between items i and j.

4.4. Negative Sampling

Effective ranking of machine learning models requires a balanced mix of positive and negative samples. Positive samples represent real events in sessions, labeled as 1 (user clicks, adds to cart, or orders). Negative samples are labeled as 0, representing instances where a project was shown but not interacted with.

Different negative sample ratios were set for clicks (0.25), carts (0.5), and orders (0.2), ensuring a balanced dataset suitable for training. This approach allowed the ranking algorithms to prioritize projects with and without user engagement effectively.

4.5. Ensemble Model

The ensemble model combines the predictions of LightGBM, XGBoost, and CatBoost, each contributing to the final recommendation score. The weights for each model in the ensemble were determined using Optuna to maximize the recall@20 metric. The final prediction score for an item i in session s is calculated as follows:

$$\mathrm{score}(s,i)=w_L·{\mathrm{pred}}_L(s,i)+w_X·{\mathrm{pred}}_X(s,i)+w_C·{\mathrm{pred}}_C(s,i)$$

(5)

where $w_{\mathrm{L}}$, $w_{\mathrm{X}}$, and $w_{\mathrm{C}}$ are the optimized weights for LightGBM, XGBoost, and CatBoost predictions, respectively.

4.6. Evaluation Metric

The effectiveness of the proposed multi-objective recommender system is assessed using the Recall@20 metric for each action type: clicks, carts, and orders. To obtain a comprehensive evaluation score that reflects the relative importance of each action type, the recall values are combined in a weighted average.

The evaluation score score is formulated as follows:

$$\mathrm{score}=0.10·R_{\mathrm{clicks}}+0.30·R_{\mathrm{carts}}+0.60·R_{\mathrm{orders}}$$

(6)

In this equation, $R_{\mathrm{clicks}}$, $R_{\mathrm{carts}}$, and $R_{\mathrm{orders}}$ represent the Recall@20 values for clicks, carts, and orders, respectively.

The recall for each action type $R_{\mathrm{type}}$ is rigorously defined by the following expression:

$$R_{\mathrm{type}}={\displaystyle \frac{1}{N}}\sum _{i=1}^N{\displaystyle \frac{\left|{\left\{\text{pred\_aids}\right\}}_{i,\mathrm{type}}\cap {\left\{\text{ground\_truth\_aids}\right\}}_{i,\mathrm{type}}\right|}{min(20,\left|{\left\{\text{ground\_truth\_aids}\right\}}_{i,\mathrm{type}}\right|)}}$$

(7)

where:

• N denotes the total number of sessions in the test set.
• ${\left\{\text{predicted\_aids}\right\}}_{i,\mathrm{type}}$ represents the set of predicted article IDs (aids) for session i and action type type.
• ${\left\{\text{ground\_truth\_aids}\right\}}_{i,\mathrm{type}}$ corresponds to the set of actual article IDs (ground truth) for session i and action type type.
• The denominator $min(20,\left|{\left\{\text{ground\_truth\_aids}\right\}}_{i,\mathrm{type}}\right|)$ normalizes the recall, ensuring that sessions with fewer than 20 ground truth article IDs are appropriately accounted for.

To compute the Recall@20 metric, the predicted article IDs for each session-type (e.g., each row in the submission file) are truncated after the first 20 predictions. This truncation ensures that the evaluation remains focused on the top 20 recommendations, which is a realistic representation of a user’s interaction with a recommendation system. The weighted average of recall values across different action types ensures a holistic evaluation of the recommender system, reflecting its ability to perform well across various user behaviors.

By employing this evaluation metric, we provide a robust and comprehensive assessment of the multi-objective recommender system, ensuring its efficacy in optimizing multiple consumer behaviors concurrently.

5. Experiments Results

To evaluate the performance of our multi-objective recommender system, we conducted experiments using the OTTO Recommender Systems Dataset. We measured the effectiveness of the individual models (LightGBM Ranker, XGBoost Ranker, and CatBoost Ranker) as well as their ensemble combination. The results are summarized in the following Table 4 .

The ensemble model combining LightGBM, XGBoost, and CatBoost rankers demonstrated superior performance across all metrics, achieving the highest CV, public, and private scores. This confirms the effectiveness of the ensemble approach in optimizing multiple consumer behaviors simultaneously, enhancing the overall recommendation system performance.

6. Conclusions

In conclusion, this study presents a significant advancement in e-commerce by developing a multi-objective recommender system that simultaneously optimizes multiple consumer behaviors, including clicks, cart additions, and completed orders. Using the extensive OTTO Recommender Systems Dataset, the proposed two-stage candidate reranker model efficiently generates and ranks potential items. The ensemble approach, combining LightGBM, XGBoost, and CatBoost Rankers, outperformed individual models, achieving higher recall rates and demonstrating the effectiveness of the multi-objective strategy. This system enhances user satisfaction and engagement while increasing conversion rates for retailers. Future work will focus on refining the model, exploring additional features, and ensuring fairness, transparency, and user privacy in recommendations.