Enhancing Collaborative Filtering with Deep Neural Networks and User Reviews

1. Introduction

Nowadays, e-commerce websites offer a great variety of items and brands for sale. They differ in shape, size, color, and other variances. The vast quantity of products available causes consumers to experience information overload. This may lead to customers feeling confused and overstimulated because of the influence of a huge amount of information that is beyond their ability to process. Developers create recommender systems (RS) to address the problem of information overload [1]. RS is a type of expert system that assists customers in obtaining personalized relevant items of interest from a large set of possible choices. Companies such as Amazon, Spotify, Netflix, and YouTube are all examples of popular recommender systems in use [2,3]. CF is deemed to be one of the most promising and widely used methods in building a RS. Collaborative Filtering (CF) techniques offer personalized recommendations to target users by identifying commonalities between the target user and other users within the community. The Corresponding users or items are identified by calculating the similarities among users' common ratings on certain items. Though, CF approaches perform well when there is sufficient rating information [4]. Nevertheless, CF approach efficiency often suffers from limitations like cold-start problems and sparsity of data [5]. Data sparsity occurs when there are few co-rated items between users or when there is no overlap in co-rated items. The cold-start problem arises when new items are introduced into the system that either have no ratings or have not received enough ratings. Another limitation is that CF approaches do not capture the reason behind user numerical ratings and, accordingly, cannot capture the accurate personal preferences of a target user [6,7].

To enrich, enhance, and handle problems associated with CF efficiently, numerous methods have been proposed namely the representation of users and items by exploiting additional knowledge sources, including user tags [8], items’ product descriptions [9], and social media aspects [10]. However, these solutions remain inadequate, particularly when dealing with high levels of rating sparsity or insufficient rating preferences for target users.

Currently, e-commerce platforms such as Yelp and Amazon provide users with the opportunity to express their opinions on items through written reviews. These review texts contain valuable information and reliable user preferences, which can enhance CF approaches. User text reviews offer semantic information that, when combined with users' rating preferences, can significantly improve the efficiency of CF recommendations [11,12].

Several studies have proposed methods to uncover the hidden aspects of user review texts using techniques such as Latent Dirichlet Allocation (LDA) or Non-Negative Matrix Factorization (NMF) [13]. These studies have shown improvements in recommendation quality compared to traditional collaborative filtering (CF), which solely relies on numerical ratings. However, LDA and NMF utilize bag-of-words (BOW) representations that overlook the contextual information embedded in the review text. Deep learning models address this limitation by learning and analyzing sequential textual features in review text [14].

The current study aims to enhance the quality of recommendations through CF systems by leveraging users' review texts with Bi-GRU deep recurrent networks. This proposed approach addresses the limitations of traditional rating-based CF methods by incorporating the content of user reviews to infer rating preferences and generate recommendations. This method improves recommendation accuracy, particularly in sparse data scenarios, by utilizing deep learning to extract richer feature representations from review texts.

The rest of the paper is organized as follows: Section 2 provides background information on traditional collaborative filtering (CF) recommender systems and recurrent neural networks. Section 3 reviews related work in CF. Section 4 introduces the proposed recommendation framework. Section 5 details the experiments, including datasets, methodology, metrics, comparisons results, and discussion. Finally, Section 6 summarize the conclusions and suggest future research directions.

2. Background Study

This section comprises two parts. The first part delves into the background of traditional collaborative filtering recommendation systems, while the second part elucidates an overview of Recurrent Neural Networks (RNN).

2.1. Traditional Collaborative Filtering

Collaborative filtering recommender systems are classified into two categories: memory-based (also known as heuristic-based) and model-based. The first category, memory-based approaches, utilizes the entire user-item database to identify the "neighborhood" of a new user or item. Based on the distance or correlation between users or items, each neighbor is assigned a weight, and the algorithm aggregates their preferences to generate predictions or recommendations for the new or target user[15]. When the objective is to provide top-N recommendations, these approaches focus on finding the most similar users or items. The memory-based approach is further classified into two methods: user-based and item-based, depending on whether the similarities are determined between users or items [16]. User-based collaborative filtering operates on the assumption that users with similar preferences will enjoy the same items. This approach typically employs metrics such as cosine similarity or Pearson's correlation coefficient to compare the target user's choices with those of other users, identifying a group of "like-minded" users. Once this group is identified, items that have received high ratings from the group are recommended to the target user [17]. Conversely, the item-based approach focuses on finding the most similar items rather than users.

In comparison to the memory-based approach, the model-based approach recommends items by creating a model of user ratings rather than relying on identifying similar users. Model-based recommendations employ machine learning and data mining techniques, such as rule-based approaches, matrix factorization, and clustering models, to develop prediction models. To estimate missing ratings in the user-item matrix, Recommender systems use a model-based approach. According to [18], model-based collaborative filtering tends to be more accurate and generates more relevant recommendations than memory-based recommender systems.

In traditional CF the recommendation process can be formulated as follows. Let $\mathrm{U}=\left\{{\mathrm{u}}_1,{\mathrm{u}}_2{,\mathrm{u}}_3{,\dots ..,\mathrm{u}}_{\mathrm{N}}\right\}$ be a set of registered users in the system, and let $\mathrm{I}=\left\{{\mathrm{i}}_1,{\mathrm{i}}_2{,\mathrm{i}}_3{,\dots ..,\mathrm{i}}_{\mathrm{M}}\right\}$ (see Table 1 for symbol definition presented in this article) be a set of all possible items users have access to in the recommender system. Furthermore, assume $\widehat{\mathrm{r}}:\mathrm{U}\times \mathrm{I}\to \mathrm{R}$, where R is a set of real or non-negative numbers to be a utility function. The utility function $\widehat{\mathrm{r}}\left(\mathrm{u},\mathrm{i}\right)$ measures the gain of how likely a user $\mathrm{u}$ is interested in item $\mathrm{i}$. According to Adomavicius and Tuzhilin [19], the recommendation problem consists of selecting unseen item $\mathrm{I}\in \mathrm{I}$ for each user $\mathrm{u}\in \mathrm{U}$, that maximizes the utility function formally, through Eq. (1),

$$\widehat{r}\left(u, i\right)={\overline{r}}_u+{\displaystyle \frac{\sum _{v\in \widehat{U}}{sim\left(u,v\right)\times (r}_v-{\overline{r}}_v)}{\sum _{v\in \widehat{U}}\left|sim\left(u,v\right)\right|}}$$

(1)

where $\mathrm{s}\mathrm{i}\mathrm{m}\left(\mathrm{u},\mathrm{v}\right)$ indicate the similarity between user $\mathrm{u}$ and user $\mathrm{v}$, ${\overline{\mathrm{r}}}_{\mathrm{u}}$ denote the average ratings of user $\mathrm{u}$ and ${\overline{\mathrm{r}}}_{\mathrm{v}}$the average ratings of user $\mathrm{v}$. There are different methods to compute the similarity between items or users. The most commonly used methods to compute the similarity between two users $\mathrm{u}$ and $\mathrm{v}$ are Cosine-based and Pearson correlation coefficient similarity measures [20]. The resemblance concerning user $\mathrm{u}$ and $\mathrm{v}$ is defined as follows, by Eq. (2),

$$Sim\left(u, v\right){\displaystyle \frac{{\sum }_{i_m\in I_{n,k}} \left(r_{n,m}-{\overline{r}}_n\right)\times \left(r_{k,m}-{\overline{r}}_k\right)}{\sqrt{{\sum }_{i_m\in I_{n,k}}{ \left(r_{n,m}-{\overline{r}}_n\right)}^2} *\sqrt{{\sum }_{i_m\in I_{n,k}}{ \left(r_{k,m}-{\overline{r}}_k\right)}^2}}}$$

(2)

where ${\mathrm{I}}_{\mathrm{n},\mathrm{k}}$ denote the co-rated items between users $\mathrm{u}$ and $\mathrm{v}.$ In other words, it refers to the items rated by both users.

The primary challenge for CF systems is data sparsity [3]. which occurs when the number of co-rated items between the target user and others is not enough. Locating comparable users to the target user in a sparse and large dataset is particularly challenging in CF. Therefore, sparsity leads to poor-quality recommendations.

2.2. Recurrent Neural Networks (RNN)

Concerning the human brain, artificial intelligence algorithms have different approaches for handling individual and sequential data. The first generation of artificial neural networks (ANNs) gained popularity in the past years. ANNs are feedforward networks proficient in learning individual parts of data, such as single images or fixed-length records of information. However, they are not suitable for sequential data, where the order of input data matters. On the other hand, a class of neural networks naturally suited to capture the sequential information present in the input data are Recurring Neural Networks[21]. RNNs are considered feedback neural networks, where the current output from neurons is used as an input for the next neuron's output each time, forming a directed forward and backward cycle between neurons' input and output.

Although basic RNNs are utilized in many applications for predicting short-term sequence data, they suffer from vanishing gradient and exploding gradient problems [22]. These problems arise when there is a large sequence of input data, making it challenging for RNNs to learn from data with long-term dependencies, such as processing text with long sentences. Advanced RNN models have been proposed in the literature to address the vanishing gradient problem inherent in standard RNNs. Hochreiter and Schmidhuber[23] introduced the long short-term memory (LSTM) model, which was one of the first successful RNN models specifically designed to tackle this issue. Despite its effectiveness, the LSTM model has a complex structure with many parameters, making it challenging to learn long-term dependencies. To address these complexities, Cho et al. [24] proposed the gated recurrent unit (GRU) as an alternative to the standard RNN. This study utilizes the GRU model. The following subsection provides a comprehensive overview of its application.

2.1.1. Gated Recurrent Unit (GRU)

The GRU model delivers excellent performance in sequence modeling, particularly in handling long-term data dependencies such as those in text modeling. The GRU architecture introduces a gating mechanism within RNNs, featuring two types of gates: update and reset gates. The reset gate determines how much past information should be discarded from the model. Conversely, the update gate decides how much past information (from earlier time steps) should be carried forward to the next state. In the text modeling process, the first step is word embedding. This involves mapping each word in a sentence or document into a low-dimensional semantic space vector. Given an input sequence ${\mathrm{s}}_{\mathrm{t}}\in {\mathrm{R}}^{{\mathrm{d}}_{\mathrm{s}}},\mathrm{t}\in (\mathrm{1,2},3,\dots ,\mathrm{T})$, a basic RNN defines the sequence of hidden states ${\mathrm{h}}_{\mathrm{t}}\in {\mathbb{R}}^{{\mathrm{d}}_{\mathrm{h}}}$ update at each time step according to the following rule in Eq (3),

$$h_t=\sigma \left(W_h{h}_{t-1}\right)+\left(W_s{s}_{\mathrm{t}}\right)+b)$$

(3)

Where ${\mathrm{W}}_{\mathrm{h}}\in {\mathbb{R}}^{{\mathrm{d}}_{\mathrm{h}}\times {\mathrm{d}}_{\mathrm{h}}}$ , ${\mathrm{W}}_{\mathrm{s}}\in {\mathbb{R}}^{{\mathrm{d}}_{\mathrm{s}}\times {\mathrm{d}}_{\mathrm{h}}}$ and $\mathrm{b}\in {\mathbb{R}}^{{\mathrm{d}}_{\mathrm{h}}}$ are the model parameters and $\mathsf{\sigma }$ is a nonlinear activation function.

In GRU the input sentence in a document is mapped into a set of hidden states ${\mathrm{h}}_{\mathrm{j}}^{\mathrm{i}}$ as follow, Eq (4), Eq (5), Eq (6) and Eq (7),

$${\mathrm{h}}_{\mathrm{t}}=\left(1-{\mathrm{z}}_{\mathrm{t}}\right)\odot {\mathrm{h}}_{\mathrm{t}-1}+{\mathrm{z}}_{\mathrm{t}}\odot \widetilde{{\mathrm{h}}_{\mathrm{t}}}$$

(4)

$${\mathrm{z}}_{\mathrm{t}}=\mathsf{\sigma }\left({\mathrm{W}}_{\mathrm{z}}{\mathrm{w}}_{\mathrm{t}}\right)\odot {\mathrm{U}}_{\mathrm{z}}{\mathrm{h}}_{\mathrm{t}-1}+{\mathrm{b}}_{\mathrm{z}})$$

(5)

$$\widetilde{{\mathrm{h}}_{\mathrm{t}}}=\mathrm{t}\mathrm{a}\mathrm{n}\mathrm{h}\left({\mathrm{W}}_{\mathrm{z}}{\mathrm{w}}_{\mathrm{t}}\right)+{\mathrm{U}}_{\mathrm{h}}({\mathrm{r}}_{\mathrm{t}}\odot {\mathrm{h}}_{\mathrm{t}-1})+{\mathrm{b}}_{\mathrm{h}})$$

(6)

$$r_t=\sigma \left(W_r{w}_t\right)\odot U_r{h}_{t-1}+b_r)$$

(7)

Figure 1 described below presents the comprehensive architecture of the basic GRU unit.

3. Related Works

Recommender systems (RS) have become ubiquitous, guiding user choices on online platforms by suggesting relevant products, services, or content tailored to individual preferences [19] . Collaborative Filtering (CF) remains a dominant approach, leveraging user-item interaction data like ratings or purchase history to generate recommendations [3]. However, CF methods struggle with data sparsity, where limited interactions hinder the generation of high-quality recommendations [3].

This section explores recent advancements in recommendation systems that address data sparsity and enhance recommendation accuracy. We focus on various approaches, including advanced matrix factorization techniques, leveraging user reviews and user behavior, deep learning architectures, and how these approaches compare to the one proposed in this paper.

3.1. Beyond Matrix Factorization: Exploring New Avenues

Matrix factorization (MF) techniques like Singular Value Decomposition (SVD) and Non-negative Matrix Factorization for Recommendation Systems with Dynamic Bias (NMFRS) have been foundational for CF, they primarily rely on explicit user-item interactions [25,26]. This dependence on explicit ratings can limit their ability to capture the nuances of user preferences, especially when dealing with data sparsity issues.

Alternative approaches explore different aspects of recommendation tasks, moving beyond the limitations of traditional MF methods. However, these methods often struggle to fully capture the richness of user preferences, particularly when dealing with sparse data.

Here, we propose the BI-GRUCF model, which addresses this limitation by incorporating user reviews alongside collaborative filtering. By leveraging the valuable insights within reviews, BI-GRUCF can provide more comprehensive user profiles and improve recommendation accuracy, even in scenarios with data sparsity.

3.2. Utilising User Reviews for Recommendations

Several approaches address data sparsity by incorporating user reviews and deep learning architectures. For instance, Vuong et al.[27] leverage Word2Vec to capture semantic relationships between items from reviews, alleviating sparsity issues. Their method offers a significant improvement over traditional MF techniques by considering semantic relationships. However, it focuses solely on item relationships and doesn't explicitly model user preferences within reviews.

Other studies mine user opinions from reviews to enrich CF models. D'Addio et al. [28] use vector representations of user reviews, while Yang et al. [30] employ sentiment analysis to extract user opinions. These approaches offer valuable insights from reviews but lack a mechanism to capture the sequential nature of user reviews and the contextual dependencies within them. Ghabayen and Ahmed [29] combine review clustering with user ratings, but their method doesn't leverage the rich sequential information embedded in review text.

While these approaches offer valuable insights into user preferences from reviews, they often fail to capture the full context and sequential nature of user opinions. This can limit their effectiveness in overcoming data sparsity challenges.

The proposed BI-GRUCF model addresses this limitation by employing Bidirectional GRUs to analyze the sequential nature of user reviews. This allows BI-GRUCF to capture richer contextual information and user preferences, leading to more accurate and personalized recommendations.

By combining the strengths of collaborative filtering and user review analysis through a deep learning approach, the BI-GRUCF model offers a significant advancement over existing methods, particularly in its ability to address data sparsity and generate more accurate recommendations.

3.3. Deep Learning and Recommendation Systems

The advancements in deep learning techniques, particularly within natural language processing and related fields, have significantly influenced the trajectory of recommendation systems (RS). Deep learning methods have been increasingly employed to address RS challenges such as sparsity, cold start issues, and the enhancement of recommendation quality through the extraction and inference of embedding features [30].

While Recurrent Neural Network (RNN) architectures like Long Short-Term Memory (LSTM) and Bidirectional LSTM (Bi-LSTM) have gained traction for sequential learning tasks involving textual data, this study underscores the suitability of Gated Recurrent Units (GRUs) for RS due to their intrinsic mechanism for processing and retaining information over sequences [30].

The integration of Convolutional Neural Networks (CNNs) with Collaborative Filtering (CF) techniques for recommendation systems has shown promising results. CF effectively captures user-item interaction data, identifying users with similar preferences for relevant recommendations. CNNs excel in extracting features from complex data such as reviews or images, enhancing the understanding of user preferences and item characteristics. Recently, Arliyanna et al. [31] proposed a film recommendation system utilizing content-based filtering and CNN classification methods. This approach demonstrated the potential of CNNs in enhancing CF techniques for more accurate and personalized film recommendations.

Illustrating the practical application of RNNs in recommendation models, Kim et al. proposed a context-aware model that integrates CNNs with probabilistic matrix factorization (PMF) to enhance rating predictions in RS [32]. Similarly, Zheng et al. introduced the DeepCoNN model, employing dual CNNs to respectively learn user preferences and item characteristics from review text, culminating in a layered approach that captures user-item interactions for accurate predictions [33].

Moreover, the classification-based deep neural architecture proposed by [34] introduces a novel approach using binary classifications to determine item relevance and user voting, streamlining the prediction process through efficient model execution. Additionally, Ahmed and Ghabayen present a framework for review rating prediction, utilizing bidirectional GRU architectures to predict sentiment polarity and subsequent user ratings, demonstrating substantial improvements in prediction metrics such as accuracy, recall, F1-score, and root-mean-square error (RMSE) [35].

While these approaches offer valuable advancements in specific aspects of recommender systems, they may not fully capture the richness of user preferences and the complex relationships between users, items, and reviews.

Despite recent advancements in recommendation systems (RS), several limitations persist. Traditional CF and matrix factorization techniques struggle with data sparsity and fail to capture the full scope of user preferences, particularly in sparse data scenarios [25,26].

Approaches leveraging user reviews, such as those utilizing Word2Vec [27] and sentiment analysis [28], often neglect the sequential nature and contextual dependencies of reviews. The proposed BI-GRUCF model addresses these limitations by employing Bidirectional GRUs to analyze the sequential nature of user reviews, capturing richer contextual information and nuanced user preferences. This leads to more accurate and personalized recommendations, effectively mitigating data sparsity issues and enhancing recommendation accuracy, thereby offering a significant advancement over existing methods.

4. Proposed Approach

This paper introduces a novel approach to CF that aims to enhance the quality of recommendations and address the sparsity issue common to CF. The proposed approach consists of two distinct phases. In the first phase, a deep learning model incorporating stacked bi-bidirectional gated recurrent units (Bi-GRUs) is utilized to infer user ratings from textual review text. The second-phase recommender system involves the CF component, which generates Top-N recommendations based on the estimated ratings, which generates Top-N recommendations based on the estimated ratings. To provide a visual representation of the proposed collaborative filtering method, refer to Figure 2. The subsequent subsection delves into the detailed steps of the proposed method.

4.1. Phase I: Deep Learning Bi-GRU Model

This section encompasses the data preprocessing steps, including the word embedding process applied to the resulting data. Additionally, it outlines the structure of the Bi-GRU neural network layers.

Figure 2. Proposed CF System Architecture.

Figure 2. Proposed CF System Architecture.

4.1.1. Data Pre-Processing

The data preprocessing phase is a crucial step in the development of recommender systems that leverage text data. The goal of data preprocessing is to prepare the text data for analysis by removing noise and irrelevant information, and by ensuring that the data is consistent and well-formatted. The following are some of the most common data preprocessing techniques used in recommender systems:

• Tokenization: The process of breaking down text into individual words or tokens is known as tokenization. This is accomplished by splitting the text based on whitespace and punctuation marks.
• Stop word removal: These are the common words that do not add much meaning to the text. Hence, these words are typically removed from the text to improve the efficiency of the analysis.
• Stemming: The process of reducing words to their base or root form is known as stemming. This is achieved by removing prefixes and suffixes.
• Lemmatization: Lemmatization is a process of grouping words that have the same base or root form, regardless of their inflection. This is done by considering the contextual meaning of words.
• Case normalization: Case normalization is the process of converting all words to either lowercase or uppercase. This is done to ensure that the text is consistent and well-formatted.
• Removal of extra white spaces: Extra white spaces are any additional spaces that are present in the text. These spaces are typically removed to improve the readability of the text.
• Removal of URLs: URLs are web addresses that are typically used to link to external websites. These URLs are typically removed from the text to improve the efficiency of the analysis.
• Removal of hashtags: Hashtags are used to tag text with keywords. These hashtags are typically removed from the text to improve the readability of the text.
• Removal of date and time references: To improve the efficiency of the analysis, Date and time references are typically removed from the text.

By applying these preprocessing techniques, the text data can be cleaned and prepared for analysis. This can improve the accuracy and effectiveness of recommender systems by removing noise and irrelevant information, thereby ensuring data consistency and its proper formating.

4.1.2. Bi-GRU Model

Bidirectional Gated Recurrent Units (Bi-GRUs) are a powerful recurrent neural network (RNN) that excels in natural language processing tasks. They have unique ability to capture long-term dependencies in sequential data, that are crucial for understanding the intricate nuances present in the user review text. Bi-GRUs offer not only superior performance but also efficient training, thereby making them highly suitable for scaling recommender systems. Research conducted by [36] has demonstrated that Bi-GRUs outperform other types of RNNs, including LSTMs, particularly when they are dealing with user review text. This advantage stems from their remarkable capacity to comprehend long-range dependencies, which enables them to have a deeper understanding of the text and ultimately leading to a more precise and accurate recommendations. The specific architecture of the Bi-GRU layers is illustrated in Figure 3.

As illustrated in Figure 3, the Bi-GRUs are trained on the input sequence in reverse order. Therefore, the output of hidden layers at time t is determined based on the outputs of both the backward and forward layers hidden at times t and t-1, by Eq (8), Eq (9), Eq (10) below:

$${\overrightarrow{h}}_{t-1}=GRU(x_t,{\overrightarrow{h}}_{t-1})$$

(8)

$${\overleftarrow{h}}_{t-1}=GRU(x_t,{\overleftarrow{h}}_{t-1})$$

(9)

$$\overleftarrow{h}=w_t{\overrightarrow{h}}_t+v_t{\overleftarrow{h}}_t+b_t$$

(10)

The GRU units apply a nonlinear transformation to the word embedding vector. As a result, this word embedding vector is encoded into a corresponding GRU unit, which resides in the hidden layer state. Where ${\mathrm{w}}_{\mathrm{t}}$ and ${\mathrm{v}}_{\mathrm{t}}$ are the weights related to the forward hidden layer state ${\overrightarrow{\mathrm{h}}}_{\mathrm{t}}$ and the backward hidden state ${\overleftarrow{\mathrm{h}}}_{\mathrm{t}}$, correspondingly at time moment t.; ${\mathrm{b}}_{\mathrm{t}}$refers to the offset value at the time moment t.

• Review Word Embedding Layer: The Word Embedding Layer is a crucial component of neural networks used for representing words as vectors. These vectors typically have dimensions ranging from 100 to 300 and are trained on extensive text corpora. By capturing the semantic meaning of words, these vectors significantly enhance the accuracy of natural language processing tasks. In the proposed CF system, the Review Word Embedding Layer plays a pivotal role. It accepts a fixed sequence of 200 words as input and transforms each word into its corresponding 300-dimensional GloVe word vector [37]. These embedding vectors are subsequently fed into the Bi-GRU layers, allowing them to learn the long-term dependencies at each specific time step present in the user review text.
• Drop-out layer: The embedded resulting word vectors can be encoded either with numerical values associated with the input words or by using one-hot vectors that is, vectors with only one position filled with a value of one and the rest filled with zeros. However, an optimized embedding matrix can cause overfitting, which can be prevented by using dropout on one-hot vectors [38]. The 1D of spatial dropout layer with a dropout rate of 0.5 plays a role in preventing the activation functions from becoming strongly correlated, leading to better generalization, and preventing overfitting of the model.
• Context Attention layer: The context attention layer is applied to more effectively predict the rating class of a given review since words do not equally contribute to indicating the importance of meaning or the message of the theme. The attention layer obtains such words that significantly demonstrate the meaning of a sentence. This model utilizes the attention mechanisms as in [39]. This mechanism initiates a context vector, which can be viewed as a fixed query, that helps to extract the informative words. The context vector is randomly initialized and jointly learned with the rest of the attention layer weights, formally: Let H be a matrix composed of output vectors $[{\mathrm{h}}_1,{\mathrm{h}}_2,{\mathrm{h}}_3,\dots ,{\mathrm{h}}_{\mathrm{T}}]$ that are produced by the last BiGRUlayer and $\mathrm{T}$ is the length of the sentence. Context attention r of the sentence is determined by the weighted sum of the produced vectors as by Eqs. (11), (12) and (13),

  $$M=\mathrm{t}\mathrm{a}\mathrm{n}\mathrm{h}\left(H\right)$$

  (11)

  $$\alpha =softmax\left({\mathcal{w}}^{T}M\right)$$

  (12)

  $$r=H{\alpha }^T$$

  (13)

  Where $\mathrm{H}\in {\mathbb{R}}^{\mathrm{d}\mathcal{w}\times \mathrm{T}}$, ${\mathrm{d}}^{\mathcal{w}}$ is the dimension of the word vectors, $\mathrm{w}$ is a trained parameter vector and ${\mathcal{w}}^{\mathrm{T}}$is the transpose. Hence the dimension of $\mathcal{w},\mathsf{\alpha },\mathrm{r}$ is, ${\mathrm{d}}^{\mathcal{w}}\mathrm{T},{\mathrm{d}}^{\mathcal{w}}$ separately. The output of this layer is calculated as by Eq (14),

  $$h^{\mathrm{*}}=\mathrm{t}\mathrm{a}\mathrm{n}\mathrm{h}\left(r\right)$$

  (14)

• Concatenation layer: The concatenation layer performs global max pooling and global average pooling to generate one feature map for each class in the classification task. The use of global average pooling requires no parameter optimization, which helps prevent overfitting. Additionally, global average pooling ignores spatial information, making it more robust against spatial variations in the input data. After applying global max pooling and global average pooling to each Bi-GRU output, the resulting vectors are concatenated into a 1D array. Finally, this concatenated output is passed through a dense output layer with a sigmoid function to provide the rating prediction class.

4.2. Phase II: Recommender System

The user-based CF methodology is employed in the second phase of the recommender system. This phase aims to leverage the predicted rating results from the previous phase to calculate the similarity between users.

The cosine similarity metric, widely used in information retrieval and recommendation systems, is employed for this purpose, and is adopted to quantify the similarity between user vectors representing their rating patterns. Following the similarity calculation, the subsequent step involves identifying the most similar users, commonly known as nearest neighbors. These users exhibit comparable preferences and behaviors based on their historical rating data. The nearest-neighbor approach enables the recommender system to harness the collective wisdom of users who share similar tastes and preferences, facilitating the capture of valuable user insights.

Figure 3. The Structure of (BiGRU) layers.

Figure 3. The Structure of (BiGRU) layers.

Once the set of K's similar or like-minded users is determined, the final step entails generating the top-N recommended items for the target user. This is accomplished by considering the preferences and choices of the identified similar users and incorporating their historical rating data and preferences. The recommended items are selected based on the collective wisdom and experiences of these users, ensuring a personalized and relevant recommendation. By incorporating user-based CF in this manner, the recommender system effectively taps into the collective intelligence and preferences of similar users, thereby improving the precision and significance of the recommendations. Thus, this collaborative approach enables the whole system to leverage the shared behaviors and tastes of a community of users, providing tailored recommendations that align with individual user preferences.

5. Experiments

This section outlines a series of experiments conducted using three real-world datasets from Amazon. The goal of these experiments is to address the following research question:

RQ1: How does the proposed approach compare to traditional baseline collaborative filtering (CF) methods?

The experiments aim to assess the effectiveness and superiority of the proposed approach by comparing it against established collaborative filtering (CF) baselines. By benchmarking the proposed approach against these traditional methods, the study aims to assess its ability to provide improved recommendations and address the limitations of existing CF techniques.

The evaluation of the proposed approach involves measuring various performance metrics, including precision, recall, accuracy, and F1-score. These metrics provide insights into the quality and effectiveness of the recommendation outcomes. By comparing these metrics between the proposed approach and the baseline methods, the study aims to determine how well the proposed approach outperforms or surpasses traditional collaborative filtering (CF) methods.

The real-world datasets selected and obtained from Amazon ensure the relevance and applicability of the findings to practical recommendation scenarios. These datasets comprise diverse user preferences, item characteristics, and rating patterns, thereby enabling a thorough assessment of the proposed approach's performance across different domains.

Through this experimental analysis, the study aims to provide empirical evidence and quantitative results that determine the dominance of the recommended approach in comparison to traditional CF baselines. These findings contribute in advancing the field of recommendation systems and provide insights into the effectiveness of novel approaches in improving recommendation quality and user satisfaction.

5.1. Experiments Parameters Settings

The dataset was randomly divided into five folds, with 80% of the data used for the training set and the remaining 20% used as the test set. To ensure unbiased recommendation results, the process was repeated five times, ensuring that no specific training or test set influenced the outcomes. The dataset was randomly divided into five folds, with 80% of the data used for the training set and the remaining 20% used as the test set. To ensure unbiased recommendation results, the process was repeated five times, ensuring that no specific training or test set influenced the outcomes. The constraint that is significant to note here is that the ratings in the test set were never included in the training set. The average values obtained from these iterations were considered the results, ensuring reliable prediction of outcomes. One of the significant advantages of this technique is its ability to maintain a consistent class distribution between the training and test sets, which is crucial in training neural networks, as they enable the model to generalize beyond the training data. By avoiding biases in the training set and addressing overfitting, the model becomes more robust in handling new and unseen data.

The proposed model uses an embedding dimension of 200, representing the size of the word vectors used to encode the input data. This dimension helps capture the semantic meaning of words and their relationships within the text.

The Review Word Embedding Layer in the proposed collaborative filtering (CF) system takes a fixed sequence of 200 words as input and converts each word into its corresponding 200-dimensional pre-trained GloVe word vector. This study uses the "Wikipedia 2014 + Gigaword 5" version of GloVe, which contains 6 billion tokens and a vocabulary size of 400,000.

The model sets the maximum number of features, or vocabulary size, to 20,000, determining the number of unique words it can handle. To prevent overfitting and enhance generalization, a spatial dropout layer is incorporated with a dropout rate of 0.5. This layer randomly sets a fraction of input units to zero during training, helping in regularization and preventing the model from overly relying on specific features. The model architecture includes a bidirectional Gated Recurrent Unit (GRU) layer with 40 units. The bidirectional aspect allows the model to process the input sequence in both forward and backward directions, capturing contextual information from both past and future contexts. The return sequences parameter is set to true, meaning the layer returns the full sequence of hidden states as output. The output layer is a dense layer with several units equal to the number of classes. The activation function used is the sigmoid function, which produces probabilities for each class, indicating the likelihood of the input belonging to each class.

Early stopping and checkpoint callbacks were configured to trigger at the end of each epoch. Early stopping automatically halts model training when the evaluation metric stops improving. In contrast, the checkpoint saves the weights from the best epoch for later use. Additionally, all experiments were conducted on a machine with an Intel Xeon CPU family 6, model 85, running at 2000.180 MHz and equipped with 13.021 GB of RAM.

5.2. Datasets

Various collaborative filtering (CF) recommendation methods are evaluated using the standard Amazon dataset [40]. This dataset includes millions of user reviews and ratings on Amazon, spanning from May 1996 to October 2018 Various collaborative filtering (CF) recommendation methods are evaluated using the standard Amazon dataset [40] This dataset includes millions of user reviews and ratings on Amazon, spanning from May 1996 to October 2018[41]. The Amazon dataset comprises 24 product categories. Due to the vast size of the dataset, handling all categories is challenging. Therefore, three product categories from different domains with the largest number of users and reviews were selected to train, develop, and evaluate the proposed method. The chosen datasets are Electronics, CDs & Vinyl, and Movies & TV. These datasets include user profiles and item numerical ratings ranging from 1 to 5, along with the corresponding plaintext reviews. Table 1 summarizes the detailed statistics of these datasets after pre-processing

Table 1. Classification of Amazon Dataset Categories.

Dataset	User #	Item #	Reviews#	Sparsity
Electronics	1,92102	63,001	16,60038	98.62
CDs &Vinyl	7,5127	64,443	10,77845	97.77
Movies &TV	1,23633	50,052	16,65265

Table 1. Classification of Amazon Dataset Categories.

Dataset	User #	Item #	Reviews#	Sparsity
Electronics	1,92102	63,001	16,60038	98.62
CDs &Vinyl	7,5127	64,443	10,77845	97.77
Movies &TV	1,23633	50,052	16,65265

5.3. Evaluation Protocol and Metrics

One of the most standard strategies for evaluating RS is predictive accuracy, which measures the predictive ability of these systems on unseen observations. This study uses the standard mean absolute error (MAE) and root-mean-square error (RMSE) to measure predictive accuracy. However, in most recommendation scenarios, users receive a ranked list of Top-N recommended items rather than predictions for specific item ratings. Therefore, classification accuracy methods such as precision, recall, and F1-measure are used to evaluate the predicted values that the recommender system can generate for the user.

5.3.1. Accuracy Metrics

According to [42] the MAE measurement is not effective when the experimental dataset is sparse. In such cases, the Root Mean Squared Error (RMSE) metric is more suitable. The RMSE metric[43] is calculated by considering the mean squared error, making large errors more pronounced than small ones. Both RMSE and MAE measure the error rate in recommendations. MAE and RMSE are computed as follows[26,42], by Eq (15) and EQ (16) below,

$$\mathrm{M}\mathrm{A}\mathrm{E}={\displaystyle \frac{{\sum }_{\mathrm{i}=1}^{\mathrm{N}}\mathrm{⎹}\mathrm{r}(\mathrm{u},\mathrm{m})-\widehat{\mathrm{r}}\left(\mathrm{u},\mathrm{m}\right)\mathrm{⎹}}{\mathrm{N}}}$$

(15)

$$\mathrm{R}\mathrm{M}\mathrm{S}\mathrm{E}=\sqrt{{\displaystyle \frac{{\sum }_{\mathrm{i}=1}^{\mathrm{N}}{\left(\mathrm{r}\left(\mathrm{u},\mathrm{i}\right)-\widehat{\mathrm{r}}\left(\mathrm{u},\mathrm{i}\right)\right)}^2}{\mathrm{N}}}}$$

(16)

where N denotes the total number of rating overall users in the test set, $\mathrm{r}(\mathrm{u},\mathrm{m})$ is the actual rating of the user for item $\mathrm{m}$, $\widehat{\mathrm{r}}\left(\mathrm{u},\mathrm{m}\right)$ is the predicted rating for the user u on item $\mathrm{m}$. The lower the value of MAE and RMSE the better the accuracy result. The sparsity level of a dataset can be computed as follows, Eq (17),

$$\mathrm{S}\mathrm{p}\mathrm{a}\mathrm{r}\mathrm{s}\mathrm{i}\mathrm{t}\mathrm{y}=100\mathrm{\%}-\mathrm{D}\mathrm{e}\mathrm{n}\mathrm{s}\mathrm{i}\mathrm{t}\mathrm{y}$$

(17)

where, Density is calculated using the following equation, Eq (18) below:

$$\mathrm{D}\mathrm{e}\mathrm{n}\mathrm{s}\mathrm{i}\mathrm{t}\mathrm{y}={\displaystyle \frac{NumberofItem\times NumberofUsers}{NumberofItem\times NumberofUsers}}\times 100$$

(18)

5.3.2. Classification Metrics

The following presents the classification accuracy metrics the Precision, Recall, and F1-measure present Eq (19), Eq (20) and Eq (21). These metrics are used to evaluate the predicted classification accuracy that RS generates for the target users.

$$\mathrm{P}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{s}\mathrm{i}\mathrm{o}\mathrm{n}\left(\mathrm{u}\right)={\displaystyle \frac{|\mathrm{T}\mathrm{e}\mathrm{s}\mathrm{t}\left(\mathrm{u}\right)\cap \mathrm{T}\mathrm{o}\mathrm{p}\mathrm{N}(\mathrm{u}\left)\right|}{\left|\mathrm{T}\mathrm{o}\mathrm{p}\mathrm{M}\left(\mathrm{u}\right)\right|}}.$$

(19)

$$\mathrm{R}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{l}\left(\mathrm{u}\right)={\displaystyle \frac{\left|\mathrm{T}\mathrm{e}\mathrm{s}\mathrm{t}\left(\mathrm{u}\right)\cap \mathrm{T}\mathrm{o}\mathrm{p}\mathrm{N}\left(\mathrm{u}\right)\right|}{\left|\mathrm{T}\mathrm{e}\mathrm{s}\mathrm{t}\left(\mathrm{u}\right)\right|}}.$$

(20)

$$\mathrm{F}1-\mathrm{m}\mathrm{e}\mathrm{a}\mathrm{s}\mathrm{u}\mathrm{r}\mathrm{e}={\displaystyle \frac{2\times \mathrm{r}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{l}\times \mathrm{p}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{s}\mathrm{i}\mathrm{o}\mathrm{n}}{\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{l}+\mathrm{p}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{s}\mathrm{i}\mathrm{o}\mathrm{n}}}.$$

(21)

where $\mathrm{T}\mathrm{e}\mathrm{s}\mathrm{t}\left(\mathrm{u}\right)$the item list of user u in the test set, and $\mathrm{T}\mathrm{o}\mathrm{p}\mathrm{N}\left(\mathrm{u}\right)$ is the $\mathrm{t}\mathrm{o}\mathrm{p}\mathrm{N}$ recommended item list for user $\mathrm{u}$.

5.4. Comparisons

This section compares several recommender systems approaches, highlighting their strengths and limitations in the context of our proposed method Bi-GRUCF, as follows:

• SVD: Singular Value Decomposition (SVD) is a sophisticated collaborative filtering (CF) technique that uses matrix factorization to transform users and items into a latent factor space. This method, widely recognized as a benchmark, effectively predicts ratings even for users with sparse data by leveraging the latent factors [26,44,45].
• NMFRS: Recommender Systems with Dynamic Bias (NMFRS) enhances traditional matrix factorization by incorporating non-negative updates and dynamic bias matrices, which improves interpretability and accuracy [25] . It focuses on probabilistic distributions and minimizes differences between observed and predicted ratings.
• Word2Vec: This approach captures semantic relationships between items from reviews, which helps alleviate data sparsity issues. It improves over traditional matrix factorization techniques by considering semantic relationships, although it primarily focuses on item relationships [27].
• RNN (LSTM): Recurrent Neural Network (RNN) architectures like Long Short-Term Memory (LSTM) are effective for sequential learning tasks involving textual data. They process and retain information over sequences, making them suitable for capturing user preferences in recommendation systems [31]
• CNN: Convolutional Neural Networks (CNNs) excel in extracting features from complex data, such as reviews or images. When integrated with CF techniques, they enhance the understanding of user preferences and item characteristics, leading to more accurate recommendations. Recent studies have demonstrated the potential of CNNs in improving CF methods for personalized recommendations [33].

In summary, while traditional methods like SVD and NMFRS provide robust frameworks for recommendation systems, deep learning approaches such as Word2Vec, RNNs, and CNNs offer significant advancements by capturing richer contextual information and addressing data sparsity more effectively.

5.5. Results and Discussion

This section investigates the performance of various recommender system approaches on the Amazon Reviews datasets (Electronics, Movies & TV, CDs & Vinyl) characterized by high sparsity levels (above 97%). Sparsity refers to the proportion of missing entries in the user-item interaction matrix.

5.5.1. Performance Comparison on Sparse Data

In recommender systems, sparse data can pose challenges as traditional methods struggle to learn robust relationships between users and items. We employ Mean Absolute Error (MAE) and Root Mean Squared Error (RMSE) as evaluation metrics to assess the accuracy of recommendations.

The performance of various recommender system approaches on the three datasets with varying sparsity levels is summarized in Table 2. Here, we focus on the MAE metric to illustrate the impact of sparsity.

The results presented in Table 2, focusing on MAE, reveal a clear trend across all datasets. Traditional collaborative filtering (CF) methods (SVD, NMFRS) consistently achieve higher MAE scores compared to deep learning-based approaches (CNN, Word2Vec, Bi-GRUCF). This suggests that regardless of the specific sparsity level (above 97% in all cases), CF methods struggle to capture the intricate relationships between users and items in sparse data scenarios. Deep learning methods, on the other hand, excel at extracting richer feature representations from limited data points, leading to recommendations with lower absolute errors (MAE).

As shown in Table 3, Bi-GRUCF consistently achieves the lowest Mean Absolute Error (MAE) across all datasets, outperforming traditional methods (SVD, NMFRS) by an average of 55.4% and deep learning approaches like CNN and Word2Vec by an average of 77.8%. This significant improvement suggests that Bi-GRUCF's ability to leverage user reviews through a deep learning architecture grants it a substantial advantage in sparse data scenarios.

Further strengthening this notion is the superior performance of the proposed Bi-GRUCF method. As shown in Table 4, Bi-GRUCF achieves the lowest MAE scores across all datasets, consistently outperforming other approaches. This significant improvement underscores the effectiveness of Bi-GRUCF in leveraging user reviews through a deep learning architecture. By incorporating textual information from reviews, Bi-GRUCF can potentially learn more nuanced user preferences and item characteristics, leading to more accurate recommendations, especially when dealing with limited data.

The high sparsity levels (above 97%) in all three datasets likely play a crucial role in the observed performance differences. Traditional CF methods, which rely primarily on user-item interaction data, might struggle to learn robust representations in such sparse environments. Deep learning approaches, like Bi-GRUCF, which can extract additional information from user reviews, might be more advantageous under these circumstances.

It's important to note the limitations of CNN and Word2Vec approaches included for comparison. As evidenced by the high MAE scores in Table 1, relying solely on item features (CNN) or word embeddings (Word2Vec) might not be as effective as incorporating user reviews through a sequential learning approach like Bi-GRUCF. CNNs might struggle to capture the sequential nature of user preferences within reviews, while Word2Vec, while effective for item relationships, might not explicitly model user behavior.

Table 5. MAE and RMSE results for the three CF models on three datasets.

Dataset	Sparsity#	Method	MAE	RMSE
Electronics	98.62	SVD	0.88	1.22
		NMFRS	0.94	1.25
		CNN	1.72	2.27
		Word2Vec	1.71	2.26
		MCNN	1.71	2.26
		Bi-GRUCF	0.37	0.81
Movies &TV vedio	97.77	SVD	0.78	1.08
		NMFRS	0.79	1.07
		CNN	2.21	2.86
		Word2Vec	2.21	2.86
		CNN	2.2	2.86
		Bi-GRUCF	0.38	0.8
CDs & Vinyl media	97.3	SVD	0.71	1.01
		NMFRS	0.84	1.14
		CNN	1.86	1.82
		Word2Vec	1.86	1.81
		CNN	1.85	1.81
		Bi-GRUCF	0.42	0.84

Table 5. MAE and RMSE results for the three CF models on three datasets.

Dataset	Sparsity#	Method	MAE	RMSE
Electronics	98.62	SVD	0.88	1.22
		NMFRS	0.94	1.25
		CNN	1.72	2.27
		Word2Vec	1.71	2.26
		MCNN	1.71	2.26
		Bi-GRUCF	0.37	0.81
Movies &TV vedio	97.77	SVD	0.78	1.08
		NMFRS	0.79	1.07
		CNN	2.21	2.86
		Word2Vec	2.21	2.86
		CNN	2.2	2.86
		Bi-GRUCF	0.38	0.8
CDs & Vinyl media	97.3	SVD	0.71	1.01
		NMFRS	0.84	1.14
		CNN	1.86	1.82
		Word2Vec	1.86	1.81
		CNN	1.85	1.81
		Bi-GRUCF	0.42	0.84

5.5.2. Evaluation Using Classification Accuracy Metrics

As mentioned earlier, classification accuracy metrics like Recall, Precision, and F-measure provide a more user-centric indicator of recommendation quality compared to MAE and RMSE. Here, items with true numerical ratings above 3.5 are considered relevant, and those below 3.5 are considered non-relevant. These values serve as a general guideline for the following experiments.

In the context of recommendation systems, the Top-N recommendation task involves presenting users with a list of N items, where N represents the number of recommended items. The value of N is crucial as it determines the amount of information presented to users. Based on extensive experimentation, we observed that a value of N=20 consistently yielded the best results across all models and datasets. This suggests that users tend to focus on the top 20 recommendations, making it a critical benchmark for evaluating recommendation quality.

The detailed results of the evaluation for Recall, Precision, and F1-measure are presented in Tables 6, 7 and 8. As shown in the tables, the Bi-GRUCF model consistently outperforms all baseline approaches (SVD, NMFRS, Word2Vec, RNN, CNN) across all three datasets (Electronics, Movies & TV, CDs & Vinyl).

A notable observation is the consistent superiority of Bi-GRUCF across different recommendation list lengths (N). For instance, in the Electronics dataset (Table 4), Bi-GRUCF achieves a Recall of 97.31% at N=20, significantly higher than the closest competitor, RNN (92.64%). This indicates that Bi-GRUCF is more effective in retrieving relevant items for users in this domain, regardless of the number of recommendations presented. Similar improvements are observed for Precision (Table 3) and F1-measure (Table 5) across all datasets, highlighting the overall superiority of the Bi-GRUCF model in terms of recommendation accuracy.

Table 9. Comparison of the Precision obtained on 3 datasets.

Dataset	SVD	NMFRS	Word2Vec	RNN	CNN	BI-GRUCF
Electronics	83.25	81.79	83.14	92.64	93.28	97.31
Movies & TV	87.16	81.88	87.05	90.7	92.39	96.82
CDs & Vinyl	84.71	82.28	84.6	90.89	92.65	95.95

Table 9. Comparison of the Precision obtained on 3 datasets.

Dataset	SVD	NMFRS	Word2Vec	RNN	CNN	BI-GRUCF
Electronics	83.25	81.79	83.14	92.64	93.28	97.31
Movies & TV	87.16	81.88	87.05	90.7	92.39	96.82
CDs & Vinyl	84.71	82.28	84.6	90.89	92.65	95.95

Table 10. Comparison of the Recall obtained on 3 datasets.

Dataset	SVD	NMFRS	Word2Vec	RNN	CNN	BI-GRUCF
Electronics	84.84	83.52	84.73	93.11	93.44	95.04
Movies & TV	87.72	85.02	87.61	92.79	93.46	94.91
CDs & Vinyl	85.59	83.8	85.48	92.9	92.65	94.05

Table 10. Comparison of the Recall obtained on 3 datasets.

Dataset	SVD	NMFRS	Word2Vec	RNN	CNN	BI-GRUCF
Electronics	84.84	83.52	84.73	93.11	93.44	95.04
Movies & TV	87.72	85.02	87.61	92.79	93.46	94.91
CDs & Vinyl	85.59	83.8	85.48	92.9	92.65	94.05

Table 11. Comparison of the F1-Meausre obtained on 3 datasets.

Dataset	SVD	NMFRS	Word2Vec	RNN	CNN	BI-GRUCF
Electronics	86.49	85.32	86.38	93.23	93.46	92.87
Movies & TV	88.28	88.41	87.05	93.62	93.56	93.08
CDs & Vinyl	86.49	85.37	85.48	92.41	92.4	92.23

Table 11. Comparison of the F1-Meausre obtained on 3 datasets.

Dataset	SVD	NMFRS	Word2Vec	RNN	CNN	BI-GRUCF
Electronics	86.49	85.32	86.38	93.23	93.46	92.87
Movies & TV	88.28	88.41	87.05	93.62	93.56	93.08
CDs & Vinyl	86.49	85.37	85.48	92.41	92.4	92.23

The superior performance of Bi-GRUCF stems from its unique architecture, which effectively captures sequential information and context from user reviews. This is achieved through two key components: GRUs and an attention mechanism. GRUs selectively remember or forget information from the sequence, allowing Bi-GRUCF to capture long-term dependencies within user reviews better than RNNs. This leads to a deeper understanding of user preferences and context. Additionally, the attention mechanism focuses on the most informative parts of reviews by dynamically assigning weights to different words or phrases. This selective focus, in contrast to RNNs that process the entire sequence uniformly, enables Bi-GRUCF to extract more meaningful insights from user reviews. Consequently, Bi-GRUCF achieves superior Recall, Precision, and F1-measure across diverse datasets.

Table 6. List of Symbols.

Symbol	Description	Category
R	User-item rating matrix	Variable
U	Set of registered users	Set
I	Set of all possible items	Set
u	User ID	Variable
i	Item ID	Variable
r_ui	Rating of item i by user u	Variable
N(u)	Neighborhood of user u (similar users)	Set
K	Number of nearest neighbors	Variable
pred_rui	Predicted rating of item i for user u	Variable
W	Weight matrix in Bi-GRU layer	Variable
b	Bias vector in Bi-GRU layer	Variable
H	Hidden state vector in Bi-GRU layer	Variable
x(t)	Input word vector at time step t	Variable
f	Forget gate function in GRU unit	Function
i	Input gate function in GRU unit	Function
o	Output gate function in GRU unit	Function
σ	Sigmoid activation function	Function
c(t)	Cell state vector in GRU unit at time step t	Variable
γ	Context vector in Attention layer	Variable
α(t)	Attention weight at time step t	Variable
φ	Embedding function (maps item/user ID to embedding vector)	Function

Table 6. List of Symbols.

Symbol	Description	Category
R	User-item rating matrix	Variable
U	Set of registered users	Set
I	Set of all possible items	Set
u	User ID	Variable
i	Item ID	Variable
r_ui	Rating of item i by user u	Variable
N(u)	Neighborhood of user u (similar users)	Set
K	Number of nearest neighbors	Variable
pred_rui	Predicted rating of item i for user u	Variable
W	Weight matrix in Bi-GRU layer	Variable
b	Bias vector in Bi-GRU layer	Variable
H	Hidden state vector in Bi-GRU layer	Variable
x(t)	Input word vector at time step t	Variable
f	Forget gate function in GRU unit	Function
i	Input gate function in GRU unit	Function
o	Output gate function in GRU unit	Function
σ	Sigmoid activation function	Function
c(t)	Cell state vector in GRU unit at time step t	Variable
γ	Context vector in Attention layer	Variable
α(t)	Attention weight at time step t	Variable
φ	Embedding function (maps item/user ID to embedding vector)	Function

6. Conclusions

The challenge of data sparsity in CF recommender systems is revisited in this study by leveraging deep learning techniques. Limitations of traditional CF methods, such as SVD and NMFRS, in sparse data environments due to their reliance solely on numerical ratings are highlighted. The proposed approach addresses these limitations by integrating users' review texts through Bi-GRU deep recurrent networks, significantly improving recommendation accuracy. Experimental results on the Amazon dataset demonstrate that the Bi-GRU-based method consistently outperforms other state-of-the-art techniques, achieving substantial gains in both MAE and RMSE. These findings emphasize the importance of incorporating textual data to capture user preferences more effectively.

In conclusion, the potential of deep learning models to enhance CF systems by extracting rich, contextual information from user reviews is underscored by this research. Future research directions include exploring the scalability of this approach and its applicability to other domains, encouraging further investigation into advanced deep learning architectures for recommender systems.