Two-path Multi-behavior Sequence Modeling

1. Related Work

1.1. Self-Supervised Representation Learning

The purpose of self-supervised learning was to learn useful representations by maximizing the mutual information between the inputs and outputs of the encoder [11]. With respect to visual representation learning multiple graphs were generated by multiple data augmentation, which maximized the mutual information of these graph features[12]. Figure characterization was based on self-supervised learning to extract features [3]. Regarding sequence recommendations, self-supervised signals were obtained by feature masks. With sparse data on the session, the mask generated sparse self-supervised signals and therefore was not applicable to sequence behavior recommendation [6].

1.2. Ggeneral Recommendation

Collaborative filtering algorithm was used as a recommendation system to predict similar users based on the interests of similar users, and to recommend interested projects to them[13]. Factorization was one of the most popular methods among collaborative filtering methods, which mapped user interactions to a representation space. Hadamard product of feature vectors of users-items was used to represent user preferences[14]. The other methods aimed to be different from the collaborative filtering method and had the same goals as the results of this method. For example, user preferences were predicted by using neural collaborative filtering[2] through multiple MLPS.

1.3. Multi-Behavior Recommendation System

In recent years, various studies had been proposed to deal with multiple user interactions [5,6]. Specifically, MBGCN effectively transmitted heterogeneous behavior information and obtained user characteristics due to the effectiveness of graph neural network [5]. MGNN-SPred proposed to learn global Item-Item relations via GNN, and integrated target embedding and current session through a gating mechanism [6].

2. Materials and Methods

In order to distinguish different user behavior perception preferences, we propose an Self-supervised Learning Method for Multi-behavior Attention Based on two paths parallel computational(TP_MB), and the model framework is shown in Figure 1. The model consists of a Multi-behavior interaction sequence path, a multi-behavior interaction graph path and path-path self-supervised learning. Each path consists of three parts: (1) Multi-Behavior initial embedding. (2) Multi-Behavior interaction dependencies. (3) Multi-Behavior Self-Supervised learning paradigm for paths. The sequence path is based on the BERT approach to model user behavior sequences capable of capturing individual user characteristics. The graph path models all users to form a user-item behavior graph, where the behavior graph contains extracted behavior subgraphs by behavior categories, and the behavior graph contains all user characteristics. Individual user features in the sequence path are not sufficient to provide enough data for the model, while all user features in the graphics path can be effectively modeled as user features. Therefore, parallel computation of sequence and graph paths can complement each other and effectively help extract user feature information. TP_MB first learns embeddings containing user behavior features through a two-path (sequence path, graph path) framework. Embeddings are learned by TP_MB that contains user behavior features through a two-path (sequence path (MBS Interaction path), graph path (MBG Interaction path)) framework. Where ’pv’ means click, ’buy’ means purchase, ’cart’ means shopping cart and ’fav’ means favorite. Then, the multi-behavior dependencies between the auxiliary behavior (click, add to cart) and the target behavior (purchase) is captured in the calculation of user multi-behavior including the calculation of the behavior of the sequence path and the calculation of the behavior of the graph path. Finally, the consistency of user interests is maximized by contrast learning between multi-behavior graph path and sequential path, which is in order to predict items that users will interact with through certain behaviors.

2.1. Problem Formulation

User multi-behavior interaction items are used as input, and the dependencies between different behaviors is analyzed from a fine-grained perspective by considering the interaction between different behaviors. Defined $U=\left\{u_1,u_2,\dots ,u_s\right\}$ to represent the set of users, $I=\left\{i_1,i_2,\dots ,i_t\right\}$ represents the set of items and $B=\left\{b_1,b_2,\dots ,b_k\right\}$ represents the set of user behavior types. User interaction behaviors such as click, shopping cart, favorite and purchase.

2.2. Multi-Behavior Interactive Sequence Path

2.2.1. Multi-Behavior Interaction Sequence Embedding(MBS Embedding)

We designed a multi-behavior attention network to help users collaborate on different behaviors. In order to make the behavior interaction context as the input of the sequence path, the embedding layer of the behavior context sequence path is designed. Multi Behavioral Interaction Sequence (MB Sequence) is used as the input to the sequence path, and we design the embedding layer for the behavioral contextual sequence path. Joint embedding of individual user, project information and corresponding interactive behavior context information. Given $u_q\in U$, we set each element in the sequence of user interaction as a triple $s_{u_q,i_x}^{b_k}=\left\{(u_q,b_k,i_x)|u_q\in U,b_k\in B,i_x\in I\right\}$, which indicates that user q interacts with item x with the k-th behavior. The multi-behavior sequence of a user contains the interaction information of a single user. Thus, the multi-behavior interaction sequence of the user is mapped to the initial embedding matrix $S_{u_q}=[\left(u_q,b_1,i_x\right),\left(u_q,b_2,i_x\right),\dots ,\left(u_q,b_K,i_x\right)]$ as shown in Figure 2.

2.2.2. Multi-Behavior Interaction Sequence Attention Representation(Attention Fusion)

In order to better extract user preferences and simulate the dependence between different types of behaviors, we use attention in user behavior interaction to obtain the commonality of individual behaviors and the relationship of different behaviors. Considering the correlation between different behaviors of users, this correlation will help e-commerce platforms to combine the correlation of behaviors to more accurately recommend products to users, and the attention can capture which behavior has a greater relationship with the purchase behavior, hence the name "multi-behavior attention". The relationship between multiple user behaviors is used to model the fine-grained interaction between users and items, as shown in Figure 2. We propose a Multi-Behavior Interaction Sequence-Dependent Encoder(MB Dependency Encoder), which is an implicit or explicit user interest about items that users interact with by using within behaviors and between behaviors. Different behaviors of users differently contribute to items that users may be interested in. Therefore, we propose to capture the different modal relationships using user behaviors in attention. Based on self-attention mechanism capture fine-grained correlations between different types of auxiliary behaviors and target behaviors, modeling multi-behavior dependencies. Multiple behaviors are put into attention can make behaviors no longer independent behaviors, and form a vector of features related to the purchase behavior. Specifically, $S_{u_q}$ is a sequence of historical interactions of each user. The input of attention is an auxiliary behavior feature vector $s_{u_q,i_1}^{b_k}\in R^{n*d}$ and the target behavior feature vector $s_{u_q,i_1}^{b_{k^{\prime }}}$ of the same user, $k\in [1,2,\dots ,K]$, where $i_1$ represents the first item i in the item collection I. The feature vector $s_{u_q,i_1}^{b_k}$ is an element in the sequence $S_{u_q}$. In addition, we calculate the weight matrix of each auxiliary behavior feature vector $s_{u_q,i_1}^{b_k}$ and target behavior feature vector $s_{u_q,i_1}^{b_{k^{\prime }}}$, which is to understand the correlation between behaviors from users’ historical behaviors:

$$A_{k,k^{\prime }}^s=\frac{{\left(W^Q·{s_{u_q,i_1}^{b_k}}^T\right)}^T\left(W^K·{s_{u_q,i_1}^{b_{k^{\prime }}}}^T\right)}{{\sum }_{k=1}^K{\left(W^Q·{s_{u_q,i_1}^{b_k}}^T\right)}^T\left(W^K·{s_{u_q,i_1}^{b_{k^{\prime }}}}^T\right)}$$

(1)

$${\alpha }_k^s=dropout\left(softmax\left(A_{k,k^{\prime }}^s\right)\right)$$

(2)

$$s_q^{b_{k,k^{\prime }}}={\alpha }_k^s·s_{u_q,i_1}^{b_k}·W^V$$

(3)

where $W^Q$,$W^K$,$W^V$$\in R^{d*n}$ are weight matrices of learnable behavior vectors. As $A_{k,k^{\prime }}^s\in R^{n*n}$ represents an correlation matrix between the auxiliary behavior k and the target behavior $k^{\prime }$. This correlation matrix represents the correlation between the auxiliary and target behaviors. ${\alpha }_k^s$ represents the attention score. $A_{k,k^{\prime }}^s$ represents calculated by $softmax$ function in order to find auxiliary behavior that is closer to the target behavior. To prevent overfitting problems, we use dropout method to obtain ${\alpha }_k^s$. The sequence $s_q^{b_{k,k^{\prime }}}$ contain the closer relationship between the auxiliary behavior and the target behavior ,which is obtained by multiplying the attention fraction ${\alpha }_k^s$ with the behavior feature vector $s_{u_q,i}^{b_k}$.

2.2.3. Multi-Behavior Interaction Sequence Modeling

The history of multiple user interactions in the real world contains a wealth of information. Past studies have modeled user interactions chronologically to predict the next item that a user will interact with. However, due to the existence of many external uncertain factors, it is impossible for users to interact according to the strict sequence of interactive behaviors. We developed a multi-behavior Attention Fusion to extract different user behavior characteristics, which is to capture personalized user interests by modeling the underlying relationships of different types of behavior. We develop an Attention Fusion to extract different user behavior features. Inspired by BERT4Rec [7] in terms of text, we introduce a semantic encoder of behavior to capture personalized user interests:

$$s_q^{b_{k,k^{\prime }}}=GELU(s_q^{b_{k,k^{\prime }}}*W+b)$$

(4)

where we use $Gaussianerrorlinearelement$ (GELU) activation function. W represents the weight matrix of the GELU activation $function$, b means bias. $s_q^{b_{k,k^{\prime }}}$ represents the user partial long-term interest captured by k behaviors in the user long-term history. Softmax, as the activation function of output, normalizes the results of the concatenation of various behavior sequences:

$$s_q^{b_k}=softmax(s_q^{b_{k,k^{\prime }}}\left|\right|s_q^{b_{k+1,k^{\prime }}}\left|\right|\dots \left|\right|s_q^{b_{k+n,k^{\prime }}})$$

(5)

For different users, users have different behavioral interaction sequences leading to different encoding results.

2.2.4. Multi-Behavior Interaction Sequence Contrast Loss(Sequence Contrast Learning)

In TP_MB, it is our goal to captures the commonality of multiple behaviors of the same user, the differences of multiple behaviors of different users. Inspired by EHCF, users’ multi-behavioral relationships reflect their implicit intentions[13]. Self-supervised learning is a kind of contrast learning in machine learning. Therefore, we design intra-path self-supervised learning of multi-behavior interaction sequences to enhance supervision signals of multiple behavioral data through self-supervised learning. Specifically, we regard the different behaviors of the same user as positive sample pairs$\left\{\left(s_q^{b_k},s_q^{b_{k+1}}\right)|q\in U\right\}$, and the different behaviors of different users as negative sample pairs$\left\{\left(s_q^{b_k},s_p^{b_{k+1}}\right)|q,p\in U,q\ne p\right\}$. We define the TP_MB objective $function$ according to InfoNCE:

$$L_{SCL}=\sum _{q\in U}-log\left(\frac{exp\left((s_q^{b_k}·s_q^{b_{k+1}})\right)}{exp\left((s_q^{b_k}·s_q^{b_{k+1}})\right)+exp\left((s_q^{b_k}·s_p^{b_{k+1}})\right)}\right)$$

(6)

2.3. Multi-Behavior Interaction Graph Path

2.3.1. Multi-Behavior Interaction Graph Embedding

In general, using a single information of the user cannot obtain enough available information. Since users have multiple behaviors, including clicking, shopping cart, favorate and purchase. In addition, for the user’s purchase behavior, other behaviors have great value in helping extract user interest. Therefore, we set the purchase behavior as the target behavior, and other behaviors as auxiliary behaviors. Learning user interests with the help of auxiliary behaviors is to predict user purchasing behavior. Here we build a user-item graph of different interaction behaviors through all items that users interact with in different behaviors. Multi-behavior interaction graph(MB Interaction Graph) contain all information about user interactions. First, we define $G=(V,E)$, V means that the node set includes user set $u\in U$ and item set $i\in I$, namely $(U,I)\in V$. E represents different interaction behaviors between user nodes and item nodes. Furthermore, the embedding of a multi-behavior graph is composed of multiple behavior subgraph embeddings, so our behavior subgraph embedding can be expressed as $G_b=(V_b,E_b)$. For example, the behavior subgraph composed of the items clicked by the user, $G_{click}=\left(V_{click},E_{click}\right)$,$G_{click}$ represents the item graph representation that the user interacts with by clicking. $V_{click}$ represents the user and item node connected with the click behavior, and $E_{click}$ represents the user’s click behavior.

2.3.2. Attention Representation of Multi-Behavior Interaction Graph in Two-path(Graph-inner Attention)

We design Multi-Behavior Interaction Graph Information Aggregation. Research results on graph convolution showed that general research focused on node characterization of simple undirected graphs[8]. The above work ignored edges between nodes that also had multiple properties, such as behavior. Inspired by graph neural network processing user multi-relational data GHCF [2]. We propose a multi-behavior interaction graph path, and use neural networks to obtain the global representation of user interaction, as shown in Figure 3. Specifically, it first focuses on the node representation of the learning graph. For each behavior subgraph, it is embedded into the adjacency matrix $A_k$, which is composed of the matrix $R\_k$:

$$A_k=\left(\begin{array}{cc}0& R_k\\ {R_k}^T& 0\end{array}\right)$$

(7)

$$L_k=I_k-D_k^{-\frac{1}{2}}{A}_k{D}_k^{-\frac{1}{2}}$$

(8)

where $D_k^{-\frac{1}{2}}$ represents the degree matrix of k behavior, and $I_k$ represents the identity matrix of k behavior. $L_k$ is the normalized Laplacian matrix of k behavior. For the multi-behavior interaction graph of users, GCN can be used to better obtain the global representation of all users. We follow LightGCN:

$$x_{b_k}^{(l+1)}=sigmoid(L_k·x_{b_k}^{\left(l\right)}·W_k)$$

(9)

where $x_{b_k}^{\left(l\right)}$ is the node feature matrix of layer l of nodes in the figure, and $W_k$ is the transformation matrix of behavior graph information transfer. $Sigmoid$ is the activation $function$. GCN has a total of L layers, and l represents the order of neighbors we obtain. By obtaining the features of nodes with k behaviors in the graph. We design a multi-behavior interaction sequence attention encoder. The encoder constructs with the graph node representation under different behaviors. The influence coefficient of the corresponding auxiliary behavior graph with respect to the target behavior graph can be obtained. Therefore, we introduce a cross-behavior interaction graph attention encoder, auxiliary behavior graph $x_{b_k}$ and target behavior graph $x_{b_{k^{\prime }}}$ as input:

$$A_{k,k^{{}^{\prime }}}^g=\frac{{(W^Q·x_{b_k})}^T(W^K·x_{b_{k^{{}^{\prime }}}})}{{\sum }_{k=1}^K{(W^Q·x_{b_k})}^T(W^K·x_{b_{k^{{}^{\prime }}}})}$$

(10)

$${\alpha }_k^g=dropout\left(softmax\left(A_{k,k^{\prime }}^g\right)\right)$$

(11)

$$X_{b_k}={\alpha }_k^g·x_{b_k}·W^V$$

(12)

where $W^Q\in R^{d*n}$,$W^K\in R^{d*n}$,$W^V\in R^{d*n}$ is the weight matrix of the learnable behavior matrix. ${\alpha }_k^g$ g is the attention correlation coefficient, which is regarded as the weight added to the auxiliary behavior $x_{b_k}$. $A_{k,k^{\prime }}^g$ is calculated in the same way as we mentioned in section 3.2. By taking the Attention-Correlation matrix $A_{k,k^{\prime }}^g$ as the input, it is obtained by Multi-Classification. $X_{b_k}$ is the auxiliary behavior feature matrix about the target behavior as the final output of the cross-behavior interaction graph attention encoder.

2.3.3. Multi-Behavior Interaction Graph Contrast Loss(Graph Contrast Learning)

As we all known, the sparsity problem of user interaction behavior in the recommendation system was widespread[15]. There is also a long tail problem among behaviors. For example, users click too many times and make few purchases. Therefore, to alleviate this problem, in TP_MB we capture user interests by capturing complex dependencies among behaviors through unsupervised signals of multi-behavior interaction graphs, instead of utilizing a single behavior to capture user interests. Specifically, the output of our multi-behavior interaction graph attention encoder and the embedding of the graph of the target behavior are taken as input. That is to say, the embedding $X_k$ of the graph of the auxiliary behavior and the embedding $X_{k^{\prime }}$ of the graph of the target behavior are used as input. Contrastive learning is one of the method in self-supervised learning. Thus, features are learned by constructing positive and negative examples and establishing a contrastive loss. Therefore, intra-path multi-behavior interaction graph self-supervised learning is designed to enhance multiple behavioral data supervision signals through self-supervised learning. Graph self-supervised learning based on InfoNCE can effectively alleviate the long-tail distribution problem of different behaviors among users. In TP_MB, different behavior graph of the same user are regarded as positive sample pairs$\left\{\left(X_q^k,X_q^{k+1}\right)|q\in U\right\}$, and different behaviors of different users are regarded as negative sample pairs$\left\{\left(X_q^k,X_p^{k+1}\right)|q,p\in U,q\ne p\right\}$. Maximize mutual information between users by comparing positive and negative sample pairs:

$$L_{GCL}=\sum _{q\in U^{+},p\in U^{-}}-log\left(\frac{exp(X_q^k·X_q^{k+1})}{exp(X_q^k·X_q^{k+1})+(X_q^k·X_p^{k+1})}\right)$$

(13)

The consistency of two behavioral graph is maximized via the self-supervised loss defined above. And maximize the difference between different user behaviors to get behavior data enhancement.

2.4. Contrast Loss for Two-Path Enhancement

Since the problem of user interaction data sparsity has always existed, even the sequence self-supervision proposed in Section 3.2.4 and the graph self-supervision proposed in Section 3.3.3 can effectively alleviate this problem. However, using only these two self-supervision modalities will not perform better than combining the two supervision modalities to complement each other. Specifically, multi-behavior interactions between different multi-behavior paths is based on use InfoNCE loss. We regard the sequence path and graph path of the same user as positive samples, expressed by $\left\{\left(s_q^{b_k},X_q^{k^{\prime }}\right)|q\in U^{+}\right\}$. The sequence path and graph path of different users are regarded as negative samples expressed by $\left\{\left(s_q^{b_k},X_p^{k^{\prime }}\right)|q\in U^{+},p\in U^{-}\right\}$. Self-supervised Losses for InfoNCE -based sequences and Graphs $L_{SGCL}$:

$$L_{SGCL}=\sum _{q\in U^{+},p\in U^{-}}-log\frac{exp((s_q^{b_k}·X_q^{k+1})/\tau )}{exp((s_q^{b_k}·X_p^{k+1})/\tau )+exp((s_q^{b_k}·X_q^{k+1})/\tau )}$$

(14)

$$L_{CL}=L_{SCL}+L_{GCL}+L_{SGCL}$$

(15)

$\tau$ is the temperature coefficient that balances the intensity of learning between the two paths. $L_{SCL}$ is the sequence self-supervised learning of multi-behavior interactions within two paths we mentioned in Section 4.2.4. $L_{GCL}$ is the self-supervised learning of multi-behavior interaction graphs within two paths we mentioned in Section 4.3.3. Therefore, We propose the two contrastive learning losses as the supplement. The final loss function list $L_{CL}$ is composed of sequence loss function $L_{SCL}$, graph loss function $L_{GCL}$ and sequence graph loss function $L_{SGCL}$ for each pair of behaviors.

3. Results

We conduct experiments on two public datasets to verify the effectiveness of TP_MB. Our aim is to study the following questions:

• RQ1: Whether TP_MB can achieve better performance compared to the State-Of-The-Art model?
• RQ2: In our designed TP_MB, analyze the influence of each component on each other.
• RQ3: How different hyperparameters affect model performance?

3.1. Experimental Results and Analysis

Experiments on two real datasets in the real world demonstrate the effectiveness of our proposed TP_MB. And compare it with other baseline models and mainstream models. We evaluated TP_MB using a series of parameterization and ablation studies.

3.2. Datasets

We evaluate the effectiveness of our model on two different types of datasets, as shown in Table 1. Tmall: One of the largest E-Commerce platforms in China. This data set contains user interaction data: click, add to cart, purchase. We follow the method in [2] for data processing. CIKM2019 EComm AI: It is the E-Commerce data provided by CIKM2019 E-Commerce Artificial Intelligence Challenge. Each piece of data consists of a user, an item, and an interaction. Among them, user interaction includes (click, add to shopping cart, purchase), and various user interaction behaviors reflect the user’s intention for the item.

3.3. Implementation Details and Environment

We choose pytorch as the deep learning framework. The Adam optimizer is used to optimize the parameters during the training process, and the maximum sequence length is set to 100. The adjustment range of learning rate is $\left\{{0.6e}^{-3},{1e}^{-3},{2e}^{-3},{3e}^{-3}\right\}$. The size of each batch entered from the dataset is 256. The decay rate at each epoch in the training phase is ${0.1e}^{-4}$. The adjustment range of the number of layers for the information transmission of the graph is 1-4. In addition, due to the small number of behavior types, Over-Fitting problems are prone to occur during the experiment. We use dropout in the graph convolutional network to solve this problem. In our evaluation, four evaluation indicators are selected, which are hit rate (Hit), normalized loss cumulative gain (NDCG), mean reciprocal rank (MRR), and area under the metric (AUC). The first three are used to measure the accuracy of our recommendation model. The area under the metric (AUC) is a performance indicator to measure the pros and cons of the learner.

3.4. Baseline

In order to fully verify the validity of the model in this paper, we selected three different types of push line system models as the baseline models for comparative experiments. These include: Single-behavior recommender system approaches, sequential recommender methods, and Multi-behavior recommender system approaches.

Single-behavior Recommended Methods:

• LightGCN [9]:A simplified method of traditional GCN. This model removed the feature transformation and Non-Linear activation operations of the aggregation layer and propagation layer in the standard GCN, and only consisted of neighborhood aggregation, making it more concise and more suitable for recommendation systems.
• BERT4Rec [7]: To Modeled user behavior through Self-Attention mechanism, and fused project neighbor information for recommendation.

General Sequence Recommendations:

• Caser [9]:For the most recent project, graph convolution was embedded in time and space to model the dynamic change of user interest over time.

Multi-behavior Recommendation Methods:

• GHCF [2]:The model encoded the behavior of users interacting with products, and the behavior pattern was embedded into the overall heterogeneous graph as a node when composing the graph. At the same time, LightGCN was used to realize aggregation and propagation between nodes.
• DHSL-GM [10]:Similar users and similar items constituted hypergraphs respectively. The gated neural network was used to predict less popular items in a more balanced manner. It was the latest use of Self-Supervised learning to model the representation of users and products on the dual hypergraph. This model was a relatively advanced method at present, and was also the main comparison object of this paper.

3.5. Performance Comparison

In order to fully verify the validity of the model in this paper, we select three different types of recommendation system models as the baseline models for comparative experiments. These include: Single-behavior recommendation system approaches, sequential recommendation methods, and Multi-behavior recommendation system approaches. We summarize the detailed evaluation results of all methods on different dataset in Table 2, where TP_MB results are compared with baseline as well as State-Of-The-Art models. The results of the State-Of-The-Art model and TP_MB comparison are underlined and bold.

The main observations are two main reasons for TP_MBs outperforms other baselines on two datasets:

1) Multi-behavior recommendation methods (such as GHCF, DHSL-GM) perform better than Single-behavior recommendation methods (such as LightGCN, BERT4Rec, Caser). This demonstrates the effectiveness of modeling user preferences by exploring Multi-behavior information.

2) The improvement of TP_MB is due to the exploration of multi-behavior dependencies that are ignored by existing methods. This suggests that modeling different behavioral dependencies can make the model better. It is verified that our two-path comparison method gives the model the ability to deal with behavioral dependence. Unlike the Graph-Based Self-Supervised method GHCF. The two-path contrastive learning paradigm included in TP_MB incorporates auxiliary Self-Supervised signals from different types of behavioral dimensions.

3) Unlike the Graph-Based Self-Supervised method GHCF. The two-path contrastive learning paradigm included in TP_MB incorporates auxiliary Self-Supervised signals from different types of behavioral dimensions. The direction of parameter optimization is provided for the model.

These models not only do not extract the interactions between multiple behaviors but also none of the other models have parallel computation methods, except for DHSL_GM which is a dual hypergraph convolution model. And our model outperforms DHSL_GM because our two path parallel computational is in the form of sequence path and graph path contrast, and the possibility of overfitting is less than that of DHSL_GM. The superiority of our model can be demonstrated compared to these baseline models. As shown in Figure 4(a) and Figure 4(b), TP_MB outperforms the other models both on the Tmall and CIKM datasets.

3.6. Ablation Study

We performed ablation studies on the model to justify the key components. The analysis details are summarized as follows. In particular, we describe the model variables in the experiment:

• Remove path attention $w/o$-Att: Different modal relationships between multiple behaviors are not captured using attention in the two paths.
• Remove two-path Self-Supervision $w/o$-DCL: No two-path contrastive learning strategy is used to supplement sparse data.

Table 3 shows the results of TP_MB ablation study. We summarize the following findings:

1)TP_MB compared with w/o-Att. Combining interdependencies between different types of interactions on all items can greatly improve performance. It demonstrates the legitimacy of our Self-Attention structures for learning explicit paired relationships between different types of behavior.

2)The effect of TP_MB after the removal of two path Self-Supervision (w/o-DCL) decreased significantly. Thus illustrating the importance of considering dense contextual data across multiple behavior types when analyzing user preferences.

3.7. Hyper-Parameters Analysis

In this section of the experiment, we investigate the effects of different hyperparameter settings in our designed TP_MB framework. The experimental results are shown in Figure 5(a) and Figure 5(b). We study the effect of one hyperparameter at a time and keep the other parameters at their default settings.

$\beta$ hyperparameter. It controls the importance of comparing learning tasks and recommendation tasks between paths as shown in Figure 5(a). When the value of $\beta$ is low, the model pays less attention to the recommendation task. The model cannot obtain enough supervisory signals from the training data to update the parameters. With the increase of $\beta$, the performance of the model also improves. When $\beta$=0.3, the performance is the best. When $\beta$ is larger than a certain value ($\beta$ = 0.4), the performance of the model decreases slightly in both datasets.

Embedding dimension d. When the embedding dimension is 256 on the Tmall dataset, the dimension on the CIKM dataset is 128, The model has good performance as shown in Figure 5(b). When the embedding dimension is set too high, it will cause the tensor dimension to be too large. While further increasing the amount of calculation, it will also lead to non-discriminative characterization results. So the embedding dimension is also crucial. Experimental results show that this model can improve learning performance in a higher dimension. This shows that the model is capable of learning from User-Item multiple interactions.

4. Conclusion

In this paper, we propose a new Multi-Behavior Self-Supervised attention learning method based on two paths. It explicitly captures multiple behavioral dependencies. Due to the many types of interactions between users and items are easily overlooked. TP_MB designs a two-path multi-behavior dependent encoder to encode relational context signals from different types of user behaviors. In addition, we propose a framework for incorporating Self-Supervised learning. Multi-behavior model modeling in a customized way, capture different Multi-behavior modes of users for recommendation.