A Hybrid LLM and Graph-Enhanced Transformer Framework for Cold-Start Session-Based Fashion Recommendation

1. Introduction

Session-based recommendation is important in areas where content changes quickly and user interactions are short. But standard models often fail in cold-start settings and in capturing detailed and changing user needs. This is more difficult when the products include rich and structured data. Models like RNNs and Transformers can learn the order of actions, but they do not understand meaning well. Graph-based models help with learning item transitions, but they need a lot of user interaction data.

Large language models such as LLaMA-2-7B add new tools to this field. They can use prompts and provide semantic understanding of items. But using LLMs alone does not work well, because they do not model time and user sessions clearly. To fix this, we present RecAgent-LLaMA. It combines language understanding with structure-aware methods. The system includes a prompt-based generator to get candidates, a session encoder built from Transformer-XL and GAT, a module inspired by the Deep Interest Network for fine-grained attention, and a cross-attention reranker. With a combined training method and a special data preprocessing step, RecAgent-LLaMA learns both what the items mean and how user behavior changes. This makes it useful for real-time and cold-start recommendation tasks.

2. Related Work

The use of large language models (LLMs) in recommendation has led to many new ideas. Wu et al. [1] showed how LLMs can help with meaning and generalization. Li et al. [2,3] discussed how LLMs are used in generative and conversational settings. But they also pointed out problems such as low control and unclear structures. These studies show that LLMs are useful, but they also need to be better connected to user modeling and system design.

In conversational recommendation, Zhang et al. [4] found that many systems with multiple turns cannot give stable and helpful suggestions. Yang et al. [5] suggested using user behavior alignment for evaluation and showed that LLM-based models still do not match real user interests well.

Sequential and structure-based modeling is also important. He et al. [6] created GAT4Rec, which uses graph attention and recurrent units to learn item transitions, but it does not focus on meaning. Li et al. [7] and Wang et al. [8] added personalization and prompt-based interaction, but they depend too much on rich user data, so they are not good for cold-start cases.

Work on Transformer explainability and dialogue modeling also helps. Ferrando et al. [9] explained how Transformers work, and Thoppilan et al. [10] built LaMDA for open dialogue. These works are useful, but they are not made for recommendation. In comparison, RecAgent-LLaMA brings together LLM-based search, long-range session modeling, and interest-based reranking. This gives a more flexible and complete solution for recommendation systems.

3. Methodology

RecAgent-LLaMA is a multi-stage framework for session-based fashion recommendation, designed to address cold-start conditions, evolving user intent, and rich item metadata. Its components are:

1)

LLaMA-based Semantic Retriever for candidate generation;

2)

Transformer-XL Long-Sequence Encoder with graph-enhanced item transitions;

3)

Fine-Grained Interest Module inspired by Deep Interest Network (DIN);

4)

Multi-Head Cross-Attention Reranker.

The model employs domain-specific prompt construction, multi-level fusion of item features and identifiers, and a hybrid ranking objective combining Bayesian Personalized Ranking (BPR) and focal losses. Experimental results show that RecAgent-LLaMA outperforms strong baselines in Recall@20 and NDCG@20, demonstrating its scalability in dynamic, cold-start environments.

3.1. Semantic Candidate Generation with LLaMA-2-7B

The frozen LLaMA-2-7B model serves as a domain-aware semantic retriever. Each item i is converted into a structured prompt:

$$\mathrm{Prompt}\left(i\right)="\mathrm{A}\mathrm{dress}\mathrm{with}\left[\mathrm{color}\right],\left[\mathrm{neckline}\right],[\mathrm{sleeve}\mathrm{length}]"$$

(1)

These prompts are processed by LLaMA and mean-pooled:

$${\mathbf{z}}_i={\mathrm{LLaMA}}_{\mathrm{mean}}\left(\mathrm{Prompt}\left(i\right)\right),{\mathbf{z}}_S={\mathrm{LLaMA}}_{\mathrm{mean}}\left(\mathrm{Prompt}\left(S\right)\right)$$

(2)

A task-specific projection adapts item embeddings:

$${\mathbf{z}}_i^{\mathrm{proj}}=W_z{\mathbf{z}}_i+b_z$$

(3)

Cosine similarity identifies the top-K candidates:

$${\mathcal{C}}_S=\mathrm{TopK}\left(\frac{{\mathbf{z}}_S^{\top }{\mathbf{z}}_i^{\mathrm{proj}}}{\parallel {\mathbf{z}}_S\parallel \parallel {\mathbf{z}}_i^{\mathrm{proj}}\parallel }\right)$$

(4)

Figure 1 depicts the retrieval pipeline.

3.2. Long-Sequence Session Encoder

Session behaviors are encoded by fusing sequential dynamics and graph structure (Figure 2). Each item is first mapped to

$${\mathbf{x}}_i={\mathrm{MLP}}_{\mathrm{fuse}}\left([{\mathbf{e}}_i;{\mathbf{f}}_i]\right)\in {\mathbb{R}}^{256},$$

where ${\mathbf{e}}_i\in {\mathbb{R}}^{128}$ and ${\mathbf{f}}_i\in {\mathbb{R}}^{64}$, and MLP_fuse has layers 128→256→256 with ReLU activations. These representations pass through a 4-layer Transformer-XL (hidden size 256, memory length 32, dropout 0.1) to produce ${\mathbf{h}}_i$. A directed session graph $G_S$ is built over consecutive items, and each node is refined by a residual GAT:

$${\mathbf{g}}_i=\mathrm{GAT}({\mathbf{h}}_i,\mathcal{N}\left(i\right))+{\mathbf{h}}_i,$$

with two GAT layers, four attention heads, hidden dimension 128, and LeakyReLU(0.2). Finally, we obtain the session embedding via soft-attention pooling:

$${\beta }_i={\displaystyle \frac{exp\left({\mathbf{w}}^{\top }{\mathbf{g}}_i\right)}{{\sum }_{j\in S}exp\left({\mathbf{w}}^{\top }{\mathbf{g}}_j\right)}},{\mathbf{s}}_G=\sum _{i\in S}{\beta }_i{\mathbf{g}}_i.$$

3.3. Fine-Grained Interest Modeling with DIN

We further refine ranking precision via a localized interaction mechanism using Deep Interest Network (DIN). For each candidate c, we compute personalized attention across the session graph nodes:

$${\alpha }_i^{\left(c\right)}={\mathrm{MLP}}_{\mathrm{DIN}}\left([{\mathbf{g}}_i;\mathbf{c};{\mathbf{g}}_i-\mathbf{c};{\mathbf{g}}_i\odot \mathbf{c}]\right)$$

(5)

where:

• $\mathbf{c}\in {\mathbb{R}}^{256}$: candidate item embedding
• MLP_DIN = [256 → 128 → 1]

The interest-aware context vector for each candidate is:

$${\mathbf{s}}_c=\sum _{i\in S}{\alpha }_i^{\left(c\right)}{\mathbf{g}}_i$$

(6)

This adaptive mechanism enables modeling of diverse user intents, with the ability to highlight relevant sub-sequences depending on each candidate.

3.4. Multi-Head Cross-Attention Reranking

Finally, we combine session-level interest ${\mathbf{s}}_c$ and candidate representation $\mathbf{c}$ using multi-head attention:

$${\mathbf{o}}_c=\mathrm{MultiHeadAttn}({\mathbf{s}}_c,\mathbf{c})$$

(7)

We use 4 attention heads, each projecting into 64 dimensions, followed by residual and layer norm layers. The final prediction is computed via sigmoid scoring:

$${\widehat{y}}_c=\sigma ({\mathbf{W}}_o^{\top }{\mathbf{o}}_c+b)$$

(8)

This cross-attention mechanism ensures that the model captures nuanced user-item alignment, benefiting from both long-term structure and real-time preference signals.Figure 3 illustrates the pipline of din and reranking.

3.5. Loss Function

The training objective balances pairwise ranking and calibration:

$$\mathcal{L}={\mathcal{L}}_{\mathrm{BPR}}+\alpha {\mathcal{L}}_{\mathrm{Focal}},\alpha =1.$$

(9)

The pairwise BPR loss

$${\mathcal{L}}_{\mathrm{BPR}}=-{\mathbb{E}}_{(c^{+},c^{-})\sim {\mathcal{C}}_S}\left[log\sigma ({\widehat{y}}_{c^{+}}-{\widehat{y}}_{c^{-}})\right]$$

(10)

ensures positive items score above negatives. The focal loss

$${\mathcal{L}}_{\mathrm{Focal}}=-{\mathbb{E}}_{c^{+}\sim {\mathcal{C}}_S}\left[{(1-{\widehat{y}}_{c^{+}})}^{\gamma }log\left({\widehat{y}}_{c^{+}}\right)\right],\gamma =2$$

(11)

emphasizes hard positives by down-weighting well-classified ones.

3.5.1. Final Loss Aggregation

We combine both objectives with a regularization term:

$${\mathcal{L}}_{\mathrm{total}}={\lambda }_1{\mathcal{L}}_{\mathrm{BPR}}+{\lambda }_2{\mathcal{L}}_{\mathrm{Focal}}+{\lambda }_3{\parallel \Theta \parallel }_2^2$$

(12)

With ${\lambda }_1=0.6$, ${\lambda }_2=0.35$, and ${\lambda }_3=0.05$, this hybrid formulation enhances both ranking order and score discrimination. Figure 4 presents a comprehensive analysis of our hybrid loss function’s training dynamics.

3.6. Data Preprocessing

Raw fashion sessions undergo the following steps:

Feature Fusion for Item Encoding

Each item i is represented by:

• Learnable ID embedding ${\mathbf{e}}_i\in {\mathbb{R}}^{128}$
• One-hot categorical feature ${\mathbf{f}}_i\in {\mathbb{R}}^{64}$

• These are combined via a two-layer MLP:

  $${\mathbf{x}}_i=\mathrm{ReLU}\left(W_2\mathrm{ReLU}(W_1[{\mathbf{e}}_i;{\mathbf{f}}_i]+b_1)+b_2\right)$$

  (13)

Prompt Construction for LLaMA Embedding

Item metadata are converted into structured prompts:

$$\mathrm{Prompt}\left(i\right)="\mathrm{A}\mathrm{dress}\mathrm{with}\left[\mathrm{color}\right],\left[\mathrm{neckline}\right],\left[\mathrm{sleeve}\right]"$$

(14)

These prompts are encoded by the frozen LLaMA-2 model:

$${\mathbf{z}}_i={\mathrm{LLaMA}}_{\mathrm{mean}}\left(\mathrm{Prompt}\left(i\right)\right)$$

(15)

Temporal Splitting and Hard Negative Sampling

Sessions are partitioned chronologically into train (80%), validation (10%), and test (10%). For each session, the candidate set ${\mathcal{C}}_S$ includes one positive and 49 hard negatives sampled from the same category:

$${\mathcal{C}}_S=\left\{c^{+}\right\}\cup \{c_1^{-},\dots ,c_{49}^{-}\}$$

(16)

Figure 5 illustrates the preprocessing pipeline.

3.7. Experiment Results

We compare the performance of our proposed model, RecAgent-LLaMA, with several baselines and ablated variants. All experiments are conducted on the cleaned Dressipi dataset with consistent data splits. The main results are shown in Table 1. Figure 6 presents the training dynamics of our RecAgent-LLaMA model across 100 epochs, demonstrating consistent convergence across all evaluation metrics.

3.7.1. Ablation Study

To assess the contribution of each module, we conduct ablation experiments by selectively removing components from RecAgent-LLaMA:

As shown in Table 2, removing any component of the model leads to a noticeable drop in performance, confirming that each module—LLaMA retrieval, GAT-based session encoding, and DIN-style interest matching—contributes positively to the final performance.

4. Conclusion

We present RecAgent-LLaMA, a large language model-enhanced framework tailored for session-based fashion recommendation. By combining LLaMA-driven semantic retrieval, Transformer-XL long-range modeling, GAT-enhanced session encoding, and DIN-style interest reranking, our model achieves state-of-the-art results on the RecSys 2022 dataset. Extensive experiments and ablation studies validate the effectiveness and complementarity of each module. This work demonstrates the potential of integrating LLMs with structured recommender system architectures in real-world e-commerce applications.