Submitted:
24 October 2026
Posted:
26 October 2026
You are already at the latest version
Abstract
[Objective] Many comparative studies in recommender systems are evaluated on a single dataset and lack statistical validation against published baselines. This study asks whether model rankings and the contribution of content features transfer across domains.[Methods] We implement six recommenders of increasing complexity (UserCF, ItemCF, SVD, NeuMF, LightGCN and MultiModalNCF), reproduce two published baselines (BPR-MF and VBPR) under the same leave-last-5-out protocol, and evaluate them on MovieLens 100K and an Amazon Video_Games slice. Neural models are reported as five-seed mean±standard deviation, together with modality-ablation and cold-start studies.[Results] The ladder ordering (UserCF < ItemCF < SVD) and the neural band's lead over SVD hold on both domains; MultiModalNCF is statistically tied with VBPR. Removing all content costs about one seed standard deviation on MovieLens and nothing measurable on Amazon, whereas in the cold-start study the content model clearly outranks the content-blind model.[Limitations] Conclusions rest on a single data split without formal significance testing, epoch budgets are not equalised across models, and the Amazon slice covers heavy users only.[Conclusions] A simple concatenation-based multi-modal extension of NeuMF is competitive with a published content-aware baseline, and content features matter mainly for cold-start rather than for overall ranking quality. The full codebase and evaluation scripts are released for reproduction.
Keywords:
recommender systems
; multi-modal recommendation
; cold-start
; cross-dataset valida- tion
; reproducibility
Copyright: This open access article is published under a Creative Commons CC BY 4.0 license, which permit the free download, distribution, and reuse, provided that the author and preprint are cited in any reuse.