Self-Evolving Recommender Systems: A Survey and an Evaluation Audit

In 2026 a new kind of recommender research appeared: large language model (LLM) agents that propose changes to a recommender's configuration or source code, train the changed model, and keep what the evaluation says is better. Within ten months, more than thirty such systems were reported, from open-source research harnesses to production loops at Google, Meta, Amazon, Alibaba, Kuaishou, Tencent and NetEase. This survey organises them, together with search-space AutoML and the agentic recommenders that update their own memory or weights, along two axes: what the loop changes and which signal decides whether a change survives. Because a self-evolving system is only as good as the signal it selects on, we then audit how 27 configuration- and code-level recommender systems are evaluated, coding each paper's full text against nine criteria. Three of the 27 evaluate with full-catalogue ranking. Of the 17 that select candidates offline, nine do not say which data split makes the decision, two select on the data they report, and three state that the test set was used once. None reports its headline result with the spread over repeated evolution sessions, and none runs an equal-budget random search over its own edit space; the few studies that measure transfer to held-out data find that part of the selected gain, sometimes most of it, does not survive. Four systems report that their agent inflated the metric or leaked the target, caught by a human or by monitoring rather than by the acceptance rule. We distil these findings into a reporting standard and describe recforge, an open benchmark under development that implements the API side of a sealed-test, full-ranking, equal-budget protocol.