From Trajectories to Experience: A Survey of Experience-Driven LLM Agents

Large language model (LLM) agents can plan, use tools, and recover from errors within a task, yet many carry little from one task to the next and repeat mistakes they have already resolved. The root cause is a lack of experience accumulation: trajectories record what failed and how the agent recovered, but this information is seldom turned into guidance for later tasks. Research on this capability spans reflection, agent memory, workflow learning, and skill acquisition, but these lines describe similar content in different terms and lack a common account of experience itself, leaving basic questions open: what counts as experience, how it should be accumulated, used, and revised, and how to tell whether it improves later behavior. This survey addresses these questions with a unified framework. We define agent experience as action-guiding information grounded in task trajectories and retained to guide later episodes where it applies, whether as a trajectory precedent, a semantic lesson, or a reusable procedure. We focus on explicitly accessible experience, including its use as a training signal for internalization into model parameters. We describe each unit of experience by its guidance, applicability conditions, execution evidence, and reliability evaluation, so that memory entries, workflows, and skills become directly comparable, and follow experience through four lifecycle functions---formation, organization, utilization, and maintenance---that determine how it is accumulated, used, and revised. To judge reported gains, we distinguish three improvement claims---fixed experience reuse, feedback-driven sustained improvement, and improvement of the learning mechanism---by the evidence each requires. Current studies most clearly support fixed experience reuse, chiefly on tasks with tests or reference answers; evidence for the other two claims remains partial. Our analysis yields recommendations for seven application settings and a central design principle: keep guidance connected to its applicability conditions and to the evidence that supports or challenges it. Finally, we outline eight open problems, from learning with human expert records to testing whether gains accumulate, toward agents whose improvement is cumulative, transferable, and correctable.