Endogenous Exploration in Reinforcement Learning with Intrinsic Curiosity

We propose a reinforcement learning framework in which exploration is driven by intrinsic curiosity,designed for scenarios where environments are non-stationary and rewards are sparse, delayed, unin-formative, or absent. In our model, action selection is guided by a combination of external rewardsand an epistemic motivation mechanism that biases the agent toward structured exploratory directions.The central hypothesis is that effective exploration emerges at intermediate levels of incoherence, whileperformance degrades under both overly rigid and overly disordered dynamics. To test this idea, weimplement the framework on top of a Liquid State Machine (LSM) substrate and evaluate it on twostandard benchmarks—the discrete-action LunarLander-v2 and the continuous-control BipedalWalker-v3. The proposed method achieves competitive performance on both tasks relative to established deepRL algorithms, including Proximal Policy Optimization (PPO) and Intrinsic Curiosity Module (ICM).We further show that the curiosity window is not recovered in Active Inference agents under the sameanalysis, suggesting that the proposed dynamics capture a distinct exploration regime.