A Comparative Evaluation of Value-Based and Policy-Gradient Deep Reinforcement Learning Algorithms: Benchmarking on LunarLander-v2 and a Custom Partially Observable Drone-Rescue Environment

Deep Reinforcement Learning (DRL) algorithms are frequently compared on standard bench mark environments, yet the practical implications of algorithmic choice for engineered, par tially observable tasks are less often examined side by side with such benchmarks. In this work we present a two-part empirical study. First, we implement ve DRL algorithms en tirely from scratch in PyTorch Deep Q-Network (DQN), Double DQN, REINFORCE, Proximal Policy Optimization (PPO), and Advantage Actor-Critic (A2C) and evaluate them under matched conditions on the LunarLander-v2 control task from the Gymnasium suite. We report learning curves, sample e ciency, stability, and nal policy quality, and nd that the value-based methods (DQN, Double DQN) solve the environment reliably (in 461 and 798 episodes respectively, reaching average scores above 200), whereas the on-policy methods REINFORCE, PPO, and A2C fail to solve the task within their allotted training budgets despite requiring substantially more environment interactions. Second, we design and evaluate Snow Drone Rescue, a custom, partially observable, stochastic environment built in pygame that models an autonomous drone performing a multi-objective search-and rescue mission under battery constraints and adverse weather. A DQN agent trained with a dense-plus-sparse shaped reward learns energy-aware navigation and obstacle avoidance despite receiving only a six-dimensional partial observation. Taken together, the two stud ies indicate that, for discrete-action tasks with reasonably dense reward signals, algorithmic stability mechanisms (experience replay, target networks) matter more for nal performance than the choice between value-based and policy-gradient families, while reward and envi ronment design remain at least as consequential as the learning algorithm itself. We discuss the theoretical basis for these observations, algorithmic and environment-design trade-o s, and directions for future work, including entropy-regularized policy optimization, systematic hyperparameter search, and extension to continuous-control and multi-agent settings.