Multi-objective optimization of best management practices in agricultural watersheds using single-step deep reinforcement learning

Optimal placement and intensity of best management practices (BMPs) in watershed management needs balancing pollutant reduction with implementation cost. We proposed a deep reinforcement learning (DRL) framework for cost-aware, SWAT-based BMP implementation, applied to the Namyang estuarine reservoir watershed in South Korea. The framework jointly optimized continuous application intensities of four structural BMPs, vegetative filter strip (VFS), grassed waterway (GWW), terrace, and detention pond (DTP), across 31 subbasins in a single-step optimization environment. Three DRL algorithms and NSGA-Ⅱ were compared under an identical budget of 1,000 SWAT simulations. Among the algorithms, proximal policy optimization (PPO) achieved the highest final reward, approximately 66% and 68% higher than those of TD3 and SAC, respectively. The best PPO solution achieved 92.4% of the maximum attainable combined reduction of suspended sediments (10.2%) and total phosphorus (82.2%) at a cost of $5.02 million, corresponding to 9.6% of the full implementation cost, whereas NSGA-Ⅱ required $14.62 million to reach the same reduction level. Consequently, PPO was approximately 2.9 times more cost-efficient than NSGA-Ⅱ under the identical evaluation budget. Additionally, the learned PPO policy showed a spatially differentiated pattern. The VFS was broadly applied across the watershed, GWW was selectively combined with VFS in steeper subbasins, and DTP was concentrated in a single key subbasin. Thus, PPO-based DRL provides a cost-efficient and interpretable alternative to conventional multi-objective evolutionary optimization for direct SWAT-in-the-loop applications.