Abstract:

[Objective] To address the shortcomings of differential evolution (DE) algorithms in cascade reservoir optimization, this study proposes an intelligent algorithm that couples reinforcement learning and differential evolution (RLDE). [Methods] The RLDE algorithm improved the standard DE algorithm through three key strategies: chaotic mapping to enhance initial solution quality, Q-learning-based adaptive parameter adjustment, and a variable step-size strategy. Specifically, (1) chaotic mapping enhanced the initial solution quality. Logistic mapping with the best experimental performance was selected and applied to the population initialization of the RLDE algorithm. (2) The adaptive parameter adjustment was conducted based on the Q-learning algorithm. (3) A variable step-size strategy was designed for the actions in the Q-table, where the precision of action rows gradually increased with the number of iterations. To validate the feasibility and effectiveness of the RLDE algorithm, it was applied to optimize the power generation scheduling model for four major cascade reservoirs (Wudongde, Baihetan, Xiluodu, and Xiangjiaba) on the lower Jinsha River. [Results] (1) The chaotic initialization strategy effectively improved the initial solution quality. The adaptive parameter adjustment strategy based on the Q-learning algorithm enabled the algorithm to continuously adapt by receiving feedback from the environment. This process enhanced population diversity, greatly mitigated problems such as premature convergence or population evolutionary stagnation found in the traditional DE algorithm, thereby improving optimization performance. The variable step-size strategy allowed the algorithm to better respond to environmental feedback, further strengthening the optimization capability of the algorithm. (2) Compared with the traditional DE algorithm and adaptive genetic algorithm, the RLDE algorithm achieved an average annual power generation increase of 2.02% and 2.06%, respectively, under three typical inflow scenarios (wet, normal, and dry). Moreover, the average standard deviation of the proposed algorithm after multiple runs was reduced by an average of 729 million kW·h compared with the traditional DE algorithm, and by 844 million kW·h compared with the adaptive genetic algorithm. [Conclusions] This study proposes an intelligent algorithm that integrates reinforcement learning with differential evolution, effectively addressing issues such as premature convergence and search stagnation in the traditional DE algorithm. The proposed method provides an efficient and reliable solution for the optimal scheduling of cascade reservoirs.

Key words: cascade reservoirs, optimal scheduling, differential evolution, reinforcement learning, adaptive parameter adjustment