Optimal Scheduling Method for Power Generation of Cascade Reservoirs Based on RLDE Algorithm

CHEN Jia-wen, ZHU Xin, TANG Zheng-yang, SHEN Ke-yan, CHEN Xiao-lin, QIN Hui

Journal of Changjiang River Scientific Research Institute ›› 2025, Vol. 42 ›› Issue (6) : 210-218.

PDF(7048 KB)
PDF(7048 KB)
Journal of Changjiang River Scientific Research Institute ›› 2025, Vol. 42 ›› Issue (6) : 210-218. DOI: 10.11988/ckyyb.20240431
Multi-Objective Optimization Scheduling for Reservoir Groups

Optimal Scheduling Method for Power Generation of Cascade Reservoirs Based on RLDE Algorithm

Author information +
History +

Abstract

[Objective] To address the shortcomings of differential evolution (DE) algorithms in cascade reservoir optimization, this study proposes an intelligent algorithm that couples reinforcement learning and differential evolution (RLDE). [Methods] The RLDE algorithm improved the standard DE algorithm through three key strategies: chaotic mapping to enhance initial solution quality, Q-learning-based adaptive parameter adjustment, and a variable step-size strategy. Specifically, (1) chaotic mapping enhanced the initial solution quality. Logistic mapping with the best experimental performance was selected and applied to the population initialization of the RLDE algorithm. (2) The adaptive parameter adjustment was conducted based on the Q-learning algorithm. (3) A variable step-size strategy was designed for the actions in the Q-table, where the precision of action rows gradually increased with the number of iterations. To validate the feasibility and effectiveness of the RLDE algorithm, it was applied to optimize the power generation scheduling model for four major cascade reservoirs (Wudongde, Baihetan, Xiluodu, and Xiangjiaba) on the lower Jinsha River. [Results] (1) The chaotic initialization strategy effectively improved the initial solution quality. The adaptive parameter adjustment strategy based on the Q-learning algorithm enabled the algorithm to continuously adapt by receiving feedback from the environment. This process enhanced population diversity, greatly mitigated problems such as premature convergence or population evolutionary stagnation found in the traditional DE algorithm, thereby improving optimization performance. The variable step-size strategy allowed the algorithm to better respond to environmental feedback, further strengthening the optimization capability of the algorithm. (2) Compared with the traditional DE algorithm and adaptive genetic algorithm, the RLDE algorithm achieved an average annual power generation increase of 2.02% and 2.06%, respectively, under three typical inflow scenarios (wet, normal, and dry). Moreover, the average standard deviation of the proposed algorithm after multiple runs was reduced by an average of 729 million kW·h compared with the traditional DE algorithm, and by 844 million kW·h compared with the adaptive genetic algorithm. [Conclusions] This study proposes an intelligent algorithm that integrates reinforcement learning with differential evolution, effectively addressing issues such as premature convergence and search stagnation in the traditional DE algorithm. The proposed method provides an efficient and reliable solution for the optimal scheduling of cascade reservoirs.

Key words

cascade reservoirs / optimal scheduling / differential evolution / reinforcement learning / adaptive parameter adjustment

Cite this article

Download Citations
CHEN Jia-wen , ZHU Xin , TANG Zheng-yang , et al . Optimal Scheduling Method for Power Generation of Cascade Reservoirs Based on RLDE Algorithm[J]. Journal of Changjiang River Scientific Research Institute. 2025, 42(6): 210-218 https://doi.org/10.11988/ckyyb.20240431

References

[1]
沈柯言. 考虑径流不确定性的梯级水库中长期优化调度及评价决策研究[D]. 武汉: 华中科技大学, 2022.
SHEN Ke-yan. Research on Medium-and Long-term optimal Dispatch and Evaluation Decisions of Cascaded Reservoirs Considering Runoff Uncertainty[D]. Wuhan: Huazhong University of Science and Technology, 2022. (in Chinese)
[2]
马宇航, 黄媛, 刘俊勇, 等. 考虑日内来水不确定和电网断面约束的梯级水电日前调度[J]. 电力建设, 2020, 41(9):39-49.
Abstract
日内来水的不确定性和传输通道断面约束是影响大容量梯级水电日前调度计划制定的重要因素。为提高日前调度对不同来水情景的适应性,首先在日均径流预测基础上,结合历史数据分析与概率分布拟合获取日内来水量的概率密度函数,通过准确刻画各时段来水规律,建立不同情景下的梯级发电量偏差来反映各时刻来水的不确定性。其次,在模型断面约束中引入模糊变量,并通过离线仿真将断面约束转化为发电机有功出力约束。针对含梯级电站发电量偏差的日前优化调度模型,采用多核并行动态规划算法进行求解,保证求解的收敛性及求解效率。最后,以西南地区某梯级水电站接入IEEE 39节点系统为例,验证了所提算法的合理性和可行性,所得调度计划在4种典型来水场景下均未出现水位及断面功率越限。
(MA Yu-hang, HUANG Yuan, LIU Jun-yong, et al. Day-ahead Dispatching of Cascade Hydropower Stations Considering Daily Interval Flow Uncertainty and the Section Constraint of Power Grid[J]. Electric Power Construction, 2020, 41(9):39-49. (in Chinese))

The uncertainties of daily interval flow and the restriction of the transmission section are important factors that affect the day-ahead dispatching plan of large-capacity cascade hydropower stations. In order to improve the adaptability of the daily dispatching to different inflow scenarios, the probability density function of daily interval flow is obtained on the basis of the forecast daily average interval flow firstly, combining historical data analysis and probability distribution fitting. Through accurately describing the rule of interval flow in each period, the deviation of cascade power generation under different scenarios is established to reflect the uncertainty of the interval flow at each period. Secondly, fuzzy variables are established to express the complex section constraint relationship in the day-ahead dispatching, and the section constraint is converted to the generator’s active power constraint through off-line simulation. The multi-core parallel dynamic programming algorithm is adopted to solve the day-ahead dispatching model to ensure the convergence and efficiency of the solution. Finally, the rationality and feasibility of the proposed algorithm are verified through the example of cascade hydropower stations in southwest China connected to IEEE 39-node system, and the water level and transmission section power are not out of limits in the four typical inflow scenarios.

[3]
陶湘明. 水库中长期发电优化调度的解析方法及应用[D]. 武汉: 武汉大学, 2017.
(TAO Xiang-ming. Derivation and Application of Analytic Method for Reservoir Mid-Long Term Optimal Operation[D]. Wuhan: Wuhan University, 2017. (in Chinese))
[4]
廖灵明, 钟芳仁. 水库优化调度方法研究进展[J]. 机电技术, 2022, 45(6): 33-35, 45.
(LIAO Ling-ming, ZHONG Fang-ren. Research Progress of Reservoir Optimal Operation Method[J]. Mechanical & Electrical Technology, 2022, 45(6): 33-35, 45. (in Chinese))
[5]
郭生练, 陈炯宏, 刘攀, 等. 水库群联合优化调度研究进展与展望[J]. 水科学进展, 2010, 21(4):496-503.
(GUO Sheng-lian, CHEN Jiong-hong, LIU Pan, et al. State-of-the-art Review of Joint Operation for Multi-reservoir Systems[J]. Advances in Water Science, 2010, 21(4): 496-503. (in Chinese))
[6]
张琪, 任明磊, 王凯, 等. 基于改进遗传算法的水库群防洪联合优化调度研究及其应用[J]. 中国防汛抗旱, 2022, 32(6): 21-26.
(ZHANG Qi, REN Ming-lei, WANG Kai, et al. Research and Application of Joint Optimal Operation for Flood Defense of Reservoir Group Based on Improved Genetic Algorithm[J]. China Flood & Drought Management, 2022, 32(6): 21-26. (in Chinese))
[7]
丁青锋, 尹晓宇. 差分进化算法综述[J]. 智能系统学报, 2017, 12(4): 431-442.
(DING Qing-feng, YIN Xiao-yu. Research Survey of Differential Evolution Algorithms[J]. CAAI Transactions on Intelligent Systems, 2017, 12(4): 431-442. (in Chinese))
[8]
刘孟桦, 何中政, 盛卫荣, 等. 基于改进差分进化算法的万安水库防洪优化调度[J]. 水电能源科学, 2023, 41(6): 54-58.
(LIU Meng-hua, HE Zhong-zheng, SHENG Wei-rong, et al. Optimal Operation of Flood Control in Wan’an Reservoir Based on Improved Differential Evolution Algorithm[J]. Water Resources and Power, 2023, 41(6): 54-58. (in Chinese))
[9]
王冠中, 王士军, 冉川东. 基于改进差分进化算法的自由曲面测量路径优化[J]. 制造技术与机床, 2024(3):51-56.
(WANG Guan-zhong, WANG Shi-jun, RAN Chuan-dong. Optimization of Free Surface Measurement Path Based on Improved Differential Evolution Algorithm[J]. Manufacturing Technology & Machine Tool, 2024(3): 51-56. (in Chinese))
[10]
邓先礼. 时滞对梯级水电站经济运行的影响[J]. 重庆大学学报(自然科学版), 1982, 5(2): 91-102.
(DENG Xian-li. On the Effect of Time Lag Upon the Economical Operation of Cascade Hydroelectric Power Stations[J]. Journal of Chongqing University (Natural Science), 1982, 5(2): 91-102. (in Chinese))
[11]
STORN R, PRICE K. Differential Evolution—A Simple and Efficient Heuristic for Global Optimization over Continuous Spaces[J]. Journal of Global Optimization, 1997, 11(4): 341-359.
[12]
郑肇葆. 混沌映射在优化计算中的应用[J]. 武汉大学学报(信息科学版), 2007, 32(11): 998-1000.
(ZHENG Zhao-bao. Application of Chaos Mappings in Optimization Calculation[J]. Geomatics and Information Science of Wuhan University, 2007, 32(11): 998-1000. (in Chinese))
[13]
ZHU C, LI S, Q. Pseudo-random Number Sequence Generator Based on Chaotic Logistic-Tent System. 2019, 547-551.
[14]
王素琴, 张洋, 蒋浩, 等. 针对新用户冷启动问题的改进Epsilon-greedy算法[J]. 计算机工程, 2018, 44(11):172-177.
Abstract
在解决新用户冷启动问题时,固定不变的Epsilon参数会使传统Epsilongreedy算法收敛缓慢。为此,提出一种改进的Epsilon-greedy算法。利用免疫反馈模型动态调整Epsilon参数,从而使算法快速收敛。使用蒙特卡罗模拟方法对算法进行实验验证,结果表明,该算法能够在用户与推荐系统交互较少的情况下为用户进行有效推荐,且推荐效果优于传统的Epsilon-greedy、Softmax和UCB算法。
(WANG Su-qin, ZHANG Yang, JIANG Hao, et al. Improved Epsilon-greedy Algorithm for Cold-start Problem of New Users[J]. Computer Engineering, 2018, 44(11): 172-177. (in Chinese))
When solving the cold-start problem of new users,fixed and invariant Epsilon parameters will slow the convergence of traditional Epsilon-greedy algorithm.Therefore,an improved Epsilon-greedy algorithm is proposed.Immune feedback model is used to dynamically adjust the Epsilon parameters so that the algorithm converges quickly.Monte Carlo simulation is used to validate the proposed algorithm.Results show that this algorithm can effectively recommend to users when they have little interaction with the recommendation system,and the recommendation effect is better than the traditional Epsilon-greedy algorithm,Softmax algorithm and UCB algorithm.
[15]
谢雨祚, 郭生练, 钟斯睿, 等. 金下-三峡-葛洲坝梯级水库联合优化调度方案比较研究[J]. 水力发电学报, 2024, 43(11):39-48.
(XIE Yu-zuo, GUO Sheng-lian, ZHONG Si-rui, et al. Comparative Study of Joint and Optimal Operation Schemes for Lower Jinsha-Three Gorges-Gezhouba Cascade Reservoirs[J]. Journal of Hydroelectric Engineering, 2024, 43(11): 39-48. (in Chinese))
[16]
陈进. 长江梯级水库群联合调度成效、挑战及对策[J]. 长江科学院院报, 2024, 41(5): 1-7.
Abstract
经过70 a的建设,长江流域控制性水库群基本建成。以三峡水库为核心的梯级水库群联合调度已经开展10 a。在总结控制性水库群在防洪、供水、生态和应急联合调度取得成绩的基础上,分析控制性水库群及与水网工程联合调度面临的问题和挑战,提出了相应的对策建议。研究表明:水库群联合调度因流域防洪调度而启,目前在防洪、抗旱供水、生态环境和应急等公益调度中发挥了重要作用。气候变化及水文中长期预报不确定性是联合调度面临的主要挑战,其次是水库群联合调度与单个水库多目标调度利益协调、定量评价调度效果及合理的补偿等管理问题。加强“四预”是解决气候变化和人类活动不确定性的主要对策。水库群与水网工程联合调度可以更广泛地发挥水利工程水安全保障作用。建立科学合理的水工程群联合调度体制和机制是解决联合调度管理问题的有效措施。
(CHEN Jin. Achievements, Challenges and Measures of Joint Operation of Cascade Reservoirs on the Yangtze River[J]. Journal of Changjiang River Scientific Research Institute, 2024, 41(5): 1-7. (in Chinese))
PDF(7048 KB)

Accesses

Citation

Detail

Sections
Recommended

/