基於強化式學習之複合電力電動機車能量管理系統

Abstract

本論文針對鋰電池搭配超級電容發展一複合式能量管理系統，利用基本規則庫控制策略、強化式學習控制策略、Q學習全域搜索法控制策略以及全域搜索法控制策略，在不同需求功率和超級電容殘電量的情況下，計算出最佳功率分配比以達到最小化能量消耗。強化式學習之特點為可於系統運作過程中，不斷學習以找到最佳解，相較於一般演算法更具有靈活性，而在學習方法上本論文採用Q-Learning和Deep Q Network (DQN)兩種策略，其中DQN引用深度學習類神經網路的概念，以解決傳統Q學習在狀態維度上之缺點。此外本論文也將Q學習與全域搜索法結合，利用Q學習調整濾波參數以抑制全域搜索法之鋰電池輸出電流，達到保護電池之目的。實驗平台以電源端、數位訊號處理器、雙向直流-直流轉換器和電子負載組成，其中負載端選用新歐洲地區行車型態 (NEDC) 與全球機車測試型態 (WMTC) 作為測試依據，另外為了增加系統之輸出功率，直流轉換器以三台並聯並搭配均流技術，可確保轉換器模組間輸出電流一致，提高系統穩定度。同時為了使負載端更貼近真實情況，本論文也使用一直流無刷馬達，透過馬達和磁粉剎車搭配驅動器進行控制，於負載端模擬車輛之行駛狀態。結果比較中以基本規則庫控制策略為基準，分別探討於不同控制策略下之能耗改善幅度，由實驗結果可知，全域搜索法為最佳解，而所提出之強化式學習控制策略亦能有效的改善能耗。This study developed a Hybrid Energy Management System (HEMS) with a lithium battery-supercapacitor for an electric vehicle. Using Rule-Base (RB) control strategy, Reinforcement Learning (RL) control strategy, Q-learning Global Search Algorithm (QGSA), and Global Search Algorithm (GSA). The optimal power-split ratio is calculated to minimize energy consumption with different demand power and supercapacitor state-of-charge. Due to the characteristic of RL, it can continuously learn to find the best solution during the system's operation. Compared with the general algorithm, it is more flexible. In terms of learning methods, this thesis adopts two strategies, Q-Learning and DQN. DQN refers to the concept of deep learning neural network to solve the shortcomings of traditional Q learning in the state dimension. In addition, this thesis also combines Q-Learning and the Global Search Algorithm. It uses Q-Learning to adjust the filtering parameters to inhibit the lithium battery output current of the Global Search Algorithm to protect the battery.The experiment platform consists of a power supply, a digital signal processor, a bidirectional DC-DC converter, and an electronic load. The load terminal uses New European Driving Cycle (NEDC) and Worldwide Motorcycle Test Cycle (WMTC) as the demand power. In addition, to increase the system power output, three DC converters are connected in parallel and matched with current sharing technology to ensure consistent output current between converter modules and improve system stability. At the same time, to make the load side closer to the actual situation, this thesis also uses a DC motor for performing Motor and Brake control with the driver to simulate the driving cycle of the vehicle on the load side.In the experiment results, the RB control strategy is used as the benchmark to discuss energy consumption improvement under different control strategies. The experimental results show that the GSA is the best solution, and the proposed RL control strategy can also be effective in improving energy consumption.