SCENARIO-LEVEL HIERARCHICAL ENERGY MANAGEMENT CONTROL STRATEGY FOR HYBRID ELECTRIC VEHICLE

Feng Xiao, Ruiqiang Zhang, Chunyang Qi

DOI Number
10.22190 FUME250508031X
First page
Last page

Abstract


To address the challenge that single-scenario solutions often fail to adequately handle the complex and dynamic mixed environments that drivers regularly navigate, this study aims to leverage learning-based algorithms to design energy management control strategies tailored to the unique characteristics of different driving scenarios. The goal is to achieve precise matching and efficient execution of the driving strategies. The main research focus of this study is developing a scene-level hierarchical energy management control strategy (SHEMS) framework for hybrid electric vehicles in mixed driving environments. In the car-following scenario, to address the challenges of reward function design and the impact of environment and driver habits, an adaptive strategy learning strategy with imitation learning is proposed. To overcome issues of suboptimal expert knowledge and the curse of dimensionality, optimization factors are added. For the intersection scenario, aiming at the challenge of reward sparsity caused by the scarcity of traffic signals and safety incentives, an additional reward mechanism is innovatively proposed to enrich the reward function. The experimental results demonstrate that the SHEMS significantly reduces fuel consumption. Specifically, the PPO, A2C, A3C, and DQN-based strategies achieved 7.52%, 5.29%, 9.6%, and 5.93% reductions, respectively. Taking the DQN algorithm as an example, the emissions of harmful gases CO, HC, PM and NOx are reduced by 18.3%, 14.23%, 16.94%, and 20.9%, respectively, after layering.


Keywords

Hybrid electric vehicle, Mixed scenarios, Intersection scenarios, Reinforcement learning

Full Text:

PDF

References


Huang, Y., Ng, E., Zhou, J., Surawski, N., Lu, X., Du, B., Forehead, H., Perez, P., Chan, E., 2021, Impact of drivers on real-driving fuel consumption and emissions performance, Science of the Total Environment, 798, 149297.

Li, J., Fotouhi, A., Liu, Y., Zhang, Y.,Chen, Z., 2024, Review on eco-driving control for connected and automated vehicles, Renewable & Sustainable Energy Reviews, 189, 114025.

Ahmad, A., Al-Sumaiti, A., Byon, Y.,Al Hosani, K., 2024, Eco-Driving Framework for Autonomous Vehicles at Signalized Intersection in Mixed-Traffic Environment, IEEE Access, 12, pp. 85291-85305.

Yuan, W., Frey, H., 2020, Potential for metro rail energy savings and emissions reduction via eco-driving, Applied Energy, 268, 114944.

Wu, F., Ye, H., Bektas, T., Dong, M., 2025, New and tractable formulations for the eco-driving and the eco-routing-and-driving problems, European Journal of Operational Research, 321(2), pp. 445-461.

Yu, M., Long, J., 2022, An Eco-Driving Strategy for Partially Connected Automated Vehicles at a Signalized Intersection, IEEE Transactions on Intelligent Transportation Systems, 23(9), pp. 15780-15793.

Guo, Q., Angah, O., Liu, Z., Ban, X., 2021, Hybrid deep reinforcement learning based eco-driving for low-level connected and automated vehicles along signalized corridors, Transportation Research Part C-Emerging Technologies, 124, 102980.

He, W., Huang, Y., 2021, Real-time Energy Optimization of Hybrid Electric Vehicle in Connected Environment Based on Deep Reinforcement Learning, 6th IFAC Conference on Engine Powertrain Control, Simulation and Modeling (E-COSM), 54, pp. 176-181.

Tadić, D., Lukić, J., Komatina, N., Marinković, D., Pamučar, D., 2025, A Fuzzy Decision-Making Approach to Electric Vehicle Evaluation and Ranking, Tehnicki Vjesnik, 32(3), pp. 1066-1075.

Hu, J., Shao Y., Sun, Z., Wang M., Bared, J., Huang, P., 2016, Integrated optimal eco-driving on rolling terrain for hybrid electric vehicle with vehicle-infrastructure communication, Transportation Research: Part C, 68 pp. 228-244.

Liu, L., Neng, L., Peng, Z., Zhan, S., Gao, J., Wang, H., 2024, Modeling and Verification of Eco-Driving Evaluation, Journal of Information Processing Systems, 20(3), pp. 296-306.

Li, J., Fotouhi, A., Pan, W., Liu, Y., Zhang, Y., Chen, Z., 2023, Deep reinforcement learning-based eco-driving control for connected electric vehicles at signalized intersections considering traffic uncertainties, Energy, 279, 128139.

Bai, Z., Hao, P., Shangguan, W., Cai, B., Barth, M., 2022, Hybrid Reinforcement Learning-Based Eco-Driving Strategy for Connected and Automated Vehicles at Signalized Intersections, IEEE Transactions on Intelligent Transportation Systems, 23(9), pp. 15850-15863.

Xie, S., Hu, X., Liu, T., Qi, S., Lang, K., Li, H., 2019, Predictive vehicle-following power management for plug-in hybrid electric vehicles, Energy, 166, pp. 701-714.

Qi, C., Zhu, Y., Song, C., Yan, G., Wang, D., Xiao, F., Zhang, X., Cao, J., Song, S., 2022, Hierarchical reinforcement learning based energy management strategy for hybrid electric vehicle, Energy, 238, 121703.

Marinković, D., Dezső, G., Milojević, S., 2024, Application of machine learning during maintenance and exploitation of electric vehicles, Advanced Engineering Letters, 3(3), pp. 132-140.

Yeom, K., 2022, Model predictive control and deep reinforcement learning based energy efficient eco-driving for battery electric vehicles, Energy Reports, 8, pp. 34-42.

He, T., Liang, C., Zheng, C., Liu, Y., Zhang, Y., Hu, J., 2025, Multi-Objective Autonomous Eco-Driving Strategy: A Pathway to Future Green Mobility, Green Energy and Intelligent Transportation, 4(4), 100279.

Ozatay, E., Onori, S., Wollaeger, J., Ozguner, U., Rizzoni, G., Filev, D., Michelini, J., Di Cairano, S., 2014, Cloud-Based Velocity Profile Optimization for Everyday Driving: A Dynamic-Programming-Based Solution, IEEE Transactions on Intelligent Transportation Systems, 15(6), pp. 2491-2505.

Wang, S., Lin, X., 2020, Eco-driving control of connected and automated hybrid vehicles in mixed driving scenarios, Applied Energy, 271, 115233.

Guo, L., Zhang, X., Zou, Y., Han, L., Du, G., Guo, N., Xiang, C., 2022, Co-optimization strategy of unmanned hybrid electric tracked vehicle combining eco-driving and simultaneous energy management, Energy, 246, 123309.

Liu, Y., Huang, B., Yang, Y., Lei, Z., Zhang, Y., Chen, Z., 2022, Hierarchical speed planning and energy management for autonomous plug-in hybrid electric vehicle in vehicle-following environment, Energy, 260, 125212.

Peng, J., Zhang, F., Coskun, S., Hu, X., Yang, Y., Langari, R., He, J., 2023, Hierarchical Optimization of Speed Planning and Energy Management for Connected Hybrid Electric Vehicles Under Multi-Lane and Signal Lights Aware Scenarios, IEEE Transactions on Intelligent Transportation Systems, 24(12), pp. 14174-14188.

Li, J., Liu, Y., Zhang, Y., Lei, Z., Chen, Z., Li, G., 2021, Data-driven based eco-driving control for plug-in hybrid electric vehicles, Journal of Power Sources, 498, 229916.

Qiu, S., Qiu, L., Qian, L., Pierluigi, P., 2019, Hierarchical energy management control strategies for connected hybrid electric vehicles considering efficiencies feedback, Simulation Modelling Practice and Theory, 90, pp. 1-15.

Almannaa, M., Chen, H., Rakha, H., Loulizi, A., El-Shawarby, I., 2019, Field implementation and testing of an automated eco-cooperative adaptive cruise control system in the vicinity of signalized intersections, Transportation Research Part D-Transport and Environment, 67, pp. 244-262.

Zheng, Y., Guo, R., Ma, D., Zhao, Z., Li, X., 2020, A Novel Approach to Coordinating Green Wave System with Adaptation Evolutionary Strategy, IEEE Access, 8, pp. 214115-214127.

Chen, Z., Xiong, S., Chen, Q., Zhang, Y., Yu, J., Jiang, J., Wu, C., 2022, Eco-Driving: A Scientometric and Bibliometric Analysis, IEEE Transactions on Intelligent Transportation Systems, 23(12), pp. 1-21.

Ma, H., Xie, H., Brown, D., 2018, Eco-Driving Assistance System for a Manual Transmission Bus Based on Machine Learning, IEEE Transactions on Intelligent Transportation Systems, 19(2), pp. 572-581.

Chen, H., Rakha, H., Loulizi, A., El-Shawarby, I., Almannaa, M., 2016, Development and Preliminary Field Testing of an In-Vehicle Eco-Speed Control System in the Vicinity of Signalized Intersections, 14th IFAC Symposium on Control in Transportation Systems (CTS), 49(3), pp. 249-254.

Dong, H., Zhuang, W., Chen, B., Lu, Y., Liu, S., Xu, L., Pi, D., Yin, G., 2022, Predictive energy-efficient driving strategy design of connected electric vehicle among multiple signalized intersections, Transportation Research Part C-Emerging Technologies, 137, 103595.

Yao, H., Li, X., 2020, Decentralized control of connected automated vehicle trajectories in mixed traffic at an isolated signalized intersection, Transportation Research Part C-Emerging Technologies, 121, 102846.

Kebria, P., Khosravi, A., Salaken, S., Nahavandi, S., 2020, Deep imitation learning for autonomous vehicles based on convolutional neural networks, IEEE-Caa Journal of Automatica Sinica, 7(1), pp. 82-95.

Zhang, C., Bai, W., Du, X., Liu, W., Zhou, C., Qian, H., 2023, Survey of imitation learning：tradition and new advances, Journal of Image and Graphics, 28(6), pp. 1585-1607.

Hussein, A., Gaber, M., Elyan, E., Jayne, C., 2017, Imitation Learning: A Survey of Learning Methods, ACM Computing Surveys, 50(2), 21.

Zhong, Q., Zhi, J., Xu, Y., Gao, P., Feng, S., 2024, Applying an Extended Theory of Planned Behavior to Predict Young Drivers' In-Vehicle Information System (IVIS) Use Intention and Behavior While Driving: A Longitudinal Two-Wave Survey, Sustainability, 16(20), 8908.

Huang, H., Zeng, Z., Yao, D., Pei, X., Zhang, Y., 2022, Spatial-temporal ConvLSTM for vehicle driving intention prediction, Tsinghua Science and Technology, 27(3), pp. 599-609.

Emerson, R., 2024, K-Means Clustering Explained, Journal of Visual Impairment & Blindness, 118(1), pp. 65-66.

Savitzky, A., Golay, M., 1964, Smoothing and Differentiation of Data by Simplified Least Squares Procedures, Analytical Chemistry, 36(8), pp. 1627-1639.

Brockfeld, E., Kühne, R., Wagner, P., 2005, Calibration and Validation of Microscopic Traffic Flow Models, Transportation Research Record, 1934, pp. 62-70.


Refbacks

  • There are currently no refbacks.


ISSN: 0354-2025 (Print)

ISSN: 2335-0164 (Online)

COBISS.SR-ID 98732551

ZDB-ID: 2766459-4