关键词: 深度强化学习, 电动救护车, 鲁棒Actor-Critic算法, 重部署, 不确定性环境

Abstract: Ambulance is a scarce resource in the emergency medical service (EMS) system, and the rational deployment of ambulance has been the most important means to improve the quality of EMS. This study addresses the intelligent redeployment of electric ambulances, specifically how to dynamically and intelligently reselect deployment stations after completing an emergency task, in response to the uncertainty in call request arrival rates (λ). In this study, "intelligent" refers to the use of deep reinforcement learning (DRL) methods to automatically optimize redeployment decisions to address environmental uncertainties and dynamic changes. Existing mainstream redeployment decision-making methods typically rely on precise estimates of λ. However, these methods perform poorly in highly uncertain environments, especially when λ cannot be accurately predicted. To address this challenge, this paper proposes a dynamic decision-making method for the intelligent redeployment of electric ambulances, considering the uncertainty in state observations. First, the EMS system is modeled as a Markov decision process, and an approximate dynamic programming approach is used to compress the system's state variables to mitigate the curse of dimensionality. Second, uncertainty is incorporated into the observation of state variables, and a new policy gradient calculation formula is derived. The robust Actor-Critic algorithm is then designed to intelligently learn the optimal redeployment strategy. Experimental results show that the robust Actor-Critic algorithm significantly outperforms existing methods in both average response time and on-time response rate. Building on this, the paper explores the impact of different charging strategies on ambulance performance and provides management recommendations for electric ambulance redeployment, offering theoretical support and practical guidance for optimizing EMS.

Key words: deep reinforcement learning, electric ambulance, robust actor-critic algorithm, redeployment, uncertain environments