Abstract:

Ambulances， as scarce resources in the Emergency Medical Service （EMS） system， require rational deployment to significantly improve EMS quality. The intelligent redeployment problem of electric ambulances-specifically， how to intelligently reselect deployment stations based on dynamically changing demand after completing emergency missions is studied， in order to address the uncertainty in call request rates （\mathit{\lambda }）. In this study， “intelligence” refers to the use of deep reinforcement learning methods to automatically optimize redeployment decisions in response to environmental uncertainties and dynamic changes. Current mainstream redeployment decision methods typically rely on precise estimation of\mathit{\lambda }. However， in highly uncertain environments， especially when \mathit{\lambda } cannot be accurately predicted， these methods perform poorly. To address this， an intelligent dynamic decision-making method for electric ambulance redeployment is proposed that considers state observation uncertainty. First， the EMS system is modeled as a Markov Decision Process （MDP）， and basis functions are introduced based on approximate dynamic programming theory to compress system state variables， thereby solving the dimensionality curse problem. Second， uncertainty is incorporated into the observation of state variables， a new policy gradient calculation formula is derived， and a robust Actor-Critic algorithm is designed to learn optimal redeployment strategies. Experimental results show that the robust Actor-Critic algorithm demonstrates significant advantages in both average response time and on-time response rate. Building on this， the impact of different charging strategies on ambulance performance is explored， management suggestions for electric ambulance redeployment is proposed， and theoretical support and practical guidance for optimizing EMS systems is provided.

Key words: deep reinforcement learning, electric ambulance, robust actor-critic algorithm, redeployment, uncertain environments