考虑需求时间和空间不确定的电动救护车智能重部署

doi:10.16381/j.cnki.issn1003-207x.2024.2042

摘要/Abstract

摘要：

救护车作为紧急医疗服务（emergency medical service，EMS）系统中的稀缺资源，其合理部署对于提高EMS质量至关重要。本文研究电动救护车的智能重部署问题，即在完成急救任务后如何根据动态变化的需求智能地重新选择部署站点，以应对呼叫请求率 $λ$ 的不确定性。在本研究中，“智能”指的是通过深度强化学习方法，自动优化重部署决策以应对环境的不确定性和动态变化。现有的主流重部署决策方法通常依赖于对 $λ$ 的精确估计，然而，在高度不确定的环境下，尤其是 $λ$ 无法准确预测时，这类方法表现不佳。为此，本文提出了一种考虑状态观测不确定性的电动救护车智能重部署动态决策方法。首先，将EMS系统建模为马尔科夫决策过程，并基于近似动态规划理论引入基函数压缩系统状态变量，以解决维度灾难问题。其次，本文将不确定性纳入状态变量的观测中，推导出新的策略梯度计算公式，并设计了鲁棒Actor-Critic算法学习最优重部署策略。实验结果表明，鲁棒Actor-Critic算法在平均接诊时间和按时接诊率上均表现出显著优势。在此基础上，本文探讨不同充电策略对救护车性能的影响，提出针对电动救护车重部署的管理建议，为优化EMS提供理论支持和实践指导。

关键词: 深度强化学习, 电动救护车, 鲁棒Actor-Critic算法, 重部署, 不确定性环境

Abstract:

Ambulances， as scarce resources in the Emergency Medical Service （EMS） system， require rational deployment to significantly improve EMS quality. The intelligent redeployment problem of electric ambulances-specifically， how to intelligently reselect deployment stations based on dynamically changing demand after completing emergency missions is studied， in order to address the uncertainty in call request rates （ $λ$ ）. In this study， “intelligence” refers to the use of deep reinforcement learning methods to automatically optimize redeployment decisions in response to environmental uncertainties and dynamic changes. Current mainstream redeployment decision methods typically rely on precise estimation of $λ$ . However， in highly uncertain environments， especially when $λ$ cannot be accurately predicted， these methods perform poorly. To address this， an intelligent dynamic decision-making method for electric ambulance redeployment is proposed that considers state observation uncertainty. First， the EMS system is modeled as a Markov Decision Process （MDP）， and basis functions are introduced based on approximate dynamic programming theory to compress system state variables， thereby solving the dimensionality curse problem. Second， uncertainty is incorporated into the observation of state variables， a new policy gradient calculation formula is derived， and a robust Actor-Critic algorithm is designed to learn optimal redeployment strategies. Experimental results show that the robust Actor-Critic algorithm demonstrates significant advantages in both average response time and on-time response rate. Building on this， the impact of different charging strategies on ambulance performance is explored， management suggestions for electric ambulance redeployment is proposed， and theoretical support and practical guidance for optimizing EMS systems is provided.

Key words: deep reinforcement learning, electric ambulance, robust actor-critic algorithm, redeployment, uncertain environments

中图分类号:

U492

刘嘉,操洁,熊毅. 考虑需求时间和空间不确定的电动救护车智能重部署[J]. 中国管理科学, 2026, 34(9): 109-119.

Jia Liu,Jie Cao,Yi Xiong. Intelligent Redeployment of Electric Ambulances Considering Temporal and Spatial Uncertainty of Demand[J]. Chinese Journal of Management Science, 2026, 34(9): 109-119.

图/表 9

图1

图2

表1

图3

图4

图5

图6

表2

图7

参考文献 35

[1]	Ahrens M， Evarts B. Fire loss in the United States ［R］. Discussion Paper， National Fire Protection Association， 2020.
[2]	World Health Organization. Global status report on road safety 2023［EB/OL］.（2023-12-23）［2025-09-02］..
[3]	Clinic Cleveland. Sudden cardiac statistics［EB/OL］. （2020-06-11）［2025-09-02］..
[4]	Foundation Heart. Sudden cardiac statistics［EB/OL］. （2024-10-25）［2025-09-02］..
[5]	Satterfield C， Schefter K. Electric vehicle sales and the charging infrastructure required through 2030［R］. Discussion Paper， Edison Electric Institute， 2024.
[6]	周德群，程雪熠，王群伟，等. 异质性政策组合对新能源汽车产业发展的影响效应分析［J］. 中国管理科学， 2024， DOI： 10.16381/j.cnki.issn1003-207x.2024.0879 .
	Zhou D Q， Cheng X Y， Wang Q W， et al. Asymptotically optimal policies for dynamic ambulance Dispatch［J］. Chinese Journal of Management Science， 2024， DOI： 10.16381/j.cnki.issn1003-207x.2024.0879 .
[7]	Ji S， Zheng Y， Wang Z， et al. A deep reinforcement learning-enabled dynamic redeployment system for mobile ambulances［J］. Proceedings of the ACM on Interactive， Mobile， Wearable and Ubiquitous Technologies， 2019， 3（1）： 1-20.
[8]	Maxwell M S， Restrepo M， Henderson S G， et al. Approximate dynamic programming for ambulance redeployment［J］. INFORMS Journal on Computing， 2010， 22（2）： 266-281.
[9]	Ali Nasrollahzadeh A， Khademi A， Mayorga M E. Real-time ambulance dispatching and relocation［J］. Manufacturing & Service Operations Management， 2018， 20（3）： 467-480.
[10]	Gao X， Kong N， Griffin P. Shortening emergency medical response time with joint operations of uncrewed aerial vehicles with ambulances［J］. Manufacturing & Service Operations Management，2023，26（2）： 447-464.
[11]	Wang W， Wang S， Zhen L， et al. EMS location-allocation problem under uncertainties［J］. Transportation Research Part E： Logistics and Transportation Review， 2022， 168： 102945.
[12]	Grot M. Decision support framework for tactical emergency medical service location planning［J］. Omega， 2024， 125： 103036.
[13]	Boutilier J J， Chan T C Y. Ambulance emergency response optimization in developing countries［J］. Operations Research， 2020， 68（5）： 1315-1334.
[14]	Jong N D， Aslan A， Bakir I.Dynamic service of geographically dispersed time-sensitive demands［J］.Transportation Research Part C： Emerging Technologies， 2024， 163（c）：104625.
[15]	Yoon S， Albert L A. Dynamic dispatch policies for emergency response with multiple types of vehicles［J］. Transportation Research Part E： Logistics and Transportation Review， 2021， 152： 102405.
[16]	Li M， Carter A， Goldstein J， et al. Determining ambulance destinations when facing offload delays using a Markov decision process［J］. Omega， 2021， 101： 102251.
[17]	Bertsimas D， Ng Y. Robust and stochastic formulations for ambulance deployment and dispatch［J］. European Journal of Operational Research， 2019， 279（2）： 557-571.
[18]	Daskin M S. Application of an expected covering model to emergency medical service system design［J］. Decision Sciences， 1982， 13（3）： 416-439.
[19]	Daskin M S. A maximum expected covering location model： Formulation， properties and heuristic solution［J］. Transportation Science， 1983， 17（1）： 48-70.
[20]	Gendreau M， Laporte G， Semet F. A dynamic model and parallel tabu search heuristic for real-time ambulance relocation［J］. Parallel Computing， 2001， 27（12）： 1641-1653.
[21]	Naoum-Sawaya J， Elhedhli S. A stochastic optimization model for real-time ambulance redeployment［J］. Computers & Operations Research， 2013， 40（8）： 1972-1978.
[22]	Jagtenberg C J， Bhulai S， van der Mei R D. An efficient heuristic for real-time ambulance redeployment［J］. Operations Research for Health Care， 2015， 4： 27-35.
[23]	Gendreau M， Laporte G， Semet F. The maximal expected coverage relocation problem for emergency vehicles［J］. Journal of the Operational Research Society， 2006， 57（1）： 22-28.
[24]	Andersson T， Värbrand P. Decision support tools for ambulance dispatch and relocation［J］. Journal of the Operational Research Society， 2007， 58（2）： 195-201.
[25]	Sudtachat K， Mayorga M E， McLay L A. A nested-compliance table policy for emergency medical service systems under relocation［J］. Omega， 2016， 58： 154-168.
[26]	Schmid V. Solving the dynamic ambulance relocation and dispatching problem using approximate dynamic programming［J］. European Journal of Operational Research， 2012， 219（3）： 611-621.
[27]	Kullman N D， Cousineau M， Goodson J C， et al. Dynamic ride-hailing with electric vehicles［J］. Transportation Science， 2022， 56（3）： 775-794.
[28]	Yan P， Yu K， Chao X， et al. An online reinforcement learning approach to charging and order-dispatching optimization for an e-hailing electric vehicle fleet［J］. European Journal of Operational Research， 2023， 310（3）： 1218-1233.
[29]	Ahadi R， Ketter W， Collins J， et al. Cooperative learning for smart charging of shared autonomous vehicle fleets［J］. Transportation Science， 2023， 57（3）： 613-630.
[30]	Zhang H， Chen H， Xiao C， et al. Robust deep reinforcement learning against adversarial perturbations on state observations［J］. Advances in Neural Information Processing Systems， 2020， 33： 21024-21037.
[31]	Huang J， Huang W， Lan L， et al. Meta attention for off-policy actor-critic［J］. Neural Networks， 2023， 163： 86-96.
[32]	Xi L， Wu J， Xu Y， et al. Automatic generation control based on multiple neural networks with actor-critic strategy［J］. IEEE Transactions on Neural Networks and Learning Systems， 2021， 32（6）： 2483-2493.
[33]	Bertsimas D， Sim M. The price of robustness［J］. Operations Research， 2004， 52（1）： 35-53.
[34]	Schulman J， Levine S， Abbeel P， et al. Trust region policy optimization［C］//International conference on machine learning， 2015， 1889-1897.
[35]	Xu K， Shi Z， Zhang H， et al. Automatic perturbation analysis on general computational graphs［R］. Discussion Paper， Lawrence Livermore National Lab， 2020.

策略	收益	AvePT	RelaPT
Static and hoc	2548	425.16	0.728
Random	2611	403.93	0.746
Dynamic heuristic	2664	352.66	0.761
DRLSN	2674	355.42	0.764
R-AC	2807	341.68	0.802

策略	ambu=35		ambu=45		ambu=55		ambu=65		ambu=75
	AvePT	RelaPT	AvePT	RelaPT	AvePT	RelaPT	AvePT	RelaPT	AvePT	RelaPT
Sah	536.35	0.516	453.76	0.646	425.16	0.728	415.92	0.738	402.71	0.745
Random	502.78	0.539	436.25	0.658	403.93	0.746	388.76	0.751	381.63	0.757
DH	456.96	0.728	408.78	0.744	352.66	0.761	344.48	0.773	342.99	0.781
DRLSN	459.89	0.735	404.35	0.749	355.42	0.764	350.49	0.771	345.72	0.782
R-AC	401.89	0.759	363.72	0.788	341.68	0.802	331.75	0.810	327.66	0.816