关键词: 深度强化学习, 大规模动作空间, 网络能力控制, 超售

Abstract:

The challenges of airline network capacity control and overbooking strategies in the context of a choice-based demand model is explored in this research. Traditionally， most studies in this area have relied on independent demand models， where passenger behavior is assumed to be independent and the target product for each passenger is a fixed option. However， with the rise of low-cost carriers and increasing competition， simulating the decision-making process of passengers selecting from a range of products has become a key issue. The core of the problem lies in optimizing seat （resource） allocation across multiple flight segments， with the goal of maximizing revenue while managing the potential losses caused by passenger no-shows and overbooking. At each time step， it is necessary to decide which products to offer， and as the number of product types increases， the number of possible product combinations grows exponentially， leading to a large action space. Additionally， passenger behavior is uncertain， and the state transition rules are complex. To address this problem， the research models the issue as a Markov Decision Process （MDP）， where the state represents the total booking amount of each product at each time step. The action space consists of determining which products to open at each time step， considering resource constraints and overbooking limits. To efficiently solve this problem， the research proposes a deep reinforcement learning （DRL） algorithm based on the Branching Dueling Q-Network （BDQ） framework， named Max-BDQ （MBDQ）. This algorithm significantly reduces the computational complexity by decomposing the high-dimensional action space into multiple independent subspaces. It also combines techniques such as Double Deep Q-Network （Double DQN）， prioritized experience replay， and Dueling Deep Q-Network （Dueling DQN）， which improve the stability and convergence speed of the model. MBDQ incorporates several key enhancements， including a modified Q-value calculation method， an improved neural network architecture， and hyperparameter optimization， all of which enable the algorithm to adapt to dynamic and uncertain environments. A linear programming-based revenue upper bound calculation method is proposed and extensive experiments are conducted by setting different passenger arrival patterns and load factors to evaluate the performance of the MBDQ algorithm. The experimental results show that the average revenue of the MBDQ algorithm is approximately 87% of the revenue upper bound， with the highest performance exceeding 90%. Compared to a full-product open strategy， MBDQ yields about 10% higher revenue， and its performance further improves as the load factor increases. Compared to BDQ， MBDQ yields about 6% higher revenue. These demonstrate the robustness and efficiency of MBDQ in handling large-scale， high-dimensional decision spaces. A practical solution is provided for enhancing airline revenue management and offers theoretical support for dynamic optimization in high-dimensional systems. The findings also highlight the potential of deep reinforcement learning as a powerful tool for solving complex， large-scale revenue management problems.

Key words: deep reinforcement learning, large action space, network capacity control, overbooking