Abstract:

Optimization of action selection strategy for multi-agent Q-Learning model is one of the urgent problems to be solved in the simulation process of complex economics games. The forced ε-greedy action selection policy is introduced into the multi-agent Nash Q-Learning model， the effect of this action selection policy is compared with the classical ε-greedy policy through game experiments， and the effect of this action selection policy on the computational speed and convergence of the algorithm is explored； at the same time， theoretical verification of the veracity of the algorithm is carried out according to the experimental results， and a theoretical verification of the algorithm based on the nature of the multi-agent model is given. The simulation results show that forced ε-greedy is suitable for more complex games involving more state actions and more rounds， at which time it can effectively improve the running performance of the multi-agent Q-Learning algorithm， but due to its nature of initially increasing the exploration of actions， which will consume some rounds and lead to a decrease in the equilibrium convergence rate. Therefore， the performance improvement brought by enforcing ε-greedy versus the lost equilibrium convergence rate is an issue that users need to weigh when applying this policy.The main contributions of this paper are：1） comparing the model performance under the classical action selection strategy and the extended action selection policy， and conducting theoretical validation of the algorithm's veracity based on the simulation results， and summarizing the characteristics of the application of this extended policy in the multi-agent Nash Q model； 2） based on the analysis of the theoretical properties of multiple multi-agent models and the process of proof of convergence， the conclusion of the generalization of this action selection policy is given for the multi-agent reinforcement learning algorithms with action selection link provide a strategy to enhance the model computing speed and improve the convergence results. This promotes the operation efficiency of complex reinforcement learning models in economics， and to a certain extent， it can reduce the cost of experiments， and increase the number of experiments within the allowable range of computational volume to improve the reliability of experimental results.

Key words: Nash Q-Learning, \mathrm{\epsilon }-\mathrm{g}\mathrm{r}\mathrm{e}\mathrm{e}\mathrm{d}\mathrm{y}')">forced\mathrm{\epsilon }-\mathrm{g}\mathrm{r}\mathrm{e}\mathrm{e}\mathrm{d}\mathrm{y}, action selection