关键词: 大规模创新型竞赛, 评分误差, 交叉熵, 专家权重

Abstract:

The judging of innovative works is somewhat subjective， and ratings may vary considerably from one expert to another. The establishment of a scientific evaluation method for innovative competitions is of great significance in promoting the fairness of competitions and increasing the motivation for innovation. Some of the problems of the current large-scale innovative competition evaluation program are addressed in this paper， including the problem of rational allocation of competition works， the problem of scientific adjustment of abnormal scoring， and the problem of determining expert weights， etc.， and a complete set of evaluation program is established that can effectively reduce the harm of scoring errors.Firstly， a mathematical model of “bundled cross-assignment” of competition entries is developed. On the one hand， the model requires that every two entries be bundled together and distributed to the experts for evaluation， which ensures that each entry has a comparable object so that unreasonable scores can be adjusted. On the other hand， the model requires that there be as much cross-evaluation of works between any two experts as possible， thus ensuring that the workload of each expert is as close as possible， and at the same time， the scoring characteristics of any two experts can be compared and analyzed to provide reliable and objective data for the determination of experts' weights. In addition， a property of the optimal solution of the model is discussed， and a greedy algorithm for solving the local optimal solution is designed. Secondly， according to the principle of majority rule， a simple and practical model for adjusting the scores of a few experts is established based on the ratio of the scores of the majority of experts on the two works. The model can reduce the impact of random errors to a certain extent， and can effectively attenuate the unfairness caused by individual experts' misjudgment and deliberate high or low scores. Thirdly， a cross-entropy-based weighting method is established， which characterizes the experts' judging scores by means of “intrinsic information” and cross-entropy， and then designs a reliable formula for calculating experts' weights. The weights， to some extent， can improve the defect that the premise assumptions of the standardized scoring method may not be valid in the evaluation of large-scale competitions， and at the same time， the weights can further minimize the range of the scores for most of the works. Finally， in order to test the effectiveness of the method proposed in this paper， the mathematical modeling competition works of H university are used as experiments， and the five evaluation schemes are compared and analyzed by four evaluation indexes， namely， ranking difference degree， Spearman’s rank correlation coefficient， experts' scoring error degree， and works' controversy degree， and the results show that our method improves the Spearman’s rank correlation coefficient， works' controversy degree， and experts' scoring error degree and decreases the ranking difference degree， which shows that our method is more scientific and reasonable than others， and is able to make a fairer and more objective evaluation and ranking for the works of the competition.Four avenues for further investigation on this subject matter. （1） The competition entries’ distribution model proposed in this paper is a mathematical model of a class of assignment problems， and further research is required to develop an algorithm for its optimal solution. （2） The adjustments are introduced to the original scores， and the calculation formula can be discussed for transforming these scores （e.g.， standard scores）. （3） In order to address systematic and random errors in expert scoring， it is necessary to explore reasonable methods for assigning weights to experts. （4） Can the scores of t experts for each competition entry be changed into the interval number according to reasonable rules， and then the entries are evaluated and ranked based on the method proposed in literature ［11］.

Key words: large-scale innovative competitions, scoring error, cross-entropy, experts’ weights