Abstract: It is well known that the traditional maximum likelihood estimation of GARCH model is severely biased in high dimensions. In this paper, the nonlinear shrinkage method proposed by Ledoit and Wolf is used to estimate DCC and BEKK models. In particular, the initial sample covariance estimator in maximum m-profile quasi-likelihood estimation (MMLE) proposed by Engle et al. is substituted by the nonlinear shrinkage estimator, which turns out to largely improve the estimation efficiency of high dimensional DCC and BEKK models, and for the first time, makes the valid estimation possible when the sample size is larger than the time series dimension. Based on the Percentage Relative Improvement in Average Loss (PRIAL), the Monte-Carlo simulations verify the obvious superiority of the nonlinear shrinkage substitution over the usual DCC and BEKK, which even strengthens as the ratio between sample size and time series dimension increases. Besides, for both DCC and BEKK, the performance of nonlinear shrinkage estimation is better than that of linear shrinkage, while linear shrinkage estimation is better than the usual estimation. Furthermore, the performance of DCC is better than BEKK, and the optimizing effect of nonlinear shrinkage on DCC is more significant than on BEKK. Finally, in the empirical part, using daily stock return data from the Center for Research in Security Prices (CRSP), the global minimum variance (GMV) portfolios of stocks traded in NYSE and NASDAQ are constructed based on various methods, and their real variances are compared. The empirical result supports the important role nonlinear shrinkage plays in promoting the estimation of high dimensional conditional covariance matrix, and thus in optimizing the portfolio selection.

Key words: nonlinear shrinkage, linear shrinkage, conditional covariance matrix