Hierarchical deep neural network for multivariate regression. (March 2017)

• Record Type: Journal Article
• Authors: Du, Jun; Xu, Yong
• Is Part Of: Pattern recognition. Volume 63(2017:Mar.)
• Page Start: 149
• Page End: 157
• Publication Date: 2017-03
• Subjects: Divide and Conquer -- Hierarchical Deep Neural Network -- Multivariate Regression -- Speech Enhancement -- Handwritten Chinese Character Recognition; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2016.10.003
• Languages: English
• ISSNs: 0031-3203

Abstract

Abstract: This paper presents the novel hierarchical deep neural network (HDNN) for the general multivariate regression problem. The recent insight of deep neural network (DNN) is the deep architecture with large training data can bring the best performance in many research areas. The architecture design of our proposed HDNN focuses on both "depth" and "width" of artificial neural network. Specifically for the multivariate regression, HDNN consists of multiple subnets, which is empirically more powerful than DNN by using a divide and conquer strategy. The effectiveness of HDNN as the regression model is verified on two tasks, namely speech enhancement and Chinese handwriting recognition. For the speech enhancement task, our experiments show that HDNN significantly outperforms DNN in terms of perceptual evaluation of speech quality (PESQ), which is an objective measure highly correlated to subjective testing of listening quality. And for Chinese handwriting recognition task, as a nonlinear feature mapping function, we have a very interesting observation that DNN-based approach can not even bring performance gain while HDNN-based approach yields significant improvements of recognition accuracy. Abstract : Highlights: Hierarchical deep neural network focuses on both deep and wide architectures. Decomposing a complicated regression problem into multiple subproblems to be solved. More powerful modeling capability and better learning convergence than DNN. Successful application for one regression problem (speech enhancement). Successful application for one classification problem (Chinese handwriting recognition).