Learning a good representation with unsymmetrical auto-encoder. Issue 5 (July 2016)

• Record Type: Journal Article
• Authors: Sun, Yanan; Mao, Hua; Guo, Quan; Yi, Zhang
• Is Part Of: Neural computing & applications. Volume 27:Issue 5(2016)
• Page Start: 1361
• Page End: 1367
• Publication Date: 2016-07
• Subjects: Auto-encoder -- Neural networks -- Feature learning -- Deep learning -- Unsupervised learning; Neural networks (Computer science) -- Periodicals; Neural circuitry -- Periodicals; Artificial intelligence -- Periodicals; Neural Networks (Computer) -- Periodicals; Réseaux neuronaux (Informatique) -- Périodiques; Réseaux nerveux -- Périodiques; Intelligence artificielle -- Périodiques; 006.32
• DOI: 10.1007/s00521-015-1939-3
• Languages: English
• ISSNs: 0941-0643

Abstract

Abstract Auto-encoders play a fundamental role in unsupervised feature learning and learning initial parameters of deep architectures for supervised tasks. For given input samples, robust features are used to generate robust representations from two perspectives: (1) invariant to small variation of samples and (2) reconstruction by decoders with minimal error. Traditional auto-encoders with different regularization terms have symmetrical numbers of encoder and decoder layers, and sometimes parameters. We investigate the relation between the number of layers and propose an unsymmetrical structure, i.e., an unsymmetrical auto-encoder (UAE), to learn more effective features. We present empirical results of feature learning using the UAE and state-of-the-art auto-encoders for classification tasks with a range of datasets. We also analyze the gradient vanishing problem mathematically and provide suggestions for the appropriate number of layers to use in UAEs with a logistic activation function. In our experiments, UAEs demonstrated superior performance with the same configuration compared to other auto-encoders.