Representation learning with deep sparse auto-encoder for multi-task learning. (September 2022)

• Record Type: Journal Article
• Authors: Zhu, Yi; Wu, Xindong; Qiang, Jipeng; Hu, Xuegang; Zhang, Yuhong; Li, Peipei
• Is Part Of: Pattern recognition. Volume 129(2022)
• Publication Date: 2022-09
• Subjects: Deep sparse auto-encoder -- Multi-task learning -- RICA -- Labeled and unlabeled data; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2022.108742
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: We propose a novel representation l earning method DSML to achieve a better performance based on deep sparse auto encoder for Multi task Learning. We propose a new stacked sparse auto encoder for feature reconstruction which can learn higher level and better representations by using the deep learning method and overcome the overfitting problem effectively. The training of parameters in DSML has lower computational cost than other common deep learning methods. Experimental studies of compared with seven representation l earning methods show that DSML is superior to traditional and state of the art representation l earning for Multi task Learning. Abstract: We demonstrate an effective framework to achieve a better performance based on Deep Sparse auto-encoder for Multi-task Learning, called DSML for short. To learn the reconstructed and higher-level features on cross-domain instances for multiple tasks, we combine the labeled and unlabeled data from all tasks to reconstruct the feature representations. Furthermore, we propose the model of Stacked Reconstruction Independence Component Analysis (SRICA for short) for the optimization of feature representations with a large amount of unlabeled data, which can effectively address the redundancy of image data. Our proposed SRICA model is developed from RICA and is based on deep sparse auto-encoder. In addition, we adopt a Semi-Supervised Learning method (SSL for short) based on model parameter regularization to build a unified model for multi-task learning. There are several advantages in our proposed framework as follows: 1) The proposed SRICA makes full use of a large amount of unlabeled data from all tasks. It is used to pursue an optimal sparsity feature representation, which can overcome the over-fitting problem effectively. 2) The deep architecture used in our SRICA model is applied for higher-level and better representation learning, which is designed to train on patches for sphering the input data. 3) Training parameters in our proposed framework has lower computational cost compared to other common deep learning methods such as stacked denoising auto-encoders. Extensive experiments on several real image datasets demonstrate our proposed framework outperforms the state-of-the-art methods.