Deep collaborative multi-task network: A human decision process inspired model for hierarchical image classification. (April 2022)

• Record Type: Journal Article
• Authors: Zhou, Yu; Li, Xiaoni; Zhou, Yucan; Wang, Yu; Hu, Qinghua; Wang, Weiping
• Is Part Of: Pattern recognition. Volume 124(2022)
• Publication Date: 2022-04
• Subjects: Hierarchical image classification -- Deep multi-task network -- Collaborative learning -- Decision uncertainty evaluation; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2021.108449
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: We propose a deep collaborative multi-task learning framework for hierarchical classification, where each prediction problem in the hierarchy is regarded as a sub-task to obtain multi-granularity intermediate predictions. To well utilize the relations among different sub-tasks, a novel fusion function is designed based on the confidence degree and the uncertainty degree, which can adaptively adjust the weights of the intermediate predictions from all the sub-tasks, acquiring better final predictions. We evaluate the performance of the proposed model on three image datasets. The experimental results demonstrate that considering the relations among different sub-tasks can improve the classification results, and our proposed model can achieve state-of-the-art performance. Abstract: Hierarchical classification is significant for big data, where the original task is divided into several sub-tasks to provide multi-granularity predictions based on a tree-shape label structure. Obviously, these sub-tasks are highly correlated: results of the coarser-grained sub-tasks can reduce the candidates for the fine-grained sub-tasks, while results of the fine-grained sub-tasks provide attributes describing the coarser-grained classes. A human can integrate feedbacks from all the related sub-tasks instead of considering each sub-task independently. Therefore, we propose a deep collaborative multi-task network for hierarchical image classification. Specifically, we first extract the relationship matrix between every two sub-tasks defined by the hierarchical label structure. Then, the information of each sub-task is broadcasted to all the related sub-tasks through the relationship matrix. Finally, to combine this information, a novel fusion function based on the task evaluation and the decision uncertainty is designed. Extensive experimental results demonstrate that our model can achieve state-of-the-art performance.