Split 'n' merge net: A dynamic masking network for multi-task attention. (June 2022)

• Record Type: Journal Article
• Authors: Fernando, Tharindu; Sridharan, Sridha; Denman, Simon; Fookes, Clinton
• Is Part Of: Pattern recognition. Volume 126(2022)
• Publication Date: 2022-06
• Subjects: Multi-task learning -- Attention -- Cuffless blood pressure measurement -- Biomedical signal processing -- Deep learning -- Emotion recognition; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2022.108551
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: A novel outlook on multi-task learning as a feature separation problem. A simplified, interpretable, and dynamic framework for multi-task learning. Multi-head attention based dynamic feature masking to select task specific and shared features from an embedding. Achieves state-of-the-art results in multiple public benchmarks. Abstract: In this paper we propose a novel Multi-Task Learning (MTL) framework, Split 'n' Merge Net. We draw the inspiration from the multi-head attention formulation of Transformers and propose a novel, simple and interpretable pathway to process information captured and exploited by multiple tasks. In particular, we propose a novel splitting network design, which is empowered with multi-head attention, and generates dynamic masks to filter task specific information and task agnostic shared factors from the input. To drive this generation, and to avoid the oversharing of information between the tasks, we propose a novel formulation of the mutual information loss which encourages the generated split embeddings to be distinct as possible. A unique merging network is also introduced to fuse the task specific, and shared information and generate an augmented embedding for the individual downstream tasks in the MTL pipeline. We evaluate the proposed Split 'n' Merge Network on two distinct MTL tasks where we achieve state-of-the-art results for both. Our primary, ablation and interpretation evaluations indicate the robustness and flexibility of the propose approach and demonstrates its applicability to numerous, diverse real-world MTL applications.