Task weighting based on particle filter in deep multi-task learning with a view to uncertainty and performance. (August 2023)

• Record Type: Journal Article
• Authors: Aghajanzadeh, Emad; Bahraini, Tahereh; Mehrizi, Amir Hossein; Yazdi, Hadi Sadoghi
• Is Part Of: Pattern recognition. Volume 140(2023)
• Publication Date: 2023-08
• Subjects: Multi task learning -- Uncertainty -- Hyper-parameter tuning -- Deep learning -- Particle filter -- Bayesian estimation; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2023.109587
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: Deep multi-task networks have high uncertainty despite encouraging performance. Multi-task learning system performance is sensitive to weighting strategy for tasks. Paper proposes a novel weighting strategy to improve model uncertainty and accuracy. This strategy enhances quality of weights by Bayesian estimator and particle filter. Proposed tasks' weights make model have low uncertainty and high performance. Abstract: Recently multi-task learning (MTL) has been widely used in different applications to build more robust models by sharing knowledge across several related tasks. However, one challenge that arises is the variability in the learning pace of different tasks causing the inefficiency of naively training all tasks. Therefore, it is of great importance to consider some coefficients to balance tasks in the process of learning, but, due to the large search space and the significance of setting them properly, conventional search methods such as grid or random search are no longer effective. In this paper, we propose a learning mechanism for these coefficients based on the high efficiency of the particle filter (PF) algorithm to deal with nonlinear search problems. PF considers each state of the tasks' coefficients as a particle and recursively converges coefficients to an optimum point. While in most previous works coefficients were evaluated to only increase performance, to address the recent concerns related to applying AI in real-world applications, we also incorporate uncertainty alongside our method to prevent learning coefficients leading to unstable outcomes. This mechanism is independent of the models main learning process and can be easily added to every learning system without changing its training algorithm. Extensive experiments on real-world data sets demonstrate the superiority of the proposed method over the state-of-the-art methods on both performance and uncertainty. We also proved the acceptable performance of the method using Cramer Rao lower bound theory.