A Feature Sparse Co-Attention Network for Visual internet of things (VIoT) sensing. (July 2022)

• Record Type: Journal Article
• Authors: Dong, Feng; Wang, Xiaofeng; Oad, Ammar; Talpur, Mir Sajjad Hussain; Khuhro, Mansoor Ahmed; Sun, Bowen
• Is Part Of: Computers & electrical engineering. Volume 101(2022)
• Publication Date: 2022-07
• Subjects: Visual internet of things (VIoT), Feature sparse co-attention network (FSCAN) -- Bi-LSTM -- Image self-attention unit (ISAU) -- Question self-attention unit (QSAU) -- Guided attention unit (GAU); Computer engineering -- Periodicals; Electrical engineering -- Periodicals; Electrical engineering -- Data processing -- Periodicals; Ordinateurs -- Conception et construction -- Périodiques; Électrotechnique -- Périodiques; Électrotechnique -- Informatique -- Périodiques; Computer engineering; Electrical engineering; Electrical engineering -- Data processing; Periodicals; Electronic journals; 621.302854
• DOI: 10.1016/j.compeleceng.2022.107988
• Languages: English
• ISSNs: 0045-7906

Abstract

Abstract: Visual internet of things (VIoT) is a cross-modal learning task that requires a simultaneous understanding of image and question text information. The attention mechanism simulates human vision to retain the important part of the information and discard the unimportant part. However, since both important and unimportant features participate in the weighted summation, there is no true retention and discarding of information features. Furthermore, in many VIoT solutions, the utilization of the attention mechanism does not achieve the purpose of improving VIoT performance by aligning the key regions of image and question. To solve the problems mentioned above, this paper proposes an attention mechanism that sparse features to achieve attention with true retention and discard functions. At the same time, a feature sparse co-attention network is constructed to align the key regions of vision and text. The network is composed of image self-attention unit, question self-attention unit, and guiding attention unit. Each self-attention unit has a feature sparse function. These units are cascaded in depth to form a hierarchical structure that, as a whole, realizes co-attention. Several experiments conducted on the VQA-v2 dataset show that the proposed method outperforms the latest methods.