Improving reasoning with contrastive visual information for visual question answering. Issue 20 (11th June 2021)

• Record Type: Journal Article
• Authors: Long, Yu; Tang, Pengjie; Wang, Hanli; Yu, Jian
• Is Part Of: Electronics letters. Volume 57:Issue 20(2021)
• Page Start: 758
• Page End: 760
• Publication Date: 2021-06-11
• Subjects: Electronics -- Periodicals; 621.381
• DOI: 10.1049/ell2.12255
• Languages: English
• ISSNs: 0013-5194

Abstract

Abstract: Visual Question Answering (VQA) aims to output a correct answer based on cross‐modality inputs including question and visual content. In general pipeline, information reasoning plays the key role for a reasonable answer. However, visual information is commonly not fully employed in many popular models nowadays. Facing this challenge, a new strategy is proposed in this work to make the best of visual information during reasoning. In detail, visual information is divided into two subsets: (1) question‐relevant visual set, and (2) question‐irrelevant visual set. Then, both of these two sets are employed by reasoning to generate reasonable outputs. Experiments are conducted on the benchmark VQAv2 dataset, which demonstrate the effectiveness of the proposed strategy. The project page can be found in https://mic.tongji.edu.cn/e6/8d/c9778a190093/page.htm .