Owner name entity recognition in websites based on multiscale features and multimodal co-attention. (15th August 2023)

• Record Type: Journal Article
• Authors: Ren, Yimo; Li, Hong; Liu, Peipei; Liu, Jie; Zhu, Hongsong; Sun, Limin
• Is Part Of: Expert systems with applications. Volume 224(2023)
• Publication Date: 2023-08-15
• Subjects: Owner NER -- Multimodal NER -- Multiscale features -- Multimodal co-attention -- Network security; Expert systems (Computer science) -- Periodicals; Systèmes experts (Informatique) -- Périodiques; Electronic journals; 006.33
• DOI: 10.1016/j.eswa.2023.120014
• Languages: English
• ISSNs: 0957-4174

Abstract

Abstract: Identifying the owners of online devices on the Internet can enable numerous network security applications. For example, fast and accurate Owner Name Entity Recognition (ONER) of websites is critical to find influenced owners in light of new security threats. In this situation, as a specific task of Multimodal Named Entity Recognition (MNER), ONER is essential and helpful for network security. Currently, most of the existing MNER models only use texts and images, so they cannot effectively utilize the multimodal data of devices to achieve ONER accurately and fast. In order to improve performance and training speed simultaneously, the paper proposes a framework MFMCA: Multiscale Features and Multimodal Co-Attention. MFMCA is based on a two-step gated co-attention with fewer transformer blocks and simultaneously uses texts, images, and domains. Also, MFMCA extracts multiscale image features to get a fine-grained hint for ONER. Moreover, due to the lack of MNER datasets, the paper manually labels a multimodal dataset containing texts, images, and domains for MNER research. The experiments show that MFMCA achieves 0.8211 F1 scores on the recognition of owner entities, which is competitive compared with 0.8288, the best performance of existing state-of-the-art MNER models. However, MFMCA saves about 34% training time on the proposed dataset. Highlights: This paper raises the problem of Owner Name Entity Recognition of websites. This paper constructs a manually labelled multimodal dataset, containing about 15, 000 samples. This paper designs a two-step gated co-attention to improve the performance of ONER. The results show the competitive performance of our method with less cost time. This paper builds multiscale image features to realize feature alignment well.