A news picture geo-localization pipeline based on deep learning and street view images. Issue 1 (31st December 2022)

• Record Type: Journal Article
• Authors: Chu, Tianyou; Chen, Yumin; Su, Heng; Xu, Zhenzhen; Chen, Guodong; Zhou, Annan
• Is Part Of: International journal of digital earth. Volume 15:Issue 1(2022)
• Page Start: 1485
• Page End: 1505
• Publication Date: 2022-12-31
• Subjects: Street view images -- geo-localization -- image retrieval -- social media; Geographic information systems -- Periodicals; Sustainable development -- Information technology -- Periodicals; Social planning -- Information technology -- Periodicals; 910.285
• DOI: 10.1080/17538947.2022.2121437
• Languages: English
• ISSNs: 1753-8947

Abstract

ABSTRACT: Numerous news or event pictures are taken and shared on the internet every day that have abundant information worth being mined, but only a small fraction of them are geotagged. The visual content of the news image hints at clues of the geographical location because they are usually taken at the site of the incident, which provides a prerequisite for geo-localization. This paper proposes an automated pipeline based on deep learning for the geo-localization of news pictures in a large-scale urban environment using geotagged street view images as a reference dataset. The approach obtains location information by constructing an attention-based feature extraction network. Then, the image features are aggregated, and the candidate street view image results are retrieved by the selective matching kernel function. Finally, the coordinates of the news images are estimated by the kernel density prediction method. The pipeline is tested in the news pictures in Hong Kong. In the comparison experiments, the proposed pipeline shows stable performance and generalizability in the large-scale urban environment. In addition, the performance analysis of components in the pipeline shows the ability to recognize localization features of partial areas in pictures and the effectiveness of the proposed solution in news picture geo-localization.