Computer vision-enhanced selection of geo-tagged photos on social network sites for land cover classification. (June 2020)

• Record Type: Journal Article
• Authors: ElQadi, Moataz Medhat; Lesiv, Myroslava; Dyer, Adrian G.; Dorin, Alan
• Is Part Of: Environmental modelling & software. Volume 128(2020)
• Publication Date: 2020-06
• Subjects: land cover -- Social network -- Geo-tagged photos -- Computer vision -- Machine learning; Environmental monitoring -- Computer programs -- Periodicals; Ecology -- Computer simulation -- Periodicals; Digital computer simulation -- Periodicals; Computer software -- Periodicals; Environmental Monitoring -- Periodicals; Computer Simulation -- Periodicals; Environnement -- Surveillance -- Logiciels -- Périodiques; Écologie -- Simulation, Méthodes de -- Périodiques; Simulation par ordinateur -- Périodiques; Logiciels -- Périodiques; Computer software; Digital computer simulation; Ecology -- Computer simulation; Environmental monitoring -- Computer programs; Periodicals; Electronic journals; 363.70015118
• DOI: 10.1016/j.envsoft.2020.104696
• Languages: English
• ISSNs: 1364-8152

Abstract

Abstract: Land cover maps are key elements for understanding global climate and land use. They are often created by automatically classifying satellite imagery. However, inconsistencies in classification may be introduced inadvertently. Experts can reconcile classification discrepancies by viewing satellite and high-resolution images taken on the ground. We present and evaluate a framework to filter relevant geo-tagged photos from social network sites for land cover classification tasks. Social network sites offer massive amounts of potentially relevant data, but its quality and fitness for research purposes must be verified. Our framework uses computer vision to analyse the content of geo-tagged photos on social network sites to generate descriptive tags. These are used to train artificial neural networks to predict a photo's relevance for land cover classification. We apply our models to four African case studies and their neighbours. The framework has been implemented within Geo-Wiki to fetch relevant photos from Flickr. Highlights: A framework for finding geotagged photos for land cover classification is proposed. We build a classification model based on text tags generated from computer vision. We evaluate the framework against 4 African countries, and 3 of their neighbours. Photos relevant to land cover classification found to be only 30–70%. Our models save time to check 72–90% of irrelevant photos in case study countries.