Real‐time insect tracking and monitoring with computer vision and deep learning. Issue 3 (30th November 2021)

• Record Type: Journal Article
• Authors: Bjerge, Kim; Mann, Hjalte M. R.; Høye, Toke Thomas
• Editors: Sankey, Temuulen; Ahumada, Jorge
• Is Part Of: Remote sensing in ecology and conservation. Volume 8:Issue 3(2022)
• Page Start: 315
• Page End: 327
• Publication Date: 2021-11-30
• Subjects: Computer vision -- deep learning -- insects -- pollinators -- real‐time -- tracking; Remote sensing -- Periodicals; Ecology -- Research -- Periodicals; Ecology -- Methodology -- Periodicals; Ecology -- Remote sensing -- Periodicals; Nature conservation -- Methodology -- Periodicals; 577.0723
• DOI: 10.1002/rse2.245
• Languages: English
• ISSNs: 2056-3485

Abstract

Abstract: Insects are declining in abundance and diversity, but their population trends remain uncertain as insects are difficult to monitor. Manual methods require substantial time investment in trapping and subsequent species identification. Camera trapping can alleviate some of the manual fieldwork, but the large quantities of image data are challenging to analyse. By embedding the image analyses into the recording process using computer vision techniques, it is possible to focus efforts on the most ecologically relevant image data. Here, we present an intelligent camera system, capable of detecting, tracking, and identifying individual insects in situ . We constructed the system from commercial off‐the‐shelf components and used deep learning open source software to perform species detection and classification. We present the Insect Classification and Tracking algorithm (ICT) that performs real‐time classification and tracking at 0.33 frames per second. The system can upload summary data on the identity and movement track of insects to a server via the internet on a daily basis. We tested our system during the summer 2020 and detected 2994 insect tracks across 98 days. We achieved an average precision of 89% for correctly classified insect tracks of eight different species. This result was based on 504 manually verified tracks observed in videos during 10 days with varying insect activities. Using the track data, we could estimate the mean residence time for individual flower visiting insects within the field of view of the camera, and we were able to show a substantial variation in residence time among insect taxa. For honeybees, which were most abundant, residence time also varied through the season in relation to the plant species in bloom. Our proposed automated system showed promising results in non‐destructive and real‐time monitoring of insects and provides novel information about phenology, abundance, foraging behaviour, and movement ecology of flower visiting insects. Abstract : We present an intelligent camera system, capable of detecting, tracking, and identifying individual insects in situ . We constructed the system from commercial off‐the‐shelf components, and it uses deep learning open source software to perform species detection and classification. We present the Insect Classification and Tracking algorithm (ICT) that performs real‐time classification and tracking at 0.33 frames per second. Using the track data, we could estimate the mean residence time for individual insects within the field of view of the camera and we were able to show substantial variation in residence time among insect taxa. For honeybees, which were most abundant, residence time also varied throughout the season in relation to the species of flowers in bloom. Our proposed automated system showed promising results in non‐destructive and real‐time monitoring of insects, and provides novel information on foraging behaviour and movement ecology of flower visiting insects in particular.