Diminished reality system with real-time object detection using deep learning for onsite landscape simulation during redevelopment. (September 2020)

• Record Type: Journal Article
• Authors: Kido, Daiki; Fukuda, Tomohiro; Yabuki, Nobuyoshi
• Is Part Of: Environmental modelling & software. Volume 131(2020)
• Publication Date: 2020-09
• Subjects: Landscape simulation -- Design support system -- Diminished reality (DR) -- Augmented reality (AR) -- Real-time object detection -- Deep learning; Environmental monitoring -- Computer programs -- Periodicals; Ecology -- Computer simulation -- Periodicals; Digital computer simulation -- Periodicals; Computer software -- Periodicals; Environmental Monitoring -- Periodicals; Computer Simulation -- Periodicals; Environnement -- Surveillance -- Logiciels -- Périodiques; Écologie -- Simulation, Méthodes de -- Périodiques; Simulation par ordinateur -- Périodiques; Logiciels -- Périodiques; Computer software; Digital computer simulation; Ecology -- Computer simulation; Environmental monitoring -- Computer programs; Periodicals; Electronic journals; 363.70015118
• DOI: 10.1016/j.envsoft.2020.104759
• Languages: English
• ISSNs: 1364-8152

Abstract

Abstract: Landscape simulation is necessary for stakeholders to discuss future landscapes with new designs in order to preserve good landscapes. Augmented reality can be used to study the future landscape on a large scale by adding a three-dimensional design model to the real world. On the other hand, diminished reality (DR) can simulate the virtual demolition and removal of structures in redevelopment. However, it has not been possible to visually remove moving landscape objects such as vehicles and pedestrians in real time for accurate landscape simulation. This research develops a DR system that can virtually remove moving landscape objects by implementing real-time object detection using deep learning with a game engine, as well as immobile objects such as structures. In addition to evaluating the performance of detecting the size of moving landscape objects, the developed DR system is applied to large-scale landscape simulation at two sites, and its utility is validated. Graphical abstract: Image 1 Highlights: A diminished reality (DR) with real-time moving object detection is proposed. A prototype system was developed on a game engine combined with deep learning. Accuracy of detecting moving objects such as cars and people was evaluated. Landscape simulations using the DR method were feasible and effective.