Deep learning based virtual point tracking for real-time target-less dynamic displacement measurement in railway applications. (1st March 2022)

• Record Type: Journal Article
• Authors: Shi, Dachuan; Šabanovič, Eldar; Rizzetto, Luca; Skrickij, Viktor; Oliverio, Roberto; Kaviani, Nadia; Ye, Yunguang; Bureika, Gintautas; Ricci, Stefano; Hecht, Markus
• Is Part Of: Mechanical systems and signal processing. Volume 166(2022)
• Publication Date: 2022-03-01
• Subjects: Point tracking -- Computer vision -- Displacement measurement -- Photogrammetry -- Deep learning -- Railway; Structural dynamics -- Periodicals; Vibration -- Periodicals; Constructions -- Dynamique -- Périodiques; Vibration -- Périodiques; Structural dynamics; Vibration; Periodicals; 621
• DOI: 10.1016/j.ymssp.2021.108482
• Languages: English
• ISSNs: 0888-3270

Abstract

Highlights: A novel approach of virtual point tracking for target-less displacement measurement. Real-time point detection using lightweight convolutional neural network. A rule engine based on railway domain knowledge is defined for point tracking. Implementation of the proposed approach for real-time edge computing. Our codes and data are available at the Github repository. Abstract: In the application of computer-vision-based displacement measurement, an optical target is usually required to prove the reference. If the optical target cannot be attached to the measuring objective, edge detection and template matching are the most common approaches in target-less photogrammetry. However, their performance significantly relies on parameter settings. This becomes problematic in dynamic scenes where complicated background texture exists and varies over time. We propose virtual point tracking for real-time target-less dynamic displacement measurement, incorporating deep learning techniques and domain knowledge to tackle this issue. Our approach consists of three steps: 1) automatic calibration for detection of region of interest; 2) virtual point detection for each video frame using deep convolutional neural network; 3) domain-knowledge based rule engine for point tracking in adjacent frames. The proposed approach can be executed on an edge computer in a real-time manner (i.e. over 30 frames per second). We demonstrate our approach for a railway application, where the lateral displacement of the wheel on the rail is measured during operation. The numerical experiments have been performed to evaluate our approach's performance and latency in a harsh railway environment with dynamic complex backgrounds. We make our code and data available at https://github.com/quickhdsdc/Point-Tracking-for-Displacement-Measurement-in-Railway-Applications .