A Triplet network framework based automatic assessment of simulation quality for respiratory droplet propagation. (November 2021)

• Record Type: Journal Article
• Authors: Hu, Jinlong; Xu, Songhua; Ding, Xiangdong
• Is Part Of: Pattern recognition. Volume 119(2021)
• Publication Date: 2021-11
• Subjects: Simulation quality assessment -- Respiratory droplet propagation -- Triplet network -- Multi-scale CNN-BiLSTM -- Attentive temporal pooling; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2021.108060
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: A Triplet network framework with attentive temporal pooling is proposed, which is the first trial to assess the quality of simulation for droplet respiratory propagation automatically. A regularization constraint for triplet loss boosts the performance. The proposed approach has generalization for other applications such as 3D object retrieval. Abstract: Respiratory droplet propagation has been extensively explored with simulation and experimental methods. However, there still exists a huge gap between these methods, making automatic assessment of simulation quality quantitatively being a challenge. To address above problem, in this work, a triplet neural network framework with multi-scale CNN-BiLSTM network is developed. Firstly, Conditional Variational Auto-Encoder (CVAE) is utilized to generate multi-view simulations. Secondly, YOLOv3 is adopted to extract droplet regions of real image and simulation results. Then, a multi-scale CNN-BiLSTM network with attentive temporal pooling is designed to extract and aggregate temporal information across consecutive frames. Finally, all above networks are constructed into a triplet structure with triplet loss, and a regularization constraint being denoted as reconstruction term and prediction term is proposed. To demonstrate the performance of our approach, a new dataset is established including real sequences of cough droplets and simulation results. We validate the effectiveness and feasibility of our proposed framework using our dataset and two benchmarks, the PSB dataset and the ETH dataset, for 3D object retrieval. Our approach outperforms state-of-the-arts on our dataset and achieves comparative performance on PSB and ETH for 3D object retrieval, given quantitative quality assessment of simulation for droplet respiratory propagation automatically.