COVID-19 detection in cough, breath and speech using deep transfer learning and bottleneck features. (February 2022)

• Record Type: Journal Article
• Authors: Pahar, Madhurananda; Klopper, Marisa; Warren, Robin; Niesler, Thomas
• Is Part Of: Computers in biology and medicine. Volume 141(2022)
• Publication Date: 2022-02
• Subjects: COVID-19 -- Cough -- Breath -- Speech -- Transfer learning -- Bottleneck features; Medicine -- Data processing -- Periodicals; Biology -- Data processing -- Periodicals; 610.285
• DOI: 10.1016/j.compbiomed.2021.105153
• Languages: English
• ISSNs: 0010-4825

Abstract

Abstract: We present an experimental investigation into the effectiveness of transfer learning and bottleneck feature extraction in detecting COVID-19 from audio recordings of cough, breath and speech. This type of screening is non-contact, does not require specialist medical expertise or laboratory facilities and can be deployed on inexpensive consumer hardware such as a smartphone. We use datasets that contain cough, sneeze, speech and other noises, but do not contain COVID-19 labels, to pre-train three deep neural networks: a CNN, an LSTM and a Resnet50. These pre-trained networks are subsequently either fine-tuned using smaller datasets of coughing with COVID-19 labels in the process of transfer learning, or are used as bottleneck feature extractors. Results show that a Resnet50 classifier trained by this transfer learning process delivers optimal or near-optimal performance across all datasets achieving areas under the receiver operating characteristic (ROC AUC) of 0.98, 0.94 and 0.92 respectively for all three sound classes: coughs, breaths and speech. This indicates that coughs carry the strongest COVID-19 signature, followed by breath and speech. Our results also show that applying transfer learning and extracting bottleneck features using the larger datasets without COVID-19 labels led not only to improved performance, but also to a marked reduction in the standard deviation of the classifier AUCs measured over the outer folds during nested cross-validation, indicating better generalisation. We conclude that deep transfer learning and bottleneck feature extraction can improve COVID-19 cough, breath and speech audio classification, yielding automatic COVID-19 detection with a better and more consistent overall performance. Highlights: Coughs carry the strongest COVID-19 signature followed by breath and speech. Transfer learning improves overall classifier performance on all types of vocal audio. Bottleneck features were extracted from a pre-trained model, trained on large data. Transfer learning reduces standard deviation across the cross-validation folds.