An ensemble multi-stream classifier for infant needs detection. Issue 4 (April 2023)

• Record Type: Journal Article
• Authors: Fahmy, Hesham Ahmed; Fahmy, Sherif Fadel; Del Barrio García, Alberto A.; Botella Juan, Guillermo
• Is Part Of: Heliyon. Volume 9:Issue 4(2023)
• Publication Date: 2023-04
• Subjects: 68T05 -- 68T07 -- 68T10 -- 68U10; Video classifier -- Infant needs -- Dunstan baby language -- Machine learning -- Deep learning -- Ensemble classifier; Research -- Periodicals; Medical sciences -- Periodicals; Natural history -- Periodicals; Social sciences -- Periodicals; Earth sciences -- Periodicals; Physical sciences -- Periodicals; 507.2
• DOI: 10.1016/j.heliyon.2023.e15098
• Languages: English
• ISSNs: 2405-8440

Abstract

Abstract: In this paper, we propose a novel multi-stream video classifier for infant needs detection. The proposed system is an ensemble-based system that combines several machine learning to improve the overall result of the state-of-the-art algorithms. It is a multi-stream in the sense that it combines the output predictions of both audio and images of infants from every single classifier employed in the system for a unified result. This produces better performance and results compared to the previous other research techniques, which relied on only one of these modalities. For training and testing the proposed system, from the Dunstan Baby Language video collection, we built three separate datasets for videos, images, and sounds encompassing the five primary infant needs that require predicting. These are: hunger, have wind, uncomfortable (require diaper change), wants to burp or tired, with a total of 3348 samples. We used four different ensemble algorithms for the best reachable performance. The proposed algorithm improves the overall accuracies of each single classifier from a low of 51% to a high of 99%. The proposed method also improves the accuracy of the classification process by about 9% compared to the state-of-the-art approaches, which was 90%. Highlights: Video classification. Infant needs detection. Machine Learning Algorithms. Deep Learning Algorithms. Ensemble Classification.