A facial expression controlled wheelchair for people with disabilities. (October 2018)

• Record Type: Journal Article
• Authors: Rabhi, Yassine; Mrabet, Makrem; Fnaiech, Farhat
• Is Part Of: Computer methods and programs in biomedicine. Volume 165(2018)
• Page Start: 89
• Page End: 105
• Publication Date: 2018-10
• Subjects: Smart wheelchair -- Facial expression -- Engineering rehabilitation -- Artificial intelligence; Medicine -- Computer programs -- Periodicals; Biology -- Computer programs -- Periodicals; Computers -- Periodicals; Medicine -- Periodicals; Médecine -- Logiciels -- Périodiques; Biologie -- Logiciels -- Périodiques; Biology -- Computer programs; Medicine -- Computer programs; Periodicals; Electronic journals; 610.28
• DOI: 10.1016/j.cmpb.2018.08.013
• Languages: English
• ISSNs: 0169-2607

Abstract

Highlights: A new intelligent and semi-autonomous control wheelchair is developed in order to overcome the shortcomings of the standard joystick. The control of EWC is determined by the emotion of the face selected by the user, while it's going and stopping. This smart control system is suitable for a large patient group, who cannot use the wheelchair manipulated by a standard joystick. The user can customize the control by choosing the appropriate facial expressions for him/her using an intelligent algorithm, based on a neural network controller (NN). This EWC presents an efficient hands-free option that doesn't require sensors or contraptions attached to the user's body. A three-dimensional simulation environment has been performed to illustrate the new control devise with extreme security. Abstract: Background and Objectives: In order to improve assistive technologies for people with reduced mobility, this paper develops a new intelligent real-time emotion detection system to control equipment, such as electric wheelchairs (EWC) or robotic assistance vehicles. Every year, degenerative diseases and traumas prohibit thousands of people to easily control the joystick of their wheelchairs with their hands. Most current technologies are considered invasive and uncomfortable such as those requiring the user to wear some body sensor to control the wheelchair. Methods: In this work, the proposed Human Machine Interface (HMI) provides an efficient hands-free option that does not require sensors or objects attached to the user's body. It allows the user to drive the wheelchair using its facial expressions which can be flexibly updated. This intelligent solution is based on a combination of neural networks (NN) and specific image preprocessing steps. First, the Viola-Jones combination is used to detect the face of the disability from a video. Subsequently, a neural network is used to classify the emotions displayed on the face. This solution called "The Mathematics Behind Emotion" is capable of classifying many facial expressions in real time, such as smiles and raised eyebrows, which are translated into signals for wheelchair control. On the hardware side, this solution only requires a smartphone and a Raspberry Pi card that can be easily mounted on the wheelchair. Results: Many experiments have been conducted to evaluate the efficiency of the control acquisition process and the user experience in driving a wheelchair through facial expressions. The classification accuracy can expect 98.6% and it can offer an average recall rate of 97.1%. Thus, all these experiments have proven that the proposed system is able of accurately recognizing user commands in real time. Indeed, the obtained results indicate that the suggested system is more comfortable and better adapted to severely disabled people in their daily lives, than conventional methods. Among the advantages of this system, we cite its real time ability to identify facial emotions from different angles. Conclusions: The proposed system takes into account the patient's pathology. It is intuitive, modern, doesn't require physical effort and can be integrated into a smartphone or tablet. The results obtained highlight the efficiency and reliability of this system, which ensures safe navigation for the disabled patient.