ISeePlus : A cost effective smart assistance archetype based on deep learning model for visually impaired. Issue 7 (2nd October 2020)

• Record Type: Journal Article
• Authors: Virmani, Deepali; Gupta, Charu; Bamdev, Pakhi; Jain, Prateek
• Is Part Of: Journal of information & optimization sciences. Volume 41:Issue 7(2020)
• Page Start: 1741
• Page End: 1756
• Publication Date: 2020-10-02
• Subjects: 68T11; Deep learning -- Computer vision -- Machine learning -- Android application -- Raspberry Pi3 -- Visual impairment -- Assistive technology -- Convolution neural network; Electronic data processing -- Periodicals; Information science -- Periodicals; Mathematical optimization -- Periodicals; 519.6
• DOI: 10.1080/02522667.2020.1812198
• Languages: English
• ISSNs: 0252-2667

Abstract

Abstract: In this paper, a unique smart assistance aid, iSeePlus, for visually impaired people is presented. iSeePlus, as a prototype product, is equipped with smart glasses (iSee Glasses), a smart stick (iStick) and an android application which will be initially used to set up the device. The smart glasses are trained on proposed deep learning models to provide features like face detection and recognition, emotion detection and image captioning with a click of a specifically placed buttons on the device. The proposed prototype product also supports reading texts from images via Google Cloud Vision API and allows the user to send alert texts with his current location to the saved emergency contacts. The iStick is an integrated accessory that comes with iSeePlus which creates vibrations to alert the user of any incoming obstacle within the range of 25 cm. The android application will be used to set up the device at initial stages as it helps to add a database of known people and the emergency contacts. Presently, iSeePlus, has an accuracy of 98.03% on face detection and recognition model, FDREnet, 68.5% on emotion detection with ExpressNet and 65.7% on image captioning with CaptionMeNet BLUE-1: 65.7). With the help of iSeePlus, people suffering from visual impairment will now be able to do their day-to-day chores independently up to a significant degree.