HMMCF:A human-computer collaboration algorithm based on multimodal intention of reverse active fusion. Issue 169 (January 2023)

• Record Type: Journal Article
• Authors: Lang, Xujie; Feng, Zhiquan; Yang, Xiaohui; Xu, Tao
• Is Part Of: International journal of human-computer studies. Issue 169(2023)
• Publication Date: 2023-01
• Subjects: Human-computer collaboration -- Human-computer interaction -- Multimodal -- Interactive intentions -- Intention understanding; Human-machine systems -- Periodicals; Systems engineering -- Periodicals; Human engineering -- Periodicals; Human engineering; Human-machine systems; Systems engineering; Periodicals; Electronic journals; 004.019
• DOI: 10.1016/j.ijhcs.2022.102916
• Languages: English
• ISSNs: 1071-5819

Abstract

Highlights: The focus of this research is to develop a human-computer interaction system suitable for the elderly care robot. It can extract the intention of fuzzy expression, and the accuracy rate is 97.7%. It can avoid the wrong intention of the elderly due to forgetfulness. A human-computer collaborative division of labor decision is proposed to reduce the user interaction compliance. This paper puts forward a valuable research path for a series of challenging problems in the environment of helping the elderly. Abstract: This paper proposes a fusion model of an improved Hidden Markov Model (HMM) and an improved CF trust evaluation algorithm, and a human-computer collaboration mechanism embedded in the X-arm manipulator. It can correctly predict user intentions of interaction, correct erroneous intentions, and carry out human-computer collaboration under the user's multimodal fuzzy expression or wrong expression, and finally complete the intention task. It can correctly predict the user interactive intentions, correct the wrong intention, and carry out human-computer collaboration in the case of user multimodal fuzzy expression or wrong expression. The main contributions of this paper are: when humans input intentions to robots, users no longer interact strictly through discrete and explicit interactive operations, but can use non-deterministic and unclear multimodal data to express interactive intentions; When the model predicts user's intentions, it comprehensively considers modal information such as user's voice, gesture, posture, and extracts the user's intention from the user's daily habits and the context of the interactive environment. The process of trust evaluation, reverse analysis, and active acquisition of necessary information on the user's intention determines whether the intention is feasible, and then analyzes the intention through the human-computer collaboration mechanism, reasonably assigns the tasks that the human and the computer need to complete. Finally, human and computer cooperate together to achieve the user's intention. The algorithm proposed in this paper can be considered as a new method of intention understanding in human-computer interaction and a research of exploring the computational principle of natural interaction. The algorithm proposed in this paper has been verified in many daily life scenarios such as helping the elderly to drink water, take medicine, read, watch TV and so on, and it has also proposed a valuable research path for the challenging problem of human-computer interaction.