Automatic detection of behavioural codes in team interactions. (July 2022)

• Record Type: Journal Article
• Authors: Hasan, Madina; Jefferson, Nicholas; Hain, Thomas; Dawson, Jeremy
• Is Part Of: Computer speech & language. Volume 74(2022)
• Publication Date: 2022-07
• Subjects: Behaviour codes recognition -- Teamwork -- Small group interaction -- Spoken discourse understanding -- CRF -- RNN; Speech processing systems -- Periodicals; Automatic speech recognition -- Periodicals; Computers -- Periodicals; Linguistics -- Periodicals; Speech-Language Pathology -- Periodicals; Traitement automatique de la parole -- Périodiques; Reconnaissance automatique de la parole -- Périodiques; Automatic speech recognition; Speech processing systems; Electronic journals; Periodicals; 006.454
• DOI: 10.1016/j.csl.2021.101339
• Languages: English
• ISSNs: 0885-2308

Abstract

Abstract: This paper investigates the feasibility of the task of automatic behaviour coding of spoken interactions in teamwork settings. We introduce the coding schema used to classify the behaviours of the group members and the corpus we collected to assess the coding schema reliability in real teamwork meetings. The behaviours embedded in spoken utterances are modelled using a discriminative approach based on conditional random fields, and state-of-the-art neural networks based models. Moreover, we fine-tune publicly available language models to fit our target domain and task and demonstrate how this type of knowledge transfer improves classification models' generalisation capacity. To utilise public resources, the AMI corpus was used for deploying the proposed framework. However, the models were evaluated on both AMI (matched task) and recordings of students solving an engineering challenge (mismatched task). Evaluation results reveal that neural networks are the best performing models in matched tasks, but that C R F models outperform them in mismatched tasks. Mitigating the effect of noisy data, by implementing a lightly supervised approach leads to relative improvements of 32% and 22%, in F 1 measures of C R F and B E R T, respectively. The proposed classifiers are used as a part of technological support to the training programme in collaborative skills for undergraduate students. Highlights: The viability of automatic verbal behaviour coding using speech transcriptions is investigated. A new corpus with annotations following a precise metadiscourse coding scheme is introduced to the field of automatic behaviour analysis in teamwork meetings. The applied coding scheme is based on a well-studied psychological theory that is designed to wisely observe to provide valuable feedback. A lightly supervised approach that produces a better version of manually coded training data is also proposed. Lexical and contextual text features are used to train sequential and neural network automatic metadiscourse tagging models. The performance of these models is compared under different hierarchical levels of behaviour analysis. To evaluate the proposed approach, AMI (a matched set) and a real application set taken from the ENG Challenge data (unmatched set), test sets are used.