Understanding social relationships in egocentric vision. Issue 12 (December 2015)

• Record Type: Journal Article
• Authors: Alletto, Stefano; Serra, Giuseppe; Calderara, Simone; Cucchiara, Rita
• Is Part Of: Pattern recognition. Volume 48:Issue 12(2015:Dec.)
• Page Start: 4082
• Page End: 4096
• Publication Date: 2015-12
• Subjects: Egocentric vision -- Social interactions -- Group detection -- Video analysis -- Head pose estimation; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2015.06.006
• Languages: English
• ISSNs: 0031-3203

Abstract

Abstract: The understanding of mutual people interaction is a key component for recognizing people social behavior, but it strongly relies on a personal point of view resulting difficult to be a-priori modeled. We propose the adoption of the unique head mounted cameras first person perspective (ego-vision) to promptly detect people interaction in different social contexts. The proposal relies on a complete and reliable system that extracts people׳s head pose combining landmarks and shape descriptors in a temporal smoothed HMM framework. Finally, interactions are detected through supervised clustering on mutual head orientation and people distances exploiting a structural learning framework that specifically adjusts the clustering measure according to a peculiar scenario. Our solution provides the flexibility to capture the interactions disregarding the number of individuals involved and their level of acquaintance in context with a variable degree of social involvement. The proposed system shows competitive performances on both publicly available ego-vision datasets and ad hoc benchmarks built with real life situations. Abstract : Highlights: A head pose estimation method designed to work in ego-vision scenarios is provided. We define a 3D people localization method that works without any camera calibration. We estimate social groups with supervised correlation clustering and structural SVM. A tracking state-of-the-art evaluation applied to first person videos is provided.