VSTAR: Visual Semantic Thumbnails and tAgs Revitalization. (1st May 2022)

• Record Type: Journal Article
• Authors: Carta, Salvatore; Giuliani, Alessandro; Piano, Leonardo; Podda, Alessandro Sebastian; Reforgiato Recupero, Diego
• Is Part Of: Expert systems with applications. Volume 193(2022)
• Publication Date: 2022-05-01
• Subjects: Machine learning -- Video tagging -- Thumbnail enrichment -- Google trends -- Semantic enrichment; Expert systems (Computer science) -- Periodicals; Systèmes experts (Informatique) -- Périodiques; Electronic journals; 006.33
• DOI: 10.1016/j.eswa.2021.116375
• Languages: English
• ISSNs: 0957-4174

Abstract

Abstract: Nowadays, video-sharing portals' popularity has entailed massive growth in data uploads over the Internet. For several applications (e.g., browsing, retrieval, or recommendation of videos), dealing with vast data volumes has become a critical issue. In a video-sharing scenario, the devising of tools and infrastructures able to completely satisfy users' interests and requests is becoming increasingly crucial to influence their online experiences. On the one hand, annotating a video with meaningful human-friendly words (i.e., tags) plays an essential role in matching users' interests. On the other hand, providing a condensed and straightforward preview of the video content (i.e., thumbnails) is crucial to capture the user's attention immediately. In this context, we propose VSTAR (Visual Semantic Thumbnails and tAgs Revitalization), a novel approach in video optimization aimed at generating both suitable tags and thumbnails from a different perspective than classical approaches. The novelty lies in: (i) exploiting image captioning to extract visual and semantic information for generating both tags and thumbnails; (ii) identifying semantically related popular search queries (i.e., trends) to be suggested as new tags; (iii) giving the final user the control on a trade-off between quality and quantity of the generated items (tags and thumbnails); (iv) creating a proper dataset and making it publicly available. Experiments demonstrate the viability of our proposal. Highlights: Exploiting image captioning to simultaneously suggest tags and thumbnails. Retrieval of semantically relevant trends to be suggested as video tags. Providing of user-driven trade-off between tags/thumbnails quality and quantity. A proper dataset of YouTube videos has been built and released. Results confirmed that the system suggests strongly relevant tags and thumbnail.