A MapReduce implementation of posterior probability clustering and relevance models for recommendation. (October 2018)

• Record Type: Journal Article
• Authors: Valcarce, Daniel; Parapar, Javier; Barreiro, Álvaro
• Is Part Of: Engineering applications of artificial intelligence. Volume 75(2018)
• Page Start: 114
• Page End: 124
• Publication Date: 2018-10
• Subjects: Recommender systems -- Collaborative filtering -- MapReduce -- Relevance-based language models -- Posterior probability clustering -- Matrix factorisation; Engineering -- Data processing -- Periodicals; Artificial intelligence -- Periodicals; Expert systems (Computer science) -- Periodicals; Ingénierie -- Informatique -- Périodiques; Intelligence artificielle -- Périodiques; Systèmes experts (Informatique) -- Périodiques; Artificial intelligence; Engineering -- Data processing; Expert systems (Computer science); Periodicals; 620.00285
• DOI: 10.1016/j.engappai.2018.08.006
• Languages: English
• ISSNs: 0952-1976

Abstract

Abstract: Relevance-Based Language Models are a formal probabilistic approach for explicitly introducing the concept of relevance in the Statistical Language Modelling framework. Recently, they have been determined as a very effective way of computing recommendations. When combining this new recommendation approach with Posterior Probabilistic Clustering for computing neighbourhoods, the item ranking is further improved, radically surpassing rating prediction recommendation techniques. Nevertheless, in the current landscape where the number of recommendation scenarios reaching the big data scale is increasing day after day, high figures of effectiveness are not enough. In this paper, we address one urging and common need of recommendation systems which is algorithm scalability. Particularly, we adapted those highly effective algorithms to the functional MapReduce paradigm, that has been previously proved as an adequate tool for enabling recommenders scalability. We evaluated the performance of our approach under realistic circumstances, showing a good scalability behaviour on the number of nodes in the MapReduce cluster. Additionally, as a result of being able to execute our algorithms distributively, we can show measures in a much bigger collection supporting the results presented on the seminal paper.