Mining and searching app reviews for requirements engineering: Evaluation and replication studies. Issue 114 (March 2023)

• Record Type: Journal Article
• Authors: Dąbrowski, Jacek; Letier, Emmanuel; Perini, Anna; Susi, Angelo
• Is Part Of: Information systems. Issue 114(2023)
• Publication Date: 2023-03
• Subjects: Mining user reviews -- Software engineering -- Feature extraction -- Sentiment analysis -- Searching for feature-related reviews -- Empirical study; Database management -- Periodicals; Electronic data processing -- Periodicals; Bases de données -- Gestion -- Périodiques; Informatique -- Périodiques; Database management; Electronic data processing; Periodicals; 005.7
• DOI: 10.1016/j.is.2023.102181
• Languages: English
• ISSNs: 0306-4379

Abstract

Abstract: App reviews provide a rich source of feature-related information that can support requirement engineering activities. Analyzing them manually to find this information, however, is challenging due to their large quantity and noisy nature. To overcome the problem, automated approaches have been proposed for 'feature-specific analysis'. Unfortunately, the effectiveness of these approaches has been evaluated using different methods and datasets. Replicating these studies to confirm their results and to provide benchmarks of different approaches is a challenging problem. We address the problem by extending previous evaluations and performing a comparison of these approaches. In this paper, we present two empirical studies. In the first study, we evaluate opinion mining approaches; the approaches extract features discussed in app reviews and identify their associated sentiments. In the second study, we evaluate approaches searching for feature-related reviews. The approaches search for users' feedback pertinent to a particular feature. The results of both studies show these approaches achieve lower effectiveness than reported originally, and raise an important question about their practical use. Highlights: Study the performance of mining and searching techniques applied to app reviews. The new dataset can improve the quality of evaluation and replication studies. Opinion mining approaches achieve lower effectiveness than reported originally. Existing searching tools can be useful for requirements engineering use cases. Researchers should pay more attention to the efficiency of their approaches.