Quality evaluation models or frameworks for open source software: A systematic literature review. Issue 6 (30th April 2022)

• Record Type: Journal Article
• Authors: Yılmaz, Nebi; Kolukısa Tarhan, Ayça
• Is Part Of: Journal of software. Volume 34:Issue 6(2022)
• Page Start: n/a
• Page End: n/a
• Publication Date: 2022-04-30
• Subjects: open source software -- OSS -- quality attribute -- quality evaluation -- quality model -- software quality; Software engineering -- Periodicals; Computer software -- Development -- Periodicals; Software maintenance -- Periodicals; 005.1
• DOI: 10.1002/smr.2458
• Languages: English
• ISSNs: 2047-7473

Abstract

Abstract: As open project repositories have become widespread, evaluating the quality of open source software (OSS) has gained attention in the software community. Despite the variety in quality evaluation models or frameworks (QEMoF) proposed for OSS, their adoption in practice remains limited. To further investigate this argument, a systematic literature review (SLR) has been conducted to characterize the existing QEMoF for OSS, based on particular properties, and to examine comprehensively their content and structure for identifying the gap between theory and practice. In this context, a total of 36 primary studies in the period of 2003–2020 have been analyzed. It has been identified that the majority of QEMoF takes ISO/IEC 9126 as a reference, evaluates community‐based aspects, mostly evaluates maintainability and usability as quality attributes, supports subjective and quantitative evaluations, and faces challenges due to diverse and dynamic nature of OSS products. For a better overview of the state of the art in the field, evaluation criteria for comparing the overall quality of the QEMoF are identified and applied. Finally, overall findings are discussed, and suggestions are provided for practitioners and researchers to formulate more robust studies in the future, which can advance the current situation for the benefit of the software community. Abstract : In this study, it has been identified that the majority of models/frameworks takes ISO/IEC 9126 as reference, evaluates community‐based aspects, mostly evaluates maintainability and usability as quality attributes, support subjective and quantitative evaluations, and faces challenges due to diverse and dynamic nature of OSS products. The results are presented together with total score (based on a number of RQs answered), which can be used to compare the overall quality of the models/frameworks to guide potential users, as the final output.