On the utilization of non‐quality assessed literature in software engineering research. Issue 7 (24th May 2022)

• Record Type: Journal Article
• Authors: Yasin, Affan; Fatima, Rubia; Liu, Lin; Ali Khan, Javed; Ali, Raian; Wang, Jianmin
• Is Part Of: Journal of software. Volume 34:Issue 7(2022)
• Page Start: n/a
• Page End: n/a
• Publication Date: 2022-05-24
• Subjects: grey literature -- indexing -- non‐quality assessed literature -- online literature -- quality assessment -- software engineering; Software engineering -- Periodicals; Computer software -- Development -- Periodicals; Software maintenance -- Periodicals; 005.1
• DOI: 10.1002/smr.2464
• Languages: English
• ISSNs: 2047-7473

Abstract

Abstract: Quality of research and knowledge sources can be a critical factor to judge the quality of the research citing them. The prevailing online dissemination of research via blogs, websites, experience reports, and white papers can be helpful as it minimizes potential publication biases. In this paper, we have (i) identified the body of software engineering (SE) literature that is non‐indexed by Web‐of‐Science but has been used in published SE systematic literature review (SLR) studies (as primary studies); (ii) proposed a checklist‐based method for quality assessment of non‐indexed and grey literature; (iii) empirically evaluated the effectiveness of the proposed method; (iv) searched primary studies retrieved from the SLRs in DBLP and Google Scholar (GS) to verify whether DBLP or Google Scholar can act as viable options to search for non‐indexed Web‐of‐Science (non‐WOS‐indexed) literature. Besides proposing our synthesized checklist method, we found that (i) SE literature reviews use significantly non‐indexed Web‐of‐Science sources; (ii) DBLP only covers a portion of the non‐WOS‐indexed reference, whereas GS has a more complete coverage of the non‐indexed reference in comparison. Preliminary evaluation of the proposed quality assessment point system‐based and checklist‐based methods shows it to be a viable way to assess quality of the non‐indexed and grey literature. Abstract : Besides proposing our synthesized checklist method, we found that (i) software engineering literature reviews use significantly non‐indexed Web‐of‐Science sources; (ii) DBLP only covers a portin of the non‐WOS‐indexed reference, whereas Google School has a more complete coverage of the non‐indexed reference in comparison. (iii) Preliminary evaluation of the proposed quality assessment point system‐based and checklist‐based methods shows it to be a viable way to assess quality of the non‐indexed and grey literature.