A classification scheme to improve conclusion instability using Bellwether moving windows. Issue 9 (20th July 2022)

• Record Type: Journal Article
• Authors: Mensah, Solomon; Kudjo, Patrick Kwaku
• Is Part Of: Journal of software. Volume 34:Issue 9(2022)
• Page Start: n/a
• Page End: n/a
• Publication Date: 2022-07-20
• Subjects: Bellwether moving window -- conclusion instability -- Eubank's optimal spacing theory -- growing portfolio -- software effort classification; Software engineering -- Periodicals; Computer software -- Development -- Periodicals; Software maintenance -- Periodicals; 005.1
• DOI: 10.1002/smr.2488
• Languages: English
• ISSNs: 2047-7473

Abstract

Abstract: Context: The use of a subset of recently completed and exemplary data, namely, Bellwether moving window (BMW) has proven successful to result in improved accuracy in software effort estimation (SEE). These outcomes were achieved based on the theory that estimation outcome of a future event depends on previous events. Thus, the existence of a BMW yield improved prediction accuracy for new project estimation. However, the conclusion instability problem across learners still threatens the reliability of SEE for new projects. Such instability concerns are attributed to the data subset considered for the training and validation needs of learners. Objective: To investigate whether the use of BMWs together with an effort classification scheme can minimize the conclusion instability problem across learners. Method: We apply a Bellwether method comprising of three operators, namely, SORT+CLUSTER, GENERATE_TPM, and APPLY to sample the BMW from a pool of chronological projects from the Maxwell and International Software Benchmarking Standards Group (ISBSG) datasets. The sampled BMW is benchmarked against the entire collection of preprocessed projects, namely, growing portfolio to evaluate prediction and classification accuracy across a set of learners–ElasticNet regression, deep neural networks, and automatically transformed linear model. Results: (1) BMW exists in the studied projects and (2) training the learners with a BMW of average window size 28.5%–75.5% of the growing portfolio (not older than 3 years) relatively minimizes the conclusion instability of prediction results. Conclusion: When BMWs are available, we recommend their use for estimating the effort for a new project to minimize the conclusion instability problem. Abstract : Bellwether data that constitute the Bellwether moving window (BMW) are evident in the studied datasetsTraining and validation needs with BMW of window size 28.5%–75.5% (of the growing portfolio), and not older than 3 years relatively minimizes the conclusion instability of prediction results across the 3 learners, namely, ElasticNet, ATLM and Deep neural networksThe deep neural networks together with the Gaussian weighted BMW resulted in improved prediction accuracy