A novel procedure to evaluate the performance of failure assessment models. (October 2022) - Lingyun, Guo; Markus, Niffenegger; Jing, Zhou
Highlights: Novel procedure for the objective and fair evaluation of failure assessment models. Procedure based on established mathematical methods. Procedure based on well-defined evaluation index system. Model performance demonstrated for 250 burst experiments on external axial cracked pipes. Evaluation of 9 well-known failure assessment models. Abstract: The quantification for overall performance of failure assessment models (FAM) is a challenging task. In the first part of this paper, we propose a novel procedure to evaluate FAM's overall performance based on reliability engineering and system safety. This procedure is composed of three parts: 1. proposing the evaluation index system for FAM's performance based on the uncertainty of Prediction Accuracy ( PA ); 2. calculating the FAM's overall performance scores, which contains three steps: determining the weights of the criteria in the evaluation index system; calculating the decision matrix for different FAMs; and computing FAM's overall performance; 3. analyzing the evaluation results to give recommendations. In the second part of this paper, we demonstrate the application of the novel procedure to evaluate the performance of 9 FAMs for the integrity assessment of pipes containing axial-oriented part-through cracks on the outer surface. This evaluation is based on 250 well-documented pipe burst experiments. The results show that: R6-Option 2A-local solution (2016) gives the best overall prediction. In addition, different Multi-Criteria Decision Making (MCDM) methods have different focuses. VIKOR and TOPSIS are more considering the impact of high weight criteria, especially VIKOR. SWM and PROMETHEE II care more about FAM's overall performance.