A recommendation system for meta-modeling: A meta-learning based approach. (15th March 2016)

• Record Type: Journal Article
• Authors: Cui, Can; Hu, Mengqi; Weir, Jeffery D.; Wu, Teresa
• Is Part Of: Expert systems with applications. Volume 46(2016)
• Page Start: 33
• Page End: 44
• Publication Date: 2016-03-15
• Subjects: Meta-learning -- Meta-model -- Simulation -- Recommendation system -- Algorithm selection -- Feature reduction; Expert systems (Computer science) -- Periodicals; Systèmes experts (Informatique) -- Périodiques; Electronic journals; 006.33
• DOI: 10.1016/j.eswa.2015.10.021
• Languages: English
• ISSNs: 0957-4174

Abstract

Highlights: A meta-learning based recommendation system for meta-modeling is proposed. Novel meta-features for geometrical characterization on black-box problems are proposed. Model-based meta-learners generally outperforms instance-based meta-leaners. Singular value decomposition boosts the performance of the recommendation system. Experimental results indicate the proposed system significantly improves the modeling efficiency and facilitates model selection. Abstract: Various meta-modeling techniques have been developed to replace computationally expensive simulation models. The performance of these meta-modeling techniques on different models is varied which makes existing model selection/recommendation approaches (e.g., trial-and-error, ensemble) problematic. To address these research gaps, we propose a general meta-modeling recommendation system using meta-learning which can automate the meta-modeling recommendation process by intelligently adapting the learning bias to problem characterizations. The proposed intelligent recommendation system includes four modules: (1) problem module, (2) meta-feature module which includes a comprehensive set of meta-features to characterize the geometrical properties of problems, (3) meta-learner module which compares the performance of instance-based and model-based learning approaches for optimal framework design, and (4) performance evaluation module which introduces two criteria, Spearman's ranking correlation coefficient and hit ratio, to evaluate the system on the accuracy of model ranking prediction and the precision of the best model recommendation, respectively. To further improve the performance of meta-learning for meta-modeling recommendation, different types of feature reduction techniques, including singular value decomposition, stepwise regression and ReliefF, are studied. Experiments show that our proposed framework is able to achieve 94% correlation on model rankings, and a 91% hit ratio on best model recommendation. Moreover, the computational cost of meta-modeling recommendation is significantly reduced from an order of minutes to seconds compared to traditional trial-and-error and ensemble process. The proposed framework can significantly advance the research in meta-modeling recommendation, and can be applied for data-driven system modeling.