The Cobb-Douglas Learning Machine. (August 2022)

• Record Type: Journal Article
• Authors: Maldonado, Sebastián; López, Julio; Carrasco, Miguel
• Is Part Of: Pattern recognition. Volume 128(2022)
• Publication Date: 2022-08
• Subjects: Cobb-Douglas -- Minimax Probability Machine -- Minimum Error Minimax Probability Machine -- Second-order Cone Programming -- Support Vector Machines; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2022.108701
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: A novel Minimum Error Minimax Probability Machine (MEMPM) method is presented. The Cobb-Douglas production function is extended to machine learning. The proposal is a robust formulation for linear and kernel-based classification. The method is solved via a self-developed two-step alternating algorithm. We prove that the optimization scheme converges to the optimal solution of the problem. Best performance is achieved in experiments carried out on 17 benchmark datasets. Abstract: In this paper, we propose a novel machine learning approach based on robust optimization. Our proposal defines the task of maximizing the two class accuracies of a binary classification problem as a Cobb-Douglas function. This function is well known in production economics and is used to model the relationship between two or more inputs as well as the quantity produced by those inputs. A robust optimization problem is defined to construct the decision function. The goal of the model is to classify each training pattern correctly, up to a given class accuracy, even for the worst possible data distribution. We demonstrate the theoretical advantages of the Cobb-Douglas function in terms of the properties of the resulting second-order cone programming problem. Important extensions are proposed and discussed, including the use of kernel functions and regularization. Experiments performed on several classification datasets confirm these advantages, leading to the best average performance in comparison to various alternative classifiers.