One-vs-One classification for deep neural networks. (December 2020)

• Record Type: Journal Article
• Authors: Pawara, Pornntiwa; Okafor, Emmanuel; Groefsema, Marc; He, Sheng; Schomaker, Lambert R.B.; Wiering, Marco A.
• Is Part Of: Pattern recognition. Volume 108(2020:Dec.)
• Publication Date: 2020-12
• Subjects: Deep learning -- Computer vision -- Multi-class classification -- One-vs-One classification -- Plant recognition; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2020.107528
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: A novel One-vs-One classification method for deep neural networks is introduced. The One-vs-One scheme is compared to the standard One-vs-All scheme on four datasets. The results of two convolutional neural networks are better with the proposed method. When fine-tuning pre-trained architectures, the One-vs-All method performs best. Abstract: For performing multi-class classification, deep neural networks almost always employ a One-vs-All (OvA) classification scheme with as many output units as there are classes in a dataset. The problem of this approach is that each output unit requires a complex decision boundary to separate examples from one class from all other examples. In this paper, we propose a novel One-vs-One (OvO) classification scheme for deep neural networks that trains each output unit to distinguish between a specific pair of classes. This method increases the number of output units compared to the One-vs-All classification scheme but makes learning correct decision boundaries much easier. In addition to changing the neural network architecture, we changed the loss function, created a code matrix to transform the one-hot encoding to a new label encoding, and changed the method for classifying examples. To analyze the advantages of the proposed method, we compared the One-vs-One and One-vs-All classification methods on three plant recognition datasets (including a novel dataset that we created) and a dataset with images of different monkey species using two deep architectures. The two deep convolutional neural network (CNN) architectures, Inception-V3 and ResNet-50, are trained from scratch or pre-trained weights. The results show that the One-vs-One classification method outperforms the One-vs-All method on all four datasets when training the CNNs from scratch. However, when using the two classification schemes for fine-tuning pre-trained CNNs, the One-vs-All method leads to the best performances, which is presumably because the CNNs had been pre-trained using the One-vs-All scheme.