A multi-task approach for contrastive learning of handwritten signature feature representations. (1st May 2023)

• Record Type: Journal Article
• Authors: Viana, Talles B.; Souza, Victor L.F.; Oliveira, Adriano L.I.; Cruz, Rafael M.O.; Sabourin, Robert
• Is Part Of: Expert systems with applications. Volume 217(2023)
• Publication Date: 2023-05-01
• Subjects: Signature verification -- Convolutional neural networks -- Feature learning -- Deep learning -- Contrastive learning; Expert systems (Computer science) -- Periodicals; Systèmes experts (Informatique) -- Périodiques; Electronic journals; 006.33
• DOI: 10.1016/j.eswa.2023.119589
• Languages: English
• ISSNs: 0957-4174

Abstract

Abstract: In spite of recent advances in computer vision, the classic problem of offline handwritten signature verification still remains challenging. The signature verification task has a high intra-class variability because a given user often shows high variability between its samples. Besides, signature verification is harder in the presence of skilled forgeries. Recently, in order to tackle these challenges, the research community has investigated deep learning methods for learning feature representations of handwritten signatures. When mapping signatures to a feature space, it is desired to obtain dense clusters of signature's representations, in order to deal with intra-class variability. Besides, not only dense clusters are required but also a larger separation between different user's clusters in the feature space. Finally, it is also desired to move away feature representations of skilled forgeries in relation to the respective dense cluster of genuine representations. This last property is hard to achieve in the real-world scenario because skilled forgeries are not readily available during training. In this work, we hypothesize that such properties can be achieved by means of a multi-task framework for learning handwritten signature feature representations based on deep contrastive learning. The proposed framework is composed of two objective-specific tasks. The first task aims to map signature examples of the same user closer within the feature space, while separating the feature representations of signatures of different users. The second task aims to adjust the skilled forgeries representations by adopting contrastive losses with the ability to perform hard negative mining. Hard negatives are examples from different classes with some degree of similarity that can be applied for training. We evaluated models obtained with the proposed framework in terms of the equal error rate on GPDSsynthetic, CEDAR and MCYT-75 datasets in writer-dependent and writer-independent verification approaches. Using synthetic and real signature datasets, Friedman tests with Bonferroni–Dunn post hoc tests were performed to compare the proposed multi-task contrastive models against the popular SigNet model as a baseline. Experiments demonstrated an statistically significant improvement in signature verification with a multi-task contrastive model based on the Triplet loss. Implementation of the method is available for download at https://github.com/tallesbrito/contrastive_sigver . Highlights: We propose a framework for the contrastive learning of signature representations. The method uses similar signatures from different users to discriminate forgeries. Models following the proposed framework generalize well to 4 different datasets. Experiments show a statistically significant improvement to the SigNet model. We found improvements for writer-dependent and writer-independent verification.