Fast writer adaptation with style extractor network for handwritten text recognition. (March 2022)

• Record Type: Journal Article
• Authors: Wang, Zi-Rui; Du, Jun
• Is Part Of: Neural networks. Volume 147(2022)
• Page Start: 42
• Page End: 52
• Publication Date: 2022-03
• Subjects: Fast writer adaptation -- Style extractor network -- Offline handwritten text recognition; Neural computers -- Periodicals; Neural networks (Computer science) -- Periodicals; Neural networks (Neurobiology) -- Periodicals; Nervous System -- Periodicals; Ordinateurs neuronaux -- Périodiques; Réseaux neuronaux (Informatique) -- Périodiques; Réseaux neuronaux (Neurobiologie) -- Périodiques; Neural computers; Neural networks (Computer science); Neural networks (Neurobiology); Periodicals; 006.32
• DOI: 10.1016/j.neunet.2021.12.002
• Languages: English
• ISSNs: 0893-6080

Abstract

Abstract: Writing style is an abstract attribute in handwritten text. It plays an important role in recognition systems and is not easy to define explicitly. Considering the effect of writing style, a writer adaptation method is proposed to transform a writer-independent recognizer toward a particular writer. This transformation has the potential to significantly increase accuracy. In this paper, under the deep learning framework, we propose a general fast writer adaptation solution. Specifically, without depending on other complex skills, a well designed style extractor network (SEN) trained by identification loss (IDL) is introduced to explicitly extract personalized writer information. The architecture of SEN consists of a stack of convolutional layers followed by a recurrent neural network with gated recurrent units to remove semantic context and retain writer information. Then, the outputs of the GRU are further integrated into a one-dimensional vector that is adopted to represent writing style. Finally, the extracted style information is fed into the writer-independent recognizer to achieve adaptation. Validated on offline handwritten text recognition tasks, the proposed fast sentence-level adaptation achieves remarkable improvements in Chinese and English text recognition tasks. Specifically, in the HETR task, a multi-information fusion network that is equipped with a hybrid attention mechanism and that integrates visual features, context features and writing style is proposed. In addition, under the same condition (only one writer-specific text line used as adaptation data), the proposed solution, without consuming extra time, can significantly outperform the previous multiple-pass decoding method. The code is available at https://github.com/Wukong90/Handwritten-Text-Recognition .