Table Structure Recognition and Form Parsing by End-to-End Object Detection and Relation Parsing. (December 2022)

• Record Type: Journal Article
• Authors: Li, Xiao-Hui; Yin, Fei; Dai, He-Sen; Liu, Cheng-Lin
• Is Part Of: Pattern recognition. Volume 132(2022)
• Publication Date: 2022-12
• Subjects: Table detection -- Table structure recognition -- Template-free form parsing -- Graph neural network -- End-to-end training; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2022.108946
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: A graph neural network (GNN) based unified framework named TSRNet is proposed to jointly detect and recognize the structures of various tables and forms. GNN is used to classify and group primitive regions into page objects and classify the relationships. The parameters of all the modules in the system is trained end-to-end to optimize the overall performance Superior performance has been achieved on table detection, table structure recognition and template-free form parsing. Abstract: The recognition of two-dimensional structure of tables and forms from document images is a challenge due to the complexity of document structures and the diversity of layouts. In this paper, we propose a graph neural network (GNN) based unified framework named Table Structure Recognition Network (TSRNet) to jointly detect and recognize the structures of various tables and forms. First, a multi-task fully convolutional network (FCN) is used to segment primitive regions such as text segments and ruling lines from document images, then a GNN is used to classify and group these primitive regions into page objects such as tables and cells. At last, the relationships between neighboring page objects are analyzed using another GNN based parsing module. The parameters of all the modules in the system can be trained end-to-end to optimize the overall performance. Experiments of table detection and structure recognition for modern documents on the POD 2017, cTDaR 2019 and PubTabNet datasets and template-free form parsing for historical documents on the NAF dataset show that the proposed method can handle various table/form structures and achieve superior performance.