A Recursive tree-structured neural network with goal forgetting and information aggregation for solving math word problems. Issue 3 (May 2023)

• Record Type: Journal Article
• Authors: Xiao, Jing; Huang, Linjia; Song, Yu; Tang, Na
• Is Part Of: Information processing & management. Volume 60:Issue 3(2023)
• Publication Date: 2023-05
• Subjects: Math word problems -- Deep learning -- Tree-Structured decoder; Information storage and retrieval systems -- Periodicals; Information science -- Periodicals; Systèmes d'information -- Périodiques; Sciences de l'information -- Périodiques; Information science; Information storage and retrieval systems; Periodicals; 658.4038
• DOI: 10.1016/j.ipm.2023.103324
• Languages: English
• ISSNs: 0306-4573

Abstract

Abstract: Most existing state-of-the-art neural network models for math word problems use the Goal-driven Tree-Structured decoder (GTS) to generate expression trees. However, we found that GTS does not provide good predictions for longer expressions, mainly because it does not capture the relationships among the goal vectors of each node in the expression tree and ignores the position order of the nodes before and after the operator. In this paper, we propose a novel Recursive tree-structured neural network with Goal Forgetting and information aggregation (RGFNet) to address these limits. The goal forgetting and information aggregation module is based on ordinary differential equations (ODEs) and we use it to build a sub-goal information feedback neural network (SGIFNet). Unlike GTS, which uses two-layer gated-feedforward networks to generate goal vectors, we introduce a novel sub-goal generation module. The sub-goal generation module could capture the relationship among the related nodes (e.g. parent nodes, sibling nodes) using attention mechanism. Experimental results on two large public datasets i.e. Math23K and Ape-clean show that our tree-structured model outperforms the state-of-the-art models and obtains answer accuracy over 86%. Furthermore, the performance on long-expression problems is promising. 1 Highlights: Constructing a novel ODE-based goal forgetting and information aggregation module. Can achieve goal forgetting, information fusion and solve the node sequencing. Mining the dependencies of child nodes by using attention mechanisms. Obtaining answer accuracy over 86% over Math23K and Ape-clean datasets.