Deep learning for haptic feedback of flexible endoscopic robot without prior knowledge on sheath configuration. (November 2019)

• Record Type: Journal Article
• Authors: Li, Xiaoguo; Tiong, Anthony Meng Huat; Cao, Lin; Lai, Wenjie; Phan, Phuoc Thien; Phee, Soo Jay
• Is Part Of: International journal of mechanical sciences. Volume 163(2019)
• Publication Date: 2019-11
• Subjects: Flexible endoscopic surgical robots -- Tendon-sheath mechanisms -- Haptic Force Feedback -- Force Hysteresis -- Deep Learning; Mechanical engineering -- Periodicals; Génie mécanique -- Périodiques; Mechanical engineering; Maschinenbau; Mechanik; Zeitschrift; Periodicals; 621.05
• DOI: 10.1016/j.ijmecsci.2019.105129
• Languages: English
• ISSNs: 0020-7403

Abstract

Highlights: Two-stage deep learning force prediction is proposed for tendon-sheath mechanisms. Sheath bending angle can be estimated in Stage I for different sheath configurations. Root-mean-square-error of distal force prediction in Stage II is 0.1711 N. The approach is validated on a flexible tendon-sheath driven surgical robot. Abstract: Distal-end force information is usually missing in flexible endoscopic robots due to the difficulties of mounting miniature force sensors on their end-effectors. This hurdle creates big challenges in providing a sense of touch for the operating surgeons. Many existing studies have developed models to calculate the distal-end forces based on the measured proximal-end forces of Tendon-Sheath Mechanisms (TSMs), but these models assume known sheath bending configuration which is unknown during real-life surgeries. This paper presents a two-stage data-driven method that makes dynamic distal-end force prediction of a flexible endoscopic robot without this assumption. In stage one, a convolutional neural network is used to estimate the sheath cumulative bending angle based on the proximal-end force responses of the robot to a probing signal; in stage two, a combination of two long-short-term-memory models pre-trained for the bending angles nearest to the estimated angle (obtained in stage one) makes dynamic estimations of the distal-end force of the robot. The proposed approach overcomes the challenges due to unknown TSM configurations and can robustly identify the correct force hysteresis phases of TSMs. The force prediction is continuous, accurate, and has a mean RMSE of 0.1711 N. This method was validated on an actual flexible surgical robot. In addition, since the proposed approach provides an estimation of the current system cumulative bending angle, it can also be used to facilitate the mathematical modeling methods which require information on the cumulative bending angle. Graphical abstract: Image, graphical abstract