Automated surgical workflow identification by artificial intelligence in laparoscopic hepatectomy: Experimental research. (September 2022)

• Record Type: Journal Article
• Authors: Sasaki, Kimimasa; Ito, Masaaki; Kobayashi, Shin; Kitaguchi, Daichi; Matsuzaki, Hiroki; Kudo, Masashi; Hasegawa, Hiro; Takeshita, Nobuyoshi; Sugimoto, Motokazu; Mitsunaga, Shuichi; Gotohda, Naoto
• Is Part Of: International journal of surgery. Volume 105(2022)
• Publication Date: 2022-09
• Subjects: Real-time surgical step identification -- Computer vision -- Laparoscopic hepatectomy -- Convolutional neural network -- Deep learning -- Artificial intelligence; AI artificial intelligence -- CAS computer-assisted surgery -- CNN convolutional neural network -- HMM Hidden Markov model -- LH Laparoscopic hepatectomy -- NCCHE National Cancer Center Hospital East; Surgery -- Periodicals; Surgical Procedures, Operative -- Periodicals; 617.005
• DOI: 10.1016/j.ijsu.2022.106856
• Languages: English
• ISSNs: 1743-9191

Abstract

Abstract: Background: To perform accurate laparoscopic hepatectomy (LH) without injury, novel intraoperative systems of computer-assisted surgery (CAS) for LH are expected. Automated surgical workflow identification is a key component for developing CAS systems. This study aimed to develop a deep-learning model for automated surgical step identification in LH. Materials and methods: We constructed a dataset comprising 40 cases of pure LH videos; 30 and 10 cases were used for the training and testing datasets, respectively. Each video was divided into 30 frames per second as static images. LH was divided into nine surgical steps (Steps 0–8), and each frame was annotated as being within one of these steps in the training set. After extracorporeal actions (Step 0) were excluded from the video, two deep-learning models of automated surgical step identification for 8-step and 6-step models were developed using a convolutional neural network (Models 1 & 2). Each frame in the testing dataset was classified using the constructed model performed in real-time. Results: Above 8 million frames were annotated for surgical step identification from the pure LH videos. The overall accuracy of Model 1 was 0.891, which was increased to 0.947 in Model 2. Median and average accuracy for each case in Model 2 was 0.927 (range, 0.884–0.997) and 0.937 ± 0.04 (standardized difference), respectively. Real-time automated surgical step identification was performed at 21 frames per second. Conclusions: We developed a highly accurate deep-learning model for surgical step identification in pure LH. Our model could be applied to intraoperative systems of CAS. Graphical abstract: Image 1 Highlights: An accurate deep-learning(DL) model for laparoscopic hepatectomy(LH) is presented. This is the first study in which a DL approach was applied to videos of LH cases. Our model could be applied to various systems of computer-assisted surgery for LH.