A novel mixed reality in breast and constructive jaw surgical tele-presence. (August 2019)

• Record Type: Journal Article
• Authors: Venkata, Haritha Sallepalli; Alsadoon, Abeer; Prasad, P.W.C.; Alsadoon, Omar Hisham; Haddad, Sami; Deva, Anand; Hsu, Jeremy
• Is Part Of: Computer methods and programs in biomedicine. Volume 177(2019)
• Page Start: 253
• Page End: 268
• Publication Date: 2019-08
• Subjects: Augmented reality -- Virtual reality -- Mixed reality -- Surgical telepresence -- Visualization; Medicine -- Computer programs -- Periodicals; Biology -- Computer programs -- Periodicals; Computers -- Periodicals; Medicine -- Periodicals; Médecine -- Logiciels -- Périodiques; Biologie -- Logiciels -- Périodiques; Biology -- Computer programs; Medicine -- Computer programs; Periodicals; Electronic journals; 610.28
• DOI: 10.1016/j.cmpb.2019.05.025
• Languages: English
• ISSNs: 0169-2607

Abstract

Highlights: State of art solution does not include any object movement information while merging. Our proposed solution helps to identify and detect the movement of objects using contour flow of previous and current tri map. State of art solution used colour rectification vector but it is calculated for only original images (first frame) and fixed for next stages. In our proposed solution, Interpolation constraint coefficient is used to alter the colour variations between the source and target images. State of art solution could not recover the covered regions. In our proposed solution, Volumetric image synthesisation is adapted to create multiple layers to the final visualization video. It helps to recover the background pixels and adjust based on the surgeon requirement. In State of art solution, the Overlay error is 1.3 mm, and Visibility of pixels 98.4%. In our proposed solution, the Overlay Error is 0.9 mm Visibility of pixels 99.1%. Abstract: Background and aim: Surgical telepresence has been implemented using Mixed reality (MR) but, MR is theory based and only used for investigating research. The Aim of this paper is to propose and implement a new solution by merging augmented video (generating in local site) and virtual expertise surgeon hand (remote site). This system is to improve the visualization of surgical area, overlay accuracy in the merged video without having any discoloured patterns on hand, smudging artefacts on surgeon hand boundary and occluded areas of surgical area. Methodology: The Proposed system consists of an Enhanced Multi-Layer Mean Value Cloning (EMLMV) algorithm that improves the overlay accuracy, visualization accuracy and the processing time. This proposed algorithm includes trimap and alpha matting as a pre-processing stage of merging process, which helps to remove the smudging and discoloured artefacts surrounded by remote surgeon hand. Results: Results showing that the proposed system improved the accuracy by reducing the overlay error of merging image from 1.3 mm (Millimeter) to 0.9 mm. Furthermore, it improves the visibility of surgeon hand in the final merged image from 98.4% (visibility of pixels) to 99.1% (visibility of pixels). Similarly, the processing time in our proposed solution is reduced, which is computed as 10 s to produce 50 frames, whilst, the state of art solution computes 11 s for the same number of frames. Conclusion: The proposed system focuses on the merging of augmented reality video (local site), and the virtual reality video (remote site) with the accurate visualization. we consider discoloured areas, smudging artefacts and occlusion as the main aspects to improve the accuracy of merged video in terms of overlay error and visualization error. So, the proposed system would produce the merged video with the removal of artefacts around the expert surgeon hand.