Precision medical image hash retrieval by interpretability and feature fusion. (July 2022)

• Record Type: Journal Article
• Authors: Guan, Anna; Liu, Li; Fu, Xiaodong; Liu, Lijun
• Is Part Of: Computer methods and programs in biomedicine. Volume 222(2022)
• Publication Date: 2022-07
• Subjects: Medical image retrieval -- Interpretability -- Attention mechanisms -- Feature fusion -- Deep hashing; Medicine -- Computer programs -- Periodicals; Biology -- Computer programs -- Periodicals; Computers -- Periodicals; Medicine -- Periodicals; Médecine -- Logiciels -- Périodiques; Biologie -- Logiciels -- Périodiques; Biology -- Computer programs; Medicine -- Computer programs; Periodicals; Electronic journals; 610.28
• DOI: 10.1016/j.cmpb.2022.106945
• Languages: English
• ISSNs: 0169-2607

Abstract

Highlights: To train a Pre-Densenet model using the C2L method on a large amount of medical image data without manual annotation, which can improve the robustness of the network model and obtain richer medical semantic information. To propose the Saliency maps as interpretable guides to improve the representation of lesion areas by focusing on lesion areas that correspond to clinical reports, which can address lesion omissions to improve the precision of retrieval results. To extract the features as the complete information representation from global network and local network and define three loss functions to optimize the hash codes, which can maintain semantics and improve the accuracy of the retrieval results. Abstract: Background and Objective: To address the problem of low accuracy of medical image retrieval due to high inter-class similarity and easy omission of lesions, a precision medical image hash retrieval method combining interpretability and feature fusion is proposed, taking chest X-ray images as an example. Methods: Firstly, the DenseNet-121 network is pre-trained on a large dataset of medical images without manual annotation using the comparison to learn (C2L) method to obtain a backbone network model containing more medical representations with training weights. Then, a global network is constructed by using global image learning to acquire an interpretable saliency map as attention mechanisms, which can generate a mask crop to get a local discriminant region. Thirdly, the local discriminant regions are used as local network inputs to obtain local features, and the global features are used with the local features by dimension in the pooling layer. Finally, a hash layer is added between the fully connected layer and the classification layer of the backbone network, defining classification loss, quantization loss and bit-balanced loss functions to generate high-quality hash codes. The final retrieval result is output by calculating the similarity metric of the hash codes. Results: Experiments on the Chest X-ray8 dataset demonstrate that our proposed interpretable saliency map can effectively locate focal regions, the fusion of features can avoid information omission, and the combination of three loss functions can generate more accurate hash codes. Compared with the current advanced medical image retrieval methods, this method can effectively improve the accuracy of medical image retrieval. Conclusions: The proposed hash retrieval approach combining interpretability and feature fusion can effectively improve the accuracy of medical image retrieval which can be potentially applied in computer-aided-diagnosis systems.