Taxonomy of multi-focal nematode image stacks by a CNN based image fusion approach. (March 2018) - Liu, Min; Wang, Xueping; Zhang, Hongzhong
Highlights: A deep convolutional neural network based image fusion approach is proposed to fuse multi-focal images. Multi-focal images within a stack are fused along 3 orthogonal directions. Multiple features extracted from the fused images along different directions are combined by CCA. The deep CNN image fusion method is embedded within a multilinear framework. The proposed classifier reaches a recognition rate of 95.7% with texture feature only. Abstract: Background and objective: In the biomedical field, digital multi-focal images are very important for documentation and communication of specimen data, because the morphological information for a transparent specimen can be captured in form of a stack of high-quality images. Given biomedical image stacks containing multi-focal images, how to efficiently extract effective features from all layers to classify the image stacks is still an open question. Methods: We present to use a deep convolutional neural network (CNN) image fusion based multilinear approach for the taxonomy of multi-focal image stacks. A deep CNN based image fusion technique is used to combine relevant information of multi-focal images within a given image stack into a single image, which is more informative and complete than any single image in the given stack. Besides, multi-focal images within a stack are fused along 3 orthogonal directions, and multiple features extracted from the fused images along different directions are combined by canonical correlation analysis (CCA). Because multi-focal image stacks represent the effect of different factors - texture, shape, different instances within the same class and different classes of objects, we embed the deep CNN based image fusion method within a multilinear framework to propose an image fusion based multilinear classifier. Results: The experimental results on nematode multi-focal image stacks demonstrated that the deep CNN image fusion based multilinear classifier can reach a higher classification rate (95.7%) than that by the previous multilinear based approach (88.7%), even we only use the texture feature instead of the combination of texture and shape features as in the previous work. Conclusions: The proposed deep CNN image fusion based multilinear approach shows great potential in building an automated nematode taxonomy system for nematologists. It is effective to classify multi-focal image stacks.