Experiments of Mutual Information Maximization on Deep Infomax. Issue 1 (1st December 2022)

• Record Type: Journal Article
• Authors: Han, Zihao; Lin, Zichao
• Is Part Of: Journal of physics. Volume 2386 Issue 1(2022)
• Publication Date: 2022-12-01
• Subjects: Deep Learning -- Mutual Information -- Deep Infomax -- Clustering -- Neural Networks; Physics -- Congresses; 530.5
• DOI: 10.1088/1742-6596/2386/1/012035
• Languages: English
• ISSNs: 1742-6588

Abstract

Abstract: Image search and photo search at the current level of science and technology have become increasingly mature, and the underlying logic of these is the clustering of pictures. Mutual information(MI) is widely used in machine learning and deep learning, its value cannot be computed directly. Therefore, there are several methods to estimate MI, such as Deep VIB, MINE, and MoCo. Deep Infomax(DIM) technique is applied to maximize the MI based on Jensen-Shannon divergence. The paper derives the mathematical formula of Deep Infomax first. If two images have the maximum MI, it could be concluded that they are similar images. In the experiment, this technique is applied in clustering of datasets that are cifar10, celeba. In the experiment, the results of cifar10 and celeba are close to expectations and can be considered to have achieved good results. In cifar10, it clusters images of similar colors. In celeba, it clusters together faces of the same skin tone and even expressions. This paper and experiment could conclude that Deep Infomax is an effective way to cluster the images and maximize the mutual information.