Application of offset estimator of differential entropy and mutual information with multivariate data. (5th September 2022)

• Record Type: Journal Article
• Authors: Frontoni, Emanuele; Marín-Franch, Iván; Sanz-Sabater, Martín; Foster, David H.
• Is Part Of: Experimental results. Volume 3(2022)
• Publication Date: 2022-09-05
• Subjects: information theory -- Kozachenko–Leonenko estimator -- mutual information -- nonparametric statistics -- R -- Python -- and MATLAB; Science -- Experiments -- Periodicals; Science -- Methodology -- Periodicals; 507.24
• DOI: 10.1017/exp.2022.14
• Languages: English
• ISSNs: 2516-712X

Abstract

Abstract: Numerical estimators of differential entropy and mutual information can be slow to converge as sample size increases. The offset Kozachenko–Leonenko (KLo) method described here implements an offset version of the Kozachenko–Leonenko estimator that can markedly improve convergence. Its use is illustrated in applications to the comparison of trivariate data from successive scene color images and the comparison of univariate data from stereophonic music tracks. Publicly available code for KLo estimation of both differential entropy and mutual information is provided for R, Python, and MATLAB computing environments at https://github.com/imarinfr/klo .