Symmetry-adapted representation learning. (February 2019)

• Record Type: Journal Article
• Authors: Anselmi, Fabio; Evangelopoulos, Georgios; Rosasco, Lorenzo; Poggio, Tomaso
• Is Part Of: Pattern recognition. Volume 86(2019:Feb.)
• Page Start: 201
• Page End: 208
• Publication Date: 2019-02
• Subjects: Representation learning -- Equivariant representations -- Invariant representations -- Dictionary learning -- Convolutional neural networks -- Regularization -- Data transformations; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2018.07.025
• Languages: English
• ISSNs: 0031-3203

Abstract

Abstract: In this paper, we propose the use of data symmetries, in the sense of equivalences under signal transformations, as priors for learning symmetry-adapted data representations, i.e., representations that are equivariant to these transformations. We rely on a group-theoretic definition of equivariance and provide conditions for enforcing a learned representation, for example the weights in a neural network layer or the atoms in a dictionary, to have the structure of a group and specifically the group structure in the distribution of the input. By reducing the analysis of generic group symmetries to permutation symmetries, we devise a regularization scheme for representation learning algorithm, using an unlabeled training set. The proposed regularization is aimed to be a conceptual, theoretical and computational proof of concept for symmetry-adapted representation learning, where the learned data representations are equivariant or invariant to transformations, without explicit knowledge of the underlying symmetries in the data.