Probabilistic generative modeling and reinforcement learning extract the intrinsic features of animal behavior. (January 2022)

• Record Type: Journal Article
• Authors: Mori, Keita; Yamauchi, Naohiro; Wang, Haoyu; Sato, Ken; Toyoshima, Yu; Iino, Yuichi
• Is Part Of: Neural networks. Volume 145(2022)
• Page Start: 107
• Page End: 120
• Publication Date: 2022-01
• Subjects: Probabilistic generative model -- Mixture density network -- Recurrent neural network -- Reinforcement learning -- Behavior analysis -- Computational ethology; Neural computers -- Periodicals; Neural networks (Computer science) -- Periodicals; Neural networks (Neurobiology) -- Periodicals; Nervous System -- Periodicals; Ordinateurs neuronaux -- Périodiques; Réseaux neuronaux (Informatique) -- Périodiques; Réseaux neuronaux (Neurobiologie) -- Périodiques; Neural computers; Neural networks (Computer science); Neural networks (Neurobiology); Periodicals; 006.32
• DOI: 10.1016/j.neunet.2021.10.002
• Languages: English
• ISSNs: 0893-6080

Abstract

Abstract: It is one of the ultimate goals of ethology to understand the generative process of animal behavior, and the ability to reproduce and control behavior is an important step in this field. However, it is not easy to achieve this goal in systems with complex and stochastic dynamics such as animal behavior. In this study, we have shown that MDN–RNN, a type of probabilistic deep generative model, is able to reproduce stochastic animal behavior with high accuracy by modeling the behavior of C. elegans . Furthermore, we found that the model learns different dynamics in a disentangled representation as a time-evolving Gaussian mixture. Finally, by combining the model and reinforcement learning, we were able to extract a behavioral policy of goal-directed behavior in silico, and showed that it can be used for regulating the behavior of real animals. This set of methods will be applicable not only to animal behavior but also to broader areas such as neuroscience and robotics. Highlights: Probabilistic neural network is suitable for behavior modeling. MDN–RNN can successfully simulate the stochastic behavior. MDN–RNN disentangles the dynamics underlying animal behavior. Combination of generative models and reinforcement learning extracts behavioral strategies automatically.