Uncertainty maximization in partially observable domains: A cognitive perspective. (May 2023)

• Record Type: Journal Article
• Authors: Ramicic, Mirza; Bonarini, Andrea
• Is Part Of: Neural networks. Volume 162(2023)
• Page Start: 456
• Page End: 471
• Publication Date: 2023-05
• Subjects: Partially observable Markov decision process -- Cognitive modeling -- Entropy -- Reinforcement learning -- Attention mechanisms and development -- Neural networks for development; Neural computers -- Periodicals; Neural networks (Computer science) -- Periodicals; Neural networks (Neurobiology) -- Periodicals; Nervous System -- Periodicals; Ordinateurs neuronaux -- Périodiques; Réseaux neuronaux (Informatique) -- Périodiques; Réseaux neuronaux (Neurobiologie) -- Périodiques; Neural computers; Neural networks (Computer science); Neural networks (Neurobiology); Periodicals; 006.32
• DOI: 10.1016/j.neunet.2023.02.044
• Languages: English
• ISSNs: 0893-6080

Abstract

Abstract: Faced with an ever-increasing complexity of their domains of application, artificial learning agents are now able to scale up in their ability to process an overwhelming amount of data. However, this comes at the cost of encoding and processing an increasing amount of redundant information. This work exploits the possibility of learning systems, applied in partially observable domains, to selectively focus on the specific type of information that is more likely related to the causal interaction among transitioning states. A temporal difference displacement criterion is defined to implement adaptive masking of the observations. It can enable a significant improvement of convergence of temporal difference algorithms applied to partially observable Markov processes, as shown by experiments performed under a variety of machine learning problems, ranging from highly complex visuals as Atari games to simple textbook control problems such as CartPole. The proposed framework can be added to most RL algorithms since it only affects the observation process, selecting the parts more promising to explain the dynamics of the environment and reducing the dimension of the observation space.