A Heterogeneously Integrated Spiking Neuron Array for Multimode‐Fused Perception and Object Classification. Issue 24 (13th May 2022)

• Record Type: Journal Article
• Authors: Zhu, Jiaxue; Zhang, Xumeng; Wang, Rui; Wang, Ming; Chen, Pei; Cheng, Lingli; Wu, Zuheng; Wang, Yongzhou; Liu, Qi; Liu, Ming
• Is Part Of: Advanced materials. Volume 34:Issue 24(2022)
• Page Start: n/a
• Page End: n/a
• Publication Date: 2022-05-13
• Subjects: memristors -- multimode‐fused perception -- object classification -- sensors -- spiking neurons; Materials -- Periodicals; Chemical vapor deposition -- Periodicals; 620.11
• DOI: 10.1002/adma.202200481
• Languages: English
• ISSNs: 0935-9648

Abstract

Abstract: Multimode‐fused sensing in the somatosensory system helps people obtain comprehensive object properties and make accurate judgments. However, building such multisensory systems with conventional metal–oxide–semiconductor technology presents serious device integration and circuit complexity challenges. Here, a multimode‐fused spiking neuron (MFSN) with a compact structure to achieve human‐like multisensory perception is reported. The MFSN heterogeneously integrates a pressure sensor to process pressure and a NbOx ‐based memristor to sense temperature. Using this MFSN, multisensory analog information can be fused into one spike train, showing excellent data compression and conversion capabilities. Moreover, both pressure and temperature information are distinguished from fused spikes by decoupling the output frequencies and amplitudes, supporting multimodal tactile perception. Then, a 3 × 3 MFSN array is fabricated, and the fused frequency patterns are fed into a spiking neural network for enhanced tactile pattern recognition. Finally, a larger MFSN array is simulated for classifying objects with different shapes, temperatures, and weights, validating the feasibility of the MFSNs for practical applications. The proof‐of‐concept MFSNs enable the building of multimodal sensory systems and contribute to the development of highly intelligent robotics. Abstract : An artificial multimodal sensory system based on a multimode‐fused spiking neuron (MFSN) is reported. The MFSN is capable of multimodal sensing, data fusion, and spiking conversion through heterogeneous integration of pressure sensors and memristors. The application feasibility of the system is demonstrated through enhanced pattern recognition and object classification. The proof‐of‐concept MFSNs contribute to the development of highly intelligent robotics.