Impact of Phase‐Change Memory Flicker Noise and Weight Drift on Analog Hardware Inference for Large‐Scale Deep Learning Networks. (23rd February 2022)

• Record Type: Journal Article
• Authors: Han, Jin-Ping; Rasch, Malte J.; Liu, Zuoguang; Solomon, Paul; Brew, Kevin; Cheng, Kangguo; Ok, Injo; Chan, Victor; Longstreet, Michael; Kim, Wanki; Bruce, Robert L.; Cheng, Cheng-Wei; Saulnier, Nicole; Narayanan, Vijay
• Is Part Of: Advanced intelligent systems. Volume 4:Number 5(2022)
• Page Start: n/a
• Page End: n/a
• Publication Date: 2022-02-23
• Subjects: analog RPU inference -- deep-learning networks -- flicker noise -- hardware awareness training -- PCM -- weight drift; Artificial intelligence -- Periodicals; Robotics -- Periodicals; Control theory -- Periodicals; 006.3
• DOI: 10.1002/aisy.202100179
• Languages: English
• ISSNs: 2640-4567

Abstract

Abstract : The analog AI core concept is appealing for deep‐learning (DL) because it combines computation and memory functions into a single device. Yet, significant challenges such as noise and weight drift will impact large‐scale analog in‐memory computing. Here, effects of flicker noise and drift on large DL systems are explored using a new flicker‐noise model with memory, which preserves temporal correlations, including a flicker noise figure of merit (FOM) A r to quantify impacts on system performance. Flicker noise is characterized for G e 2 S b 2 T e 5 (GST) based phase‐change memory (PCM) cells with a discovery of read‐noise asymmetry tied to shape asymmetry of mushroom cells. This experimental read polarity dependence is consistent with Pirovano's trap activation and defect annihilation model in an asymmetric GST cell. The impact of flicker noise and resistance drift of analog PCM synaptic devices on deep‐learning hardware is assessed for six large‐scale deep neural networks (DNNs) used for image classification, finding that the inference top‐1 accuracy degraded with the accumulated device flicker noise and drift as ∝ A r × t wait, and ∝ t wait − ν, respectively, where ν is the drift coefficient. These negative impacts could be mitigated with a new hardware‐aware (HWA) (pre)‐training of the DNNs, which is applied before programming to the analog arrays. Abstract : Flicker noise with memory is included in simulations of large image classification neural networks. Figures of merit (FOM) are derived and compared with flicker noise and drift measurements on phase‐change memory (PCM) cells. A new noise asymmetry is found, correlated with the cell's structural asymmetry. New hardware training aware algorithms are explored to mitigate noise impacts.