Space‐address decoupled scratchpad memory management for neural network accelerators. (13th October 2020)

• Record Type: Journal Article
• Authors: Zhang, Zhenxing; Sun, Shiyan; Chen, Xunyu; Zhi, Tian; Guo, Qi; Chen, Yunji
• Is Part Of: Concurrency and computation. Volume 33:Number 6(2021)
• Page Start: n/a
• Page End: n/a
• Publication Date: 2020-10-13
• Subjects: deep neural network -- memory management -- scratchpad memory; Parallel processing (Electronic computers) -- Periodicals; Parallel computers -- Periodicals; 004.35
• DOI: 10.1002/cpe.6046
• Languages: English
• ISSNs: 1532-0626

Abstract

Summary: Deep neural networks have been demonstrated to be useful in varieties of intelligent tasks, and various specialized NN accelerators have been proposed recently to improve the hardware efficiency, which are typically equipped with software‐managed scratchpad memory (SPM) for high performance and energy efficiency. However, traditional SPM management techniques cause memory fragmentation for NN accelerators, and thus lead to low utilization of precious SPM. The main reason is that traditional techniques are originally designed for managing fixed‐length registers rather than variable‐length memory blocks . In this article, we propose a novel SPM management approach for NN accelerators. The basic intuition is that NN computation/memory behaviors are predictable and relatively regular compared with traditional applications, and thus most information can be determined at compile time. In addition, by exploiting the variable‐length feature of SPM, we propose to divide the allocation process into two passes: the space assignment and the address assignment pass, which are simultaneously (and implicitly) performed in traditional one‐pass allocation techniques. Experimental results on the memory requests of a representative NN accelerator demonstrate that the proposed approach can significantly reduce the memory consumption by 30% at most compared with state‐of‐the‐art SPM management techniques, and the memory usage is only 2% larger than that of the theoretical optimal allocation.