Deep learning with transfer functions: new applications in system identification⁎This work was partially supported by the European H2020-CS2 project ADMITTED, Grant agreement no. GA832003. Issue 7 (2021)

• Record Type: Journal Article
• Authors: Piga, Dario; Forgione, Marco; Mejari, Manas
• Is Part Of: IFAC-PapersOnLine. Volume 54:Issue 7(2021)
• Page Start: 415
• Page End: 420
• Publication Date: 2021
• Subjects: System identification -- neural networks -- quantized data -- prediction error methods -- deep learning; Automatic control -- Periodicals; 629.805
• DOI: 10.1016/j.ifacol.2021.08.395
• Languages: English
• ISSNs: 2405-8963

Abstract

Abstract: This paper presents a linear dynamical operator described in terms of a rational transfer function, endowed with a well-defined and efficient back-propagation behavior for automatic derivatives computation. The operator enables end-to-end training of structured networks containing linear transfer functions and other differentiable units by exploiting standard deep learning software. Two relevant applications of the operator in system identification are presented. The first one consists in the integration of prediction error methods in deep learning. The dynamical operator is included as the last layer of a neural network in order to obtain the optimal one-step-ahead prediction error. The second one considers identification of general block-oriented models from quantized data. These block-oriented models are constructed by combining linear dynamical operators with static nonlinearities described as standard feed-forward neural networks. A custom loss function corresponding to the log-likelihood of quantized output observations is defined. For gradient-based optimization, the derivatives of the log-likelihood are computed by applying the back-propagation algorithm through the whole network. Two system identification benchmarks are used to show the effectiveness of the proposed methodologies.