A deep transfer regression method based on seed replacement considering balanced domain adaptation. (October 2022)

• Record Type: Journal Article
• Authors: Zhang, Teng; Sun, Hao; Peng, Fangyu; Zhao, Shengqiang; Yan, Rong
• Is Part Of: Engineering applications of artificial intelligence. Volume 115(2022)
• Publication Date: 2022-10
• Subjects: Deep transfer learning -- Regression -- Seed replacement -- Balanced domain adaptation; Engineering -- Data processing -- Periodicals; Artificial intelligence -- Periodicals; Expert systems (Computer science) -- Periodicals; Ingénierie -- Informatique -- Périodiques; Intelligence artificielle -- Périodiques; Systèmes experts (Informatique) -- Périodiques; Artificial intelligence; Engineering -- Data processing; Expert systems (Computer science); Periodicals; 620.00285
• DOI: 10.1016/j.engappai.2022.105238
• Languages: English
• ISSNs: 0952-1976

Abstract

Abstract: With the development of deep transfer learning, the generalization abilities of models in similar scenarios have been significantly improved. However, for regression tasks, either the marginal distribution or the conditional distribution is usually ignored. In addition, initiative regarding the representation and learning of domain knowledge is lacking due to the reliance on the loss function. A deep transfer regression method based on seed replacement considering balanced domain adaptation, called DTRSR, is proposed in this work. DTRSR is composed of four parts: structure freezing and parameter transfer, deep feature extraction, seed replacement and a fusion loss function. First, domain knowledge is captured at the model level through structure freezing and parameter transfer. Second, seed replacement is used for knowledge learning in the source and target domains at the data level. Finally, a fusion loss function considering balanced distribution adaptation is constructed to acquire domain knowledge at the loss level. In summary, domain knowledge is sufficiently learned through DTRSR. In addition, seed replacement improves the initiative of knowledge learning instead of relying only on the loss function to learn automatically. DTRSR is compared on three datasets, namely, Tool Wear, Battery Capacity and Robot Machining Errors, with nine other methods. The proposed method achieves excellent performance on most tasks, which validates its effectiveness and great potential in regression tasks. Graphical abstract: Highlights: A deep transfer regression method based on seed replacement is proposed. Marginal and conditional distributions are considered simultaneously. Seed replacement based on clustering is applied to generate new dataset. Source and target domain knowledge is integrated and represented in the new dataset. The proposed method performs well on two public and one private datasets.