Remembering for the right reasons: Explanations reduce catastrophic forgetting. Issue 4 (19th December 2021)

• Record Type: Journal Article
• Authors: Ebrahimi, Sayna; Petryk, Suzanne; Gokul, Akash; Gan, William; Gonzalez, Joseph E.; Rohrbach, Marcus; Darrell, Trevor
• Other Names: Gunning Dave guestEditor.; Vorm Eric guestEditor.; Wang Jennifer Yunyan guestEditor.; Turek Matt guestEditor.
• Is Part Of: Applied AI Letters. Volume 2:Issue 4(2021)
• Page Start: n/a
• Page End: n/a
• Publication Date: 2021-12-19
• Subjects: catastrophic forgetting -- continual learning -- explainable AI -- lifelong learning; 006.3
• DOI: 10.1002/ail2.44
• Languages: English
• ISSNs: 2689-5595

Abstract

Abstract: The goal of continual learning (CL) is to learn a sequence of tasks without suffering from the phenomenon of catastrophic forgetting. Previous work has shown that leveraging memory in the form of a replay buffer can reduce performance degradation on prior tasks. We hypothesize that forgetting can be further reduced when the model is encouraged to remember the evidence for previously made decisions. As a first step towards exploring this hypothesis, we propose a simple novel training paradigm, called Remembering for the Right Reasons (RRR), that additionally stores visual model explanations for each example in the buffer and ensures the model has "the right reasons" for its predictions by encouraging its explanations to remain consistent with those used to make decisions at training time. Without this constraint, there is a drift in explanations and increase in forgetting as conventional continual learning algorithms learn new tasks. We demonstrate how RRR can be easily added to any memory or regularization‐based approach and results in reduced forgetting, and more importantly, improved model explanations. We have evaluated our approach in the standard and few‐shot settings and observed a consistent improvement across various CL approaches using different architectures and techniques to generate model explanations and demonstrated our approach showing a promising connection between explainability and continual learning. Our code is available at https://github.com/SaynaEbrahimi/Remembering-for-the-Right-Reasons . Abstract : We posit that the catastrophic forgetting phenomenon is due in part to not being able to rely on the same reasoning as was used for a previously seen observation. Therefore, we hypothesize that forgetting can be mitigated when the model is encouraged to remember the evidence for previously made decisions; in other words, a model which can remember its final decision and can reconstruct the same prior reasoning. Based on this approach, we develop a novel strategy to exploit explainable models for improving performance.