UTRAD: Anomaly detection and localization with U-Transformer. (March 2022)

• Record Type: Journal Article
• Authors: Chen, Liyang; You, Zhiyuan; Zhang, Nian; Xi, Juntong; Le, Xinyi
• Is Part Of: Neural networks. Volume 147(2022)
• Page Start: 53
• Page End: 62
• Publication Date: 2022-03
• Subjects: Anomaly detection -- Image transformer -- One-class learning; Neural computers -- Periodicals; Neural networks (Computer science) -- Periodicals; Neural networks (Neurobiology) -- Periodicals; Nervous System -- Periodicals; Ordinateurs neuronaux -- Périodiques; Réseaux neuronaux (Informatique) -- Périodiques; Réseaux neuronaux (Neurobiologie) -- Périodiques; Neural computers; Neural networks (Computer science); Neural networks (Neurobiology); Periodicals; 006.32
• DOI: 10.1016/j.neunet.2021.12.008
• Languages: English
• ISSNs: 0893-6080

Abstract

Abstract: Anomaly detection is an active research field in industrial defect detection and medical disease detection. However, previous anomaly detection works suffer from unstable training, or non-universal criteria of evaluating feature distribution. In this paper, we introduce UTRAD, a U -TR ansformer based A nomaly D etection framework. Deep pre-trained features are regarded as dispersed word tokens, and represented with transformer-based autoencoders. With reconstruction on more informative feature distribution instead of raw images, we achieve a more stable training process and a more precise anomaly detection and localization result. In addition, our proposed UTRAD has a multi-scale pyramidal hierarchy with skip connections that help detect both multi-scale structural and non-structural anomalies. As attention layers are decomposed to multi-level patches, UTRAD significantly reduces the computational cost and memory usage compared with the vanilla transformer. Experiments on industrial dataset MVtec AD and medical datasets Retinal-OCT, Brain-MRI, Head-CT have been conducted. Our proposed UTRAD out-performs all other state-of-the-art methods in the above datasets. Code released at https://github.com/gordon-chenmo/UTRAD .