A cross-scale framework for low-light image enhancement using spatial–spectral information. (March 2023)

• Record Type: Journal Article
• Authors: Zhang, Yichi; Liu, Hengyu; Ding, Dandan
• Is Part Of: Computers & electrical engineering. Volume 106(2023)
• Publication Date: 2023-03
• Subjects: Low-light -- Transformer -- Cross-scale -- Fast Fourier Convolution -- Image enhancement; Computer engineering -- Periodicals; Electrical engineering -- Periodicals; Electrical engineering -- Data processing -- Periodicals; Ordinateurs -- Conception et construction -- Périodiques; Électrotechnique -- Périodiques; Électrotechnique -- Informatique -- Périodiques; Computer engineering; Electrical engineering; Electrical engineering -- Data processing; Periodicals; Electronic journals; 621.302854
• DOI: 10.1016/j.compeleceng.2023.108608
• Languages: English
• ISSNs: 0045-7906

Abstract

Abstract: This paper presents a spatial–spectral low-light enhancement approach based on the neural network. Considering that the Image Signal Processor (ISP) which non-linearly maps RAW data to RGB images may introduce additional noise and artifacts, we propose to conduct the enhancement task directly on RAW data. To this end, we propose a cross-scale framework to map the input low-light RAW data to visually pleasing RGB images. The cross-scale framework consists of three branches for low-level, mid-level, and high-level representations of input images. We embed paired Fast Fourier Convolution (FFC) and Transformer in each level for global and local feature analysis and aggregation. Specifically, the FFC enlarges the receptive field of our neural network, exploring the pixel correlations in the spectral space; the following Transformer uses self-attention for feature selection and aggregation in the spatial space. As a result, spatial–spectral information within an image is jointly utilized for the final fusion. Experimental results demonstrate that the proposed approach significantly outperforms state-of-the-art low-light enhancement methods in both full reference assessment metrics, including PSNR, MPSNR, and SSIM, and no-reference metrics, such as NIMA. Moreover, the proposed method produces more aesthetically pleasing RGB images than other methods. Graphical abstract: Highlights: We propose a cross-scale framework to map the RAW data to RGB images. We utilize FFC and Transformer to extract latent features in each scale of our framework. Results show that the proposed method remarkably outperforms sota works.