Learning a referenceless stereopair quality engine with deep nonnegativity constrained sparse autoencoder. (April 2018)

• Record Type: Journal Article
• Authors: Jiang, Qiuping; Shao, Feng; Lin, Weisi; Jiang, Gangyi
• Is Part Of: Pattern recognition. Volume 76(2018:Apr.)
• Page Start: 242
• Page End: 255
• Publication Date: 2018-04
• Subjects: Image quality assessment -- No-reference/referenceless -- Stereoscopic 3D image -- Deep learning -- Nonnegativity constrained -- Sparse autoencoder; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2017.11.001
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: A three-column deep non-negativity constrained sparse autoencoder is proposed for BSIQA. Both feature evolution and feature mapping are addressed in a unified framework for BSIQA. A Bayesian inference-based quality combination framework is used to derive 3D quality score. Abstract: This paper proposes a no-reference (NR)/referenceless quality evaluation method for stereoscopic three-dimensional (S3D) images based on deep nonnegativity constrained sparse autoencoder (DNCSAE). To address the quality issue of stereopairs whose perceived quality is not only determined by the individual left and right image qualities but also their interactions, a three-column DNCSAE framework is customized with individual DNCSAE module coping with the left image, the right image, and the cyclopean image, respectively. In the proposed framework, each individual DNCSAE module shares the same network architecture consisting of multiple stacked NCSAE layers and one Softmax regression layer at the end. The contribution of our model is that hierarchical feature evolution and nonlinear feature mapping are jointly optimized in a unified and perceptual-aware deep network (DNCSAE), which well resembles several important visual properties, i.e., hierarchy, sparsity, and non-negativity. To be more specific, for each DNCSAE, by taking a set of handcrafted natural scene statistic (NSS) features as inputs in the visible layer, the features in hidden layers are successively evolved to deeper levels producing increasingly discriminative quality-aware features (QAFs). Then, QAFs in the last NCSAE layer are summarized to their corresponding quality score by Softmax regression. Finally, three individual yet complementary quality scores estimated by each DNCSAE model are combined based on a Bayesian framework to obtain an overall 3D quality score. Experiments on three benchmark databases demonstrate the superiority of our method in terms of both prediction accuracy and generalization capability.