Residual objectness for imbalance reduction. (October 2022)

• Record Type: Journal Article
• Authors: Chen, Joya; Liu, Dong; Luo, Bin; Peng, Xuezheng; Xu, Tong; Chen, Enhong
• Is Part Of: Pattern recognition. Volume 130(2022)
• Publication Date: 2022-10
• Subjects: Object detection -- Class imbalance -- Residual objectness; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2022.108781
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: We discover that the foreground-background imbalance in object detection could be addressed in a learning-based manner, without any hard-crafted resampling and reweighting schemes. We propose a novel Residual Objectness (ResObj) mechanism to address the foreground-background imbalance in training object detectors. With a cascade architecture to gradually refine the objectness estimation, our ResObj module could address the imbalance in an endto- end way, thus avoiding laborious hyper-parameters tuning required by resampling and reweighting schemes. We validate the proposed method on the COCO dataset with thorough ablation studies. For various detectors, our Residual Objectness steadily improves relative 3 % ∼ 4 % detection accuracy. Abstract: As most object detectors rely on dense candidate samples to cover objects, they have always suffered from the extreme imbalance between very few foreground samples and numerous background samples during training, i.e., the foreground-background imbalance. Although several resampling and reweighting schemes ( e.g., OHEM, Focal Loss, GHM) have been proposed to alleviate the imbalance, they are usually heuristic with multiple hyper-parameters, which is difficult to generalize on different object detectors and datasets. In this paper, we propose a novel Residual Objectness (ResObj) mechanism that adaptively learns how to address the foreground-background imbalance problem in object detection. Specifically, we first formulate the imbalance problems on all object classes as an imbalance problem on an "objectness" class. Then, we design multiple cascaded objectness estimators with residual connections for that objectness class to progressively distinguish the foreground samples from background samples. With our residual objectness mechanism, object detectors can learn how to address the foreground-background problem in an end-to-end way, rather than rely on hand-crafted resampling or reweighting schemes. Extensive experiments on the COCO benchmark demonstrate the effectiveness and compatibility of our method for various object detectors: the RetinaNet-ResObj, YOLOv3-ResObj and FasterRCNN-ResObj achieve relative 3 % ∼ 4 % Average Precision (AP) improvements compared with their vanilla models, respectively.