A new formulation of gradient boosting. Issue 4 (17th September 2021)

• Record Type: Journal Article
• Authors: Wozniakowski, Alex; Thompson, Jayne; Gu, Mile; Binder, Felix C
• Is Part Of: Machine learning: science and technology. Volume 2:Issue 4(2021)
• Publication Date: 2021-09-17
• Subjects: multi-target regression -- boosting -- ensemble learning -- stacking -- quantum computing -- prior knowledge; 006.31
• DOI: 10.1088/2632-2153/ac1ee9
• Languages: English
• ISSNs: 2632-2153

Abstract

Abstract: In the setting of regression, the standard formulation of gradient boosting generates a sequence of improvements to a constant model. In this paper, we reformulate gradient boosting such that it is able to generate a sequence of improvements to a nonconstant model, which may contain prior knowledge or physical insight about the data generating process. Moreover, we introduce a simple variant of multi-target stacking that extends our approach to the setting of multi-target regression. An experiment on a real-world superconducting quantum device calibration dataset demonstrates that our approach outperforms the state-of-the-art calibration model even though it only receives a paucity of training examples. Further, it significantly outperforms a well-known gradient boosting algorithm, known as LightGBM, as well as an entirely data-driven reimplementation of the calibration model, which suggests the viability of our approach.