Machine-learning scoring functions trained on complexes dissimilar to the test set already outperform classical counterparts on a blind benchmark. Issue 6 (24th June 2021)

• Record Type: Journal Article
• Authors: Li, Hongjian; Lu, Gang; Sze, Kam-Heung; Su, Xianwei; Chan, Wai-Yee; Leung, Kwong-Sak
• Is Part Of: Briefings in bioinformatics. Volume 22:Issue 6(2021)
• Publication Date: 2021-06-24
• Subjects: scoring function -- machine learning -- random forest -- scoring power -- binding affinity -- blind benchmark; Genetics -- Data processing -- Periodicals; Molecular biology -- Data processing -- Periodicals; Genomes -- Data processing -- Periodicals; 572.80285
• DOI: 10.1093/bib/bbab225
• Languages: English
• ISSNs: 1467-5463

Abstract

Abstract: The superior performance of machine-learning scoring functions for docking has caused a series of debates on whether it is due to learning knowledge from training data that are similar in some sense to the test data. With a systematically revised methodology and a blind benchmark realistically mimicking the process of prospective prediction of binding affinity, we have evaluated three broadly used classical scoring functions and five machine-learning counterparts calibrated with both random forest and extreme gradient boosting using both solo and hybrid features, showing for the first time that machine-learning scoring functions trained exclusively on a proportion of as low as 8% complexes dissimilar to the test set already outperform classical scoring functions, a percentage that is far lower than what has been recently reported on all the three CASF benchmarks. The performance of machine-learning scoring functions is underestimated due to the absence of similar samples in some artificially created training sets that discard the full spectrum of complexes to be found in a prospective environment. Given the inevitability of any degree of similarity contained in a large dataset, the criteria for scoring function selection depend on which one can make the best use of all available materials. Software code and data are provided at https://github.com/cusdulab/MLSF for interested readers to rapidly rebuild the scoring functions and reproduce our results, even to make extended analyses on their own benchmarks.