Robustly interrogating machine learning-based scoring functions - what are they learning?

Aug 5, 2026

Upload by Riley

Machine learning-based scoring functions (MLBSFs) can show inconsistent performance across benchmarks and may learn biases inherent in training datasets rather than transferable physical understanding. The study benchmarks diverse popular MLBSFs—RFScore, SIGN, OnionNet-2, Pafnucy, and PointVS—against baseline models limited to learning dataset biases. Results show strong competitive accuracy for baselines across proposed benchmarks, indicating bias learning dominates. Tests and the ToolBoxSF platform support robust interrogation of MLBSF predictions and bias effects.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages8

Reading Time20 min

AccessFree to download

Abstract

Motivation

Results

Availability and implementation

Introduction

Scoring functions in structure-based drug discovery

Machine learning and deep learning scoring functions

Benchmarks and dataset bias

Structural bioinformatics Robustly interrogating machine learning-based scoring functions: what are they learning? Guy Durant1 , Fergus Boyles1 , Kristian Birchall2 , Brian Marsden3 , Charlotte M. Deane 1,􀀃 1 Department of Statistics, University of Oxford, St Giles’, Oxford OX1 3LB, United Kingdom 2 LifeArc, Stevenage SG1 2FX, United Kingdom 3 Nuffield Department of Medicine, University of Oxford, Oxford OX3 7BN, United Kingdom 􀀃 Corresponding author. Department of Statistics, University of Oxford, St Giles’, Oxford OX1 3LB, United Kingdom. E-mail: [deane@stats.ox.ac.uk](deane@stats.ox.ac.uk). Associate Editor: Janet Kelso Abstract Motivation: Machine learning-based scoring functions (MLBSFs) have been found to exhibit inconsistent performance on different benchmarksand be prone to learning dataset bias. For the field to develop MLBSFs that learn a generalizable understanding of physics, a more rigorous understanding of how they perform is required. Results: In this work, we compared the performance of a diverse set of popular MLBSFs (RFScore, SIGN, OnionNet-2, Pafnucy, and PointVS) to our proposed baseline models that can only learn dataset biases on a range of benchmarks. We found that these baseline models were competitive in accuracy to these MLBSFs in almost all proposed benchmarks, indicating these models only learn dataset biases. Our tests and provided platform, ToolBoxSF, will enable researchers to robustly interrogate MLBSF performance and determine the effect of dataset biases on their predictions. Availability and implementation: [https://github.com/guydurant/toolboxsf](https://github.com/guydurant/toolboxsf). 1 Introduction Predicting the binding affinity of a protein–ligand complex from its 3D structure has been extensively researched in the past decade (Meli et al. 2022) . However, doing so accurately and for any protein–ligand complex still poses a significant challenge in computational chemistry (Mobley and Gilson 2017) . Accurately predicting binding affinity would aid in structure-based drug discovery, where the chemical structure of a drug is designed based on the structure of its target, as it would allow design hypotheses to be tested in silico. One proposed methodology, scoring functions (Goodsell et al. 1996), which estimate binding affinity based on the features of a single protein–ligand complex structure, offer fast predictions and are suited for high throughput hit identification and lead optimization (Bissantz et al. 2000) . Docking software, such as AutoDock 4 (Morris et al. 2009), AutoDock Vina (Trott and Olson 2010), GOLD (Verdonk et al. 2003), and Glide (Friesner et al. 2004) commonly use scoring functions to predict the structure of the bound ligand (the pose), its binding affinity and its rank compared to other proposed poses. These scoring functions use either molecular force fields (Huang et al. 2006), statistical potentials (Gohlke et al. 2000), or linear combinations of empirical terms (Krammer et al. 2005) . Advancements in machine learning (ML) have enabled the development of MLbased scoring functions (MLBSFs) that outperform these other scoring functions in accuracy for predicting binding affinity. Initially, these scoring functions used classical ML techniques, e.g. tree-based models, and simple features extracted from the protein–ligand complex structure (Ballester and Mitchell 2010, Durrant and McCammon 2011, Zilian and Sotriffer 2013, Ballester et al. 2014, Li et al. 2015, Wang and Zhang 2017, Meli et al. 2021) . With the emergence of deep learning techniques, scoring functions based on the convolutional neural network (CNN) architecture to predict the binding affinity only were built and trained on explicit, voxelised representations of the ligand-protein complex (Francoeur et al. 2020) [e.g. Pafnucy (Stepniewska-Dziubinska et al. 2018) and KDeep (Jimnez et al. 2018)] . Newer deep learning methods such as graph neural networks (GNNs) represented atom

FAQs

Why do machine learning-based scoring functions (MLBSFs) need more rigorous evaluation?

MLBSFs can perform inconsistently across benchmarks and may learn dataset bias instead of a generalizable understanding of physics, motivating deeper investigation into how they operate.

How did the study compare MLBSFs with baseline models?

The authors compared several popular MLBSFs (RFScore, SIGN, OnionNet-2, Pafnucy, PointVS) with proposed baseline models designed to learn only dataset biases across a range of benchmarks.

What conclusion was drawn about what MLBSFs learn?

Baseline models were competitive with MLBSFs on almost all proposed benchmarks, indicating that these MLBSFs primarily learn dataset biases rather than transferable biophysical signals.

Aug 5, 2026

Upload by Riley