AUTOMATED MACHINE LEARNING FOR POSITIVE UNLABELLED LEARNING - PhD thesis submitted to the University of Kent - Abstract

Aug 4, 2026

Positive-Unlabelled (PU) learning trains classifiers using positive class examples and unlabeled instances where each unlabeled point may be either positive or negative. The lack of explicit negative labels makes PU learning fundamentally different from standard binary classification, requiring dedicated modelling and evaluation metrics. This thesis develops three PU-specific Automated Machine Learning systems: GAAuto-PU, BO-Auto-PU, and EBO-Auto-PU. The methods are evaluated across 120 tasks using F-measure, outperforming common baselines.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages249

Reading Time10 h 27 min

AccessFree to download

Abstract

Acknowledgements

Contents

List of Tables

List of Figures

List of Procedures’ Pseudocodes

Glossary

Chapter 1

AUTOMATED MACHINE LEARNING FOR POSITIVE UNLABELLED LEARNING A THESIS SUBMITTED TO THE UNIVERSITY OF KENT IN THE SUBJECT OF COMPUTER SCIENCE FOR THE DEGREE OF PHD. By Jack Duke Saunders Abstract Positive-Unlabelled (PU) learning is a field of machine learning that involves learning classifiers from data consisting of positive class and unlabelled instances. That is, instances that may be either positive or negative, but the label is unknown. PU learning differs from standard binary classification due to the absence of negative instances. This difference is non-trivial and requires differing classification frameworks and evaluation metrics. This thesis looks to address gaps in the PU learning literature and make PU learning more accessible to non-experts by introducing Automated Machine Learning (Auto-ML) systems specific to PU learning. Three such systems have been developed, GAAuto-PU, a Genetic Algorithm (GA)-based Auto-ML system, BO-Auto-PU, a Bayesian Optimisation (BO)-based Auto-ML system, and EBO-Auto-PU, an Evolutionary/Bayesian Optimisation (EBO) hybrid-based Auto-ML system. These three Auto-ML systems are three primary contributions of this work. EBO, the optimiser component of EBO-Auto-PU, is by itself a novel optimisation method developed in this work that has proved effective for the task of Auto-ML and represents another contribution. EBO was developed with the aim of acting as a trade-off between GA, which achieved high predictive performance but at high computational expense, and BO, which, when utilised by the Auto-PU system, did not perform as well as the GA-based system but did execute much faster. EBO achieved this aim, providing high predictive performance with a computational runtime much faster than the GA-based system, and not substantially slower than the BO-based system. The proposed Auto-ML systems for PU learning were evaluated on three versions of 40 datasets, thus evaluated on 120 learning tasks in total. The 40 datasets consist of 20 real-world biomedical datasets and 20 synthetic datasets. The main evaluation measure was the F-measure, a popular measure in PU learning. Based on the F-measure results, the three proposed systems outperformed in general two baseline PU learning methods, usually with statistically significant results. Among the three proposed systems, there was no statistically significance difference between their results in general, whilst a version of the EBO-Auto-PU system performed overall slightly better than the other systems, in terms ofF-measure. The two other main contributions of this work relate specifically to the field of PU learning. Firstly, in this work we present and utilise a robust evaluation approach. Evaluating PU learning classifiers is non-trivial and little guidance has been provided in the literature on how to do so. In this work, we present a clear framework for evaluation and use this framework to evaluate the proposed systems. Secondly, when evaluating the proposed systems, an analysis of the most frequently selected components of the optimised PU learning algorithm is presented. That is, the components that constitute the PU learning algorithms produced by the optimisers (for example, the choice of classifiers used in the algorithm, the number of iterations, etc.) . This analysis is used to provide guidance on the construction of PU learning algorithms for specific dataset characteristics. Acknowledgements First and foremost, I would like to give thanks to Professor Alex Freitas. Attending university was not something I had ever considered an option for myself, let alone the pursuit of a PhD. The patience and time given by Professor Freitas has been unwavering and invaluable, and I am profoundly grateful for the opportunity that he has given me. I would also like to give thanks to the University of Kent computing department, in particular the members of my supervisory panel and the Director of Graduate Studies, Daniel Soria, Fe

FAQs

What makes positive-unlabelled (PU) learning different from standard binary classification?

PU learning uses positive examples and unlabeled instances, where unlabeled data can be either positive or negative but the label is unknown. This absence of explicit negative instances changes the modelling and evaluation approach.

Which Auto-ML systems for PU learning were developed in the thesis?

The thesis develops GAAuto-PU (genetic algorithm-based), BO-Auto-PU (Bayesian optimisation-based), and EBO-Auto-PU (an evolutionary/Bayesian optimisation hybrid). EBO is also presented as a novel optimisation method.

How were the proposed systems evaluated, and what metric was used?

They were evaluated on three versions of 40 datasets, covering 120 learning tasks total, including 20 real-world biomedical datasets and 20 synthetic datasets. The main evaluation measure was the F-measure, and results generally outperformed baseline PU methods.

Aug 4, 2026

Upload by Ava Thompson