Depth Functions for Partial Orders - with a Descriptive Analysis of Machine Learning Algorithms

Aug 3, 2026

Upload by Himbo

A framework is presented for descriptively analyzing sets of partial orders using depth functions. After noting that depth functions are extensively studied in linear and metric spaces, the work addresses the gap for non-standard data types by introducing an adaptation of simplicial depth for the space of all partial orders: union-free generic (ufg) depth. The ufg depth is then used to compare machine learning algorithms through multidimensional performance measures, analyzing classifier-performance distributions across benchmark data sets, and highlighting differences from traditional benchmarking.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages13

Reading Time33 min

AccessFree to download

Abstract

Introduction

Depth functions for partial orders

Comparison of machine learning algorithms with multidimensional measures

Random partial-order outcomes in data-driven settings

Depth Functions for Partial Orders with a Descriptive Analysis of Machine Learning Algorithms Hannah Blocher [hannah.blocher@stat.uni-muenchen.de](hannah.blocher@stat.uni-muenchen.de) Georg Schollmeyer [georg.schollmeyer@stat.uni-muenchen.de](georg.schollmeyer@stat.uni-muenchen.de) Christoph Jansen [christoph.jansen@stat.uni-muenchen.de](christoph.jansen@stat.uni-muenchen.de) Malte Nalenz [malte.nalenz@stat.uni-muenchen.de](malte.nalenz@stat.uni-muenchen.de) Department of Statistics, Ludiwg–Maximilians–Universität München, Munich, Germany arXiv :2304 .09872v2 [ cs .LG] 4 Jul 2023 Abstract We propose a framework for descriptively analyzing sets of partial orders based on the concept of depth functions. Despite intensive studies of depth functions in linear and metric spaces, there is very little discussion on depth functions for non-standard data types such as partial orders. We introduce an adaptation of the well-known simplicial depth to the set of all partial orders, the union-free generic (ufg) depth. Moreover, we utilize our ufg depth for a comparison of machine learning algorithms based on multidimensional performance measures. Concretely, we analyze the distribution of different classifier performances over a sample of standard benchmark data sets. Our results promisingly demonstrate that our approach differs substantially from existing benchmarking approaches and, therefore, adds a new perspective to the vivid debate on the comparison of classifiers.1 Keywords: partial orders, data depth, benchmarking, algorithm comparison, outlier detection, non-standard data 1. Introduction Partial orders – and the systematic incomparabilities of objects encoded in them – occur naturally in a variety of problems in a wide range of scientific disciplines. Examples range from decision theory, where the agents under consideration might be unable to arrange the consequences of their actions into total orders (see, e.g., [38, 22]) or have partial cardinal preferences (see, e.g., [18, 20]), over social choice theory, where a fair aggregate order might only be possible by incorporating systematic incomparabilities (see, e.g., [31, 19]), to finance, where risky assets do not always have to be comparable (see, e.g., [24, 7]) . Of course, many other relevant examples exist. 1Open Science: Reproducible implementation and data analysis are available [at:](at: www.github.com/hannahblo/23_Performance_)[ www.github.com/hannahblo/23_Performance_](at: www.github.com/hannahblo/23_Performance_)Analysis_ML_Algorithms. In the specific context of statistics and machine learning, the incompleteness of the considered orders often originates from the fact that the objects to be ordered are to be compared with respect to several criteria and/or on several instances simultaneously: only if there is unanimous dominance of one object over another, this order is included in the corresponding relation. Quite a number of research papers recently have been devoted to such comparison in the specific context of classification algorithms, either with respect to multiple quality metrics (e.g., [12, 21]) or across multiple data sets (e.g., [10, 3]) or with respect to genuinely multidimensional performance criteria like receiver operating characteristic (ROC) curves (e.g., [8]) . Another source of partial incomparability of classifiers is the case of classifiers that make only imprecise predictions, like for example the naive credal classifier (cf., [46]) or credal sumproduct networks (cf., [27]) . In this case the imprecision in the predictions may take over to incomparabilities of the then possibly interval-valued performance measures2 Within the application field of machine learning and statistics, one further aspect is of special importance: Since the instances generally depend on chance, the same is true for the partial orders considered. Consequently, instead of a single partial order, random variables must then be analyzed that map in

FAQs

What problem does the paper address about depth functions?

It addresses the lack of depth-function methodology for non-standard data types, specifically sets of partial orders, where incomparabilities play a central role.

How is the proposed union-free generic (ufg) depth defined conceptually?

It adapts the well-known simplicial depth to the set of all partial orders, yielding a depth notion tailored to poset-valued outcomes.

How is ufg depth used to compare machine learning algorithms?

The paper uses ufg depth to compare classifiers via multidimensional performance measures by studying the distribution of performance outcomes over multiple benchmark data sets.

Aug 3, 2026

Upload by Himbo