Uncertain Choices in Method Comparisons - An Illustration with t-SNE and UMAP

Aug 30, 2026

Upload by Sarah

Dimensionality reduction enables meaningful low-dimensional representations of high-dimensional data for analysis and visualization. This bachelor’s thesis studies t-SNE and UMAP by combining theoretical descriptions with a systematic empirical evaluation. Both algorithms are run with multiple hyperparameter settings on six datasets, and three embedding quality measures are computed for each run. Linear models and comparative plots are then used to assess how parameters affect local, mesoscopic, and global structure. The results show t-SNE outperforming UMAP across all three measures, despite UMAP’s common reputation for preserving global structure.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages69

Reading Time2 h 54 min

AccessFree to download

Introduction

Background

t-SNE

UMAP

Comparison

Methods

Parameters

Data Sets

Quality Measures

Analysis

Results

Local Structure

Mesoscopic Structure

Global Structure

Scatter Plots

Discussion

Appendix

Uncertain Choices in Method Comparisons: An Illustration with t-SNE and UMAP Bachelor’s Thesis Faculty of Mathematics, Informatics and Statistics Department of Statistics Ludwig–Maximilian–University Munich Submitted by Philipp Weber Supervisor: Dr. Sabine Hoffmann Munich, 21 April 2023 I hereby declare that this thesis was composed by myself and that the work contained herein is my own except where explicitly stated otherwise in the text. Munich, 21 April 2023 Signature Abstract To help with the visualization of high dimensional data, dimension reduction techniques have become essential. Two such techniques that have gained a lot of popularity in the last years are t-Distributed Neighbor Embedding (t-SNE) and Uniform Manifold Approximation and Projection (UMAP) . In this thesis we examined these two algorithms, first theoretically describing and comparing them and then analyzing their performance and the influence of certain parameters. We ran t-SNE and UMAP with different parameter settings on six datasets and calculated three quality measures for each outcome. We then analyzed these results through linear models and compared them with different plots. Most of the parameters we examined in this thesis influenced the quality of the embedding. For some parameters one setting was clearly superior, while other parameters were more of a trade-off between different quality measures. In our analysis, t-SNE performed a bit better than UMAP regarding all three quality measures, which is surprising, since UMAP is often regarded to better preserve the global structure of the data (Becht et al., 2018) . Contents 1 Introduction 1 2 Background 2 2.1 t-SNE ................................... 2 2.1.1 Algorithm ............................. 2 2.1.2 Hyperparameters ......................... 5 2.1.3 Strengths and Weaknesses .................... 6 2.2 UMAP ................................... 6 2.2.1 Algorithm ............................. 7 2.2.2 Hyperparameters ......................... 9 2.2.3 Strengths and Weaknesses .................... 9 2.3 Comparison ................................ 10 2.3.1 Theoretical Comparison ..................... 10 2.3.2 Empirical Comparison ...................... 11 3 Methods 13 3.1 Parameters ................................ 13 3.1.1 t-SNE ............................... 13 3.1.2 UMAP ............................... 14 3.2 Data Sets ................................. 14 3.3 Quality Measures ............................. 15 3.4 Analysis .................................. 15 3.4.1 Linear Models ........................... 15 3.4.2 Density Plots ........................... 16 3.4.3 Box Plots ............................. 16 3.4.4 Scatter Plots ........................... 16 4 Results 17 4.1 Local Structure .............................. 17 4.2 Mesoscopic Structure ........................... 21 4.3 Global Structure ............................. 25 4.4 Scatter Plots ............................... 29 4.4.1 MNIST .............................. 30 4.4.2 COIL-20 .............................. 32 5 Discussion 35 A 37 A.1 Linear Models ............................... 37 A.2 Diagnostic Plots .............................. 41 A.3 Scatter Plots ............................... 50 A.4 Parameters ................................ 56 Chapter 1 Introduction Dimensionality reduction is an important task in the field of data analysis and visualization, as it creates a low dimensional representation of high dimensional data, enabling one to understand and analyze complex datasets more easily. Two popular algorithms for dimensionality reduction are t-Distributed Stochastic Neighbor Embedding (t-SNE) by van der Maaten and Hinton (2008) and Uniform Manifold Approximation and Projection (UMAP) by McInnes et al. (2018), which have been shown to be effective and produce visually appealing results across many domains, such as art (Vermeulen

FAQs

What is the main objective of the thesis?

The thesis investigates which hyperparameters most strongly influence embedding quality, whether consistently good settings exist, and whether t-SNE or UMAP is generally superior.

How are t-SNE and UMAP evaluated in the study?

Both algorithms are run with different parameter settings on six datasets, and three quality measures are calculated for each embedding outcome.

What analytical tools are used to interpret the results?

The quality-measure results are analyzed using linear models and compared through different visualization plots such as diagnostic and scatter-based approaches.

Aug 30, 2026

Upload by Sarah