“What are they not telling me?” - Learning machine learning: Understanding the challenges for novices

Aug 5, 2026

Machine Learning (ML) is increasingly accessible to people with limited theoretical background, yet misusing it can cause harmful outcomes. This qualitative study investigates challenges novices face when learning core ML concepts and building their first models. Twenty participants learned classification via an interactive GUI tutorial, built a gesture-shape model, and joined semi-structured interviews. Thematic analysis highlights difficulties in problem selection, multidimensionality, defining ML, choosing algorithms, cross-validation, and interpreting visualizations, along with bias risks from input features. Findings inform the design of novice-focused ML tools.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages15

Reading Time38 min

AccessFree to download

Introduction

Study design and participant setup

Research question and thematic analysis

| ‘‘What are they not telling me?’’ Learning machine learning: Understanding the challenges for novices | | | | | --- | --- | --- | --- | | Robert Cinca a ,∗, Enrico Costanza a, Mirco Musolesi a,b, Muna Alebri c ,1 a University College London, Gower Street, London, UK
b University of Bologna, Via Zamboni, Bologna, Italy
c United Arab Emirates University, Sheik Khalifa Bin Zayed Street, Abu Dhabi, United Arab Emirates | | | | | A R T I C L E I N F O | | A B S T R A C T | | | Keywords:
Learning machine learning Machine learning Explainable AI Algorithms
Visualization
Black-box | | Machine Learning (ML) is increasingly accessible to users with limited knowledge of its theoretical foundations. However, misapplying it can lead to negative consequences. This paper reports on a qualitative study designed to reveal challenges that novices encounter when learning about basic ML concepts and building their first models. Twenty participants were introduced to fundamental ML concepts for classification through an interactive tutorial involving an off-the-shelf GUI application, built their own ML model for a shape gesture dataset, and participated in a semi-structured interview. A thematic analysis revealed insights into these challenges, particularly around problem selection and multi-dimensionality, but also around what constitutes ML, algorithm selection, cross-validation, and interpreting visualizations. Despite these and other misconceptions, participants reflected on good model building practices, discussing that algorithm selection might require knowledge and context and that input features may introduce bias. We discuss the findings’implications for the design of ML tools for novices. | | 1. Introduction Machine Learning (ML) is being used in an increasing number of domains, such as cyber-security, diagnosing disease, document classification, and language translation (Mitchell, 2019; Barreno et al., 2010; Koller and Sahami, 1997; Srividya et al., 2018). The proliferation of AutoML tools and GUI’s such as KNIME (Berthold et al., 2009), RapidMiner (Hofmann and Klinkenberg, 2016), and Orange (Demšaret al., 2013) allow novices to apply their own models even with limited theoretical knowledge of ML (Carney et al., 2020). This opens them up to the risks of misapplying ML, for example, through building models with biased outcomes for marginalized groups (Angwin et al., 2016; Noble, 2018; Keyes, 2018). Although programs built by experienced ML practitioners can also fall foul of such outcomes, those without any formal training in ML are more likely to experience challenges when designing or using ML applications, as they have limited knowledge of the theoretical foundations and the practical implications of using these techniques (Patel et al., 2008a; Amershi et al., 2014; Yang et al., 2018). We therefore present a qualitative study designed to shine light on the challenges novices in ML encounter when applying ML in an interactive environment, with implications to the design of ML tools for novices. Prior research investigating ML challenges identified challenges that more experienced ML users face (Patel et al., 2008b,a; Amershi et al., 2019; Yang et al., 2018; Veale et al., 2018), the difficulties a lay user might face when interacting with ML applications (Rader and Gray, 2015; Sanchez et al., 2021; Oh et al., 2020), or around children’s understanding of ML concepts (Hitron et al., 2019; Touretzkyet al., 2019). Instead, this work aims to introduce fundamental concepts required for building classification models to novice ML users.2 It also examines the difficulties that these novices encounter as they learn about ML. Twenty volunteers interested in learning and applying ML took part in our study. They were introduced to ML concepts through a takehome tutorial designed to be completed within two hours. The tutorial was iteratively designed and centered around a widely used graphical user interface (GUI) a

FAQs

这项研究关注的新手面临哪些主要学习挑战？

研究发现新手在选择问题、理解算法与数据的多维性、界定“机器学习”、算法选择、交叉验证以及解读可视化方面存在困难。

研究是如何让参与者学习并完成第一个 ML 模型的？

研究让 20 名参与者通过一个约两小时可完成的随带式教程学习分类概念，教程围绕一个常用图形界面（GUI）工具。随后参与者用手势形状数据集搭建自己的 ML 模型，并通过半结构化访谈接触真实场景数据。

研究对面向新手的 ML 工具设计有什么启示？

参与者在算法选择与上下文理解方面存在认知不足，同时输入特征可能引入偏差。研究据此提出，ML 工具应更好地支持新手理解这些关键环节。

Aug 5, 2026

Upload by Genevieve