On the Experiences and Challenges of Recruiting Expert Samples for Security Studies - Case Studies with Software Developers, Machine Learning Practitioners and Open-Source Software Developers

Aug 5, 2026

Modern society’s reliance on software systems makes security assurance a central pillar for resilient digital infrastructure. Security studies must therefore examine human factors, especially by recruiting experts who build, maintain, apply, or improve technologies. This dissertation investigates recruiting expert samples for security studies and reports recruitment experiences while focusing on three groups: software developers, machine learning practitioners, and open-source software developers. It analyzes recruitment platforms, surveys developers, and conducts interviews to derive barriers, traits, and recommendations for selecting recruitment channels.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages212

Reading Time8 h 54 min

AccessFree to download

Summary

Research motivation and problem statement

Dissertation scope and expert groups

Work 1: recruitment platforms for software developers

Work 2: barriers in adversarial ML defenses and recruiting ML practitioners

Work 3: threat finding and mitigation practices in open-source projects

Key findings and recruitment recommendations

On The Experiences And Challenges Of Recruiting Expert Samples For Security Studies: Case Studies With Software Developers, Machine Learning Practitioners And Open-Source Software Developers Von der Fakultät für Elektrotechnik und Informatik der Gottfried Wilhelm Leibniz Universität Hannover zur Erlangung des akademischen Grades Doktorin der Naturwissenschaften (abgekürzt: Dr. rer. nat.) genehmigte Dissertation von Frau Harjot Kaur, M.Sc. geboren am 17.11.1993 in Hazaribag, Indien 2025 Referent: Prof. Dr. Sascha Fahl Korreferentin: Prof. Dr. Alena Naiakshina Tag der Promotion: 11.03.2025 Summary Modern society’s undeniable dependency on software systems mean that software usage is deeply intertwined with nearly every facet our daily lives. This dependency makes assurance of software security one of the key pillars upholding the resiliency of the digital infrastructure that (em)powers our modern world. Researching human factors is important for improving the overall security and privacy of information technology systems. In particular, conducting security studies with experts that build, maintain, apply or improve technologies is necessary to understand the root cause(s) behind poor software security that so often leads to security pitfalls. Experts, such as developers, have garnered increased research attention within the security community. However, in contrast to end-users, recruiting experts for security studies is often quiet challenging, due to factors such as high recruitment costs or targeting expert niches. Therefore, a better overview of the (dis)advantages of different recruitment avenues, along with the characteristics of recruited experts, can definitely help (security) researchers recruit experts more efficiently. This dissertation does the same, investigating specific recruitment platforms and experts samples. In this work, I explore the recruitment of expert samples for security studies and share the findings of my investigations as well as the experiences I accrued while doing so. To achieve this, I focus on three expert samples: software developers, machine learning practitioners and open-source software developers. In addition to providing recruitmentrelated insights, in the specific case studies with machine learning practitioners and open-source software developers, I also investigate these experts in the context of different security-specific research questions. My first work explores the potentials and challenges of various recruitment platforms to recruit software developer samples for security developer studies. Using literature analysis, I first identify popular platforms used to recruit expert users in previous work. Using an online survey, I then investigate participants’ security and software development knowledge, skills, and experiences, and systematically compare developer samples across various recruitment platforms. Using semi-structured interviews with 21 machine learning practitioners, my second work investigates what barriers obstruct them from effectively implementing adversarial machine learning defenses in their machine learning systems. This work also details my experiences recruiting machine learning practitioners from the freelancing platform Upwork. My final work uses semi-structured interviews with 25 open-source software developers to investigate the threat finding and mitigating practices in open-source software projects. Open-source developers were recruited using variety of strategies and I conclude by describing my accrued experiences. Overall, my findings show that there is no one-size-fits-all model for recruiting experts. The roles of these experts and their working environments must be taken into account when selecting one or multiple recruitment channels. I also describe the traits of different recruitment platforms and strategies that may impact their suitability for different (security) study types. Based on my findings and experiences, v

FAQs

Why is recruiting experts important in security studies?

Recruiting experts enables researchers to study root causes behind poor software security, which end-user studies often cannot reveal as directly. Expert perspectives help connect security outcomes to real development and technology choices.

Which three expert groups does the dissertation focus on?

The dissertation focuses on software developers, machine learning practitioners, and open-source software developers, and examines them through different security-relevant case study questions.

What are the main insights about recruiting experts?

There is no one-size-fits-all recruitment model. The experts’ roles and working environments must be considered when choosing recruitment channels, since platform and strategy traits affect suitability for different study types.

Aug 5, 2026

Upload by Seraphina