Auditing Exposure to Harmful Content on TikTok using Multimodal Language Models - A Cross-National, Age-Stratified Study

Sep 15, 2026

Uploaded by Hazel

Online video platforms can expose young users to harmful content, yet independent audits are hindered by expensive annotation and inconsistent moderation judgments across languages. This study audits TikTok in France, Italy, and Sweden using sockpuppet accounts across four age personas (13, 16, 19, 40), collecting 36,971 videos from passive For-Youpage scrolling and active probe sessions. Four multimodal LLMs are validated on a 300-video reference set; Gemini 2.5 Flash with sampled frames and text performs best. Keyword search yields 35–56% harmful content, up to 7.5× versus passive baselines, with provider filters undercounting explicit harms.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages20

Reading Time50 min

AccessFree to download

Abstract

Introduction

Related Work

Auditing Exposure to Harmful Content on TikTok using Multimodal Language Models: A Cross-National, Age-Stratified Study Hamidreza Saffari1 and Francesco Pierri1 1Politecnico di Milano [hamidreza.saffari@mail.polimi.it](hamidreza.saffari@mail.polimi.it), [francesco.pierri@polimi.it](francesco.pierri@polimi.it) arXiv : 2608 . 17583v 1 [cs .CL] 18 Aug 2026 Abstract Online video platforms can expose young users to harmful content, but independent audits remain difficult because video annotation is costly and moderation judgments vary across languages. We audit TikTok in France, Italy, and Sweden with sockpuppet accounts representing four age personas (13, 16, 19, 40), collecting 36 ,971 videos from passive For-Youpage scrolling and active sessions that scroll, search for harm keywords, and scroll again. To scale annotation, we validate four multimodal LLMs against native-speaker labels on a 300-video reference set. Gemini 2 .5 Flash with eight sampled frames plus text performs best (aggregate κ = 0 .42), at half the per-call cost of native-video upload, and we apply it to a 10% sample for approximately $50 in total API spend across both modalities. Keyword search returns 35–56% harmful content, a 1.5–7.5 × increase over the scrolling baseline in ten of twelve country–age combinations; the spike is temporary and flattens the age differences observed in France and Sweden. Under passive scrolling, Italy has the highest harm rate at every age, with Italian age-19 reaching 48.6% . Overall, MLLM-based auditing offers a scalable approach for cross-national youth-safety audits, while provider safety filters (1 . 1% refusal rate) under-count the most explicit harms. 1 Introduction TikTok has become one of the most influential gateways through which young users encounter video content online, with a For-You feed that rapidly adapts to micro-interactions such as watch time and loop rates (Boeker and Urman, 2022 ; Baumann et al., 2025) . Independent audits have documented self-harm and suicidal-ideation content reaching newly created teen accounts within hours (Amnesty International, 2023, 2025), weightnormative and pro-eating-disorder content dominating health-adjacent feeds (Minadeo and Pope, 2022 ; Strickland et al., 2025 ; Blackburn and Hogg, 2024), and age-gating mechanisms that do not meaningfully shield underage personas relative to adults (Eltaher et al., 2025 ; Xue et al., 2025) . Public-health concerns (Murthy, 2023) and regulatory frameworks like the EU Digital Services Act make independent, reproducible audits of shortvideo platforms increasingly urgent. One response to this moderation burden is to enlist multimodal large language models (MLLMs) as automated annotators: recent work shows aligned hate-speech judgments with humans (Davidson et al., 2025 ; Jo et al., 2024) and improved alignment from policy-and rule-grounded MLLMs (Wu et al., 2025) . At the same time, MLLMs disagree substantially with each other on the same items (Fasching and Lelkes, 2025) and can exploit language priors to answer without looking at the video (Apple Machine Learning Research, 2025); how these strengths and failure modes interact with age (which shifts the harm distribution the algorithm exposes), country (which introduces cross-lingual moderation challenges), and input modality, the three dimensions most relevant to youth-safety audits of TikTok, remains largely unexplored. We address three research questions: • RQ1: Which MLLM configuration agrees with native-speaker annotators well enough to run at scale? • RQ2: What is the marginal value of sampled frames and native video over text-only input? • RQ3: How does harm exposure differ across age personas, countries, and platform signals? We tackle them with a two-stage audit on three EU countries (France, Italy, Sweden) and four age personas (13, 16, 19, 40, spanning TikTok’s stated minimum, mid-adolescence, young adulthood in the platform’s 18+ tier, and an adult con

FAQs

How did the study measure exposure to harmful content on TikTok?

It used sockpuppet accounts representing four age personas to collect videos from passive For-Youpage scrolling and from active sessions involving harm-keyword searches followed by additional scrolling.

What role do multimodal LLMs play in the auditing pipeline?

Multimodal LLMs act as automated annotators. Four model configurations were validated against native-speaker labels on a 300-video reference set, and the best-performing configuration was applied to the full corpus using a sampled subset for scalability.

What impact did keyword search have compared with passive scrolling?

Keyword search produced substantially higher harm rates—35–56%—with increases of 1.5–7.5× over the passive scrolling baseline in most country–age combinations. The spike was temporary and returned toward baseline after the post-search snapshot.

Sep 15, 2026

Uploaded by Hazel