UCSC NLP at SemEval-2026 Task 10 Boundary-Aware Span Extraction and RoBERTa Classification for Conspiracy Detection

Jul 16, 2026

Systems for SemEval-2026 Task 10 (PsyCoMark) address conspiracy marker extraction and document-level conspiracy detection. Marker extraction is modeled as multi-label span classification over enumerated candidate spans, using IoU≥0.95 positive labeling, hard-negative sampling, and containment-based nonmaximum suppression with boundary-aware span representations. Document classification uses an independent sequence classifier with label smoothing and a stratified train–validation split. Results show robust Actor/Victim detection while abstract roles remain sensitive to boundary criteria; best submissions rank 7th on Subtask 1 and 12th on Subtask 2.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages6

Reading Time15 min

Abstract

Introduction

Task overview

Core methods

Background and Task Setup

PsyCoMark Task and Data

Related Work and Modeling Motivation

UCSC NLP at SemEval-2026 Task 10: Boundary-Aware Span Extraction and RoBERTa Classification for Conspiracy Detection Dom Marhoefer UC Santa Cruz [dmarhoef@ucsc.edu](dmarhoef@ucsc.edu) Milos Suvakovic UC Santa Cruz [msuvakov@ucsc.edu](msuvakov@ucsc.edu) Glenn Grant-Richards UC Santa Cruz [ggranri@ucsc.edu](ggranri@ucsc.edu) Aidan Pinero UC Santa Cruz [apinero@ucsc.edu](apinero@ucsc.edu) Ryan King UC Santa Cruz [rytking@ucsc.edu](rytking@ucsc.edu) arXiv :2607 .05689v 1 [ cs .CL] 6 Jul 2026 Abstract We present our systems for SemEval-2026 Task 10 (PsyCoMark), addressing conspiracy marker extraction (Subtask 1) and document-level conspiracy detection (Subtask 2) . For marker extraction, we formulate the task as multi-label span classification over enumerated candidate spans, using IoU≥0.95 positive labeling, hardnegative sampling, and containment-based nonmaximum suppression (NMS) with boundaryaware span representations. Document classification is modeled independently using a sequence classifier with label smoothing and a stratified train–validation split. Analysis shows that entity-like roles (Actor, Victim) are detected robustly, while abstract roles (Action, Effect, Evidence) remain sensitive to boundary criteria. On the official test set, our systems rank 7th in Subtask 1 (0.2251 macro F1) and 12th in Subtask 2 (0.7694 weighted F1) . 1 Introduction Conspiracy narratives are often described in terms of recurring narrative roles: a perceived Actor performs an Action against a Victim, justified by Evidence and producing harmful Effects. SemEval- 2026 Task 10 (PsyCoMark) operationalizes this perspective through two subtasks: extracting conspiratorial roles from text and classifying whether a document expresses a conspiracy narrative (Samoryet al., 2026) . • Subtask 1: Conspiracy-marker extraction. Given a document, systems predict labeled spans corresponding to five semantic roles (Actor, Action, Effect, Evidence, Victim) . The output is a set of markers of the form {type, start, end} per document. • Subtask 2: Document-level conspiracy classification. Each document is assigned one of three labels ( Yes, No, Can’t tell) . A central feature of the PsyCoMark subtasks is that the previously defined role structures occur in both conspiratorial and non-conspiratorial discourse (critical narrative) . Documents labeled as non-conspiratorial may contain complete Actor–Action–Victim structures without conspiratorial intent, while conspiratorial documents differ primarily in framing and epistemic stance. As a result, marker extraction and conspiracy detection are intrinsically related but not equivalent: success requires the precise boundary identification of abstract semantic roles while also modeling document-level stance. For Subtask 1, we use multi-label span classification over enumerated candidate spans with IoU≥0.95 labeling, hard-negative mining, and containment-based NMS; for Subtask 2, we use a document classifier with label smoothing and a stratified train–validation split. On the SemEval evaluation server, our best submissions ranked 7thin Subtask 1 (macro F1 0 .2251) and 12th in Subtask 2 (weighted F1 0.7694) . Both models achieved competitive performance relative to other sharedtask submissions. Analysis indicates that boundary sensitivity remains a primary failure mode for abstract roles (Action, Effect, Evidence), particularly under stricter token-level overlap criteria. Entity-like roles (Actor, Victim) were comparatively robust, suggesting that semantic abstraction and variable span length dominate marker difficulty rather than topical content. 2 Background and Task Setup 2.1 PsyCoMark Task and Data Dataset characteristics. The PsyCoMark corpus contains over 4,100 English Reddit submission statements drawn from more than 190 subreddits (Samory et al., 2026) . The document-level conspiracy labels are moderately imbalanced, with 35.3% Yes, 46 .6% No, and 18 . 1%

FAQs

What are the two subtasks in SemEval-2026 Task 10 (PsyCoMark)?

Subtask 1 extracts conspiracy markers by predicting labeled spans for semantic roles (Actor, Action, Effect, Evidence, Victim). Subtask 2 classifies each document with one of three labels: Yes, No, or Can’t tell.

How is conspiracy marker extraction modeled?

It uses multi-label span classification over enumerated candidate spans, with IoU≥0.95 for positive labeling, hard-negative sampling, and containment-based NMS. Boundary-aware span representations are used to improve span boundary precision.

Which roles are most affected by boundary criteria, and what do the results show?

Entity-like roles such as Actor and Victim are detected more robustly, while abstract roles like Action, Effect, and Evidence are sensitive to boundary criteria. This indicates that boundary sensitivity, rather than topical content, drives marker difficulty for abstract roles.

Jul 16, 2026

Upload by Theodore