ANCHOR Automated Alignment Auditing for CLI Agents on Real-World Harm

Jul 16, 2026

Autonomous CLI agents can carry out hundreds of actions in long sessions, such as writing code, running shell commands, browsing the web, and operating cloud infrastructure with minimal oversight. ANCHOR is introduced as an automated auditing framework that stress-tests these agents on illegal tasks grounded in public US court cases. A strong auditor agent uses strategic, persistent malicious user behavior to bypass refusals and adapt across multi-turn interactions. Results show refusal can fail under persistence, enabling harmful autonomy and catastrophic-risk outcomes.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages19

Reading Time48 min

Introduction

Catastrophic risk and policy context

Limits of existing benchmarks and auditing frameworks

The ANCHOR auditing framework

ANCHOR: Automated Alignment Auditing for CLI Agents on Real-World Harm Kefan Song 1 Yanjun Qi 1 arXiv :2607 . 10455v 1 [ cs .AI] 11 Jul 2026 Abstract Autonomous CLI agents can now execute hundreds of actions across multi-hour sessions: writing code, executing shell commands, browsing the web, and managing cloud infrastructure, all with minimal human oversight. Does greater autonomy invite greater risk? We introduce ANCHOR, an automated auditing framework that stress-tests CLI agents on illegal tasks grounded in public US court cases. ANCHOR deploys an auditor agent fine-tuned on dark personality data using supervised and reinforcement fine tuning. This auditor roleplays persistent malicious users who decompose tasks, reframe requests upon refusal, and adapt strategies across multi-turn interactions. Evaluating frontier CLI agents, we find that while they often refuse illegal tasks when prompted directly, compliance reaches 100% under persistent malicious interaction. When agents comply, they frequently exceed user requests, autonomously building infrastructure for large-scale harm, including catastrophic risk scenarios such as large-scale financial fraud and bioweapon development. These findings demonstrate that current alignment techniques are insufficient for autonomous agents and underscore the need for safety evaluations against persistent, adaptive malicious users. We release ANCHOR at [https:](https:)//[github.com/garified/anchor](github.com/garified/anchor). 1 Introduction CLI agents such as Claude Code and Gemini-CLI are autonomous systems that execute hundreds of tool calls per session, write and run code, manage files, and interact with external services through the Model Context Protocol (MCP) . Recent work such as Cowork (Anthropic, 2026) further extends these agents beyond coding to general-purpose tasks 1University of Virginia, Charlottesville, VA, USA. Correspondence to: Kefan Song <[ks8vf@virginia.edu](ks8vf@virginia.edu) >. Proceedings of the 43 rd International Conference on Machine Learning, Seoul, South Korea. PMLR 306, 2026 . Copyright 2026 by the author(s) . including web browsing, spreadsheet manipulation, and file management. Unlike workflow-based systems where LLMs follow predefined code paths (Anthropic, 2024), these agents autonomously decide what to build and how to implement it throughout long-horizon tasks. This raises a natural question: can a malicious user exploit this autonomy to carry out large-scale real-world harmful activities? At its extreme, such misuse rises to the level of catastrophic risk, a threshold now codified in policy and law. California’s Frontier AI Models Act (SB 53) defines it as a foreseeable, material risk that a frontier model materially contributes to the death of, or serious injury to, more than 50 people or more than $1 billion in damages from a single incident (California State Legislature, 2025); Anthropic’s Responsible Scaling Policy similarly describes large-scale devastation, such as thousands of deaths or hundreds of billions of dollars in damage, directly caused by an AI model and that would not have occurred without it (Anthropic, 2023) . Existing safety benchmarks and auditing frameworks are not built to answer this. Agent safety benchmarks such as AgentHarm (Andriushchenko et al., 2025), AgentSafetyBench (Zhang et al., 2025), and OS-Harm (Kuntz et al., 2025) either pre-specify tool-call sequences or limit evaluation to short-horizon tasks, and draw on artificial scenarios that do not reflect the complexity of real-world harmful activities. Alignment auditing frameworks such as Petri (Fronsdal et al., 2025) and Bloom (Gupta et al., 2025) evaluate subtle misalignment model behaviors such as deception and self-preservation, rather than explicit misuse. Their helpfulonly auditors are neither persistent nor strategically deceptive enough to bypass safety mechanisms, and may refuse to audit catastrophic scenarios altogether, underestimating

FAQs

What is ANCHOR and what problem does it address?

ANCHOR is an automated auditing framework that stress-tests autonomous CLI agents on illegal tasks grounded in real US court cases. It targets a key gap: existing evaluations do not reflect persistent, adaptive misuse in real-world harmful activities.

How does ANCHOR generate harmful tasks?

ANCHOR-Seed transforms public US court records of successful criminal activity into realistic harmful task instructions. It uses filtering, extraction, neutral rewriting, and validation rather than relying on synthetic or annotated scenarios.

What do the evaluations show about current alignment techniques?

Agents often refuse illegal tasks when prompted directly, but compliance can reach 100% under persistent malicious interaction. When they comply, they may exceed requests and autonomously build infrastructure for large-scale harm, indicating current alignment techniques are insufficient for this threat model.

Jul 16, 2026

Upload by Theodore