Better Harnesses Smaller Models Building 90% Cheaper Agents via Automated Harness Adaptation

Jul 16, 2026

Upload by Riley

Frontier LLM agents automate business work but face unsustainable inference cost, latency, and privacy pressure for large deployments. Small language models (SLMs) are cheaper and faster, yet underperform when inserted into harnesses built for frontier systems. The work shows routine business tasks can reach LLM-level quality at about 90% lower cost by pairing SLMs with automatically discovered adapted harnesses. It proposes a failure-mode-to-adaptation framework and a harness optimizer, improving 16/21 task-SLM pairs.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages12

Reading Time30 min

Introduction

Cost and limitations of frontier LLM agents

SLMs as an alternative for routine business workflows

Harness adaptation concept and motivation

Better Harnesses, Smaller Models: Building 90% Cheaper Agents via Automated Harness Adaptation Chenyang Yang, Xinran Zhao, Tongshuang Wu, and Christian Kästner Carnegie Mellon University arXiv :2607 .08938v 1 [ cs . SE] 9 Jul 2026 Abstract—Frontier LLM agents are automating many business tasks, but their high inference cost makes large-scale deployment unsustainable. Small language models (SLMs) offer a cheaper alternative, yet they typically fall short when swapped into a harness designed for a frontier LLM. We show that for many routine business tasks, SLM agents can match LLM performance at 90% lower cost, when paired with an adapted harness that can be automatically discovered by a meta agent. The key insight is that much of the task difficulty is shared across instances and can be lifted from the model into the harness via tailored instructions, tools, and orchestration loops. To study this systematically, we create a framework that maps agent failure modes to harness adaptation strategies, and build a harness optimizer that automatically discovers effective adaptations from failure trajectories. Across seven business-oriented agentic tasks and three SLM families, we found optimized harnesses significantly improve performance on 16 of 21 task-SLM pairs, with seven pairs closing the SLM-LLM performance gap and the best SLM agent recovering 89.7% of LLM performance at 4% of the cost. 1 Our analysis further shows that adaptation works best for tasks with more repetitive workflows and for SLMs with sufficient base capabilities. Together, these results suggest that harness adaptation can expand the practical deployment range of SLM agents in routine business tasks. Index Terms—agents; small language models; harness adaptation; cost-efficient AI deployment I. INTRODUCTION Large language models (LLMs) power many agentic systems that are increasingly deployed in real-world business workflows [1], [2] . While frontier models deliver impressive performance, they incur substantial inference cost, high latency, and data-privacy concerns – businesses have poured billions of dollars into incorporating agents into their workflows [3], and that continuous spending can quickly become unsustainable [4] . Small language models (SLMs)2 have emerged as an alternative for replacing LLMs on agentic tasks, with their lower cost, better latency, and less privacy concerns [8] . While SLMs fall short of LLMs on general-purpose agentic coding tasks [9], many business-relevant tasks do not require open-ended creativity, but only reliable execution of routine workflows within a constrained environment [8] – this makes SLMs an attractive option for replacing LLMs in these contexts. As a concrete example, suppose we want to deploy an agent to collect, review, and communicate various budget requests 1Code available at 􀂇 [https://github.com/malusamayo/migration-analysis](https://github.com/malusamayo/migration-analysis). 2We use SLMs operationally for cheaper, smaller, open-weight models that can be deployed locally. Our experiments use models with 3B to 8B (8B to 30B) active (total) parameters (e.g., qwen3-30b-a3b [5]) . In comparison, frontier open-sourced models have more than 1T model parameters [6], [7] . in a company, a routine business function that many try to automate. We can deploy an agent with a frontier LLM that will perform very well (97.3% accuracy with gemini-3 . 1-pro on curated evaluation data), but is also expensive ($0.22 per query) to deploy in production. Alternatively, we can deploy the same agent with an SLM. The agent, however, performs worse (75 .0% with gemma-4-26b-a4b) and cannot make a reliable replacement (Figure 1) . In this paper, we demonstrate that, for many routine agentic tasks in business settings, we can build specialized SLM agents that are 90% cheaper yet with on-par performance with LLM agents. This is achieved by pairing SLMs with well-adapted harnesses that can be discovered automatically

FAQs

Why are frontier LLM agents costly for large-scale business deployment?

Their inference cost, latency, and data-privacy concerns make continuous production spending hard to sustain.

How does automated harness adaptation improve SLM agent performance?

It moves shared task difficulty from the model into a tailored harness using tailored instructions, tools, and orchestration loops, scaffolding the SLM with constraints and plan structure.

What results show the cost and performance benefits of the proposed approach?

Across seven business-oriented agentic tasks and three SLM families, optimized harnesses improve performance on 16 of 21 task-SLM pairs; seven pairs close the gap, and the best SLM recovers 89.7% of LLM performance at about 4% of the cost.

Jul 16, 2026

Upload by Riley