GLM-5 Serving Parameter Tuning for OpenClaw Single-Deployment MaaS Inference Optimization for Long-Context Agent Workloads

Jul 16, 2026

Upload by Aurora

OpenClaw requests are dominated by long, tool-augmented prefixes with large system prompts, accumulated conversation history, and tool outputs fed back into the context window. With roughly 28k–30k input tokens and ~500 output tokens per request, serving quality depends on throughput, time-to-first-token (TTFT), and tail latency. The report places tuning within a MaaS multi-model inference optimization architecture, focusing on the Single-Node Optimization block for one GLM-5 deployment using a two-node, sixteen-GPU cluster.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages10

Reading Time25 min

Abstract

1 Background and Problem Definition

2 MaaS Multi-Model Inference Optimization Architecture

arXiv :2607 .02518v1 [ cs .DC] 27 Apr 2026 Technical Report GLM-5 Serving Parameter Tuning for OpenClaw Single-Deployment MaaS Inference Optimization for Long-Context Agent Workloads Minjie Hua, Ning Wang, Peijun Yang, Kai Wang, and Shiguo Lian China Unicom July 7, 2026 [huamj5@chinaunicom.cn](huamj5@chinaunicom.cn), [wangn85@chinaunicom.cn](wangn85@chinaunicom.cn), [yangpj16@chinaunicom.cn](yangpj16@chinaunicom.cn) [wangk115@chinaunicom.cn](wangk115@chinaunicom.cn), [liansg@chinaunicom.cn](liansg@chinaunicom.cn) Abstract OpenClaw requests are dominated by long, tool-augmented prefixes: a sizable system prompt, accumulated conversation history, and tool outputs that are fed back into the context window [5, 6] . Under this workload shape—approximately 28000 to 30000 input tokens per request and about 500 output tokens—serving quality is governed jointly by throughput, time-to-first-token (TTFT), and tail latency rather than by raw short-prompt throughput alone [7, 11] . This report positions the tuning work inside a Model-as-a-Service (MaaS) multi-model inference optimization architecture. The broader architecture spans domain applications, MaaS platform performance optimization, multi-model orchestration, inference optimization, and compute-stack adaptation; the scope of this report is the Single-Node Optimization block in the inference-optimization layer, where serving parameters such as chunked prefill, tensor parallelism (TP), pipeline parallelism (PP), and max-running-request concurrency are tuned for one GLM-5 serving deployment. In this report, SingleNode Optimization is the architecture-block label, not a restriction to one physical server; the measured deployment uses the two-node, sixteen-GPU cluster described in Section 4. Within the tested parameter space, the best measured configuration is chunked-prefill-size=3072 , tp=4 , pp-size=4, and max-running-requests=24 . Relative to the conservative 2048/4/4/16 baseline, this setting improves request throughput from 0.43 to 0.48 req/s and total token throughput from 9029.64 to 9993.23 tok/s, while reducing TTFT average from 8.98 to 6.69 seconds and latency P90 from 40.23 to 32.64 seconds. Under a fixed hardware footprint, that corresponds to an estimated 10 .4% lower serving cost per request and 9 .6% lower serving cost per token. Relative to the 4096/4/4/16 comparison profile, normalized serving cost falls by about 20 .8% per request and 21 . 1% per token. Accordingly, we recommend 3072 / tp4 / pp4 / max24 as the default deployment profile for OpenClaw. The data also show that the optimum is not monotonic: larger chunk sizes (4096, 6144) and deeper queueing (32, 48) do not further improve this workload. The results indicate a workload-specific sweet spot rather than a simple “larger is always better” rule. 1 Background and Problem Definition Optimizing serving parameters using “raw model throughput”(e.g., short prompts, single-turn, synthetic load) can be misleading for OpenClaw. OpenClaw constructs a per-run context that includes: (i) a system prompt describing tools/skills/runtime constraints, (ii) the full conversation history, and (iii) tool calls and tool outputs; all of these consume tokens inside the model context window [5, 6] . As a result, even when the user prompt is brief, the effective model input can become very long, especially as multi-turn interactions accumulate and tool outputs (command output, file snippets, web/tool results) are reinjected [5] . This yields three business-relevant characteristics for OpenClaw agent serving: Long system prompt and high context overhead. Tool schemas and injected workspace content add non-trivial context cost each turn [6] . This increases the probability of “long prefix”requests even in routine usage. Multi-turn accumulation and tool-heavy traces. Multi-round agent sessions are inherently prefix-growing: each turn appends assistant outputs and tool transcripts [5] . Long-prefix reques

FAQs

Why can raw model throughput be misleading for OpenClaw serving?

OpenClaw constructs per-run context from a system prompt, full conversation history, and reinjected tool calls and tool outputs. This inflates effective input length and changes prefill and KV-cache pressure even when the user prompt is short.

Which serving parameters are tuned in this report for the GLM-5 deployment?

The tuned parameters include chunked prefill size, tensor parallelism (TP), pipeline parallelism (PP), and max-running-request concurrency, optimized within the Single-Node Optimization block.

What is the best measured configuration and how does it affect performance?

The best configuration is chunked-prefill-size=3072, tp=4, pp-size=4, and max-running-requests=24. Compared with a 2048/4/4/16 baseline, it raises throughput and token throughput while reducing average TTFT and P90 latency, leading to lower estimated serving cost per request and per token.

Jul 16, 2026

Upload by Aurora