[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-84670-en":3,"doc-seo-84670-105":29,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},84670,4810365810221,"Aurora","https://ap-avatar.wpscdn.com/davatar_155a257f0dc6eb9ab79c44ca47cae57d",6,"Technology","A Workflow-Aware Serving Layer for Agentic Applications","Agentic AI applications create workflow-shaped serving requests where each step is a node in a DAG of LLM and tool calls, with per-node model choices and optional quality operators like verifiers. This serving workload sits between model-serving engines and agent frameworks, leaving no single component that jointly selects model-verifier-backend decisions under live serving conditions. Dyserve fills the gap by compiling per-workflow node strategies into an ILP over heterogeneous backends using skill-conditioned offline profiles, then shifting uncommitted workflow suffixes under load while keeping solvers off the critical path.","A Workflow-Aware Serving Layer for Agentic  \nApplications  \nJiayi Qian∗ Georgia Institute of Technology Atlanta, GA, USA  \nChun Tao  \nIntel Santa Clara, CA, USA  \nZishen Wan∗ Columbia University New York, NY, USA  \nSouvik Kundu  \nIntel Los Angeles, CA, USA  \nHanchen Yang  \nGeorgia Institute of Technology Atlanta, GA, USA  \nTushar Krishna  \nGeorgia Institute of Technology Atlanta, GA, USA  \narXiv :2607 .02942v 1 [ cs .DC] 3 Jul 2026  \nAbstract  \nAgentic AI applications form an emerging serving workload in which a request creates a workflow: a directed acyclic graph of LLM and tool calls that exposes per-node model choices and optional quality operators such as verifiers. This workload falls between two existing layers. Model-serving engines execute individual calls efficiently but cannot see workflow structure, while agent frameworks fix the workflow but cannot see backend load, so neither jointly chooses each node’s model, verifier, and backend under serving-time conditions.  \nWe present Dyserve, a workflow-aware serving layer that fills this gap. Dyserve compiles each workflow’s pernode (model, verifier) choices in one integer linear program (ILP) over a heterogeneous backend pool, priced by skillconditioned offline profiles that transfer across workflows. This couples with hardware entering only through per-model throughput sweeps, and weighted to concentrate strong models and verification on the nodes whose errors propagate the furthest. Because no single latency-quality preference fits every workload mix, Dyserve pre-solves the program at several pressure levels at admission and shifts a workflow’s uncommitted suffix among these strategies under load, keeping the solver off the load-shift path; a failed tool call triggers a one-time residual re-solve that preserves committed work. On LiveCodeBench, GAIA, ComplexFuncBench, and SWE-bench, Dyserve’s compiled strategies achieve the highest accuracy on every workload, 3 to 10 points above the highest-accuracy baseline, at 1. 1 to 6 . 8× lower latency. Under multi-tenant bursts, a balanced compiled plan gains 4.7 accuracy points over round-robin routing at 2. 5× lower burst tail latency, and the precomputed ladder restores an oversubscribing plan’s SLO goodput from 18 to 67%, within 6.5 points of the best measured static plan, while leaving stable plans untouched. Event-driven recovery rescues 84% of injected tool failures against 55 for a flat retry, and admission compilation takes under 60ms per request at the 95th percentile.  \n∗Corresponding authors: Jiayi Qian ([jiayiqian@gatech.edu](jiayiqian@gatech.edu)) and Zishen Wan ([zishenwan@seas.harvard.edu](zishenwan@seas.harvard.edu)) .  \n1 Introduction  \nLarge language models are increasingly applied to complex tasks in code generation, scientific reasoning, and toolaugmented problem solving, yet a single model invocation rarely suffices. Agentic systems address this by orchestrating multiple model and tool calls into a structured computation.  \nThese systems follow two broad paradigms. Dynamic agents such as ReAct [42] and AutoGPT [30] construct the next step only after observing the previous result. Predefined agents instead execute an operator graph authored manually [11, 33–35, 41] or generated by an LLM planner [38, 44, 45], run by frameworks such as LangGraph [2], AutoGen [41], and DSPy [16] that resolve branches and unroll bounded iterations into a materialized graph of dependent LLM and tool calls. We focus on this latter setting, where the workflow is known at admission, and treat the workflow rather than the single call as the unit of serving optimization: logical structure is preserved while the physical choices that execute it adapt.  \nWorkflow structure makes final accuracy an end-to-end property rather than the property of any one call, so different nodes warrant different serving choices. A difficult synthesis node may require a strong model or verification, whereas a peripheral node whose error has little downstr","cbCailDJlPCYz58b","https://ap.wps.com/l/cbCailDJlPCYz58b","pdf",3329546,1,15,"English","en",105,"# Abstract\n# Introduction\n## Agentic AI serving paradigms\n## Resource elasticity and fleet-level scheduling\n## Dyserve position in the serving stack","[{\"question\":\"What problem does Dyserve address in agentic AI serving?\",\"answer\":\"Dyserve addresses the gap where model-serving engines cannot see workflow structure and agent frameworks cannot see backend load, preventing joint selection of each node’s model, verifier, and backend under serving-time conditions.\"},{\"question\":\"How does Dyserve decide serving strategies for each workflow?\",\"answer\":\"Dyserve compiles each workflow’s per-node (model, verifier) choices into an integer linear program over a heterogeneous backend pool, priced using skill-conditioned offline profiles that transfer across workflows.\"},{\"question\":\"What mechanisms allow Dyserve to adapt under load while keeping runtime efficient?\",\"answer\":\"Dyserve pre-solves the plan at multiple pressure levels during admission and shifts the workflow’s uncommitted suffix among these strategies under load. It also performs a one-time residual re-solve only when a tool call fails, preserving already committed work.\"}]",1784197581,38,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":27},"a-workflow-aware-serving-layer-for-agentic-applications","",{"@graph":35,"@context":85},[36,53,68],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/technology/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/a-workflow-aware-serving-layer-for-agentic-applications/84670/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-17","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What problem does Dyserve address in agentic AI serving?","Question",{"text":75,"@type":76},"Dyserve addresses the gap where model-serving engines cannot see workflow structure and agent frameworks cannot see backend load, preventing joint selection of each node’s model, verifier, and backend under serving-time conditions.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"How does Dyserve decide serving strategies for each workflow?",{"text":80,"@type":76},"Dyserve compiles each workflow’s per-node (model, verifier) choices into an integer linear program over a heterogeneous backend pool, priced using skill-conditioned offline profiles that transfer across workflows.",{"name":82,"@type":73,"acceptedAnswer":83},"What mechanisms allow Dyserve to adapt under load while keeping runtime efficient?",{"text":84,"@type":76},"Dyserve pre-solves the plan at multiple pressure levels during admission and shifts the workflow’s uncommitted suffix among these strategies under load. It also performs a one-time residual re-solve only when a tool call fails, preserving already committed work.","https://schema.org",{"og:url":51,"og:type":87,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":89,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,113,118,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":45,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":111,"slug":112},50,"technology",{"id":114,"doc_module":4,"doc_module_name":45,"category_name":115,"show_sort_weight":116,"slug":117},7,"Healthcare",40,"healthcare",{"id":119,"doc_module":4,"doc_module_name":45,"category_name":120,"show_sort_weight":121,"slug":122},8,"Research & Report",30,"research-report",{"id":124,"doc_module":4,"doc_module_name":45,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":45,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":45,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":45,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]