[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-83010-en":3,"doc-seo-83010-105":29,"detail-sidebar-cat-0-en-105":82},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},83010,7971461740886,"Theodore","https://ap-avatar.wpscdn.com/davatar_3d24733baf745e90a7e4bdd5f77d97b2",8,"Research & Report","Think Before You Grid-Search Floor-First Triage for LLM Serving","Production teams optimizing large language model (LLM) serving face a configuration space spanning parallelism layout, batching, quantization, sparse attention, and kernel-level decisions. Conventional practice benchmarks many configurations and opens heavy profilers once a latency target is missed. The paper proposes a residual-driven workflow: build an analytical floor first, reconcile measurements against that floor, and escalate to profiling only when the residual demands it. Floor First estimates profiling value analytically, preventing grid-search degeneration and enabling general triage across autoregressive transformer deployments.","arXiv :2607 .05876v2 [ cs .PF] 8 Jul 2026  \nThink Before You Grid-Search: Floor-First Triage for LLM Serving  \nYihua Liu  \nTaikang Insurance Group  \n[liuyihua1994@gmail.com](liuyihua1994@gmail.com)  \nJuly 2026  \nAbstract  \nProduction teams optimizing large language model (LLM) serving face a configuration space spanning parallelism layout, batching, quantization, sparse attention, and kernel-level work. The common operational response is to benchmark many configurations and open heavy profilers whenever a latency target is missed. This paper argues for a different workflow: build an analytical floor first, reconcile benchmarks against that floor, and escalate to profiling only when the residual justifies it. Estimation is the analytical layer of profiling: without it, optimization degenerates to grid search.  \nWe present Floor First, a residual-driven triage methodology with a zero-dependency artifact: a floor calculator plus an agent skill that makes the discipline enforceable in agentic optimization loops. The account is compositional—new attention or state-space variants enter by declaring one module, not by rewriting the framework—so the workflow applies to any autoregressive transformer served on accelerators. Floor First models each decode step as a five-dimensional resource vector (HBM bytes, FLOPs, network bytes, network messages, KV capacity) . Terms that use the same resource add; independent resources can overlap. This gives two numbers: an optimistic floor, max, and a no-overlap floor, sum. Where a measurement falls inside this [max, sum] interval is already a diagnostic: it tells us how much overlap the system is getting before any profiler is opened. Deployment alternatives are then compared by wall ordering—which resource wall binds first as load grows—rather than by point benchmarks.  \nAs a case study, we analyze a DeepSeek-V3.2-style 671B MoE/MLA model on 16 NVIDIA H20 GPUs, a hardware point whose ridge point of ∼74 FLOP/byte (versus ∼590 for H100) makes it an extreme decode-oriented part that no published analysis characterizes. The floors show that TP16 decoding at batch 64 and 8K context is KV-capacity-limited to ∼70 concurrent requests, that DSA-style sparse attention removes the KV-bandwidth term but not the capacity wall, and that an EP16+DP-attention layout trades slightly worse same-batch weight traffic for an order-of-magnitude higher capacity wall (∼644 requests)—while, at cluster-calibrated communication constants, single-stream latency favors TP by 2 .4 × . The judgment between layouts is thus a computable function of the operating point, which explains why production deployments on identical hardware have shipped opposite attention layouts.  \n1 Introduction  \nIn 2025, production teams serving DeepSeek-family models made opposite parallelism choices on comparable hardware. The official DeepSeek deployment decodes with large-scale expert parallelism and data-parallel attention [11]; vLLM’s default DeepSeek recipe now enables the same layout [36]; and a joint LMSYS– Ant Group deployment on the same 16×H20-96G configuration we study reached the same conclusion for decode [24] . Yet TP-sharded attention remains the serving frameworks’ historical default, and the production 2×8 H20 cluster whose deployment this paper models serves DeepSeek-V3.2 with plain TP16 attention. The Ant report states the qualitative rationale—H20 is memory-rich and compute-poor, so decode should exploit bandwidth—but none of these reports derives the choice from an explicit resource account, and none can say at what concurrency the answer flips. This paper’s position is that the disagreement was predictable from a small resource account—roughly five numbers per GPU plus the model dimensions—and that making this account systematic gives a general triage workflow for serving optimization.  \nThe operational problem is broader than one layout choice. Serving optimization spans model variants, hardware SKUs, tensor parallelism (TP),","cbCaicjkk46smyiP","https://ap.wps.com/l/cbCaicjkk46smyiP","pdf",411147,1,16,"English","en",105,"# Abstract\n# 1 Introduction\n## Serving-optimization loop\n## Problem with benchmarking-only workflows","[{\"question\":\"How is the parallelism layout comparison made in this workflow?\",\"answer\":\"Instead of relying on single point benchmarks, it compares deployment alternatives by wall ordering as load grows—identifying which resource wall binds first—so the choice can be explained as a computable function of the operating point.\"}]",1784184648,40,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":77,"head_meta":79,"extra_data":81,"updated_unix":27},"think-before-you-grid-search-floor-first-triage-for-llm-serving","",{"@graph":35,"@context":76},[36,53,67],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/think-before-you-grid-search-floor-first-triage-for-llm-serving/83010/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":61,"encodingFormat":60,"isAccessibleForFree":62,"interactionStatistic":63},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-16",true,{"@type":64,"interactionType":65,"userInteractionCount":4},"InteractionCounter",{"@type":66},"ViewAction",{"@type":68,"mainEntity":69},"FAQPage",[70],{"name":71,"@type":72,"acceptedAnswer":73},"How is the parallelism layout comparison made in this workflow?","Question",{"text":74,"@type":75},"Instead of relying on single point benchmarks, it compares deployment alternatives by wall ordering as load grows—identifying which resource wall binds first—so the choice can be explained as a computable function of the operating point.","Answer","https://schema.org",{"og:url":51,"og:type":78,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":80,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":83},[84,88,92,96,101,106,110,113,118,121,125],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":85,"show_sort_weight":86,"slug":87},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":89,"show_sort_weight":90,"slug":91},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":93,"show_sort_weight":94,"slug":95},"Exam",70,"exam",{"id":97,"doc_module":4,"doc_module_name":45,"category_name":98,"show_sort_weight":99,"slug":100},5,"Comic",60,"comic",{"id":102,"doc_module":4,"doc_module_name":45,"category_name":103,"show_sort_weight":104,"slug":105},6,"Technology",50,"technology",{"id":107,"doc_module":4,"doc_module_name":45,"category_name":108,"show_sort_weight":28,"slug":109},7,"Healthcare","healthcare",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":111,"slug":112},30,"research-report",{"id":114,"doc_module":4,"doc_module_name":45,"category_name":115,"show_sort_weight":116,"slug":117},9,"Religion & Spirituality",20,"religion-spirituality",{"id":116,"doc_module":4,"doc_module_name":45,"category_name":119,"show_sort_weight":116,"slug":120},"World Cup","world-cup",{"id":122,"doc_module":4,"doc_module_name":45,"category_name":123,"show_sort_weight":122,"slug":124},10,"Lifestyle","lifestyle",{"id":126,"doc_module":4,"doc_module_name":45,"category_name":127,"show_sort_weight":97,"slug":128},19,"General","general"]