[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-81686-en":3,"doc-seo-81686-105":30,"detail-sidebar-cat-0-en-105":92},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":21,"is_downloadable":21,"audit_status":21,"page_count":22,"language":23,"language_code":24,"site_id":25,"html_lang":24,"table_of_contents":26,"faqs":27,"seo_title":13,"seo_description":14,"update_tm":28,"read_time":29},81686,1649267921044,"Ava Thompson","https://us-avatar.wpscdn.com/avatar/1800007509477c92dfb?_k=1782875107921204101",8,"Research & Report","The Serialized Bridge: Understanding and Recovering LLM Serving Performance under Blackwell GPU Confidential Computing","GPU Confidential Computing (GPU-CC) preserves GPU-local compute performance, but LLM serving in confidential environments still suffers significant end-to-end slowdowns. On Blackwell platforms (RTX Pro 6000 and B300 HGX), dense decode throughput drops 13–27%, KV-cache restore latency can more than double, and model loading becomes dramatically slower. The dominant cause is the confidential VM–GPU bridge: transfers become serialized, asynchronous behavior is revoked at the runtime boundary, and each crossing includes substantial fixed setup cost. Mitigations include scheduling adjustments, worker-thread drainage, and a CC-aware model loader, plus analysis of confidential multi-GPU NVSwitch tenancy limits.","The Serialized Bridge: Understanding and Recovering LLM Serving Performance under Blackwell GPU Confidential Computing  \nHang Yin [hangyin@phala. network](hangyin@phala. network)  \nKevin Wang [kvinwang@phala. network](kvinwang@phala. network)  \narXiv :2606 .23969v2 [ cs .DC] 9 Jul 2026  \nMay 2026  \nAbstract  \nGPU Confidential Computing (GPU-CC) now preserves GPU-local performance: on NVIDIA B300, BF16 matmul runs at 0.998x of non-confidential performance. Yet LLM serving under Intel TDX plus GPU-CC still loses 13-27% of throughput, and KV-cache restore latency can more than double. This paper studies that gap on two Blackwell platforms, RTX Pro 6000 and B300 HGX, and identifies its dominant cause: the confidential VM-GPU bridge, not GPU compute.  \nWe find that GPU-CC turns host/device movement into a serialized, high-setup-cost channel. Secure copies do not gain CUDA-stream concurrency within a context, asynchronous transfers block at the runtime boundary, and small crossings pay a fixed toll. This violates the assumptions of modern inference runtimes, where DMA is expected to be cheap, concurrent, and asynchronous. In vLLM dense decode, the gap closes around 44xslower small alloc-and-copy operations; targeted patches reject alternative explanations. A scheduling flag recovers 57% of the gap, while a worker-thread drain recovers up to 92% in qualified high-concurrency runs. The same bridge model explains a +131% KV-restore penalty and a 34x model-load slowdown.  \nBlackwell also changes the confidential tenancy unit. We qualify confidential multi-GPU NVSwitch tenants on B300, including 510 GB/s NVLink P2P inside a CVM and concurrent isolated tenants, and identify the remaining fabric-attestation gap for production confidential AI platforms.  \n1 Introduction  \nThe question that launched GPU confidential computing benchmarking- \"how much slower is the GPU in confidential mode?\" -now has a settled answer: barely. Hopper-era studies found GPU-local LLM compute within a few percent of native [Zhu et al. , 2024 , Martínez Ibarra et al. , 2025], and our Blackwell measurements agree to three decimal places: BF16 matmul on B300 is 0.998x under CC, and a CUDA graph chaining 96,000 matmuls back-to-back is 1.0012x. The GPU, its tensor cores, and its HBM are not where confidential computing costs anything.  \nAnd yet the end-to-end numbers are not settled at all. On the same B300, dense LLM decode loses 13-22% under CC depending on configuration, mixture-of-experts serving loses up to 25%, KV-cache restore latency rises 131%, and loading GPT-OSS-120B takes 287 seconds with the default loader-on hardware that can move the same bytes in well under a minute. Worse, the relative penalty is growing across hardware generations: the same vLLM stack that loses about 10% on an RTX Pro 6000 loses about 26% on the much faster B300, because everything the GPU accelerates shrinks except the confidential data path. An H200 boundary experiment separates mechanism from consequence: the secure-copy floor is already present on Hopper, but Blackwell is where the relative tax becomes large enough to flip runtime policy.  \nThis paper locates all of these losses in one place and explains them with one mechanism: under GPU-CC, the bridge between the confidential VM and the GPU is serialized. Three measured properties define the behavior:  \n1. No concurrency within a context. All cross-device traffic in a CUDA context shares a small fixed pool of secure copy channels. Sixteen CUDA streams each issuing 32-byte device-to-host copies achieve approximately zero parallelism under CC (40 to 39 microseconds per copy from one stream to sixteen),  \nversus 24% scaling with CC off. Additional bandwidth is only reachable by adding CUDA contexts: one context sustains 0 .2x of native transfer bandwidth, while 24 contexts recover to 0 .6-0.7x.  \n2. Asynchrony is silently revoked. copy_ ( . . . , non_blocking=True) to pinned memory-the standard PyTorch idiom for overlapping transf","cbCailrBj4OkJaHN","https://ap.wps.com/l/cbCailrBj4OkJaHN","pdf",543004,5,1,17,"English","en",105,"# Abstract\n# Introduction\n## GPU-local compute vs end-to-end penalties\n## Bridge serialization mechanism\n## Transfer concurrency, asynchrony, and fixed tolls\n## Policy inversion and software-visible cause\n## Recovery hierarchy and fixes\n## Blackwell confidential tenancy changes","[{\"question\":\"What performance penalty remains when using GPU-CC for LLM serving on Blackwell GPUs?\",\"answer\":\"Even though GPU-local compute is nearly unchanged, end-to-end LLM serving can lose 13–27% throughput for dense decode, KV-cache restore latency can more than double, and model loading can take far longer under confidential mode.\"},{\"question\":\"What is the paper’s main cause of the slowdown under GPU-CC?\",\"answer\":\"The confidential VM–GPU bridge is modeled as serialized. The bridge prevents effective concurrency within a context, disables the expected overlap from non-blocking transfers, and charges a large fixed setup cost per crossing.\"},{\"question\":\"What solutions does the paper propose to recover LLM serving performance under GPU-CC?\",\"answer\":\"A scheduling flag (–no-async-scheduling) recovers about 57% of the dense-decode gap and essentially all of the KV-restore gap. A worker-thread drain patch recovers up to 92% in high-concurrency runs, and a CC-aware model loader greatly reduces large-model load time.\"}]",1784175416,43,{"code":4,"msg":31,"data":32},"ok",{"site_id":25,"language":24,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":87,"head_meta":89,"extra_data":91,"updated_unix":28},"the-serialized-bridge-understanding-and-recovering-llm-serving-performance-under-blackwell-gpu-confidential-computing","",{"@graph":36,"@context":86},[37,54,69],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":21},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/the-serialized-bridge-understanding-and-recovering-llm-serving-performance-under-blackwell-gpu-confidential-computing/81686/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":24,"description":14,"dateModified":62,"datePublished":63,"encodingFormat":61,"isAccessibleForFree":64,"interactionStatistic":65},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-07-24","2026-07-16",true,{"@type":66,"interactionType":67,"userInteractionCount":20},"InteractionCounter",{"@type":68},"ViewAction",{"@type":70,"mainEntity":71},"FAQPage",[72,78,82],{"name":73,"@type":74,"acceptedAnswer":75},"What performance penalty remains when using GPU-CC for LLM serving on Blackwell GPUs?","Question",{"text":76,"@type":77},"Even though GPU-local compute is nearly unchanged, end-to-end LLM serving can lose 13–27% throughput for dense decode, KV-cache restore latency can more than double, and model loading can take far longer under confidential mode.","Answer",{"name":79,"@type":74,"acceptedAnswer":80},"What is the paper’s main cause of the slowdown under GPU-CC?",{"text":81,"@type":77},"The confidential VM–GPU bridge is modeled as serialized. The bridge prevents effective concurrency within a context, disables the expected overlap from non-blocking transfers, and charges a large fixed setup cost per crossing.",{"name":83,"@type":74,"acceptedAnswer":84},"What solutions does the paper propose to recover LLM serving performance under GPU-CC?",{"text":85,"@type":77},"A scheduling flag (–no-async-scheduling) recovers about 57% of the dense-decode gap and essentially all of the KV-restore gap. A worker-thread drain patch recovers up to 92% in high-concurrency runs, and a CC-aware model loader greatly reduces large-model load time.","https://schema.org",{"og:url":52,"og:type":88,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":90,"canonical":52},"index,follow",{"doc_id":7,"site_id":25},{"code":4,"msg":5,"data":93},[94,98,102,106,110,115,120,123,128,131,135],{"id":21,"doc_module":4,"doc_module_name":46,"category_name":95,"show_sort_weight":96,"slug":97},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":99,"show_sort_weight":100,"slug":101},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":103,"show_sort_weight":104,"slug":105},"Exam",70,"exam",{"id":20,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":46,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":46,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":46,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":46,"category_name":137,"show_sort_weight":20,"slug":138},19,"General","general"]