[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-85996-en":3,"doc-seo-85996-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":21,"is_downloadable":21,"audit_status":21,"page_count":22,"language":23,"language_code":24,"site_id":25,"html_lang":24,"table_of_contents":26,"faqs":27,"seo_title":13,"seo_description":14,"update_tm":28,"read_time":29},85996,1374391975076,"Riley","https://ap-avatar.wpscdn.com/avatar/14000253ca4ec9f6853?x-image-process=image/resize,m_fixed,w_180,h_180&k=1783305029341752051",8,"Research & Report","Unlocking Parallelism in Autoregressive Language Models via Speculative Decoding with Progressive Tree Drafting","Speculative decoding accelerates Large Language Model inference by reducing memory-bound bottlenecks, yet traditional approaches often depend on auxiliary draft modules that add training and communication cost. Even endogenous, training-free methods may underuse the latent parallel capacity due to weak structural coordination. Progressive Tree Drafting (PTD) uses a guided parallel tree with stepwise pruning to explore multiple semantic paths per forward pass, maintaining diversity and coherence. Experiments show up to 2× speedups across benchmarks with no training and model-agnostic design.","arXiv :2607 . 1066 1v 1 [ cs .CL] 12 Jul 2026  \nUnlocking Parallelism in Autoregressive Language Models via Speculative Decoding with Progressive Tree Drafting  \nZipeng Gao1 Zhi Zheng1 ∗ Qingrong Xia2 Junda Lin1 Ziwei Zhao1 Tong Xu1 Zhefeng Wang2 Enhong Chen1  \n1 State Key Laboratory of Cognitive Intelligence, University of Science and Technology of China  \n2 Huawei Technologies Co., Ltd.  \n{gaozp619,zhengzhi97,linjunda,[zzw22222](zzw22222}@mail.ustc.edu.cn)[}](zzw22222}@mail.ustc.edu.cn)[@mail.ustc.edu.cn](zzw22222}@mail.ustc.edu.cn)  \n{zhengzhi97,tongxu,[cheneh](cheneh}@ustc.edu.cn)[}](cheneh}@ustc.edu.cn)[@ustc.edu.cn](cheneh}@ustc.edu.cn)  \n{xiaqingrong,[wangzhefeng](wangzhefeng}@huawei.com)[}](wangzhefeng}@huawei.com)[@huawei.com](wangzhefeng}@huawei.com)  \nAbstract  \nSpeculative decoding has significantly accelerated Large Language Model (LLM) inference by alleviating memory-bound bottlenecks. However, traditional speculative decoding typically relies on auxiliary draft modules, incurring significant training and communication overhead. Although recent methods attempt to generate drafts within the target model itself, they often fail to fully exploit its latent parallel capacity due to a lack of structural coordination. In this paper, we propose Progressive Tree Drafting (PTD), which employs a structured, guided parallel drafting strategy to harness the model’s parallel potential. By coupling a progressive tree structure with a stepwise pruning mechanism, PTD actively guides the LLM to explore multiple semantic paths in a single forward pass, ensuring both draft diversity and coherence. Experiments demonstrate that PTD achieves up to 2 × decoding speedup across various benchmarks while remaining training-free and model-agnostic. Our code is available at:  \n[https://github.com/MINE-USTC/PTD](https://github.com/MINE-USTC/PTD).  \n1 Introduction  \nSpeculative decoding has emerged as an important paradigm for accelerating Large Language Model (LLM) inference, mitigating the memory-bound bottlenecks inherent in autoregressive generation (Yuan et al., 2024; Xia et al., 2024) . By transforming the inefficient token-by-token processing into a candidate parallel verification process, this paradigm enhances computational utility and inference speed without compromising generation quality.  \nThe key to speculative decoding is obtaining high-quality drafts. Traditional approaches typically employ smaller auxiliary modules to generate drafts, which often incur significant communication overhead and require substantial training effort for model alignment (Xia et al., 2023; Leviathan et al., 2023; Chen et al., 2023; Miao et al., 2024; Yang et al., 2024; Cai et al., 2024; Stern et al., 2018; Li et al., 2025) . To mitigate these issues, recent research has shifted toward training-free, model-agnostic strategies that generate drafts directly within the target LLM itself. For instance, Lookahead Decoding (LADE) (Fu et al., 2024) supplements the primary decoding objective with a parallel Jacobi iteration task to predict and refine linear candidate sequences, while Self-Draft (Gao et al., 2025) leverages the intrinsic robustness of LLMs to incorporate an auxiliary drafting task, generating multiple candidate branches through input perturbations.  \n∗ Corresponding author  \nFigure 1: Paradigms of speculative decoding. (Left) Traditional methods using auxiliary draft modules. (Right) Endogenous, training-free methods.  \n…  \nDuplicate  \nD raft branches  \nFigure 2: A drafting example (left) and the branch similarity analysis (right) of SelfDraft (Gao et al., 2025) . The curve means the proportion of steps with at least two branches above the similarity threshold.  \nWhile these methods confirm the feasibility of endogenous acceleration, we argue that the latent parallel processing capabilities inherent in AR models remain under-exploited, primarily due to the independent and unstructured nature of their candidate generation. Specifically, our in-de","cbCaiuoVAxTtQTFJ","https://ap.wps.com/l/cbCaiuoVAxTtQTFJ","pdf",1003091,4,1,19,"English","en",105,"# Introduction\n## Speculative Decoding Background\n## Draft Quality and Redundancy Bottlenecks\n## Progressive Tree Drafting (PTD)\n## Contributions","[{\"question\":\"What problem does Progressive Tree Drafting (PTD) address in speculative decoding?\",\"answer\":\"PTD targets the under-exploitation of autoregressive models’ latent parallel capacity caused by independent, unstructured draft generation, which leads to high semantic redundancy.\"},{\"question\":\"How does PTD generate drafts differently from prior endogenous approaches?\",\"answer\":\"PTD drafts using a structured progressive tree and a stepwise pruning mechanism, merging redundant prefixes while guiding exploration of multiple distinct semantic paths in a single forward pass.\"},{\"question\":\"What performance benefits does PTD report in experiments?\",\"answer\":\"Experiments demonstrate up to 2× decoding speedup across various benchmarks while remaining training-free and model-agnostic, and the method maintains both draft diversity and coherence.\"}]",1784207662,48,{"code":4,"msg":31,"data":32},"ok",{"site_id":25,"language":24,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"unlocking-parallelism-in-autoregressive-language-models-via-speculative-decoding-with-progressive-tree-drafting","",{"@graph":36,"@context":85},[37,53,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":21},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":20},"https://docshare.wps.com/document/unlocking-parallelism-in-autoregressive-language-models-via-speculative-decoding-with-progressive-tree-drafting/85996/",{"url":52,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":24,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":41,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-26","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What problem does Progressive Tree Drafting (PTD) address in speculative decoding?","Question",{"text":75,"@type":76},"PTD targets the under-exploitation of autoregressive models’ latent parallel capacity caused by independent, unstructured draft generation, which leads to high semantic redundancy.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"How does PTD generate drafts differently from prior endogenous approaches?",{"text":80,"@type":76},"PTD drafts using a structured progressive tree and a stepwise pruning mechanism, merging redundant prefixes while guiding exploration of multiple distinct semantic paths in a single forward pass.",{"name":82,"@type":73,"acceptedAnswer":83},"What performance benefits does PTD report in experiments?",{"text":84,"@type":76},"Experiments demonstrate up to 2× decoding speedup across various benchmarks while remaining training-free and model-agnostic, and the method maintains both draft diversity and coherence.","https://schema.org",{"og:url":52,"og:type":87,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":89,"canonical":52},"index,follow",{"doc_id":7,"site_id":25},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":21,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":20,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":46,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":46,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":46,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":22,"doc_module":4,"doc_module_name":46,"category_name":136,"show_sort_weight":106,"slug":137},"General","general"]