[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-82033-en":3,"doc-seo-82033-105":31,"detail-sidebar-cat-0-en-105":93},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":21,"is_downloadable":21,"audit_status":21,"page_count":22,"language":23,"language_code":24,"site_id":25,"html_lang":24,"table_of_contents":26,"faqs":27,"seo_title":28,"seo_description":14,"update_tm":29,"read_time":30},82033,7971461740886,"Theodore","https://ap-avatar.wpscdn.com/davatar_3d24733baf745e90a7e4bdd5f77d97b2",8,"Research & Report","DominoTree Conditional Tree-Structured Drafting with Domino for Speculative Decoding","Speculative decoding speeds up large language model inference by drafting multiple tokens and verifying them in parallel, improving over autoregressive decoding’s sequential cost while addressing block-diffusion drafters’ marginal-only limitation. DominoTree introduces a training-free best-first draft tree that scores candidates using Domino’s GRU-based conditional correction along each root-to-node path, implemented via a per-node top-M restriction. Across Qwen3-4B benchmarks, DominoTree achieves up to 6.6× speedup and the highest mean accept length (up to 10.7 tokens/round) while using a GPU-native CUDA-graph tree builder for cheap, unchanged acceptance.","arXiv :2607 .08642v2 [ cs .CL] 10 Jul 2026  \nDominoTree: Conditional Tree-Structured Drafting with Domino  \nfor Speculative Decoding ∗ Saw S. Lin (Zhiqi Zhang) Jyh-Shing Roger Jang  \nAbstract  \nSpeculative decoding accelerates LLM inference by drafting several tokens and verifying them in parallel. Block-diffusion drafters such as DFlash produce a draft block in one pass but model only per-position marginals; best-first tree methods such as DDTree expand candidate trees from those marginals. The released Domino drafter adds a GRU-based causal correction that makes each draft token’s distribution path-dependent, a structure DDTree’s factorized formulation cannot represent. We introduce DominoTree, a training-free best-first draft tree scored by Domino’s conditional (non-factorized) correction along each root-to-node path, made practical by restricting the per-node correction to a candidate top-M. On Qwen3-4B across eight benchmarks, DominoTree reaches up to 6.6 × speedup over autoregressive decoding and the highest mean accept length of any evaluated method (up to 10.7 tokens per round) at every temperature we test. DominoTree constructs its tree with a GPU-native, CUDA-graph builder  \n—bit-identical to a reference Python implementation, so acceptance is unchanged—that keeps per-round tree construction cheap. With this builder as default, DominoTree wins throughput over the released Domino decoder—the very drafter it is built on, run at its own CUDA-graph best—at every temperature (9–10% overall on Qwen3-4B, up to +22% on Alpaca), and over DDTree/CaDDTree at every temperature we test (CI-clean paired-bootstrap improvements), not only at greedy decoding. On Qwen3-8B, DominoTree keeps the highest accepted length at every temperature and adds a decisive throughput win at T=0 (+24% over DDTree); at higher temperature that edge over DDTree/CaDDTree narrows to a tie and a small loss, while its Overall (aggregate) wins over DFlash and Domino persist.  \n1 Introduction  \nEvery speculative decoding method lives inside the same efficiency identity: end-to-end speedup over autoregressive (AR) decoding is proportional to accepted tokens per round (τ) divided by the wall-clock cost of drafting and verifying that round ((1), Section 2.1) . Draft quality raises τ; draft cost lowers the denominator — and the two trade off. Autoregressive drafters (e.g. the EAGLE line (Li et al. , 2024a,b, 2025)) sit at one extreme: each drafted token conditions on every token drafted before it, but that costs γ strictly sequential forward passes for a length-γ draft. Blockdiffusion drafters such as DFlash (Chen et al. , 2026) sit at the other extreme: an entire block is proposed in a single parallel forward pass, but every position’s logits are then a marginal over tokens that will actually be realized elsewhere in the block, not a conditional on them, which caps how high τ can climb (Section 2.2) .  \nDomino (Huang et al. , 2026) closes much of this gap without paying the autoregressive drafter’s full sequential cost: it keeps DFlash’s parallel backbone unchanged and adds a lightweight sequential correction—a GRU tracks which tokens have actually been sampled so far along the current draft  \n∗ Code: [https://github.com/slin-zhq/Domino-Tree](https://github.com/slin-zhq/Domino-Tree)  \nDepartment of Computer Science and Information Engineering, National Taiwan University. Correspondence to: Saw S. Lin (Zhiqi Zhang) \u003C[r13922176@ntu.edu.tw](r13922176@ntu.edu.tw) >.  \nDFlash DDTree(16) CaDDTree Domino DominoTree(16)  \n(b)  \n(a)  \nMean accept length τ  \n12  \n10  \n8  \n6  \n4  \n2  \n0  \nGSM8KMATH-500  \nAIME25  \nHumanEval  \nMBPP  \nLCB  \nMT-Bench Alpaca  \nSpeedup over AR (x)  \n8  \n7  \n6  \n5  \n4  \n3  \n2  \n1  \n0  \nGSM8KMATH-500  \nAIME25  \nHumanEval  \nMBPP  \nLCB  \nMT-Bench Alpaca  \nFigure 1 . DominoTree vs. DFlash, DDTree, CaDDTree, and Domino, Qwen3-4B, T=0, across the full eight-dataset grid (Table 1) . Left: mean accept length τ . Right: speedup over autoregressive (AR) dec","cbCaibMmOijUGCE2","https://ap.wps.com/l/cbCaibMmOijUGCE2","pdf",865780,5,1,23,"English","en",105,"# Abstract\n# Introduction","[{\"question\":\"What problem does speculative decoding address in LLM inference?\",\"answer\":\"It accelerates inference by drafting several tokens and verifying them in parallel, aiming to increase accepted tokens per round while balancing drafting and verification cost.\"},{\"question\":\"How does DominoTree differ from factorized tree methods like DDTree and CaDDTree?\",\"answer\":\"DominoTree uses Domino’s GRU-based causal correction, which makes draft token distributions path-dependent; it evaluates candidates using conditional scoring along each root-to-node path rather than assuming factorized marginals.\"},{\"question\":\"How is high throughput achieved in DominoTree during decoding?\",\"answer\":\"DominoTree constructs the draft tree with a GPU-native CUDA-graph builder that is bit-identical to a reference implementation, keeping per-round tree construction cheap and maintaining acceptance behavior.\"}]","DominoTree Conditional Tree-Structured Drafting with Domino for Speculative Decoding | PDF",1784177704,58,{"code":4,"msg":32,"data":33},"ok",{"site_id":25,"language":24,"slug":34,"title":13,"keywords":35,"description":14,"schema_data":36,"social_meta":88,"head_meta":90,"extra_data":92,"updated_unix":29},"dominotree-conditional-tree-structured-drafting-with-domino-for-speculative-decoding","",{"@graph":37,"@context":87},[38,55,70],{"@type":39,"itemListElement":40},"BreadcrumbList",[41,45,49,52],{"item":42,"name":43,"@type":44,"position":21},"https://docshare.wps.com","Home","ListItem",{"item":46,"name":47,"@type":44,"position":48},"https://docshare.wps.com/document/","Document",2,{"item":50,"name":12,"@type":44,"position":51},"https://docshare.wps.com/document/research-report/",3,{"item":53,"name":13,"@type":44,"position":54},"https://docshare.wps.com/document/dominotree-conditional-tree-structured-drafting-with-domino-for-speculative-decoding/82033/",4,{"url":53,"name":13,"@type":56,"author":57,"headline":13,"publisher":59,"fileFormat":62,"inLanguage":24,"description":14,"dateModified":63,"datePublished":64,"encodingFormat":62,"isAccessibleForFree":65,"interactionStatistic":66},"DigitalDocument",{"name":9,"@type":58},"Person",{"url":42,"name":60,"@type":61},"DocShare","Organization","application/pdf","2026-07-29","2026-07-16",true,{"@type":67,"interactionType":68,"userInteractionCount":20},"InteractionCounter",{"@type":69},"ViewAction",{"@type":71,"mainEntity":72},"FAQPage",[73,79,83],{"name":74,"@type":75,"acceptedAnswer":76},"What problem does speculative decoding address in LLM inference?","Question",{"text":77,"@type":78},"It accelerates inference by drafting several tokens and verifying them in parallel, aiming to increase accepted tokens per round while balancing drafting and verification cost.","Answer",{"name":80,"@type":75,"acceptedAnswer":81},"How does DominoTree differ from factorized tree methods like DDTree and CaDDTree?",{"text":82,"@type":78},"DominoTree uses Domino’s GRU-based causal correction, which makes draft token distributions path-dependent; it evaluates candidates using conditional scoring along each root-to-node path rather than assuming factorized marginals.",{"name":84,"@type":75,"acceptedAnswer":85},"How is high throughput achieved in DominoTree during decoding?",{"text":86,"@type":78},"DominoTree constructs the draft tree with a GPU-native CUDA-graph builder that is bit-identical to a reference implementation, keeping per-round tree construction cheap and maintaining acceptance behavior.","https://schema.org",{"og:url":53,"og:type":89,"og:title":13,"og:site_name":60,"og:description":14},"article",{"robots":91,"canonical":53},"index,follow",{"doc_id":7,"site_id":25},{"code":4,"msg":5,"data":94},[95,99,103,107,111,116,121,124,129,132,136],{"id":21,"doc_module":4,"doc_module_name":47,"category_name":96,"show_sort_weight":97,"slug":98},"Story & Novel",90,"story-novel",{"id":48,"doc_module":4,"doc_module_name":47,"category_name":100,"show_sort_weight":101,"slug":102},"Literature",80,"literature",{"id":54,"doc_module":4,"doc_module_name":47,"category_name":104,"show_sort_weight":105,"slug":106},"Exam",70,"exam",{"id":20,"doc_module":4,"doc_module_name":47,"category_name":108,"show_sort_weight":109,"slug":110},"Comic",60,"comic",{"id":112,"doc_module":4,"doc_module_name":47,"category_name":113,"show_sort_weight":114,"slug":115},6,"Technology",50,"technology",{"id":117,"doc_module":4,"doc_module_name":47,"category_name":118,"show_sort_weight":119,"slug":120},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":47,"category_name":12,"show_sort_weight":122,"slug":123},30,"research-report",{"id":125,"doc_module":4,"doc_module_name":47,"category_name":126,"show_sort_weight":127,"slug":128},9,"Religion & Spirituality",20,"religion-spirituality",{"id":127,"doc_module":4,"doc_module_name":47,"category_name":130,"show_sort_weight":127,"slug":131},"World Cup","world-cup",{"id":133,"doc_module":4,"doc_module_name":47,"category_name":134,"show_sort_weight":133,"slug":135},10,"Lifestyle","lifestyle",{"id":137,"doc_module":4,"doc_module_name":47,"category_name":138,"show_sort_weight":20,"slug":139},19,"General","general"]