[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-81499-en":3,"doc-seo-81499-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":21,"is_downloadable":21,"audit_status":21,"page_count":22,"language":23,"language_code":24,"site_id":25,"html_lang":24,"table_of_contents":26,"faqs":27,"seo_title":13,"seo_description":14,"update_tm":28,"read_time":29},81499,1099513958607,"Jiven","https://ap-avatar.wpscdn.com/avatar/100002390cf8733938c?x-image-process=image/resize,m_fixed,w_180,h_180&k=1778829742770036399",8,"Research & Report","Accelerating Large Language Model Inference with Dynamic Self-Speculative Decoding","The document presents Dynamic Self-Speculative Decoding (DSSD), a plug-and-play approach to accelerate large language model (LLM) inference while minimizing retraining. It introduces early-exit “heads” placed across transformer layers, which terminate computation once calibrated confidence thresholds are met for each token. A modular framework and experimental study evaluate training strategies and confidence metrics across model sizes from 70M to 2.8B on Pythia. DSSD integrates early exit with speculative decoding, achieving 1.66× higher token acceptance rates versus layer-skip baselines with minimal hyperparameter tuning.","Accelerating Large Language Model Inference with Dynamic Self-Speculative Decoding  \nFlorian Valade  \nUniversit´e Gustave Eiffel Fujitsu  \narXiv :2407 .2 1082v 3 [ cs .CL] 10 Jul 2026  \n1 Introduction  \nLarge language models (LLMs) have become central to advancing capabilities in natural language processing (NLP), delivering remarkable performance across a range of tasks. The trend towards scaling up these models correlates strongly with improved performance, understanding, and generality. This relationship has been formalized through empirical scaling laws, which demonstrate that model performance improves predictably with increased model size, datasetsize, and compute budget Kaplan et al. (2020); Hoffmann et al. (2022) . However, the computational cost associated with these larger models is substantial, often necessitating the use of powerful server infrastructure Samsi et al. (2023) . This not only limits local usability but also raises significant privacy concerns and requires considerable investment to scale in response to user demand. Solutions exist to reduce the computational demands of these models, but they often impact the model’s performance by reducing its accuracy Zhu et al. (2023) .  \nDespite their effectiveness, these models often operate inefficiently. The nature of language itself contributes to this inefficiency; namely, not all tokens generated during the inference process contribute equally to the overall meaning or require the same level of computational resources. Some tokens are inherently simpler and can be predicted with high confidence early in the computation process, while others, contributing more significantly to the context or meaning, may require deeper processing.  \nIn response to these challenges, we develop a method that can be easily integrated into existing pre-trained models to enhance their inference speed without extensive retraining. Our solution focuses on the strategic placement of early exit ”heads” Scardapane et al.(2020); Teerapittayanon et al. (2017) within the transformer layers of an LLM. These heads terminate the inference process when a calibrated confidence threshold is met, based on the complexity and predictability  \nof the token being processed.  \nOur contributions are twofold:  \n1. We provide a detailed experimental study and modular framework for training and deploying early exit heads on top of LLMs. We analyze multiple training strategies, confidence metrics, and demonstrate scalability across model sizes from 70M to 2.8B parameters on the Pythia suite.  \n2. We adapt our early exit mechanism to speculative decoding, introducing Dynamic SelfSpeculative Decoding (DSSD), which achieves 1.66 × higher token acceptance rates than manually-tuned LayerSkip baselines while requiring minimal hyperparameter tuning—only a single accuracy threshold ϵ .  \n2 Related Work  \nTransformers Vaswani et al. (2017) scaled into today’s LLM families such as BERT, GPT-3, PaLM, LaMDA, LLaMA and OPT Devlin et al. (2019); Brown et al. (2020); Chowdhery et al. (2022); Thoppilan et al.(2022); Touvron et al. (2023); Zhang et al. (2022), powering vision Dosovitskiy et al. (2020), speech Radford et al. (2023) and multimodal models. Their billionparameter footprints, however, make every token generation costly. Static compression—quantisation, pruning and distillation Shen et al. (2019); Sun et al.(2020); Yao et al. (2022); Fan et al. (2019); Sun et al.(2019); Bai et al. (2021)—slashes model size but still expends identical compute on easy and hard inputs. Early-exit methods address this imbalance by attaching lightweight classifiers to intermediate layers and halting computation once a confidence criterion is met.  \nIn computer vision, BranchyNet Teerapittayanon et al.(2017) and MSDNet Huang et al. (2018) or EERO methodology Valade et al. (2024) established the two key components still used today: deeply supervised branches and an entropy-based exit rule. Transferring  \nthe idea to Transformers, DeeBE","cbCaicYLvzhgfEwK","https://ap.wps.com/l/cbCaicYLvzhgfEwK","pdf",1298462,3,1,18,"English","en",105,"# Introduction\n# Related Work\n# Methodology","[{\"question\":\"What problem does the method target in large language model inference?\",\"answer\":\"Inference is computationally expensive, especially for larger models, and tokens vary in predictability and contribution to meaning. The method addresses inefficiency by exiting earlier when token confidence is high enough.\"},{\"question\":\"How do early-exit heads improve inference speed?\",\"answer\":\"The approach places lightweight heads on transformer layers and stops inference when a calibrated confidence threshold is reached for the current token.\"},{\"question\":\"What is DSSD and how does it differ from prior LayerSkip approaches?\",\"answer\":\"DSSD adapts early exit to speculative decoding and selects exit layers dynamically using calibrated confidence. It avoids the exhaustive hyperparameter search required by manually tuned LayerSkip.\"}]",1784173827,45,{"code":4,"msg":31,"data":32},"ok",{"site_id":25,"language":24,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"accelerating-large-language-model-inference-with-dynamic-self-speculative-decoding","",{"@graph":36,"@context":85},[37,53,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,50],{"item":41,"name":42,"@type":43,"position":21},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":20},"https://docshare.wps.com/document/research-report/",{"item":51,"name":13,"@type":43,"position":52},"https://docshare.wps.com/document/accelerating-large-language-model-inference-with-dynamic-self-speculative-decoding/81499/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":24,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":41,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-20","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What problem does the method target in large language model inference?","Question",{"text":75,"@type":76},"Inference is computationally expensive, especially for larger models, and tokens vary in predictability and contribution to meaning. The method addresses inefficiency by exiting earlier when token confidence is high enough.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"How do early-exit heads improve inference speed?",{"text":80,"@type":76},"The approach places lightweight heads on transformer layers and stops inference when a calibrated confidence threshold is reached for the current token.",{"name":82,"@type":73,"acceptedAnswer":83},"What is DSSD and how does it differ from prior LayerSkip approaches?",{"text":84,"@type":76},"DSSD adapts early exit to speculative decoding and selects exit layers dynamically using calibrated confidence. It avoids the exhaustive hyperparameter search required by manually tuned LayerSkip.","https://schema.org",{"og:url":51,"og:type":87,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":89,"canonical":51},"index,follow",{"doc_id":7,"site_id":25},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":21,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":46,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":46,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":46,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":46,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]