[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-83460-en":3,"doc-seo-83460-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":21,"is_downloadable":21,"audit_status":21,"page_count":22,"language":23,"language_code":24,"site_id":25,"html_lang":24,"table_of_contents":26,"faqs":27,"seo_title":13,"seo_description":14,"update_tm":28,"read_time":29},83460,1099513958762,"Logic","https://ap-avatar.wpscdn.com/avatar/1000023916a998db790?x-image-process=image/resize,m_fixed,w_180,h_180&k=1784791008015729253",8,"Research & Report","SLIM-RL: Risk-Budgeted Random-Masking RL for Diffusion LLMs Without Trajectory Slicing","Reinforcement learning for diffusion large language models (dLLMs) has shifted toward trajectory-aware training, where random masking is regarded as mismatched with inference-time decoding. TraceRL reconstructs trajectories by slicing each rollout into trajectory-aligned samples, but slicing cost grows with block size. SLIM-RL mitigates this mismatch without trajectory reconstruction by bounding commit risk per step via a τ-budget decoder, enabling a trace-free random-masking objective with variance reduction. Experiments on SDAR-4B match or improve accuracy while using fewer samples, and transfer training-free across multiple dLLM families.","SLIM-RL: Risk-Budgeted Random-Masking RL for Diffusion LLMs Without Trajectory Slicing  \nRuikang Zhao ∗  \nTechnical University of Denmark  \nZhenting Wang  \nMBZUAI Institute of Foundation Models  \nHan Gao  \nIowa State University  \n*  \nLigong Han  \nRed Hat AI Innovation & MIT–IBM Watson AI Lab  \narXiv :2607 .00208v1 [ cs .CL] 30 Jun 2026  \nAbstract  \nReinforcement learning for diffusion large language models (dLLMs) has largely moved to trajectory-aware methods. The current state of the art, TraceRL, holds that random masking is mismatched with the model’s inference trajectory, and it reconstructs that trajectory during training by slicing each rollout into up to ⌈K/s⌉ trajectory-aligned training samples, a cost that grows with the block size K. We show that this mismatch can be mitigated without reconstructing the trajectory. Our method, SLIM-RL, bounds the commit risk of each rollout step with a τ-budget decoder, reducing aggregate commit risk in the training data. During optimization, SLIM-RL trains on these risk-controlled rollouts with a trace-free random-masking objective that adapts variance-reduction tools, combining sequence-level importance sampling, deterministic quadrature over masking levels under a mean-preserving, monotonically decreasing per-block mask schedule that we introduce. On SDAR-4B, SLIM-RL matches TraceRL’s best MATH500 accuracy on only 0.46× its training samples at block size 16, improving over TraceRL by 6.32% on MATH500 and 11.05% on GSM8K under matched dynamic sampling. At block size 4, the 4B SLIM-RL surpasses the larger LLaDA-8B and Dream-7B dLLMs on math, exceeding LLaDA-8Bby 10.76% on MATH500 while staying below the autoregressive Qwen2.5-7B. On code, it improves over TraceRL by 4.20% on MBPP and 3.65% on HumanEval. The τ-budget decoder transfers training-free across LLaDA, Dream, and SDAR.  \nThe source code is available at [https://github.com/laolaorkkkkk/SLIM-RL](https://github.com/laolaorkkkkk/SLIM-RL)  \n.  \n1 Introduction  \nDiffusion Large Language Models (dLLMs) (Nie et al., 2026; Sahoo et al., 2024; Ye et al., 2025) generate text by iteratively denoising a masked sequence, refining many positions at once instead of strictly left to right. Block-wise variants (Arriola et al., 2025; Cheng et al., 2026) decode consecutive blocks autoregressively while denoising within each block, which restores KV-cache reuse. Reinforcement learning has become the standard way to improve their reasoning (Zhao et al., 2026; Wang et al., 2025b; Hu et al., 2026; He et al., 2025; Zhu et al., 2025b; Tang et al., 2026; Liu et al., 2026; Zhu et al., 2025a) .  \nEarly work applies RL to randomly masked targets (Zhao et al., 2026; Yang et al., 2026), but random masking yields high-variance gradient estimates (Zhu et al., 2025a), and recent work has therefore moved toward trajectory-aware methods (Huang et al., 2026; Wang et al., 2026) . The current state of the art, TraceRL (Wang et al., 2025b), trains not on random masks but on the model’s exact decoding trajectory, on the view that under random masking the post-training objective is mismatched with the trajectory the model follows at inference.  \n∗ Correspondence to: Ruikang Zhao ([ruikangzhao@gmail.com](ruikangzhao@gmail.com)) and Ligong Han ([hanligong@gmail.com](hanligong@gmail.com)).  \nPreserving the trajectory is not free. To train on each decoding step in order, TraceRL slices a single rollout into up to ⌈K/s⌉ trajectory-aligned training samples, one forward each, where K is the block size and s≥1 a shrinkage factor (Wang et al., 2025b) . At full fidelity (s=1) one rollout becomes up to K samples, so the data grows with K. Raising the shrinkage s bounds the cost but aggregates s consecutive decoding steps into one slice and discards their internal order, trading trajectory fidelity for cost. A larger block therefore makes a trade-off between slicing cost and trajectory fidelity, which we quantify in Section 4.5 .  \nThe question is whether reconstructing the exa","cbCaigO8cVVkj8VJ","https://ap.wps.com/l/cbCaigO8cVVkj8VJ","pdf",783936,3,1,17,"English","en",105,"# Introduction\n## Background: trajectory-aware RL and TraceRL\n## Problem: slicing cost and rollout weaknesses\n## Proposed method: SLIM-RL components\n## Contributions and results","[{\"question\":\"What problem does SLIM-RL address in diffusion LLM reinforcement learning training?\",\"answer\":\"SLIM-RL addresses the mismatch introduced by using random masking with inference trajectories, as well as two weaknesses at rollout generation and optimization when trajectories are not reconstructed via slicing.\"},{\"question\":\"How does SLIM-RL control rollout commitment risk without trajectory slicing?\",\"answer\":\"It uses a τ-budget dynamic-unmasking decoder to bound each rollout step’s cumulative confidence-based uncertainty, reducing over-commit errors during rollout generation.\"},{\"question\":\"What performance advantages does SLIM-RL show compared with TraceRL?\",\"answer\":\"On SDAR-4B, SLIM-RL matches TraceRL’s best MATH500 accuracy using only 0.46× the training samples at block size 16, and improves over TraceRL on MATH500 and GSM8K under matched dynamic sampling; it also improves code benchmarks like MBPP and HumanEval.\"}]",1784188118,43,{"code":4,"msg":31,"data":32},"ok",{"site_id":25,"language":24,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"slim-rl-risk-budgeted-random-masking-rl-for-diffusion-llms-without-trajectory-slicing","",{"@graph":36,"@context":85},[37,53,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,50],{"item":41,"name":42,"@type":43,"position":21},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":20},"https://docshare.wps.com/document/research-report/",{"item":51,"name":13,"@type":43,"position":52},"https://docshare.wps.com/document/slim-rl-risk-budgeted-random-masking-rl-for-diffusion-llms-without-trajectory-slicing/83460/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":24,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":41,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-24","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What problem does SLIM-RL address in diffusion LLM reinforcement learning training?","Question",{"text":75,"@type":76},"SLIM-RL addresses the mismatch introduced by using random masking with inference trajectories, as well as two weaknesses at rollout generation and optimization when trajectories are not reconstructed via slicing.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"How does SLIM-RL control rollout commitment risk without trajectory slicing?",{"text":80,"@type":76},"It uses a τ-budget dynamic-unmasking decoder to bound each rollout step’s cumulative confidence-based uncertainty, reducing over-commit errors during rollout generation.",{"name":82,"@type":73,"acceptedAnswer":83},"What performance advantages does SLIM-RL show compared with TraceRL?",{"text":84,"@type":76},"On SDAR-4B, SLIM-RL matches TraceRL’s best MATH500 accuracy using only 0.46× the training samples at block size 16, and improves over TraceRL on MATH500 and GSM8K under matched dynamic sampling; it also improves code benchmarks like MBPP and HumanEval.","https://schema.org",{"og:url":51,"og:type":87,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":89,"canonical":51},"index,follow",{"doc_id":7,"site_id":25},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":21,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":46,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":46,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":46,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":46,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]