[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-84840-en":3,"doc-seo-84840-105":29,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},84840,8796095462418,"Noah","https://ap-avatar.wpscdn.com/avatar/80000253c1241d02b47?x-image-process=image/resize,m_fixed,w_180,h_180&k=1778826106357471780",8,"Research & Report","A Coin Flip Per Token: Bernoulli Sparse Steering of Large Language Models","Activation steering via sparse autoencoders (SAEs) enables behavioral control of large language models without task-specific fine-tuning, but applying the steering signal at every generated token introduces constant per-token perturbation that can harm fluency. This work replaces dense intervention with stochastic gating. Stochastic Token Steering (STS) gates tokens independently with probability p, and Stochastic Block Steering (SBS) gates an initial window once per sequence. Across two model families and two tasks, steering 50% of tokens preserves most of the dense effect and maintains fluency, while steering as few as 30% can outperform prompt-based control. The best magnitude scales inversely with the intervention ratio, showing rate-limited SAE control.","A Coin Flip Per Token: Bernoulli Sparse Steering of Large Language  \nModels  \nNima Eshraghi 1 , Lovedeep Gondara 1 , Yuqing Huang 1 , Sagarika Suresh 1 , Leizer Teran 1 , Jithin Pradeep 1 , Xiaotong (Tone) Xu2 , Fanny Chevalier2  \n1The Vanguard Group, Inc. 2University of Toronto  \n{nima_eshraghi, lovedeep_gondara, yuqing_huang, sagarika_thimmanayakanapalya, leizer_teran, [jithin_pradeep}@vanguard.com](jithin_pradeep}@vanguard.com)  \n{tonexu, [fanny}@cs.toronto.edu](fanny}@cs.toronto.edu)  \narXiv :2607 .056 15v 1 [ cs .LG] 6 Jul 2026  \nAbstract  \nActivation steering via sparse autoencoders (SAEs) enables behavioral control of large language models without task-specific fine-tuning, but standard methods apply the steering signal at every generated token, incurring constant per-token perturbation that risks degrading fluency. We ask: is dense intervention necessary? We introduce Stochastic Token Steering (STS), which gates each token independently with probability p, and Stochastic Block Steering (SBS), which gates a leading window once per sequence; neither requires a reward model or learned gating policy. Across two model families and two behavioral tasks, steering only 50% of the tokens recovers most of the densesteering effect while preserving fluency, and steering as few as 30% surpasses prompt-based control. The optimal steering magnitude scales inversely with the intervention ratio, revealing that SAE-mediated control is rate-limited: the behavioral outcome depends on cumulative signal dosage across a sequence.  \nContent Warning. This paper studies inferencetime control of language model behavior, including the reduction of toxic generation. The appendix contains examples of toxic, offensive, and emotionally distressing model outputs that are shown to illustrate the method. Reader discretion is advised.  \n1 Introduction  \nBehavioral control over large language models (LLMs) reducing toxicity, enforcing a persona, suppressing hallucinations, and adjusting refusal has become a significant deployment concern (Ouyang et al., 2022 ; Bai et al., 2022) . The default mechanism is the system prompt, but prompt-based steering is fragile: lexical perturbations shift output distributions (Mizrahi et al., 2024), alignment priors can be overridden through crafted inputs or adversarial suffixes (Wei et al., 2023 ; Zou et al.,  \n2023b), and steerability through prompting is asymmetric and bounded across many behavioral dimensions (Wolf et al., 2024) .  \nActivation-level methods intervene directly in the residual stream. ActAdd (Turner et al., 2024), Inference-Time Intervention (Li et al., 2023), and Contrastive Activation Addition (Rimsky et al., 2024) compute a steering vector by contrasting hidden states across behavioral exemplars and inject it during the forward pass. Sparse Autoencoders (SAEs) decompose dense, polysemantic activations into a monosemantic feature dictionary (Cunningham et al., 2023 ; Bricken et al., 2023), then clamp or amplify features that mediate a target behavior (Labonne, 2024 ; Templeton et al., 2024) . The Golden Gate Claude exemplifies the appeal: amplifying a single interpretable feature substantially redirects model behavior without any weight updates (Templeton et al., 2024) .  \nA property shared by these methods is uniform, every-token intervention: the steering signal is added at every position throughout generation (Turner et al., 2024 ; Rimsky et al., 2024 ; Li et al., 2023) . Persistent perturbation pushes generations off the model’s native activation manifold, degrading fluency and downstream performance (Turner et al., 2024 ; Labonne, 2024); in multi-attribute settings, conflicting signals overcorrect the primary target and regress on unrelated behaviors (Zou et al., 2023a) . These costs raise a natural question: is intervention at every token necessary? We answer this by replacing deterministic every-token intervention with stochastic gating, applying the signal to random subsets of tokens and askin","cbCaikucSB4j7h7C","https://ap.wps.com/l/cbCaikucSB4j7h7C","pdf",445582,1,15,"English","en",105,"# Introduction\n# Related Work","[{\"question\":\"Why do dense activation steering methods risk degrading model fluency?\",\"answer\":\"Dense methods apply the steering signal at every generated token, creating constant perturbations that can push outputs away from the model’s natural activation manifold, reducing fluency and downstream performance.\"},{\"question\":\"How does Stochastic Token Steering (STS) apply interventions?\",\"answer\":\"STS samples an independent Bernoulli(p) gate at every token position and applies the steering signal only to the gated subset of tokens.\"},{\"question\":\"How does Stochastic Block Steering (SBS) differ from STS?\",\"answer\":\"SBS draws a single Bernoulli(p) gate per sequence and applies the steering uniformly to a leading window, treating early-token influence as an atomic intervention site.\"}]",1784198678,38,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":27},"a-coin-flip-per-token-bernoulli-sparse-steering-of-large-language-models","",{"@graph":35,"@context":85},[36,53,68],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/a-coin-flip-per-token-bernoulli-sparse-steering-of-large-language-models/84840/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-17","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"Why do dense activation steering methods risk degrading model fluency?","Question",{"text":75,"@type":76},"Dense methods apply the steering signal at every generated token, creating constant perturbations that can push outputs away from the model’s natural activation manifold, reducing fluency and downstream performance.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"How does Stochastic Token Steering (STS) apply interventions?",{"text":80,"@type":76},"STS samples an independent Bernoulli(p) gate at every token position and applies the steering signal only to the gated subset of tokens.",{"name":82,"@type":73,"acceptedAnswer":83},"How does Stochastic Block Steering (SBS) differ from STS?",{"text":84,"@type":76},"SBS draws a single Bernoulli(p) gate per sequence and applies the steering uniformly to a leading window, treating early-token influence as an atomic intervention site.","https://schema.org",{"og:url":51,"og:type":87,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":89,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":45,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":45,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":45,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":45,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":45,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":45,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":45,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]