[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-84512-en":3,"doc-seo-84512-105":29,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},84512,962075006959,"Anda","https://ap-avatar.wpscdn.com/avatar/e0002397efbe92a78e?_k=1776741047341049297",8,"Research & Report","Rush: Scalable Asynchronous Distributed Computing via Shared State in R","Many statistics and machine learning algorithms can be parallelized asynchronously, where workers coordinate through shared state instead of executing independent, controller-dispatched tasks. Existing R parallel packages rely on a centralized controller–worker design, which cannot express this pattern. The rush R package introduces a shared-state coordination layer using Redis as a shared key–value store, offering a high-level task API, sub-millisecond overhead, robust lost-worker detection, and efficient caching. It integrates with mlr3 for asynchronous optimization and demonstrates decentralized Bayesian optimization on LightGBM.","arXiv :2606 .21430v2 [ cs .DC] 13 Jul 2026  \nrush: Scalable Asynchronous Distributed Computing via Shared State in R  \nby Marc Becker and Bernd Bischl  \nAbstract Many algorithms in statistics and machine learning can be parallelized in an asynchronous manner where workers need to communicate through shared state rather than execute independent tasks dispatched by a central controller. Especially in modern hyperparameter optimization and parallel black-box optimization with expensive objectives, this decentralized approach has become widespread, and several Python frameworks adopt it (e.g., Optuna, DeepHyper, and Hyperopt) . However, all popular R packages for parallel computing follow a centralized controller-worker architecture that does not support this pattern. We present rush, an R package that provides a shared-state coordination layer for asynchronously parallelized iterative algorithms. rush uses a Redis database as a shared key-value store: workers read and write task data through the database and independently execute their own loops. The package provides a high-level API for managing tasks and their lifecycle, featuring sub-millisecond per-task overhead, robust error handling with automatic detection of lost workers, and efficient caching. rush optionally integrates with the mlr3 ecosystem, powering asynchronous optimization in the bbotk and mlr3tuning packages. We demonstrate the practical utility of rush by implementing asynchronous decentralized Bayesian optimization (ADBO) and benchmarking it on hyperparameter optimization of LightGBM across four datasets using 448 workers.  \n1 Introduction  \nMany iterative algorithms in statistics and machine learning share a common computational structure: multiple workers operate concurrently, each maintaining local state and autonomously deciding what to compute next, while coordinating through a shared pool of (intermediate) results. This computational pattern has long-standing roots in the blackboard architecture (Nii, 1986) and the tuple space model (Gelernter, 1985), coordination paradigms in which autonomous processes communicate through a shared associative memory.  \nFrameworks for parallel computation have become extremely popular in modern machine learning in the context of expensive black-box optimization and hyperparameter optimization (HPO, Bischlet al., 2023) . Here, algorithmic approaches span a wide spectrum: from synchronous batch methods (Ginsbourger et al., 2010), through asynchronous but centralized schedulers such as ASHA (Li et al., 2020)1 and Vizier (Golovin et al., 2017), to fully decentralized approaches such as asynchronous decentralized Bayesian optimization (ADBO, Egelé et al., 2023), where each worker autonomously decides what to compute next.  \nIn contrast to these decentralized approaches, existing R packages for parallel computing follow a fundamentally different design. Packages such as parallel, future (Bengtsson, 2021), mirai (Gao, 2025), batchtools (Lang et al., 2017), and crew (Landau, 2026) adopt a controller-worker architecture (Figure 1, left): a central process proposes tasks, dispatches them to workers, and collects results. These frameworks excel at parallelizing independent computations but do not support algorithms in which workers coordinate through shared state and autonomously decide what to compute next. Beyond expressiveness—some algorithms are simply more natural when each worker has the coordination logic locally—the decentralized design also has computational advantages. Planning and generating further downstream tasks can themselves incur non-negligible cost, and moving this work from a central node to the workers allows it to run in parallel and enables more efficient task spawning, especially when many workers are available.  \nTo fill this gap, we present rush, an R package that provides a shared-state coordination layer for iterative algorithms. rush uses a Redis database as its shared store: workers communicate by readin","cbCaiaBUig7pFU72","https://ap.wps.com/l/cbCaiaBUig7pFU72","pdf",1233991,1,19,"English","en",105,"# Introduction\n## Background and motivation\n## Limitations of centralized controller–worker R packages\n## The rush shared-state coordination layer\n# Related work\n## Parallel computing in R (snow, multicore)","[{\"question\":\"What problem does the rush R package address in asynchronous distributed computing?\",\"answer\":\"It enables iterative algorithms where workers communicate via shared state and autonomously decide what to compute next, which centralized controller–worker R packages cannot express.\"},{\"question\":\"How does rush implement shared state coordination among workers?\",\"answer\":\"It uses a Redis database as a shared key–value store, letting workers read and write task inputs, outputs, and computational states while running independent loops.\"},{\"question\":\"What benefits does rush provide for practical performance and reliability?\",\"answer\":\"It targets sub-millisecond per-task overhead, includes robust error handling with automatic detection of lost workers, and provides efficient caching to reduce database operations.\"}]",1784196236,48,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":27},"rush-scalable-asynchronous-distributed-computing-via-shared-state-in-r","",{"@graph":35,"@context":85},[36,53,68],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/rush-scalable-asynchronous-distributed-computing-via-shared-state-in-r/84512/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-17","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What problem does the rush R package address in asynchronous distributed computing?","Question",{"text":75,"@type":76},"It enables iterative algorithms where workers communicate via shared state and autonomously decide what to compute next, which centralized controller–worker R packages cannot express.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"How does rush implement shared state coordination among workers?",{"text":80,"@type":76},"It uses a Redis database as a shared key–value store, letting workers read and write task inputs, outputs, and computational states while running independent loops.",{"name":82,"@type":73,"acceptedAnswer":83},"What benefits does rush provide for practical performance and reliability?",{"text":84,"@type":76},"It targets sub-millisecond per-task overhead, includes robust error handling with automatic detection of lost workers, and provides efficient caching to reduce database operations.","https://schema.org",{"og:url":51,"og:type":87,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":89,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":45,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":45,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":45,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":45,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":45,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":45,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":21,"doc_module":4,"doc_module_name":45,"category_name":136,"show_sort_weight":106,"slug":137},"General","general"]