[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-84059-en":3,"doc-seo-84059-105":29,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},84059,1374391975076,"Riley","https://ap-avatar.wpscdn.com/avatar/14000253ca4ec9f6853?x-image-process=image/resize,m_fixed,w_180,h_180&k=1783305029341752051",8,"Research & Report","Scalable Perturbation Learning for Online Self-Supervised Echo State Networks","Intelligent systems must both solve tasks and adapt under real-world constraints, requiring self-supervised learning, online sequential adaptation, and memory-efficient implementation. For high-dimensional echo state networks, these goals conflict because perturbation-based learning suffers variance that grows with the dimension of perturbed variables. The study proposes a perturbation-based online learning rule derived from an orthogonal decomposition of the self-supervised cost, perturbing only the input-dependent component. This reduces effective perturbation dimension from reservoir size to input size, maintaining adaptation while avoiding reservoir-dependent variance growth, enabling scalable and hardware-compatible learning design.","arXiv :2607 .06079v 1 [ cs .LG] 7 Jul 2026  \nScalable Perturbation Learning for Online Self-Supervised Echo State Networks  \nTaiki Yamadaa,∗, Kantaro Fujiwaraa  \na Graduate School of Information Science and Technology, The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, 113-8656, Tokyo, Japan  \nAbstract  \nIntelligent systems should not only solve tasks but also adapt under real-world constraints. Autonomous adaptation via self-supervised learning, sequential adaptation via online learning, and memory-efficient implementation via perturbationbased learning are important requirements for such systems. However, these requirements are generally in tension for high-dimensional systems, because perturbation-based learning suffers from variance that grows with the dimension of the perturbed variables.  \nIn this study, we focus on echo state networks (ESNs), where this tension naturally arises in large reservoirs. We propose a perturbation-based learning rule for online self-supervised learning in ESNs. The proposed rule is derived from an orthogonal decomposition of the self-supervised learning cost, which separates an input-dependent component from a redundant component determined by the fixed ESN parameters. By perturbing only the input-dependent component, the effective perturbation dimension is reduced from the reservoir dimension to the input dimension.  \nThus, the proposed method preserves self-supervised adaptation, online learning, and scalar-feedback perturbation learning, while avoiding reservoir-sizedependent variance growth. This suggests a design principle for scalable and hardware-compatible learning: online learning should be restricted to the dynamically necessary low-dimensional component of the objective.  \n1. Introduction  \nBuilding intelligent systems that can continuously adapt to complex realworld dynamics is an important goal in machine learning and neuromorphic engineering [20, 19] . Such systems must process temporal information under limited computational and memory resources [18, 8] . Reservoir computing follows this direction by using a recurrent dynamical system as a temporal feature  \n∗ Corresponding author  \nEmail addresses: [yamada-taiki@g.ecc.u-tokyo.ac.jp](yamada-taiki@g.ecc.u-tokyo.ac.jp) (Taiki Yamada), [kantaro@g.ecc.u-tokyo.ac.jp](kantaro@g.ecc.u-tokyo.ac.jp) (Kantaro Fujiwara)  \nextractor and reducing the trainable part of the model [17, 26] . Echo state networks [9] are a representative model of reservoir computing, in which the reservoir is a fixed recurrent neural network.  \nRealizing such continuously adapting systems with echo state networks requires satisfying three conditions. First, learning should be online and selfsupervised, so that the system can adapt without external target labels in unpredictable environments [20] . Second, the reservoir dimension should be scalable, because large reservoirs are often used to represent high-dimensional dynamics and long temporal dependencies [10, 3] . Third, this scaling should not rely on large auxiliary memory or complex error routing [8, 14, 15] . This requirement is especially important for hardware implementations that exploit reservoir dynamics directly, rather than relying on general-purpose computation and full memory access.  \nThese requirements are in tension. To explain this point, we denote the input, reservoir, and output dimensions by nin , nr , and nout , respectively. Although echo state networks can be applied to various temporal tasks by supervised readout learning, only specific tasks admit a self-supervised formulation. In this paper, we consider the task studied in our prior work [29], where the system learns to recover the external input from the reservoir dynamics without external teaching signals. For this task, nout = nin. Our prior work reformulated this problem not as the training of a usual nin × nr output map, but asthe training of an nr × nr map that reconstructs the reservoir state itself. As a result of consideration o","cbCailbHXymikdml","https://ap.wps.com/l/cbCailbHXymikdml","pdf",1320726,1,19,"English","en",105,"# Introduction\n## Online and self-supervised adaptation requirements\n## Echo state networks and scalable reservoirs\n## Challenges of online learning and memory cost\n## Variance scaling in perturbation-based learning\n## Proposed variance-scaling solution","[{\"question\":\"What problem does the document address for echo state networks in online self-supervised settings?\",\"answer\":\"It addresses the tension between online self-supervised adaptation, scalable reservoir size, and memory-efficient implementation, which conflicts because perturbation-based learning variance increases with the dimension of perturbed variables.\"},{\"question\":\"How does the proposed method reduce effective perturbation dimensionality?\",\"answer\":\"It derives a learning rule from an orthogonal decomposition of the self-supervised cost, then perturbs only the input-dependent component rather than the full reservoir-dependent part.\"},{\"question\":\"Why is perturbation-based learning inefficient for high-dimensional reservoirs in previous approaches?\",\"answer\":\"Because compressing the multi-dimensional error into a scalar feedback increases the variance of the gradient estimate, causing signal-to-noise ratio degradation as reservoir dimension grows.\"}]",1784192291,48,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":27},"scalable-perturbation-learning-for-online-self-supervised-echo-state-networks","",{"@graph":35,"@context":85},[36,53,68],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/scalable-perturbation-learning-for-online-self-supervised-echo-state-networks/84059/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-17","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What problem does the document address for echo state networks in online self-supervised settings?","Question",{"text":75,"@type":76},"It addresses the tension between online self-supervised adaptation, scalable reservoir size, and memory-efficient implementation, which conflicts because perturbation-based learning variance increases with the dimension of perturbed variables.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"How does the proposed method reduce effective perturbation dimensionality?",{"text":80,"@type":76},"It derives a learning rule from an orthogonal decomposition of the self-supervised cost, then perturbs only the input-dependent component rather than the full reservoir-dependent part.",{"name":82,"@type":73,"acceptedAnswer":83},"Why is perturbation-based learning inefficient for high-dimensional reservoirs in previous approaches?",{"text":84,"@type":76},"Because compressing the multi-dimensional error into a scalar feedback increases the variance of the gradient estimate, causing signal-to-noise ratio degradation as reservoir dimension grows.","https://schema.org",{"og:url":51,"og:type":87,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":89,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":45,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":45,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":45,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":45,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":45,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":45,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":21,"doc_module":4,"doc_module_name":45,"category_name":136,"show_sort_weight":106,"slug":137},"General","general"]