[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-seo-142133-105":3,"detail-sidebar-cat-0-en-105":81,"doc-detail-142133-en":131},{"code":4,"msg":5,"data":6},0,"ok",{"site_id":7,"language":8,"slug":9,"title":10,"keywords":11,"description":12,"schema_data":13,"social_meta":74,"head_meta":76,"extra_data":78,"updated_unix":80},105,"en","finite-time-decoupled-convergence-in-nonlinear-two-time-scale-stochastic-approximation","Finite-Time Decoupled Convergence in Nonlinear Two-Time-Scale Stochastic Approximation","","Finite-time decoupled convergence is studied for nonlinear two-time-scale stochastic approximation, where two coupled iterates evolve at different speeds via distinct step sizes. Prior linear results show that mean-square error convergence rates depend only on each step size, a property known as decoupled convergence; the nonlinear case is less understood. Under a nested local linearity assumption, finite-time decoupled convergence rates are established through cross-term analysis and fourth-order moment bounds. An illustrative counterexample shows that nonlinearity in the slow update alone can break decoupled convergence.",{"@graph":14,"@context":73},[15,34,56],{"@type":16,"itemListElement":17},"BreadcrumbList",[18,23,27,31],{"item":19,"name":20,"@type":21,"position":22},"https://docshare.wps.com","Home","ListItem",1,{"item":24,"name":25,"@type":21,"position":26},"https://docshare.wps.com/document/","Document",2,{"item":28,"name":29,"@type":21,"position":30},"https://docshare.wps.com/document/research-report/","Research & Report",3,{"item":32,"name":10,"@type":21,"position":33},"https://docshare.wps.com/document/finite-time-decoupled-convergence-in-nonlinear-two-time-scale-stochastic-approximation/142133/",4,{"url":32,"name":10,"@type":35,"image":36,"author":41,"headline":10,"publisher":44,"fileFormat":47,"inLanguage":8,"description":12,"dateModified":48,"datePublished":49,"encodingFormat":47,"isAccessibleForFree":50,"interactionStatistic":51},"DigitalDocument",{"url":37,"@type":38,"width":39,"height":40},"https://docshare.wps.com/thumbnails/finite-time-decoupled-convergence-in-nonlinear-two-time-scale-stochastic-approximation/142133.png","ImageObject",300,407,{"name":42,"@type":43},"Valentina","Person",{"url":19,"name":45,"@type":46},"DocShare","Organization","application/pdf","2026-10-27","2026-08-25",true,{"@type":52,"interactionType":53,"userInteractionCount":55},"InteractionCounter",{"@type":54},"ViewAction",13,{"@type":57,"mainEntity":58},"FAQPage",[59,65,69],{"name":60,"@type":61,"acceptedAnswer":62},"What is decoupled convergence in two-time-scale stochastic approximation?","Question",{"text":63,"@type":64},"It refers to convergence rates of the mean-square errors that depend only on the respective step sizes, even though the two iterates influence each other through coupling.","Answer",{"name":66,"@type":61,"acceptedAnswer":67},"What assumptions are used to obtain finite-time decoupled convergence for the nonlinear case?",{"text":68,"@type":64},"The analysis relies on a nested local linearity assumption, enabling control of higher-order error terms via cross-term and fourth-order moment bounds.",{"name":70,"@type":61,"acceptedAnswer":71},"Can decoupled convergence fail for nonlinear two-time-scale SA? Why?",{"text":72,"@type":64},"Yes. Even if the fast-time-scale update is linear, nonlinearity in the slow-time-scale update alone can destroy decoupled convergence, as shown by a constructed example.","https://schema.org",{"og:url":32,"og:type":75,"og:title":10,"og:site_name":45,"og:description":12},"article",{"robots":77,"canonical":32},"index,follow",{"doc_id":79,"site_id":7},142133,1787675663,{"code":4,"msg":82,"data":83},"success",[84,88,92,96,101,106,111,115,120,123,127],{"id":22,"doc_module":4,"doc_module_name":25,"category_name":85,"show_sort_weight":86,"slug":87},"Story & Novel",90,"story-novel",{"id":26,"doc_module":4,"doc_module_name":25,"category_name":89,"show_sort_weight":90,"slug":91},"Literature",80,"literature",{"id":33,"doc_module":4,"doc_module_name":25,"category_name":93,"show_sort_weight":94,"slug":95},"Exam",70,"exam",{"id":97,"doc_module":4,"doc_module_name":25,"category_name":98,"show_sort_weight":99,"slug":100},5,"Comic",60,"comic",{"id":102,"doc_module":4,"doc_module_name":25,"category_name":103,"show_sort_weight":104,"slug":105},6,"Technology",50,"technology",{"id":107,"doc_module":4,"doc_module_name":25,"category_name":108,"show_sort_weight":109,"slug":110},7,"Healthcare",40,"healthcare",{"id":112,"doc_module":4,"doc_module_name":25,"category_name":29,"show_sort_weight":113,"slug":114},8,30,"research-report",{"id":116,"doc_module":4,"doc_module_name":25,"category_name":117,"show_sort_weight":118,"slug":119},9,"Religion & Spirituality",20,"religion-spirituality",{"id":118,"doc_module":4,"doc_module_name":25,"category_name":121,"show_sort_weight":118,"slug":122},"World Cup","world-cup",{"id":124,"doc_module":4,"doc_module_name":25,"category_name":125,"show_sort_weight":124,"slug":126},10,"Lifestyle","lifestyle",{"id":128,"doc_module":4,"doc_module_name":25,"category_name":129,"show_sort_weight":97,"slug":130},19,"General","general",{"code":4,"msg":82,"data":132},{"doc_id":79,"user_id":133,"nickname":42,"user_avatar":134,"doc_module":4,"category_id":112,"category_name":29,"doc_title":10,"doc_description":12,"doc_content":135,"file_id":136,"file_url":137,"file_type":138,"file_size":139,"view_count":55,"is_deleted":4,"is_public":22,"is_downloadable":22,"audit_status":22,"page_count":140,"language":141,"language_code":8,"site_id":7,"html_lang":8,"table_of_contents":142,"faqs":143,"seo_title":144,"seo_description":12,"update_tm":80,"read_time":145,"preview_page_count":4},13056703020460,"https://ap-avatar.wpscdn.com/avatar/be000253dac470eee5d?_k=1778207105932848923","Finite-Time Decoupled Convergence in Nonlinear Two-Time-Scale Stochastic Approximation  \nYuze Han [hanyuze97@ruc.edu.cn](hanyuze97@ruc.edu.cn)  \nCenter for Applied Statistics and School of Statistics Renmin University of China  \nBeijing, China  \nXiang Li [lx10077@pku.edu.cn](lx10077@pku.edu.cn)  \nSchool of Mathematical Sciences Peking University  \nBeijing, China  \nZhihua Zhang∗ [zhzhang@math.pku.edu.cn](zhzhang@math.pku.edu.cn)  \nSchool of Mathematical Sciences Peking University  \nBeijing, China  \nEditor: Peter Richtarik  \nAbstract  \nIn two-time-scale stochastic approximation (SA), two iterates are updated at varying speeds using different step sizes, with each update influencing the other. Previous studies on linear two-time-scale SA have shown that the convergence rates of the mean-square errors for these updates depend solely on their respective step sizes, a phenomenon termed decoupled convergence. However, achieving decoupled convergence in nonlinear SA remains less understood. Our research investigates the potential for finite-time decoupled convergence in nonlinear two-time-scale SA. We demonstrate that, under a nested local linearity assumption, finite-time decoupled convergence rates can be achieved with suitable step size selection.  \nTo derive this result, we conduct a convergence analysis of the matrix cross term between the iterates and leverage fourth-order moment convergence rates to control the higher-order error terms induced by local linearity. To further investigate the necessity of local linearity for decoupled convergence, we also construct an example showing that, even when the fast-time-scale update is linear, the nonlinearity of the slow-time-scale update alone can destroy decoupled convergence.  \nKeywords: two-time-scale stochastic approximation, finite-time convergence, decoupled convergence  \n1. Introduction  \nStochastic approximation (SA), initially introduced by Robbins and Monro (1951), is an iterative method for finding the root of an unknown operator based on noisy observations. This method has gained substantial attention over the past few decades, finding applications in stochastic optimization and reinforcement learning (Kushner and Yin, 2003; Borkar, 2009; Mou et al., 2020, 2022, 2023; Li et al., 2023b,a) . However, certain scenarios require  \n∗ . Corresponding Author  \n©2026 Yuze Han, Xiang Li, and Zhihua Zhang.  \nLicense: CC-BY 4.0, see [https://creativecommons.org/licenses/by/4.0/](https://creativecommons.org/licenses/by/4.0/. Attribution)[. Attribution](https://creativecommons.org/licenses/by/4.0/. Attribution) requirements are provided at  \n[http://jmlr.org/papers/v27/25-1449.html](http://jmlr.org/papers/v27/25-1449.html).  \nHan, Li, and Zhang  \nmanaging two iterates updated at different time scales using varying step sizes. Examples include stochastic bilevel optimization (Ghadimi and Wang, 2018; Chen et al., 2021; Hong et al., 2023), temporal difference learning (Sutton et al., 2009; Xu et al. , 2019; Xu and Liang, 2021; Wang et al., 2021) and actor-critic methods (Borkar and Konda, 1997; Konda and Tsitsiklis, 2003; Wu et al., 2020; Xu et al., 2020) . These cases highlight the limitations of traditional SA and underscore the need for a two-time-scale SA approach to better capture the complexities involved.  \nIn this paper, we study the two-time-scale SA (Borkar, 1997), a variant of the classical SA algorithm, designed to identify the roots of systems comprising two coupled, potentially nonlinear equations. We focus on two unknown Lipschitz operators, F : Rd x × Rdy → Rd x and G : Rd x × Rdy → Rd y, with the aim of finding the root pair (x⋆ , y⋆ ) satisfying  \n(FG((xx ,, yy))  00, (1)  \nGiven that F and G are unknown, we assume access to a stochastic oracle that provides noisy evaluations of F (x, y) and G (x, y) at any input pair (x, y) . Specifically, for any x and y, the oracle returns F (x, y) + ξ and G (x, y) + ψ where ξ and ψ represent noise components. Utilizing this stochastic ","cbCaivrUwRhVhq55","https://ap.wps.com/l/cbCaivrUwRhVhq55","pdf",1951955,69,"English","# Introduction\n## Two-time-scale stochastic approximation and motivation\n## Problem setup and two-loop interpretation\n## Fast and slow time scales\n## Example: SGD with Polyak-Ruppert averaging\n# Main focus: finite-time decoupled convergence in nonlinear SA","[{\"question\":\"What is decoupled convergence in two-time-scale stochastic approximation?\",\"answer\":\"It refers to convergence rates of the mean-square errors that depend only on the respective step sizes, even though the two iterates influence each other through coupling.\"},{\"question\":\"What assumptions are used to obtain finite-time decoupled convergence for the nonlinear case?\",\"answer\":\"The analysis relies on a nested local linearity assumption, enabling control of higher-order error terms via cross-term and fourth-order moment bounds.\"},{\"question\":\"Can decoupled convergence fail for nonlinear two-time-scale SA? Why?\",\"answer\":\"Yes. Even if the fast-time-scale update is linear, nonlinearity in the slow-time-scale update alone can destroy decoupled convergence, as shown by a constructed example.\"}]","Finite-Time Decoupled Convergence in Nonlinear Two-Time-Scale Stochastic Approximation | PDF",174]