[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-123901-en":3,"doc-seo-123901-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},123901,4398048950312,"Violet","https://ap-avatar.wpscdn.com/avatar/400002538284de19e3c?_k=1778320343897328908",8,"Research & Report","When ancient numerical demons meet physics-informed machine learning - adjoint-based gradients for implicit differentiable modeling","Recent advances in differentiable modeling—physics-informed machine learning that couples neural networks with process equations—are improving hydrological model accuracy, interpretability, and knowledge discovery. However, explicit numerical schemes with sequential operator splitting can introduce numerical errors, clouding their effect on learned parameters and representational capacity. Implicit schemes face challenges because automatic differentiation may suffer from gradient vanishing and heavy memory use. This work introduces a discretize-then-optimize adjoint method enabling differentiable implicit modeling at large scales.","Hydrol. Earth Syst. Sci., 28, 3051–3077, 2024 [https://doi.org/10.5194/hess-28-3051-2024](https://doi.org/10.5194/hess-28-3051-2024)[ ](https://doi.org/10.5194/hess-28-3051-2024)© Author(s) 2024 . This work is distributed under the Creative Commons Attribution 4 .0 License.  \nWhen ancient numerical demons meet physics-informed machine learning: adjoint-based gradients for implicit differentiable modeling  \nYalan Song 1 , Wouter J. M. Knoben2 , Martyn P. Clark2 , Dapeng Feng 1,3 , Kathryn Lawson 1 , Kamlesh Sawadekar 1 , and Chaopeng Shen 1  \n1 Civil and Environmental Engineering, The Pennsylvania State University, University Park, PA, USA  \n2Department of Civil Engineering, Schulich School of Engineering, University of Calgary, Calgary, Alberta, Canada  \n3Department of Earth System Science, Stanford University, Stanford, CA, USA Correspondence: Yalan Song ([yxs275@psu.edu](yxs275@psu.edu)) and Chaopeng Shen ([cshen@engr.psu.edu](cshen@engr.psu.edu))  \nReceived: 31 October 2023 – Discussion started: 9 November 2023  \nRevised: 23 March 2024 – Accepted: 6 May 2024 – Published: 15 July 2024  \nAbstract. Recent advances in differentiable modeling, a genre of physics-informed machine learning that trains neural networks (NNs) together with process-based equations, have shown promise in enhancing hydrological models' accuracy, interpretability, and knowledge-discovery potential. Current differentiable models are efficient for NN-based parameter regionalization, but the simple explicit numerical schemes paired with sequential calculations (operator splitting) can incur numerical errors whose impacts on models' representation power and learned parameters are not clear. Implicit schemes, however, cannot rely on automatic differentiation to calculate gradients due to potential issues of gradient vanishing and memory demand. Here we propose a “discretize-then-optimize” adjoint method to enable differentiable implicit numerical schemes for the first time for large-scale hydrological modeling. The adjoint model demonstrates comprehensively improved performance, with Kling–Gupta efficiency coefficients, peak-flow and low-flow metrics, and evapotranspiration that moderately surpass the already-competitive explicit model. Therefore, the previous sequential-calculation approach had a detrimental impact on the model's ability to represent hydrological dynamics. Furthermore, with a structural update that describes capillary rise, the adjoint model can better describe baseflow in arid regions and also produce low flows that outperform even pure machine learning methods such as long short-term memory networks. The adjoint model rectified some parameter distortions but did not alter spatial parameter distributions, demon-  \nstrating the robustness of regionalized parameterization. Despite higher computational expenses and modest improvements, the adjoint model's success removes the barrier for complex implicit schemes to enrich differentiable modeling in hydrology.  \n1 Background  \nAccurate hydrological predictions are crucial for effective water resource management around the world under a changing climate (Hannah et al., 2011; Sivapalan et al., 2003) . In recent years, deep learning models such as long short-term memory (LSTM) networks have gained traction in hydrology due to their high predictive performance in various applications, including streamflow prediction, soil moisture estimation, and the modeling of stream temperature and dissolved oxygen (Fang et al., 2017; Feng et al., 2020; Kratzert et al., 2019; Ouyang et al., 2021; Rahmani et al., 2021a, b; Zhi et al., 2023) . Despite their impressive capabilities, deep learning models are often criticized for their limited interpretability and dependence on extensive observations. Additionally, they are unable to provide outputs for untrained variables (those not trained using observations as targets), e.g., evapotranspiration (ET), water storage, or snow water equivalent, which are of great interest to sta","cbCaiuU3SsOv9LZA","https://ap.wps.com/l/cbCaiuU3SsOv9LZA","pdf",8775201,1,27,"English","en",105,"# Background\n## Differentiable modeling in hydrology\n## Limits of deep learning and differentiable modeling\n## Motivation for implicit differentiable schemes\n# Adjoint-based gradients for implicit differentiable modeling","[{\"question\":\"What limitation motivates the proposed adjoint method for implicit differentiable modeling?\",\"answer\":\"Existing differentiable models often rely on explicit numerical schemes with operator splitting that can produce numerical errors, and implicit schemes cannot easily use automatic differentiation due to gradient vanishing and memory demand.\"},{\"question\":\"How does the adjoint model perform compared with the explicit differentiable model?\",\"answer\":\"The adjoint model shows comprehensively improved performance across Kling–Gupta efficiency, peak-flow and low-flow metrics, and evapotranspiration, moderately surpassing the already competitive explicit model.\"},{\"question\":\"What structural update helps the adjoint model represent hydrology in arid regions?\",\"answer\":\"A structural update describing capillary rise improves baseflow representation in arid regions and produces low flows that can outperform even pure machine learning methods such as LSTM.\"}]","When ancient numerical demons meet physics-informed machine learning - adjoint-based gradients for implicit differentiable modeling | PDF",1785819155,68,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"when-ancient-numerical-demons-meet-physics-informed-machine-learning-adjoint-based-gradients-for-implicit-differentiable-modeling","",{"@graph":36,"@context":85},[37,54,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/when-ancient-numerical-demons-meet-physics-informed-machine-learning-adjoint-based-gradients-for-implicit-differentiable-modeling/123901/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":62,"encodingFormat":61,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-04",true,{"@type":65,"interactionType":66,"userInteractionCount":4},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What limitation motivates the proposed adjoint method for implicit differentiable modeling?","Question",{"text":75,"@type":76},"Existing differentiable models often rely on explicit numerical schemes with operator splitting that can produce numerical errors, and implicit schemes cannot easily use automatic differentiation due to gradient vanishing and memory demand.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"How does the adjoint model perform compared with the explicit differentiable model?",{"text":80,"@type":76},"The adjoint model shows comprehensively improved performance across Kling–Gupta efficiency, peak-flow and low-flow metrics, and evapotranspiration, moderately surpassing the already competitive explicit model.",{"name":82,"@type":73,"acceptedAnswer":83},"What structural update helps the adjoint model represent hydrology in arid regions?",{"text":84,"@type":76},"A structural update describing capillary rise improves baseflow representation in arid regions and produces low flows that can outperform even pure machine learning methods such as LSTM.","https://schema.org",{"og:url":52,"og:type":87,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":89,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":46,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":46,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":46,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":46,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]