[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-123969-en":3,"doc-seo-123969-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},123969,2336464648322,"Aria","https://ap-avatar.wpscdn.com/avatar/2200025388227c56fec?_k=1778556882303663488",8,"Research & Report","AN OPERATOR PRECONDITIONING PERSPECTIVE ON TRAINING IN PHYSICS-INFORMED MACHINE LEARNING","This paper analyzes gradient descent behavior in physics-informed machine learning, focusing on PINNs that minimize PDE residuals. The central finding links training difficulty to the conditioning of a specific differential operator. That operator is tied to the Hermitian square of the PDE’s differential operator, so an ill-conditioned operator yields slow or infeasible training. Rigorous mathematical analysis and empirical experiments are used to study and compare preconditioning strategies that improve operator conditioning and thereby accelerate convergence.","AN OPERATOR PRECONDITIONING PERSPECTIVE ON TRAINING IN  \nPHYSICS-INFORMED MACHINE LEARNING  \nTim De Ryck∗  \nSeminar for Applied Mathematics, ETH Z¨urich, Switzerland  \nFlorent Bonnet  \nInstitute of Intelligent Systems and Robotics, Extrality,  \nSorbonne Universit, France  \narXiv :2310 .05801v2 [ cs .LG] 3 May 2024  \nSiddhartha Mishra  \nSeminar for Applied Mathematics, ETH AI Center,  \nETH Z¨urich, Switzerland  \nEmmanuel de Bzenac∗ Seminar for Applied Mathematics, ETH Z¨urich, Switzerland  \nABSTRACT  \nIn this paper, we investigate the behavior of gradient descent algorithms in physics-informed machine learning methods like PINNs, which minimize residuals connected to partial differential equations (PDEs) . Our key result is that the difficulty in training these models is closely related to the conditioning of a specific differential operator. This operator, in turn, is associated to the Hermitian square of the differential operator of the underlying PDE. If this operator is ill-conditioned, it results in slow or infeasible training. Therefore, preconditioning this operator is crucial. We employ both rigorous mathematical analysis and empirical evaluations to investigate various strategies, explaining how they better condition this critical operator, and consequently improve training.  \n1 INTRODUCTION  \nPartial Differential Equations (PDEs) are ubiquitous as mathematical models of interesting phenomena in science and engineering (Evans, 2010) . Traditionally, numerical methods such as finite difference, finite element etc (Quarteroni & Valli, 1994) are used to simulate PDEs. However, given the prohibitive cost of these methods for a variety of PDE problems such as those with multiple scales, in high dimensions or involving multiple calls to the PDE solver like in UQ, control and inverse problems, machine learning based alternatives are finding increasing traction as efficient PDE simulators, see Karniadakis et al. (2021) and references therein.  \nWithin the plethora of approaches that leverage machine learning techniques to solve PDEs, models which directly incorporate the underlying PDE into the loss function are widely popular. A prominent example of this framework, often referred to as physics-informed machine learning, are physicsinformed neural networks or PINNs (Dissanayake & Phan-Thien, 1994; Lagaris et al., 2000a;b; Raissi et al., 2019), which minimize the PDE residual within the ansatz space of neural networks. Related approaches in which the weak or variational form of the PDE residual is minimized include Deep Ritz (E & Yu, 2018), neural Galerkin (Bruna et al., 2022), variational PINNs (Kharazmi et al., 2019) and weak PINNs (De Ryck et al., 2022) . Similarly, PDE residual minimization methods for other ansatz spaces such as Gaussian processes (Raissi & Karniadakis, 2018), Fourier features (Tancik et al., 2020), random features (Ye et al., 2023) etc have also been considered.  \nDespite the considerable success of PINNs and their afore-mentioned variants in solving numerous types of PDE forward and inverse problems (see Karniadakis et al. (2021); Cuomo et al. (2022) and references therein for extensive reviews), significant problems have been identified with physicsinformed machine learning. Arguably, the foremost problem lies with the training these frameworks  \n*  \nThese authors contributed equally to this work.  \nwith (variants of) gradient descent methods (Krishnapriyan et al., 2021; Moseley et al., 2021; Wanget al., 2021a; 2022b) . It has been increasingly observed that PINNs and their variants are slow, even infeasible, to train even on certain model problems (Krishnapriyan et al., 2021), with the training process either not converging or converging to unacceptably large loss values.  \nWhat is the reason behind the issues observed with training physics-informed machine learning algorithms? Empirical studies such as Krishnapriyan et al. (2021) attribute failure modes to thenon-convex loss landscape, which is much m","cbCaijTHD4LgR72j","https://ap.wps.com/l/cbCaijTHD4LgR72j","pdf",2420320,1,29,"English","en",105,"# Introduction\n## Physics-informed machine learning for PDEs\n## Training challenges of PINNs\n## Goal and contributions","[{\"question\":\"What is the main problem addressed by this paper?\",\"answer\":\"The paper targets why physics-informed machine learning methods like PINNs can train slowly or fail to converge when using gradient descent.\"},{\"question\":\"How does the paper explain the cause of training difficulty?\",\"answer\":\"It shows that the speed and feasibility of training are closely tied to the conditioning of an operator related to the PDE residual minimization.\"},{\"question\":\"What role does preconditioning play in improving training?\",\"answer\":\"Preconditioning the critical operator improves its conditioning, which reduces training bottlenecks and leads to better convergence behavior.\"}]","AN OPERATOR PRECONDITIONING PERSPECTIVE ON TRAINING IN PHYSICS-INFORMED MACHINE LEARNING | PDF",1785819493,73,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"an-operator-preconditioning-perspective-on-training-in-physics-informed-machine-learning","",{"@graph":36,"@context":85},[37,54,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/an-operator-preconditioning-perspective-on-training-in-physics-informed-machine-learning/123969/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":62,"encodingFormat":61,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-04",true,{"@type":65,"interactionType":66,"userInteractionCount":4},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What is the main problem addressed by this paper?","Question",{"text":75,"@type":76},"The paper targets why physics-informed machine learning methods like PINNs can train slowly or fail to converge when using gradient descent.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"How does the paper explain the cause of training difficulty?",{"text":80,"@type":76},"It shows that the speed and feasibility of training are closely tied to the conditioning of an operator related to the PDE residual minimization.",{"name":82,"@type":73,"acceptedAnswer":83},"What role does preconditioning play in improving training?",{"text":84,"@type":76},"Preconditioning the critical operator improves its conditioning, which reduces training bottlenecks and leads to better convergence behavior.","https://schema.org",{"og:url":52,"og:type":87,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":89,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":46,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":46,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":46,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":46,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]