[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-123814-en":3,"doc-seo-123814-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},123814,4810365810221,"Aurora","https://ap-avatar.wpscdn.com/davatar_155a257f0dc6eb9ab79c44ca47cae57d",8,"Research & Report","A Functional Approach to Interpreting the Role of the Adjoint Equation in Machine Learning","Connection between numerical methods for solving differential equations and machine learning is established by interpreting deep learning architectures as continuous dynamical systems. Differential equations are treated as continuous analogues of deep neural networks, enabling tasks such as image recognition via parameter learning for systems of ODEs along trajectories. By casting parameter identification as an inverse problem, the paper formulates the adjoint equation as a continuous analogue of backpropagation, yielding gradients for loss functions that fit initial value problem solutions to discrete or continuous measurements. Numerical experiments show gradients drive gradient descent to match continuous or uncertain-time discrete noisy data under controlled conditions.","Results Math (2024) 79:43  \n􀀂c 2023 The Author(s) 1422-6383/24/010001-21 published online December 8, 2023  \n[https://doi.org/10.1007/s00025-023-02074-3](https://doi.org/10.1007/s00025-023-02074-3)  \nA Functional Approach to Interpreting the Role of the Adjoint Equation in Machine Learning  \nImre Fekete, Andra´s Molna´r, and Pe´ter L. Simon  \nAbstract. The connection between numerical methods for solving differential equations and machine learning has been revealed recently. Differential equations have been proposed as continuous analogues of deep neural networks, and then used in handling certain tasks, such as image recognition, where the training of a model includes learning the parameters of systems of ODEs from certain points along their trajectories. Treating this inverse problem of determining the parameters of a dynamical system that minimize the difference between data and trajectory by a gradient-based optimization method presents the solution of the adjoint equation as the continuous analogue of backpropagation that yields the appropriate gradients. The paper explores an abstract approach that can be used to construct a family of loss functions with the aim of fitting the solution of an initial value problem to a set of discrete or continuous measurements. It is shown, that an extension of the adjoint equation can be used to derive the gradient of the loss function as a continuous analogue of backpropagation in machine learning. Numerical evidence is presented that under reasonably controlled circumstances the gradients obtained this way can be used in a gradient descent to fit the solution of an initial value problem to a set of continuous noisy measurements, and a set of discrete noisy measurements that are recorded at uncertain times.  \nMathematics Subject Classification. 90C52, 68Q32, 34A55 .  \nKeywords. Continuous backpropagation, adjoint equation, parameter learning.  \n43 Page 2 of 21 I. Fekete et al. Results Math  \n1. Introduction  \nMachine learning has been recently connected to the field of differential equations by observing that numerical time integrators resemble formulae used in residual neural networks [9, 14] . The fast development of deep learning algorithms has led to the study of analogues of deep neural networks, and that of the discretization of continuous dynamical systems [7, 15] . The continuous analogue of backpropagation in deep residual neural networks is the adjoint equation, also used in optimal control, plays a crucial role in connecting the two fields. This recently revealed relation has been exploited along the following two directions.  \n(1) Certain tasks, typically handled by deep neural networks, such as image recognition, are treated by using the continuous analogue, that is, by learning the parameters of a system of ODEs.  \n(2) Learning the parameters of a dynamical system from certain points of its trajectories by using the continuous analogue of backpropagation, that is, by applying the gradient method by computing the derivative of the loss function by solving the adjoint equation.  \nOur work presented in this paper is mostly related to direction (2), hence we will deal with the two directions in the introduction as follows.  \n• Literature overview corresponding to direction (1) .  \n• Problem description of direction (2) .  \n• Literature overview corresponding to direction (2) .  \n• Novelties and structure of our paper.  \nThe analogue between a deep residual neural network and the numerical scheme corresponding to the discretization of an ODE is presented, and the proposed method is applied in image classification in [7, 14] . The idea is also extended to learning neural ODE for stiff systems [8] . A linear multi-step architecture (LM-architecture) is introduced in [9] as a generalization of ResNet, inspired by the linear multi-step method for solving ordinary differential equations. It is shown that it achieves higher accuracy in image recognition than ResNet and other previous neural ODE","cbCaivi0QRzzUzyM","https://ap.wps.com/l/cbCaivi0QRzzUzyM","pdf",709730,1,21,"English","en",105,"# Abstract\n# Introduction\n## Direction (1): continuous analogues of deep residual networks\n## Direction (2): adjoint-equation-based parameter learning\n## Problem formulation for learning initial value problems\n# Mathematical framework and optimization setup","[{\"question\":\"How does the paper relate differential equation solvers to machine learning?\",\"answer\":\"It treats differential equations as continuous analogues of deep neural networks and uses trajectory-based parameter learning to connect numerical integration with learning models.\"},{\"question\":\"What role does the adjoint equation play in this framework?\",\"answer\":\"The paper presents the adjoint equation as a continuous analogue of backpropagation, providing gradients needed to optimize loss functions for fitting initial value problem solutions.\"},{\"question\":\"How is the inverse problem of learning defined in the paper?\",\"answer\":\"Given a family of right-hand sides with parameters and an initial condition, the method searches for the best (initial time, initial state, parameters) that minimize the discrepancy between model trajectories and measurements, via gradient descent.\"}]","A Functional Approach to Interpreting the Role of the Adjoint Equation in Machine Learning | PDF",1785818685,53,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"a-functional-approach-to-interpreting-the-role-of-the-adjoint-equation-in-machine-learning","",{"@graph":36,"@context":85},[37,54,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/a-functional-approach-to-interpreting-the-role-of-the-adjoint-equation-in-machine-learning/123814/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":62,"encodingFormat":61,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-04",true,{"@type":65,"interactionType":66,"userInteractionCount":4},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"How does the paper relate differential equation solvers to machine learning?","Question",{"text":75,"@type":76},"It treats differential equations as continuous analogues of deep neural networks and uses trajectory-based parameter learning to connect numerical integration with learning models.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"What role does the adjoint equation play in this framework?",{"text":80,"@type":76},"The paper presents the adjoint equation as a continuous analogue of backpropagation, providing gradients needed to optimize loss functions for fitting initial value problem solutions.",{"name":82,"@type":73,"acceptedAnswer":83},"How is the inverse problem of learning defined in the paper?",{"text":84,"@type":76},"Given a family of right-hand sides with parameters and an initial condition, the method searches for the best (initial time, initial state, parameters) that minimize the discrepancy between model trajectories and measurements, via gradient descent.","https://schema.org",{"og:url":52,"og:type":87,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":89,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":46,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":46,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":46,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":46,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]