[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-118124-en":3,"doc-seo-118124-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},118124,5909877438554,"Maeve","https://ap-avatar.wpscdn.com/avatar/5600025385ad2bf12a7?_k=1778553567797529272",8,"Research & Report","Mechanistic Neural Networks for Scientific Machine Learning","Mechanistic Neural Networks provide a neural network design for scientific machine learning that explicitly learns governing differential equations through a new Mechanistic Block. The method builds an ODE representation by learning coefficients that drive modeled evolution of time-series data. Training is enabled by a Relaxed Linear Programming Solver (NeuRLP) inspired by reducing linear ODE solving to linear programs, while supporting scalable GPU-parallel computation. The approach targets interpretability and efficiency, and demonstrates strong performance for equation discovery through dynamic systems modeling.","arXiv :2402 . 13077v1 [ cs .LG] 20 Feb 2024  \nMechanistic Neural Networks  \nMechanistic Neural Networks for Scientific Machine Learning  \nAdeel Pervez [a.a.pervez@uva.nl](a.a.pervez@uva.nl)  \nInformatics Institute, University of Amsterdam Amsterdam, The Netherlands  \nFrancesco Locatello [Francesco.Locatello@ist.ac.at](Francesco.Locatello@ist.ac.at)  \nInstitute of Science and Technology Klosterneuburg, Austria  \nEfstratios Gavves [e.gavves@uva.nl](e.gavves@uva.nl)  \nInformatics Institute, University of Amsterdam Amsterdam, The Netherlands  \nAbstract  \nThis paper presents Mechanistic Neural Networks – a neural network design for machine learning applications in the sciences. It incorporates a new Mechanistic Block in standard architectures to explicitly learn governing differential equations as representations, revealing the underlying dynamics of data and enhancing interpretability and efficiency in data modeling. Central to our approach is a novel Relaxed Linear Programming Solver (NeuRLP) inspired by a technique that reduces solving linear ODEs to solving linear programs. This integrates well with neural networks and surpasses the limitations of traditional ODE solvers enabling scalable GPU parallel processing. Overall, Mechanistic Neural Networks demonstrate their versatility for scientific machine learning applications, adeptly managing tasks from equation discovery to dynamic systems modeling. We prove their comprehensive capabilities in analyzing and interpreting complex scientific data across various applications, showing significant performance against specialized state-of-the-art methods. 1  \n1 Introduction  \nUnderstanding and modeling the mechanisms underlying the evolution of data is a fundamental scientific challenge and is still largely performed by hand by domain experts, who leverage their understanding of natural phenomena to obtain equations. This process can be timeconsuming, error-prone, and limited by prior knowledge. In this paper, we introduce Mechanistic Neural Networks, a new neural network design that contains one or more Mechanistic Block that explicitly integrate governing equations as symbolic elements in the form of ODE representations. To efficiently train them, we revisit classical results on linear programs (Young, 1961; Rabinowitz, 1968) and develop a GPU-friendly solver. Together, they enable automating the discovery of best-fitting mechanisms from data in an efficient, scalable, and interpretable way.  \nMechanistic Neural Networks present a fundamentally different computing paradigm than standard neural networks that rely on scalar or vector-valued numerical representations as their building block. They are composed of two parts: a mechanistic encoder and a solver. The output of the mechanistic encoder is an explicit symbolic “ODE representation” Ux of  \n1. Source code is available at [https://github.com/alpz/mech-nn](https://github.com/alpz/mech-nn)  \nMechanistic Neural Networks  \n\n|  | Neural ODE,UDE Chen et al. (2018) Rackauckas et al. (2020) | SINDy Brunton et al. (2016) | Neural Operators Li et al. (2020c) | Mech. NN |\n| --- | --- | --- | --- | --- |\n| Linear discovery | – | ✓ | – | ✓ |\n| Nonlinear discovery | – | – | – | ✓ |\n| Physical parameters | ✓ | ✓ | – | ✓ |\n| Forecasting | ✓ | – | ✓ | ✓ |\n| Interpretability | – | ✓ | – | ✓ |\n\nFigure 1: Mechanistic Neural Networks are a new neural network design that learn explicit ODE representations. Mechanistic Blocks can used as bottlenecks in other neural networks to approximate dynamical systems and discover governing equations underlying data. Additional encoders and decoders are optional and depend on the application.  \nthe general form  \nα = fθ(x) (1)  \nUx = F (α, x) . (2)  \nIn more detail, Ux is a family of ordinary differential equations F (α, x) = 0, governed by learnable coefficients α that can be time-dependent or time-independent. Coefficients α are obtained from the mechanistic encoder fθ, and parameters θ are trained to optimally m","cbCaipHbknhVM8hT","https://ap.wps.com/l/cbCaipHbknhVM8hT","pdf",3357464,1,32,"English","en",105,"# Abstract\n# Introduction\n## Mechanistic neural network design and mechanism discovery\n## Mechanistic blocks, encoders, and differentiable solvers\n## NeuRLP for efficient training and parallel ODE solving","[{\"question\":\"What are Mechanistic Neural Networks in scientific machine learning?\",\"answer\":\"They are a neural network design that incorporates Mechanistic Blocks to explicitly learn governing differential equations as symbolic ODE representations.\"},{\"question\":\"How does NeuRLP help training Mechanistic Neural Networks?\",\"answer\":\"NeuRLP is a relaxed linear programming solver that enables differentiable optimization and more efficient learning than traditional sequential ODE solvers, supporting scalable GPU parallel processing.\"},{\"question\":\"What problem do standard ODE solvers pose for this approach?\",\"answer\":\"Sequential numerical solvers like Runge-Kutta can be inefficient for large batches of independent ODEs and may produce noisier gradients due to accumulated errors over long rollouts.\"}]","Mechanistic Neural Networks for Scientific Machine Learning | PDF",1785681729,81,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"mechanistic-neural-networks-for-scientific-machine-learning","",{"@graph":36,"@context":85},[37,54,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/mechanistic-neural-networks-for-scientific-machine-learning/118124/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":62,"encodingFormat":61,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-02",true,{"@type":65,"interactionType":66,"userInteractionCount":4},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What are Mechanistic Neural Networks in scientific machine learning?","Question",{"text":75,"@type":76},"They are a neural network design that incorporates Mechanistic Blocks to explicitly learn governing differential equations as symbolic ODE representations.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"How does NeuRLP help training Mechanistic Neural Networks?",{"text":80,"@type":76},"NeuRLP is a relaxed linear programming solver that enables differentiable optimization and more efficient learning than traditional sequential ODE solvers, supporting scalable GPU parallel processing.",{"name":82,"@type":73,"acceptedAnswer":83},"What problem do standard ODE solvers pose for this approach?",{"text":84,"@type":76},"Sequential numerical solvers like Runge-Kutta can be inefficient for large batches of independent ODEs and may produce noisier gradients due to accumulated errors over long rollouts.","https://schema.org",{"og:url":52,"og:type":87,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":89,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":46,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":46,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":46,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":46,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]