[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-119493-en":3,"doc-seo-119493-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},119493,962075114101,"Seraphina","https://ap-avatar.wpscdn.com/avatar/e000253a75eb197efd?x-image-process=image/resize,m_fixed,w_180,h_180&k=1780044092746381165",8,"Research & Report","Improving Scientific Machine Learning with Algorithmic Insights from Numerical Analysis","Scientific machine learning supports approximate, efficient solutions for difficult scientific computing tasks, including simulating physical phenomena and inferring high-dimensional parameters from data. Classical numerical analysis algorithms provide principled computational methods, yet they are not designed for learning from large datasets and can be costly under strict accuracy requirements. Neural network–based approaches can leverage abundant data, but general architectures struggle with ill-conditioned scientific problems that demand higher fidelity. This dissertation applies numerical analysis insights to develop learning methods and fast, accurate PDE solvers for inverse problems in challenging data regimes, including scarce-training scenarios and imaging applications.","THE UNIVERSITY OF CHICAGO  \nIMPROVING SCIENTIFIC MACHINE LEARNING WITH ALGORITHMIC INSIGHTS  \nFROM NUMERICAL ANALYSIS  \nA DISSERTATION SUBMITTED TO  \nTHE FACULTY OF THE DIVISION OF THE PHYSICAL SCIENCES  \nIN CANDIDACY FOR THE DEGREE OF  \nDOCTOR OF PHILOSOPHY  \nDEPARTMENT OF COMPUTER SCIENCE  \nBY  \nOWEN JAMES MELIA  \nCHICAGO, ILLINOIS  \nAUGUST 2025  \nCopyright © 2025 by Owen James Melia  \nABSTRACT  \nScientific machine learning offers methods of building approximate but efficient solutions to challenging tasks in scientific computing, such as simulating a physical phenomenon or inferring high-dimensional model parameters from data. Algorithms from numerical analysis provide classical computational solutions, but these algorithms are not designed to learn from large datasets, and they can be expensive to run for certain problems and levels of error tolerance. Machine learning solutions offer approximations to these classical methods, often using neural networks, which can learn from large datasets. While general-purpose neural network architectures and training algorithms are frequently a natural first choice for such problems, scientific computing tasks are often ill-conditioned and require a level of accuracy unattainable by general-purpose methods. At the same time, numerical analysis research provides a wealth of insights for principled methods of computing solutions to these problems.  \nThis thesis develops state-of-the-art machine learning methods for scientific computing problems by applying insights from contemporary algorithms research in numerical analysis. We consider three different problem settings in scientific computing. The first example uses insights from applied harmonic analysis to derive rotation-invariant random feature models, which provide an attractive alternative to deep neural networks or hand-designed kernels. In another setting, insights from optimization theory and recursive linearization algorithms allow us to design simple yet accurate neural networks for multi-frequency inverse scattering problems in a highly nonlinear regime, when training data is available. In the final setting, this thesis considers a broader class of partial differential equation (PDE)-based inverse problems, in a setting where training data is scarce. In this setting, the thesis contributes develops software and algorithms for accelerating highly-efficient and highly-accurate fast direct solvers of elliptic PDEs on hardware acceleration devices. The final chapter of this thesis develops optimization methods applying these novel algorithm and software tools to  \nmultiple PDE-based inverse problems in imaging.  \nTABLE OF CONTENTS  \nABSTRACT ........................................ iii  \nLIST OF FIGURES .................................... ix  \nLIST OF TABLES ..................................... xi  \nPREFACE ......................................... xii  \nACKNOWLEDGMENTS ................................. xiii  \n1 INTRODUCTION ................................... 1  \n1.1 Theme 1: Developing Machine Learning Methods with Insights from Numerical Analysis .................................... 2  \n1.2 Theme 2: Developing Numerical PDE Solvers for use in Deep Learning Settings 4  \n1.3 Theme 3: Building a Computational Toolbox for PDE-Based Inverse Problems in Scientific Imaging ............................... 5  \n1.4 Summary ..................................... 8  \n2 ROTATION-INVARIANT RANDOM FEATURES PROVIDE A STRONG BASELINE FOR MACHINE LEARNING ON POINT CLOUDS ............. 9  \n2.1 Introduction .................................... 9  \n2.1.1 Contributions ............................... 12  \n2.2 Related Work ................................... 13  \n2.3 Rotational Invariance and Spherical Harmonics ................ 18  \n2.3.1 Spherical Harmonics ........................... 19  \n2.4 Rotation-Invariant Random Features ...................... 20  \n2.4.1 Evaluating the Random Features .................... 21  \n2.5 Experiments ...............................","cbCaim5jlVdyQSlR","https://ap.wps.com/l/cbCaim5jlVdyQSlR","pdf",11431237,1,209,"English","en",105,"# Abstract\n# Introduction\n## Theme 1: Developing Machine Learning Methods with Insights from Numerical Analysis\n## Theme 2: Developing Numerical PDE Solvers for use in Deep Learning Settings\n## Theme 3: Building a Computational Toolbox for PDE-Based Inverse Problems in Scientific Imaging\n# Rotation-Invariant Random Features Provide a Strong Baseline for Machine Learning on Point Clouds\n## Introduction\n## Contributions\n## Rotational Invariance and Spherical Harmonics\n## Rotation-Invariant Random Features\n## Evaluating the Random Features\n## Experiments\n# Multi-Frequency Progressive Refinement for Learned Inverse Scattering\n## Introduction\n## Problem Setup and Notation\n## Background and Related Work\n## Recursive Linearization and Our Method\n## Experiments\n# Hardware Acceleration for HPS Algorithms in Two and Three Dimensions","[{\"question\":\"What problem does the dissertation address in scientific machine learning?\",\"answer\":\"It tackles the gap between classical numerical analysis algorithms, which are principled but expensive and not data-driven, and neural learning methods, which may be insufficiently accurate for ill-conditioned scientific computing tasks.\"},{\"question\":\"How does the thesis incorporate insights from numerical analysis?\",\"answer\":\"It derives learning models and algorithms using research perspectives from areas such as applied harmonic analysis, optimization theory, and recursive linearization methods, aligning them with the structure of scientific computing problems.\"},{\"question\":\"What are the main application settings and technical contributions?\",\"answer\":\"The work covers rotation-invariant random feature models for point clouds, learned multi-frequency inverse scattering in nonlinear regimes, and PDE-based inverse problems with scarce training data, including accelerated fast direct solvers on hardware acceleration devices.\"}]","Improving Scientific Machine Learning with Algorithmic Insights from Numerical Analysis | PDF",1785724604,527,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"improving-scientific-machine-learning-with-algorithmic-insights-from-numerical-analysis","",{"@graph":36,"@context":85},[37,54,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/improving-scientific-machine-learning-with-algorithmic-insights-from-numerical-analysis/119493/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":62,"encodingFormat":61,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-03",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What problem does the dissertation address in scientific machine learning?","Question",{"text":75,"@type":76},"It tackles the gap between classical numerical analysis algorithms, which are principled but expensive and not data-driven, and neural learning methods, which may be insufficiently accurate for ill-conditioned scientific computing tasks.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"How does the thesis incorporate insights from numerical analysis?",{"text":80,"@type":76},"It derives learning models and algorithms using research perspectives from areas such as applied harmonic analysis, optimization theory, and recursive linearization methods, aligning them with the structure of scientific computing problems.",{"name":82,"@type":73,"acceptedAnswer":83},"What are the main application settings and technical contributions?",{"text":84,"@type":76},"The work covers rotation-invariant random feature models for point clouds, learned multi-frequency inverse scattering in nonlinear regimes, and PDE-based inverse problems with scarce training data, including accelerated fast direct solvers on hardware acceleration devices.","https://schema.org",{"og:url":52,"og:type":87,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":89,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":46,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":46,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":46,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":46,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]