[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-seo-327422-105":3,"detail-sidebar-cat-0-en-105":81,"doc-detail-327422-en":130},{"code":4,"msg":5,"data":6},0,"ok",{"site_id":7,"language":8,"slug":9,"title":10,"keywords":11,"description":12,"schema_data":13,"social_meta":74,"head_meta":76,"extra_data":78,"updated_unix":80},105,"en","coding-agents-can-replicate-scientific-machine-learning-papers-research-workflow-for-paper-claim-replication","Coding-agents can replicate scientific machine learning papers - Research workflow for paper claim replication","","Scientific machine learning papers often assert tight numerical or probabilistic guarantees, yet paper-only material is insufficient to ensure reliable replication. This work introduces “Paper-replication,” a workflow that binds each selected claim to recorded evidence, reconstructs methods, runs computational experiments, and links outputs to provenance and claim comparisons. Implemented as a coding-agent skill, it validates completion via workspace evidence and checks. Across twelve runs on four papers, all workspaces complete, with every recorded target matched in the replication reports.",{"@graph":14,"@context":73},[15,34,56],{"@type":16,"itemListElement":17},"BreadcrumbList",[18,23,27,31],{"item":19,"name":20,"@type":21,"position":22},"https://docshare.wps.com","Home","ListItem",1,{"item":24,"name":25,"@type":21,"position":26},"https://docshare.wps.com/document/","Document",2,{"item":28,"name":29,"@type":21,"position":30},"https://docshare.wps.com/document/research-report/","Research & Report",3,{"item":32,"name":10,"@type":21,"position":33},"https://docshare.wps.com/document/coding-agents-can-replicate-scientific-machine-learning-papers-research-workflow-for-paper-claim-replication/327422/",4,{"url":32,"name":10,"@type":35,"image":36,"author":41,"headline":10,"publisher":44,"fileFormat":47,"inLanguage":8,"description":12,"dateModified":48,"datePublished":49,"encodingFormat":47,"isAccessibleForFree":50,"interactionStatistic":51},"DigitalDocument",{"url":37,"@type":38,"width":39,"height":40},"https://docshare.wps.com/thumbnails/coding-agents-can-replicate-scientific-machine-learning-papers-research-workflow-for-paper-claim-replication/327422.png","ImageObject",300,407,{"name":42,"@type":43},"Eliana","Person",{"url":19,"name":45,"@type":46},"DocShare","Organization","application/pdf","2026-10-30","2026-09-21",true,{"@type":52,"interactionType":53,"userInteractionCount":55},"InteractionCounter",{"@type":54},"ViewAction",10,{"@type":57,"mainEntity":58},"FAQPage",[59,65,69],{"name":60,"@type":61,"acceptedAnswer":62},"Why is replicating scientific machine learning papers from paper materials alone difficult?","Question",{"text":63,"@type":64},"Paper-only records may omit training seeds, sampled datasets, solver and preprocessing conventions, stochastic training details, and plotting rules, all of which can change results even when the algorithm description is fixed.","Answer",{"name":66,"@type":61,"acceptedAnswer":67},"What is the Paper-replication workflow?",{"text":68,"@type":64},"Paper-replication selects paper claims as targets, records evidence for each target, reconstructs the paper’s method, runs computational experiments, links outputs to provenance and comparisons with the claims, and records matched evidence in a replication report.",{"name":70,"@type":61,"acceptedAnswer":71},"How does completion and validation work in Paper-replication?",{"text":72,"@type":64},"Completion depends on workspace evidence and validation checks rather than on the agent’s final message, ensuring that reported success reflects verified, claim-supported evidence.","https://schema.org",{"og:url":32,"og:type":75,"og:title":10,"og:site_name":45,"og:description":12},"article",{"robots":77,"canonical":32},"index,follow",{"doc_id":79,"site_id":7},327422,1791207495,{"code":4,"msg":82,"data":83},"success",[84,88,92,96,101,106,111,115,120,123,126],{"id":22,"doc_module":4,"doc_module_name":25,"category_name":85,"show_sort_weight":86,"slug":87},"Story & Novel",90,"story-novel",{"id":26,"doc_module":4,"doc_module_name":25,"category_name":89,"show_sort_weight":90,"slug":91},"Literature",80,"literature",{"id":33,"doc_module":4,"doc_module_name":25,"category_name":93,"show_sort_weight":94,"slug":95},"Exam",70,"exam",{"id":97,"doc_module":4,"doc_module_name":25,"category_name":98,"show_sort_weight":99,"slug":100},5,"Comic",60,"comic",{"id":102,"doc_module":4,"doc_module_name":25,"category_name":103,"show_sort_weight":104,"slug":105},6,"Technology",50,"technology",{"id":107,"doc_module":4,"doc_module_name":25,"category_name":108,"show_sort_weight":109,"slug":110},7,"Healthcare",40,"healthcare",{"id":112,"doc_module":4,"doc_module_name":25,"category_name":29,"show_sort_weight":113,"slug":114},8,30,"research-report",{"id":116,"doc_module":4,"doc_module_name":25,"category_name":117,"show_sort_weight":118,"slug":119},9,"Religion & Spirituality",20,"religion-spirituality",{"id":118,"doc_module":4,"doc_module_name":25,"category_name":121,"show_sort_weight":118,"slug":122},"World Cup","world-cup",{"id":55,"doc_module":4,"doc_module_name":25,"category_name":124,"show_sort_weight":55,"slug":125},"Lifestyle","lifestyle",{"id":127,"doc_module":4,"doc_module_name":25,"category_name":128,"show_sort_weight":97,"slug":129},19,"General","general",{"code":4,"msg":82,"data":131},{"doc_id":79,"user_id":132,"nickname":42,"user_avatar":133,"doc_module":4,"category_id":112,"category_name":29,"doc_title":10,"doc_description":12,"doc_content":134,"file_id":135,"file_url":136,"file_type":137,"file_size":138,"view_count":55,"is_deleted":4,"is_public":22,"is_downloadable":22,"audit_status":22,"page_count":139,"language":140,"language_code":8,"site_id":7,"html_lang":8,"table_of_contents":141,"faqs":142,"seo_title":143,"seo_description":12,"update_tm":144,"read_time":109,"preview_page_count":4},4398048949847,"https://ap-avatar.wpscdn.com/avatar/400002536579ef2da7f?_k=1778318612642679267","Coding-agents can replicate scientific machine learning papers  \nAtharva Hansa,∗ and Ilias Bilionisa  \na School of Mechanical Engineering, Purdue University, West Lafayette, IN, 47907, USA  \nAbstract  \nScientific machine learning papers typically make computational claims, e.g., that the relative mean square error is less than 5% or that the 95% predictive credible interval covers the test data. A coding agent can be prompted to replicate those claims from paper materials alone, but the prompt does not by itself reliably preserve progress or check whether generated evidence supports the paper’s claims. We introduce Paper-replication, a workflow that makes each selected paper claim a target with recorded evidence, and implement it as a coding-agent skill. The workflow makes the agent record those targets, reconstruct the paper’s method, run computational experiments, link generated outputs to provenance and comparisons with the paper’s claims, record where matched evidence appears in the replication report, and pass validation checks before completion. We evaluate Paper-replication on twelve independent runs across four scientific machine learning papers. All twelve workspaces pass the completion gate, and all 158 recorded targets are matched with report coverage. Even in this completed workspace state, repeated runs differ in how papers are divided into targets, in numerical fidelity to the source papers, in elapsed replication time, in the number of intermediate executions replaced before final evidence is accepted, and in the rules used to accept evidence. Paper-replication makes completion depend on workspace evidence and validation checks rather than on the agent’s final message.  \nKeywords: Coding agents, Harness engineering, Paper replication, Scientific reproducibility, Scientific machine learning  \n1. Introduction  \nComputational claims of scientific machine learning papers depend on differentiable solvers and neural differential equations (Chen et al., 2018; Rackauckas et al., 2020; Hans et al., 2023a, 2024), physics-informed neural networks and physicsinformed learning objectives (Raissi et al., 2019; Karniadakiset al., 2021), sparse discovery of dynamical systems and partial differential equations (Brunton et al., 2016; Rudy et al., 2017), and Bayesian or sampling-based inverse-problem methods (Stuart, 2010; Hans et al., 2020, 2023b; Alberts and Bilionis, 2023) . A reader can inspect the equations and algorithms in the text, but the reported results also depend on code, data, numerical solvers, preprocessing, stochastic training, and implementation choices (Hans, 2024; Hans et al., 2025) . Reproducible-research practice therefore treats code, data, computational environments, and the record of how each result was produced as part of the scientific record (Peng, 2011; Stodden et al., 2016) . It also emphasizes versioned project organization (Wilson et al., 2017), ex-  \nrecording evidence for each claim. Replicating a paper from paper materials alone is a narrower and harder task than rerunning a released package. The available materials may include the LATEX source, figures, tables, appendices, and data references, but they need not include sampled training sets, seeds, optimizer states, sampler states, preprocessing conventions, or plotting rules. In machine learning, reproducibility taxonomies treat a paper-only record as weaker evidence than a runnable package with code, data, dependencies, and environment information (Tatman et al., 2018) . Surveys of artificial-intelligence papers also find that empirical details are often underreported (Gundersen and Kjensmo, 2018) . Stochastic training makes these omissions matter because data sampling, initialization, implementation choices, and hyperparameters can change the result even when the algorithmic description stays fixed (Henderson et al., 2018; Bouthillier et al., 2021; Pineau et al., 2021) .  \nResearch code generation methods address one part of paper replication","cbCaihWAG1f9QHy7","https://ap.wps.com/l/cbCaihWAG1f9QHy7","pdf",388341,16,"English","# Introduction\n## Background on reproducibility in scientific machine learning\n## Paper replication versus rerunning released packages\n## Code generation and paper reproduction benchmarks","[{\"question\":\"Why is replicating scientific machine learning papers from paper materials alone difficult?\",\"answer\":\"Paper-only records may omit training seeds, sampled datasets, solver and preprocessing conventions, stochastic training details, and plotting rules, all of which can change results even when the algorithm description is fixed.\"},{\"question\":\"What is the Paper-replication workflow?\",\"answer\":\"Paper-replication selects paper claims as targets, records evidence for each target, reconstructs the paper’s method, runs computational experiments, links outputs to provenance and comparisons with the claims, and records matched evidence in a replication report.\"},{\"question\":\"How does completion and validation work in Paper-replication?\",\"answer\":\"Completion depends on workspace evidence and validation checks rather than on the agent’s final message, ensuring that reported success reflects verified, claim-supported evidence.\"}]","Coding-agents can replicate scientific machine learning papers - Research workflow for paper claim replication | PDF",1789977169]