[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-81907-en":3,"doc-seo-81907-105":31,"detail-sidebar-cat-0-en-105":84},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":21,"is_downloadable":21,"audit_status":21,"page_count":22,"language":23,"language_code":24,"site_id":25,"html_lang":24,"table_of_contents":26,"faqs":27,"seo_title":28,"seo_description":14,"update_tm":29,"read_time":30},81907,8796095462418,"Noah","https://ap-avatar.wpscdn.com/avatar/80000253c1241d02b47?x-image-process=image/resize,m_fixed,w_180,h_180&k=1778826106357471780",8,"Research & Report","URSA: Chemistry-Aware Benchmark for Utilitarian Retrosynthesis Assessment","Synthesis planning for drug discovery seeks reaction pathways to produce target molecules, yet objective evaluation is hindered by rigid benchmarks lacking chemical interpretability. URSA (Utilitarian RetroSynthesis Assessment) provides a unified evaluation framework that scores routes not only by stock convergence to available starting materials, but also by chemical plausibility at the level of each reaction step. The benchmark covers end-to-end retrosynthesis systems and large language models on novel, undisclosed-route targets from realistic daily design workflows, showing LLMs achieve high-level strategy while specialized models solve more reliably.","arXiv :2607 .04688v 1 [ cs .LG] 6 Jul 2026  \nURSA: Chemistry-Aware Benchmark for Utilitarian Retrosynthesis Assessment  \nBogdan Zagribelnyy 1 ∗ Ivan Ilin 1 Nikita Bondarev 1 Anton Morgunov2 Arkadii Lin 1 Maksim Kuznetsov3 Rim Shayakhmetov 1 Vladimir Aladinskiy 1 Alex Aliper 1 Alex Zhavoronkov 1 ,3 ,4  \n1Insilico Medicine AI Limited, Masdar City, Abu Dhabi, UAE  \n2Independent researcher  \n3Insilico Medicine Canada Inc., Montreal, Quebec, Canada  \n4Insilico Medicine Hong Kong Ltd., Hong Kong SAR, China  \nAbstract  \nSynthesis planning aiming to find pathways of reactions for a target molecule isone of the most important and challenging tasks in drug discovery. Recent progress has produced both specialized deep-learning retrosynthesis systems and generalpurpose large language models, but objective comparison remains difficult due to the lack of flexible, chemically interpretable benchmarking protocols. In the current study, we are introducing the URSA (Utilitarian RetroSynthesis Assessment) evaluation framework that provides the opportunity to benchmark the synthetic routes not only from a formal perspective, such as convergence to commercially available starting materials, but also from a chemical plausibility perspective, mimicking the way expert chemists evaluate the reactions and routes. The study covers a comprehensive evaluation of both conventional end-to-end retrosynthesis solutionsand LLMs for the synthesis planning task on a set of novel, diverse target molecules with undisclosed synthetic routes, which represent realistic tasks in the daily drug design routine. We find that while LLMs can support high-level strategic planning, they currently underperform specialized retrosynthesis models in reliably solving synthesis planning tasks.  \n1 Introduction  \nComputer-Aided/Assisted Synthesis Planning (CASP) aims to design synthetic pathways to target molecules, a long-standing grand challenge in organic chemistry [1] . Since Corey and Wipke’s early logic-centered systems [2], most CASP methods have treated retrosynthesis as backward search over precursor trees: single-step models propose disconnections, and planners search for routes that terminate in available starting materials. Modern neural predictors and search algorithms [3–10] have greatly improved this navigability problem, but the dominant evaluation metric still asks a narrow question: \"Did the planner find any path to stock?\"  \nThis metric, often reported as solvability, has limited chemical meaning. Its value depends strongly on the definition of the starting-material stock, which varies from realistic commercial inventories to expansive virtual libraries. More importantly, stock termination does not imply that the individual transformations in the route are chemically plausible. Recent evaluations have shown that stockterminated routes can contain chemically invalid transformations despite satisfying the formal search objective [11, 12] . Thus, high stock-termination scores can reward graph connectivity while leaving unresolved the question a chemist actually cares about: \"Is this route chemically sound?\"  \n∗bogdan@insilicomedicine.com Preprint.  \nRoute-reproduction benchmarks such as PaRoutes provide a stronger signal by testing whether models recover known experimental syntheses [13] . However, exact reproduction is conservative: it cannot reward novel but plausible routes that differ from the reference synthesis. This leaves a measurement gap between two incomplete criteria. Stock termination can accept implausible novelty; route reproduction can reject plausible novelty. Recent work formalizes this distinction through the Solv-N[task] hierarchy, which separates syntactic (N = 0) and topological validity (N = 1) from higher-order requirements such as selectivity (N = 2) and experimental executability (N = 3) and from user-defined constraints [task] such as, e.g., termination to a specified stock of starting materials (SM), maximum allowed route length or restri","cbCaidOv1braaZMe","https://ap.wps.com/l/cbCaidOv1braaZMe","pdf",3211828,4,1,33,"English","en",105,"# Abstract\n# Introduction\n# Related Work\n## Benchmarking tools for synthesis planning","[{\"question\":\"What do the results suggest about LLMs versus specialized retrosynthesis models?\",\"answer\":\"LLMs can support high-level strategic planning, but they underperform specialized retrosynthesis systems in reliably solving synthesis planning tasks under URSA’s route-level plausibility protocol.\"}]","URSA: Chemistry-Aware Benchmark for Utilitarian Retrosynthesis Assessment | PDF",1784176984,83,{"code":4,"msg":32,"data":33},"ok",{"site_id":25,"language":24,"slug":34,"title":13,"keywords":35,"description":14,"schema_data":36,"social_meta":79,"head_meta":81,"extra_data":83,"updated_unix":29},"ursa-chemistry-aware-benchmark-for-utilitarian-retrosynthesis-assessment","",{"@graph":37,"@context":78},[38,54,69],{"@type":39,"itemListElement":40},"BreadcrumbList",[41,45,49,52],{"item":42,"name":43,"@type":44,"position":21},"https://docshare.wps.com","Home","ListItem",{"item":46,"name":47,"@type":44,"position":48},"https://docshare.wps.com/document/","Document",2,{"item":50,"name":12,"@type":44,"position":51},"https://docshare.wps.com/document/research-report/",3,{"item":53,"name":13,"@type":44,"position":20},"https://docshare.wps.com/document/ursa-chemistry-aware-benchmark-for-utilitarian-retrosynthesis-assessment/81907/",{"url":53,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":24,"description":14,"dateModified":62,"datePublished":63,"encodingFormat":61,"isAccessibleForFree":64,"interactionStatistic":65},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":42,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-07-29","2026-07-16",true,{"@type":66,"interactionType":67,"userInteractionCount":20},"InteractionCounter",{"@type":68},"ViewAction",{"@type":70,"mainEntity":71},"FAQPage",[72],{"name":73,"@type":74,"acceptedAnswer":75},"What do the results suggest about LLMs versus specialized retrosynthesis models?","Question",{"text":76,"@type":77},"LLMs can support high-level strategic planning, but they underperform specialized retrosynthesis systems in reliably solving synthesis planning tasks under URSA’s route-level plausibility protocol.","Answer","https://schema.org",{"og:url":53,"og:type":80,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":82,"canonical":53},"index,follow",{"doc_id":7,"site_id":25},{"code":4,"msg":5,"data":85},[86,90,94,98,103,108,113,116,121,124,128],{"id":21,"doc_module":4,"doc_module_name":47,"category_name":87,"show_sort_weight":88,"slug":89},"Story & Novel",90,"story-novel",{"id":48,"doc_module":4,"doc_module_name":47,"category_name":91,"show_sort_weight":92,"slug":93},"Literature",80,"literature",{"id":20,"doc_module":4,"doc_module_name":47,"category_name":95,"show_sort_weight":96,"slug":97},"Exam",70,"exam",{"id":99,"doc_module":4,"doc_module_name":47,"category_name":100,"show_sort_weight":101,"slug":102},5,"Comic",60,"comic",{"id":104,"doc_module":4,"doc_module_name":47,"category_name":105,"show_sort_weight":106,"slug":107},6,"Technology",50,"technology",{"id":109,"doc_module":4,"doc_module_name":47,"category_name":110,"show_sort_weight":111,"slug":112},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":47,"category_name":12,"show_sort_weight":114,"slug":115},30,"research-report",{"id":117,"doc_module":4,"doc_module_name":47,"category_name":118,"show_sort_weight":119,"slug":120},9,"Religion & Spirituality",20,"religion-spirituality",{"id":119,"doc_module":4,"doc_module_name":47,"category_name":122,"show_sort_weight":119,"slug":123},"World Cup","world-cup",{"id":125,"doc_module":4,"doc_module_name":47,"category_name":126,"show_sort_weight":125,"slug":127},10,"Lifestyle","lifestyle",{"id":129,"doc_module":4,"doc_module_name":47,"category_name":130,"show_sort_weight":99,"slug":131},19,"General","general"]