[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-82706-en":3,"doc-seo-82706-105":29,"detail-sidebar-cat-0-en-105":90},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},82706,4810365810221,"Aurora","https://ap-avatar.wpscdn.com/davatar_155a257f0dc6eb9ab79c44ca47cae57d",8,"Research & Report","Optimus: A Generic Operator-Level PyTorch Model Transformation Framework","Large-scale recommendation and ranking systems rely on deep learning models with complex, diverse architectures that are difficult to manually optimize. Graph-based optimization is an industry standard, yet PyTorch FX transformations depend on human-engineered module-level rewrite rules that do not scale. Optimus introduces a general operator-level model transformation framework integrated into PyTorch 2.x, using concise predefined patterns and a greedy pattern-matching and replacement search while preserving semantics. Evaluations show up to 63% speedup and reduced peak memory and compile time, and the framework is open-sourced.","Optimus: A Generic Operator-Level PyTorch Model Transformation Framework  \narXiv :2607 .02945v 1 [ cs .PF] 3 Jul 2026  \nMenglu Yu  \nMeta Bellevue, USA [mengluy@meta.com](mengluy@meta.com)  \nYanbo Liang  \nMeta Menlo Park, USA [ybliang@meta.com](ybliang@meta.com)  \nJason Ansel  \nMeta Menlo Park, USA [jansel@meta.com](jansel@meta.com)  \nBrian Hirsh  \nMeta New York, USA [hirsheybar@meta.com](hirsheybar@meta.com)  \nOguz Ulgen  \nMeta Menlo Park, USA [oulgen@meta.com](oulgen@meta.com)  \nHuaqing Xiong  \nMeta New York, USA [huaqingxiong@meta.com](huaqingxiong@meta.com)  \nJunqing Zhou  \nMeta Sunnyvale, USA [junqingz@meta.com](junqingz@meta.com)  \nJiaqi Xu  \nMeta Sunnyvale, USA [jackiexu0313@meta.com](jackiexu0313@meta.com)  \nJia Liu  \nThe Ohio State University Columbus, USA [liu@ece.osu.edu](liu@ece.osu.edu)  \nElias Ellison  \nMeta New York, USA [eellison@meta.com](eellison@meta.com)  \nJia Chen Ren  \nMeta Menlo Park, USA [bobren@meta.com](bobren@meta.com)  \nXu Zhao  \nMeta Toronto, Canada [xzhao9@meta.com](xzhao9@meta.com)  \nQuanyu Zhu  \nMeta Sunnyvale, USA [qyz@meta.com](qyz@meta.com)  \nRuilin Chen  \nMeta Bellevue, USA [ruilinchen@meta.com](ruilinchen@meta.com)  \nChi-Keung Luk  \nMeta Sunnyvale, USA [kluk@meta.com](kluk@meta.com)  \nYuzhen Huang  \nMeta Sunnyvale, USA [yuzhenhuang@meta.com](yuzhenhuang@meta.com)  \nShuai Yang  \nMeta Sunnyvale, USA [shuaiyang@meta.com](shuaiyang@meta.com)  \nEdward Yang  \nMeta New York, USA [ezyang@meta.com](ezyang@meta.com)  \nWill Feng  \nMeta Menlo Park, USA [willfeng@meta.com](willfeng@meta.com)  \nDaohang Shi  \nMeta Bellevue, USA [daohang@meta.com](daohang@meta.com)  \nMingming Ding  \nMeta Bellevue, USA [midin@meta.com](midin@meta.com)  \nYuhang Yang  \nMeta Bellevue, USA [yuhangyang@meta.com](yuhangyang@meta.com)  \nAbstract  \nIn large-scale industrial applications, deep learning models that power recommendation and ranking have complex and diverse model architectures. These models are continuously developed and  \nThis work is licensed under a Creative Commons Attribution 4 .0 International License. KDD’26, Jeju Island, Republic of Korea  \n© 2026 Copyright held by the owner/author(s) .  \nACM ISBN 979-8-4007-2259-2/2026/08  \n[https://doi.org/10.1145/3770855.3818458](https://doi.org/10.1145/3770855.3818458)  \nrefined by large teams of machine learning engineers, rendering manual optimization infeasible. Consequently, graph-based optimization techniques have become an industry standard for boosting performance, with PyTorch FX transformations leading the charge. These transformations typically rely on a set of human-engineered module-level rewrite rules which are not scalable to diverse model architectures. To address this limitation, we introduce Optimus, a general-purpose model transformation framework built in the PyTorch 2.x (PT2) machine learning compiler. With a concise set of predefined patterns, Optimus applies an efficient greedy search  \nalgorithm for pattern matching and replacement, while preserving model semantic. It is designed and implemented as a highly customizable and extensible framework integrated into the PT2 stack. Our evaluation shows that the framework can achieve up to 63% speedup, 6% peak memory reduction, and over 400 second compile time decrease for our industry-scale recommendation models compared to baselines. Optimus is open-sourced together with PyTorch 2.x as a customizable model transformation layer.  \nCCS Concepts  \n• Software and its engineering → Software performance; Justin-time compilers; • Computing methodologies → Neural networks.  \nKeywords  \nPyTorch model transformation, recommendation systems, kernel optimization  \nACM Reference Format:  \nMenglu Yu, Jiaqi Xu, Yuzhen Huang, Yanbo Liang, Jia Liu, Shuai Yang, Jason Ansel, Elias Ellison, Edward Yang, Brian Hirsh, Jia Chen Ren, Will Feng, Oguz Ulgen, Xu Zhao, Daohang Shi, Huaqing Xiong, Quanyu Zhu, Mingming Ding, Junqing Zhou, Ruilin Chen, Yuhang Yang, and Chi-Keung Luk. 2026. Optimus: A Generic Operator-Level PyTorch Model T","cbCaihxkZyzpRpvP","https://ap.wps.com/l/cbCaihxkZyzpRpvP","pdf",4021850,1,12,"English","en",105,"# Introduction\n## Motivation for graph transformations\n## PyTorch 2.x and symbolic tracing\n# Optimus Framework\n## Pattern-based operator rewriting\n## Greedy pattern matching and semantic preservation\n# Evaluation Results\n## Speedup and memory reduction\n## Compile-time improvements\n# Availability","[{\"question\":\"Why are traditional module-level PyTorch FX rewrite rules not scalable for diverse model architectures?\",\"answer\":\"They require extensive human-engineered module-level rewrite rules, which become impractical when models vary widely in architecture and structure.\"},{\"question\":\"What is Optimus, and how does it fit into PyTorch 2.x?\",\"answer\":\"Optimus is an operator-level model transformation framework integrated into the PyTorch 2.x compiler stack, using predefined patterns and an efficient greedy search for matching and replacement.\"},{\"question\":\"What performance improvements does Optimus report on industry-scale recommendation models?\",\"answer\":\"The paper reports up to 63% speedup, around 6% peak memory reduction, and a reduction of over 400 seconds in compile time compared with baselines.\"}]",1784182407,30,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":85,"head_meta":87,"extra_data":89,"updated_unix":27},"optimus-a-generic-operator-level-pytorch-model-transformation-framework","",{"@graph":35,"@context":84},[36,53,67],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/optimus-a-generic-operator-level-pytorch-model-transformation-framework/82706/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":61,"encodingFormat":60,"isAccessibleForFree":62,"interactionStatistic":63},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-16",true,{"@type":64,"interactionType":65,"userInteractionCount":4},"InteractionCounter",{"@type":66},"ViewAction",{"@type":68,"mainEntity":69},"FAQPage",[70,76,80],{"name":71,"@type":72,"acceptedAnswer":73},"Why are traditional module-level PyTorch FX rewrite rules not scalable for diverse model architectures?","Question",{"text":74,"@type":75},"They require extensive human-engineered module-level rewrite rules, which become impractical when models vary widely in architecture and structure.","Answer",{"name":77,"@type":72,"acceptedAnswer":78},"What is Optimus, and how does it fit into PyTorch 2.x?",{"text":79,"@type":75},"Optimus is an operator-level model transformation framework integrated into the PyTorch 2.x compiler stack, using predefined patterns and an efficient greedy search for matching and replacement.",{"name":81,"@type":72,"acceptedAnswer":82},"What performance improvements does Optimus report on industry-scale recommendation models?",{"text":83,"@type":75},"The paper reports up to 63% speedup, around 6% peak memory reduction, and a reduction of over 400 seconds in compile time compared with baselines.","https://schema.org",{"og:url":51,"og:type":86,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":88,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":91},[92,96,100,104,109,114,119,121,126,129,133],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":93,"show_sort_weight":94,"slug":95},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":97,"show_sort_weight":98,"slug":99},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":101,"show_sort_weight":102,"slug":103},"Exam",70,"exam",{"id":105,"doc_module":4,"doc_module_name":45,"category_name":106,"show_sort_weight":107,"slug":108},5,"Comic",60,"comic",{"id":110,"doc_module":4,"doc_module_name":45,"category_name":111,"show_sort_weight":112,"slug":113},6,"Technology",50,"technology",{"id":115,"doc_module":4,"doc_module_name":45,"category_name":116,"show_sort_weight":117,"slug":118},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":28,"slug":120},"research-report",{"id":122,"doc_module":4,"doc_module_name":45,"category_name":123,"show_sort_weight":124,"slug":125},9,"Religion & Spirituality",20,"religion-spirituality",{"id":124,"doc_module":4,"doc_module_name":45,"category_name":127,"show_sort_weight":124,"slug":128},"World Cup","world-cup",{"id":130,"doc_module":4,"doc_module_name":45,"category_name":131,"show_sort_weight":130,"slug":132},10,"Lifestyle","lifestyle",{"id":134,"doc_module":4,"doc_module_name":45,"category_name":135,"show_sort_weight":105,"slug":136},19,"General","general"]