[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-84005-en":3,"doc-seo-84005-105":30,"detail-sidebar-cat-0-en-105":92},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":21,"is_downloadable":21,"audit_status":21,"page_count":22,"language":23,"language_code":24,"site_id":25,"html_lang":24,"table_of_contents":26,"faqs":27,"seo_title":13,"seo_description":14,"update_tm":28,"read_time":29},84005,7971461740909,"Levi","https://ap-avatar.wpscdn.com/davatar_155a257f0dc6eb9ab79c44ca47cae57d",8,"Research & Report","FORGE Towards Functional Tool-Use Generalization via Keypoint Trajectory Reasoning","Functional generalization in robotic tool-use addresses the gap between human ability and robot limitations when tools change but the intended hitting function remains the same. The work formalizes the perception-to-action mismatch: visually similar functional intent does not automatically transfer to action space, where novel tools require different motor patterns. It compares intermediate representations and identifies 2D keypoint trajectories as the best balance of functional expressiveness and action groundability. A two-stage policy, FORGE, predicts keypoint trajectories from action-free data and grounds them with limited demonstrations, outperforming state of the art on a seven-tool benchmark in simulation and real settings, with over 2× average success rate improvements.","arXiv :2607 .05780v 1 [ cs .RO] 7 Jul 2026  \nFORGE: Towards Functional Tool-Use  \nGeneralization via Keypoint Trajectory Reasoning  \nChuhao Zhou 1 , Liquan Wang2 , Shuxin Cao2 , Xiangyu Chen 1 , Yuxuan Hu 1 , Boyu Ma 1 , Animesh Garg2 , Jianfei Yang 1 ,†  \n1 MARS Lab, Nanyang Technological University, 2 Georgia Institute of Technology †Corresponding Author  \nWhile humans readily repurpose a book, a stone, or a shoe to drive a nail, robots trained on specific tools fail to transfer the same function to novel ones – a gap we formalize as functional generalization. Such tools share a common functional intent that is visually recognizable, yet this perceptual similarity does not carry over to action space, where each tool demands an entirely different motor pattern. To bridge this gap, we explore intermediate representations including affordance images, human video prompts, and 2D keypoint trajectories, finding that keypoint trajectories best balance functional expressiveness and action groundability. Building on this, we propose FunctiOnal Reasoning and Grounded Execution (FORGE), a two-stage policy that decouples functional reasoning from action execution: predicting generalizable keypoint trajectories from action-free data, then grounding them into robot actions with limited demonstrations. On a seven-tool hitting-function benchmark, FORGE consistently outperforms state-of-the-art methods on unseen tools in both simulation and the real world, achieving over 2 × improvement in average success rate.  \nCorrespondence: Jianfei Yang at [jianfei.yang@ntu.edu.sg](jianfei.yang@ntu.edu.sg)[ ](jianfei.yang@ntu.edu.sg)Project Page: [https://chuhaozhou99.github.io/FORGE/](https://chuhaozhou99.github.io/FORGE/)  \n1 Introduction  \nIn open-ended real-world environments, the right tool is rarely at hand, yet humans adapt effortlessly, repurposing a book, a stone, or a shoe to drive a nail, because functionally equivalent tools share a common intent: a contact region to strike with and a motion to bring it onto the target. This capacity, which we term functional generalization, is a hallmark of human dexterity that robots have yet to achieve. Current manipulation policies overfit to the appearance and geometry of seen tools, failing entirely when handed a novel one that serves the same function Chen et al. (2025); Turpin et al. (2021); Qin et al. (2023) .  \n\n| (a) Scene Generalization\u003Cbr> |  |  |\n| --- | --- | --- |\n| Table Texture | Background Object | Light Variation |\n\n(b) Category Generalization  \nColors Materials Shapes  \n(c) Functional Generalization  \nNovel Tools and Objects with Different Hitting and Target Points  \nFigure 1 Comparison between functional generalization and existing generalization settings. (a) Scene generalization evaluates robustness to visual variations. (b) Category generalization tests transfer across objects with diverse properties. (c) Functional generalization requires using unseen tools to accomplish the same function.  \nFunctional generalization is fundamentally harder than conventional generalization. As illustrated in Fig. 1 , scene-and category-level generalization only require tolerating visual variations while the underlying motion stays the same Goyal et al. (2023); Shridhar et al. (2023, 2022); Nair et al. (2023) . Functional generalization, by contrast, demands that the motion itself change: striking a target with a novel tool requires locating its contact region, aligning it, and producing an appropriate motion, even when the tool’s shape and trajectory differ entirely from training. The core difficulty is a fundamental mismatch: functionally equivalent tools share recognizable structure in visual space, but this similarity does not transfer to action space, where each tool demands an entirely different motor pattern.  \nBridging this perception-to-action gap requires an intermediate representation that carries functional intent across tools, satisfying two competing demands: expressive enough to captur","cbCaim9Cz7PehdC4","https://ap.wps.com/l/cbCaim9Cz7PehdC4","pdf",2658526,5,1,15,"English","en",105,"# Introduction\n# Functional Generalization and the Perception-to-Action Gap\n# Intermediate Representations for Transferring Functional Intent\n# FORGE Two-Stage Policy\n## Generalizable Keypoint Trajectory Prediction\n## Grounded Execution with Limited Demonstrations\n# Seven-Tool Hitting-Function Benchmark and Results","[{\"question\":\"What does the paper call functional generalization in robotic tool-use?\",\"answer\":\"Functional generalization is the ability to reuse the same intended function when tools change, while still producing the appropriate motion to achieve the task. The paper contrasts this with robots that fail to transfer across novel tools.\"},{\"question\":\"Why does visual similarity of tools not guarantee transferable robot actions?\",\"answer\":\"Functionally equivalent tools can share recognizable structure in visual space, but this similarity does not carry over to action space. Each tool can require a different motor pattern, creating a perception-to-action mismatch.\"},{\"question\":\"How does FORGE work at a high level?\",\"answer\":\"FORGE is a two-stage policy: it first predicts generalizable 2D keypoint trajectories for unseen tools from action-free observations, then grounds those trajectories into executable robot actions using limited action-labeled demonstrations.\"}]",1784191978,38,{"code":4,"msg":31,"data":32},"ok",{"site_id":25,"language":24,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":87,"head_meta":89,"extra_data":91,"updated_unix":28},"forge-towards-functional-tool-use-generalization-via-keypoint-trajectory-reasoning","",{"@graph":36,"@context":86},[37,54,69],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":21},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/forge-towards-functional-tool-use-generalization-via-keypoint-trajectory-reasoning/84005/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":24,"description":14,"dateModified":62,"datePublished":63,"encodingFormat":61,"isAccessibleForFree":64,"interactionStatistic":65},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-07-27","2026-07-16",true,{"@type":66,"interactionType":67,"userInteractionCount":20},"InteractionCounter",{"@type":68},"ViewAction",{"@type":70,"mainEntity":71},"FAQPage",[72,78,82],{"name":73,"@type":74,"acceptedAnswer":75},"What does the paper call functional generalization in robotic tool-use?","Question",{"text":76,"@type":77},"Functional generalization is the ability to reuse the same intended function when tools change, while still producing the appropriate motion to achieve the task. The paper contrasts this with robots that fail to transfer across novel tools.","Answer",{"name":79,"@type":74,"acceptedAnswer":80},"Why does visual similarity of tools not guarantee transferable robot actions?",{"text":81,"@type":77},"Functionally equivalent tools can share recognizable structure in visual space, but this similarity does not carry over to action space. Each tool can require a different motor pattern, creating a perception-to-action mismatch.",{"name":83,"@type":74,"acceptedAnswer":84},"How does FORGE work at a high level?",{"text":85,"@type":77},"FORGE is a two-stage policy: it first predicts generalizable 2D keypoint trajectories for unseen tools from action-free observations, then grounds those trajectories into executable robot actions using limited action-labeled demonstrations.","https://schema.org",{"og:url":52,"og:type":88,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":90,"canonical":52},"index,follow",{"doc_id":7,"site_id":25},{"code":4,"msg":5,"data":93},[94,98,102,106,110,115,120,123,128,131,135],{"id":21,"doc_module":4,"doc_module_name":46,"category_name":95,"show_sort_weight":96,"slug":97},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":99,"show_sort_weight":100,"slug":101},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":103,"show_sort_weight":104,"slug":105},"Exam",70,"exam",{"id":20,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":46,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":46,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":46,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":46,"category_name":137,"show_sort_weight":20,"slug":138},19,"General","general"]