[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"detail-sidebar-cat-0-en-105":3,"doc-seo-315100-105":59,"doc-detail-315100-en":130},{"code":4,"msg":5,"data":6},0,"success",[7,13,18,23,28,33,38,43,48,51,55],{"id":8,"doc_module":4,"doc_module_name":9,"category_name":10,"show_sort_weight":11,"slug":12},1,"Document","Story & Novel",90,"story-novel",{"id":14,"doc_module":4,"doc_module_name":9,"category_name":15,"show_sort_weight":16,"slug":17},2,"Literature",80,"literature",{"id":19,"doc_module":4,"doc_module_name":9,"category_name":20,"show_sort_weight":21,"slug":22},4,"Exam",70,"exam",{"id":24,"doc_module":4,"doc_module_name":9,"category_name":25,"show_sort_weight":26,"slug":27},5,"Comic",60,"comic",{"id":29,"doc_module":4,"doc_module_name":9,"category_name":30,"show_sort_weight":31,"slug":32},6,"Technology",50,"technology",{"id":34,"doc_module":4,"doc_module_name":9,"category_name":35,"show_sort_weight":36,"slug":37},7,"Healthcare",40,"healthcare",{"id":39,"doc_module":4,"doc_module_name":9,"category_name":40,"show_sort_weight":41,"slug":42},8,"Research & Report",30,"research-report",{"id":44,"doc_module":4,"doc_module_name":9,"category_name":45,"show_sort_weight":46,"slug":47},9,"Religion & Spirituality",20,"religion-spirituality",{"id":46,"doc_module":4,"doc_module_name":9,"category_name":49,"show_sort_weight":46,"slug":50},"World Cup","world-cup",{"id":52,"doc_module":4,"doc_module_name":9,"category_name":53,"show_sort_weight":52,"slug":54},10,"Lifestyle","lifestyle",{"id":56,"doc_module":4,"doc_module_name":9,"category_name":57,"show_sort_weight":24,"slug":58},19,"General","general",{"code":4,"msg":60,"data":61},"ok",{"site_id":62,"language":63,"slug":64,"title":65,"keywords":66,"description":67,"schema_data":68,"social_meta":123,"head_meta":125,"extra_data":127,"updated_unix":129},105,"en","learning-to-execute-instructions-in-a-minecraft-dialogue-abstract-and-paper-sections","Learning to execute instructions in a Minecraft dialogue - Abstract and paper sections","","The Minecraft Collaborative Building Task is a two-player language-and-action game where an Architect instructs a Builder to construct a target 3D block structure. The work studies predicting the Builder’s action sequences—block placements and removals—under a specific game context. It shows that representing the Builder’s past actions and incorporating the Builder’s perspective produces a substantial improvement on this challenging language understanding and grounded execution setting.",{"@graph":69,"@context":122},[70,84,105],{"@type":71,"itemListElement":72},"BreadcrumbList",[73,77,79,82],{"item":74,"name":75,"@type":76,"position":8},"https://docshare.wps.com","Home","ListItem",{"item":78,"name":9,"@type":76,"position":14},"https://docshare.wps.com/document/",{"item":80,"name":40,"@type":76,"position":81},"https://docshare.wps.com/document/research-report/",3,{"item":83,"name":65,"@type":76,"position":19},"https://docshare.wps.com/document/learning-to-execute-instructions-in-a-minecraft-dialogue-abstract-and-paper-sections/315100/",{"url":83,"name":65,"@type":85,"image":86,"author":91,"headline":65,"publisher":94,"fileFormat":97,"inLanguage":63,"description":67,"dateModified":98,"datePublished":99,"encodingFormat":97,"isAccessibleForFree":100,"interactionStatistic":101},"DigitalDocument",{"url":87,"@type":88,"width":89,"height":90},"https://docshare.wps.com/thumbnails/learning-to-execute-instructions-in-a-minecraft-dialogue-abstract-and-paper-sections/315100.png","ImageObject",300,407,{"name":92,"@type":93},"Dozel","Person",{"url":74,"name":95,"@type":96},"DocShare","Organization","application/pdf","2026-11-08","2026-09-20",true,{"@type":102,"interactionType":103,"userInteractionCount":29},"InteractionCounter",{"@type":104},"ViewAction",{"@type":106,"mainEntity":107},"FAQPage",[108,114,118],{"name":109,"@type":110,"acceptedAnswer":111},"What is the Minecraft Collaborative Building Task described in the document?","Question",{"text":112,"@type":113},"It is a two-player game where an Architect provides text chat instructions and a Builder executes actions to build a target structure from 3D blocks in a simulated Blocks World environment.","Answer",{"name":115,"@type":110,"acceptedAnswer":116},"What specific prediction problem does the paper focus on?",{"text":117,"@type":113},"The paper focuses on the Builder Action Prediction (BAP) subtask: predicting the Builder’s action sequences, including block placements and removals, given the game context.",{"name":119,"@type":110,"acceptedAnswer":120},"Why do the authors emphasize using the Builder’s past actions and perspective?",{"text":121,"@type":113},"Capturing the Builder’s past actions and incorporating the Builder’s perspective is shown to significantly improve performance on the task of grounded language understanding and instruction execution.","https://schema.org",{"og:url":83,"og:type":124,"og:title":65,"og:site_name":95,"og:description":67},"article",{"robots":126,"canonical":83},"index,follow",{"doc_id":128,"site_id":62},315100,1789960358,{"code":4,"msg":5,"data":131},{"doc_id":128,"user_id":132,"nickname":92,"user_avatar":133,"doc_module":4,"category_id":39,"category_name":40,"doc_title":65,"doc_description":67,"doc_content":134,"file_id":135,"file_url":136,"file_type":137,"file_size":138,"view_count":29,"is_deleted":4,"is_public":8,"is_downloadable":8,"audit_status":8,"page_count":139,"language":140,"language_code":63,"site_id":62,"html_lang":63,"table_of_contents":141,"faqs":142,"seo_title":143,"seo_description":67,"update_tm":144,"read_time":145,"preview_page_count":4},962085662650,"https://ap-avatar.wpscdn.com/davatar_29158cc5080c5b710cf443261637dec0","Learning to execute instructions in a Minecraft dialogue  \nPrashant Jayannavar Anjali Narayan-Chen Julia Hockenmaier  \nUniversity of Illinois at Urbana-Champaign fpaj3, nrynchn2, [juliahmr](juliahmrg@illinois.edu)[g](juliahmrg@illinois.edu)[@illinois.edu](juliahmrg@illinois.edu)  \nAbstract  \nThe Minecraft Collaborative Building Task is a two-player game in which an Architect A instructs a Builder B to construct a target structure out of 3D blocks. We consider the task of predicting B's action sequences (block placements and removals) in a given game context, and show that capturing B's past actions as well as B's perspective leads to a significant improvement in performance on this challenging language understanding problem.  \n1 Introduction  \nThere is a long-standing interest in building interactive agents that can communicate with humans about and operate within the physical world (e.g. Winograd (1971)) . The goal for agents in this scenario is to not only be able to engage in rich natural language discourse with their human conversation partners, but also to ground that discourse to physical objects, and execute instructions in the real world. Traditional dialogue scenarios are either completely ungrounded (Ritter et al., 2010 ; Schrading et al., 2015), focus on slot-value filling tasks (Kim et al., 2016b,a; Budzianowski et al., 2018) which instead require grounding to entities in a knowledge base, or operate within static environments, such as images (Das et al., 2017) or videos (Pasunuru and Bansal, 2018) . Relevant efforts in robotics have largely focused on single-shot instruction following, and are mostly constrained to simple language (Roy and Reiter, 2005 ; Tellex et al., 2011) with limited resources (Thomasonet al., 2015 ; Misra et al., 2016 ; Chai et al., 2018) .  \nThe recently introduced Minecraft Collaborative Building Task and the corresponding Minecraft Dialogue Corpus (Narayan-Chen et al., 2019) isone attempt to bridge this gap within the simulated game world of Minecraft. In this task, two players, an Architect (A) instructs a Builder (B) to con-  \nstruct a target structure out of multi-colored building blocks. The corpus consists of 509 game logs between humans that perform this task. NarayanChen et al. (2019) focus on generating Architect utterances. In this paper, we explore models for building an automated Builder agent.1 We focus on the subtask of predicting the Builder's block placements, and leave the back-and-forth dialogue aspect of the overall task required of a fully interactive Builder agent to future work. We define the Builder Action Prediction (BAP) task in Section 2, describe our models in Section 3, an approach to augment the training data in Section 4, and our experiments in Section 5. We analyze results and highlight challenges of the BAP task in Section 6.  \n2 Dataset and Task  \n2.1 The Minecraft Dialogue Corpus  \nThe Minecraft Dialogue Corpus (Narayan-Chenet al., 2019) consists of 509 human-human dialogues and game logs for the Minecraft Collaborative Building Task, a two-player game in a simulated Blocks World environment between an Architect (A) and a Builder (B) . A is given a target structure (Target) and has to instruct B via a text chat interface to build a copy of Target on a given build region. A and B communicate back and forth via chat throughout the game ([e.g. to](e.g. to) resolve confusions or to correct B's mistakes), but only B can move blocks, while A observes B operating in the world. B is given access to an inventory of 120 blocks of six given colors that it can place and remove. The resulting dialogues consist mainly of A providing instructions, often involving multiple actions to betaken, and grounded in the Builder's perspective, while B executes those instructions and resolves  \n1For models and code see [http://juliahmr.cs](http://juliahmr.cs) . [illinois.edu/Minecraft](illinois.edu/Minecraft)  \n2589  \nProceedings of the 58th Annual Meeting of the Association for Comput","cbCaielfjhqNhuja","https://ap.wps.com/l/cbCaielfjhqNhuja","pdf",6726899,14,"English","# Abstract\n# Introduction\n# Dataset and Task\n## The Minecraft Dialogue Corpus\n# Models and Training Data Augmentation\n# Experiments\n# Results and Challenges","[{\"question\":\"What is the Minecraft Collaborative Building Task described in the document?\",\"answer\":\"It is a two-player game where an Architect provides text chat instructions and a Builder executes actions to build a target structure from 3D blocks in a simulated Blocks World environment.\"},{\"question\":\"What specific prediction problem does the paper focus on?\",\"answer\":\"The paper focuses on the Builder Action Prediction (BAP) subtask: predicting the Builder’s action sequences, including block placements and removals, given the game context.\"},{\"question\":\"Why do the authors emphasize using the Builder’s past actions and perspective?\",\"answer\":\"Capturing the Builder’s past actions and incorporating the Builder’s perspective is shown to significantly improve performance on the task of grounded language understanding and instruction execution.\"}]","Learning to execute instructions in a Minecraft dialogue - Abstract and paper sections | PDF",1789903259,35]