[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-124735-en":3,"doc-seo-124735-105":30,"detail-sidebar-cat-0-en-105":90},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},124735,16904993612988,"Olivia Brown","https://ap-avatar.wpscdn.com/davatar_a8503ba1806abce46bf441b54a3ca4cd",8,"Research & Report","Machine Learning in Minecraft - Proof of Concept for Object Detection Oriented Autonomous Bots in Minecraft - Symposium Poster","A project applying machine learning to Minecraft automates the early survival objective of finding and collecting wood by detecting tree objects in first-person gameplay footage. The work uses the Ultralytics YOLOv8-medium neural network with bounding-box object detection, trained on manually labeled, resized frames sampled from recorded gameplay. Training uses labeled datasets across training, validation, and test splits, achieving strong precision and overall average precision at a 50% recall threshold. A bot implementation integrates Mineflayer with a local Minecraft server, converts detections into reference angles, navigates toward detected trees, and performs collection actions.","Kennesaw State University  \nDigitalCommons@Kennesaw State University  \n\n| Symposium of Student Scholars | Fall 2023 Symposium of Student Scholars |\n| --- | --- |\n| Machine Learning in Minecraft: Proof of Concept for Object Detection Oriented Autonomous Bots in Minecraft\u003Cbr>John Merkin\u003Cbr>Follow this and additional works at: [https://digitalcommons.kennesaw.edu/undergradsymposiumksu](https://digitalcommons.kennesaw.edu/undergradsymposiumksu)[ ](https://digitalcommons.kennesaw.edu/undergradsymposiumksu) Part of the Artificial Intelligence and Robotics Commons, and the Data Science Commons |  |\n\nMerkin, John, \"Machine Learning in Minecraft: Proof of Concept for Object Detection Oriented Autonomous Bots in Minecraft\" (2023) . Symposium of Student Scholars. 106.  \n[https://digitalcommons.kennesaw.edu/undergradsymposiumksu/fall2023/presentations/106](https://digitalcommons.kennesaw.edu/undergradsymposiumksu/fall2023/presentations/106)  \nThis Poster is brought to you for free and open access by the Office of Undergraduate Research at DigitalCommons@Kennesaw State University. It has been accepted for inclusion in Symposium of Student Scholars by an authorized administrator of DigitalCommons@Kennesaw State University. For more information, please contact [digitalcommons@kennesaw.edu](digitalcommons@kennesaw.edu).  \nMachine Learning in Minecraft  \nProof of Concept for Object Detection Oriented Autonomous Bots in Minecraft  \nJohn Merkin  \nIntroduction  \nMachine learning provides innumerable opportunities for automation and data analysis. One primary objective for machine learning is object detection; object detection seeks to identify the location of objects in an image as well as determine the most probable class of each object. An Interesting challenge is to implement various machine learning models into the context of video games and other virtual environments.  \nMinecraft, an open-world sandbox game, gives players the freedom to collect resources and alter the environment as they choose. In Minecraft’s survival mode, objectives and resources must be collected according to a built-in hierarchy. For example, to collect stone, a player must first collect wood to craft a pic-axe. As such, this project seeks to automate the initial objective of finding and collecting wood in a Minecraft virtual environment through implementation of a machine learning model.  \nNeural Network Architecture  \nThis project utilizes the Ultralytics Yolov8-medium pipeline architecture, a convolutional neural network (CNN) framework pretrained on Microsoft’s COCO dataset. Yolov8 is designed to predict the location and classification of objects in images with bounding boxes.  \nThe Yolov8 backbone consists of several interwoven convolutional and pooling layers. Between layers Yolov8 utilizes mosaic augmentation, segmenting images into parts and attaching them to other image segments. Mosaic augmentation further diversifies the dataset to better generalize and stops near the end of training to improve performance (Solowetz) .  \nAlthough pre-trained models are not guaranteed to improve accuracy over initially untrained models, pretraining typically yields improved results over less iterations for smaller datasets. Pre-trained models have also been shown to have improved robustness and generalization to new data (Hendrycks et al) .  \nData and Methodology  \nTraining data was collected via sampling image frames from videos of Minecraft gameplay recorded with OBS. Images were resized into a 512x288 pixel resolution. 510 unique images were sampled and manually labeled with bounding boxes representing 6 classes of Minecraft trees. Images were then stretched into a 640x640 resolution and exported using Roboflow; 351 images were used for training, 105 for validation, and 54 for testing.  \nTraining Metrics and Results  \nRunning on an Intel i5 12600KF CPU, the network took 1 hour and 16 minutes to train over 15 epochs with stochastic gradient descent, reaching an overall avera","cbCaijgC8azWVsSx","https://ap.wps.com/l/cbCaijgC8azWVsSx","pdf",1370137,1,2,"English","en",105,"# Introduction\n## Neural Network Architecture\n## Data and Methodology\n## Training Metrics and Results\n## Application\n## Conclusion","[{\"question\":\"What problem does the project address in Minecraft?\",\"answer\":\"It aims to automate the initial survival objective of locating and collecting wood by detecting tree objects in gameplay frames.\"},{\"question\":\"Which object detection model architecture is used?\",\"answer\":\"The project uses the Ultralytics YOLOv8-medium pipeline based on a convolutional neural network pretrained on Microsoft’s COCO dataset.\"},{\"question\":\"How is the trained model connected to an autonomous Minecraft bot?\",\"answer\":\"The implementation uses Mineflayer via JSPyBridge to control a bot in a local Minecraft world, then uses Selenium and OpenCV to capture, resize, and feed frames for bounding-box predictions, converting detections into angles for navigation and wood collection.\"}]","Machine Learning in Minecraft - Proof of Concept for Object Detection Oriented Autonomous Bots in Minecraft - Symposium Poster | PDF",1785894187,5,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":85,"head_meta":87,"extra_data":89,"updated_unix":28},"machine-learning-in-minecraft-proof-of-concept-for-object-detection-oriented-autonomous-bots-in-minecraft-symposium-poster","",{"@graph":36,"@context":84},[37,53,67],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,47,50],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":21},"https://docshare.wps.com/document/","Document",{"item":48,"name":12,"@type":43,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":43,"position":52},"https://docshare.wps.com/document/machine-learning-in-minecraft-proof-of-concept-for-object-detection-oriented-autonomous-bots-in-minecraft-symposium-poster/124735/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":61,"encodingFormat":60,"isAccessibleForFree":62,"interactionStatistic":63},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":41,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-08-05",true,{"@type":64,"interactionType":65,"userInteractionCount":4},"InteractionCounter",{"@type":66},"ViewAction",{"@type":68,"mainEntity":69},"FAQPage",[70,76,80],{"name":71,"@type":72,"acceptedAnswer":73},"What problem does the project address in Minecraft?","Question",{"text":74,"@type":75},"It aims to automate the initial survival objective of locating and collecting wood by detecting tree objects in gameplay frames.","Answer",{"name":77,"@type":72,"acceptedAnswer":78},"Which object detection model architecture is used?",{"text":79,"@type":75},"The project uses the Ultralytics YOLOv8-medium pipeline based on a convolutional neural network pretrained on Microsoft’s COCO dataset.",{"name":81,"@type":72,"acceptedAnswer":82},"How is the trained model connected to an autonomous Minecraft bot?",{"text":83,"@type":75},"The implementation uses Mineflayer via JSPyBridge to control a bot in a local Minecraft world, then uses Selenium and OpenCV to capture, resize, and feed frames for bounding-box predictions, converting detections into angles for navigation and wood collection.","https://schema.org",{"og:url":51,"og:type":86,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":88,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":91},[92,96,100,104,108,113,118,121,126,129,133],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":93,"show_sort_weight":94,"slug":95},"Story & Novel",90,"story-novel",{"id":21,"doc_module":4,"doc_module_name":46,"category_name":97,"show_sort_weight":98,"slug":99},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":46,"category_name":101,"show_sort_weight":102,"slug":103},"Exam",70,"exam",{"id":29,"doc_module":4,"doc_module_name":46,"category_name":105,"show_sort_weight":106,"slug":107},"Comic",60,"comic",{"id":109,"doc_module":4,"doc_module_name":46,"category_name":110,"show_sort_weight":111,"slug":112},6,"Technology",50,"technology",{"id":114,"doc_module":4,"doc_module_name":46,"category_name":115,"show_sort_weight":116,"slug":117},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":119,"slug":120},30,"research-report",{"id":122,"doc_module":4,"doc_module_name":46,"category_name":123,"show_sort_weight":124,"slug":125},9,"Religion & Spirituality",20,"religion-spirituality",{"id":124,"doc_module":4,"doc_module_name":46,"category_name":127,"show_sort_weight":124,"slug":128},"World Cup","world-cup",{"id":130,"doc_module":4,"doc_module_name":46,"category_name":131,"show_sort_weight":130,"slug":132},10,"Lifestyle","lifestyle",{"id":134,"doc_module":4,"doc_module_name":46,"category_name":135,"show_sort_weight":29,"slug":136},19,"General","general"]