[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-117614-en":3,"doc-seo-117614-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},117614,962075114765,"Quinn","https://ap-avatar.wpscdn.com/davatar_a8503ba1806abce46bf441b54a3ca4cd",8,"Research & Report","SELA - TREE-SEARCH ENHANCED LLM AGENTS FOR AUTOMATED MACHINE LEARNING","SELA introduces Tree-Search Enhanced LLM Agents to improve Automated Machine Learning by replacing static pipeline searches and limited LLM trial generation. The method represents pipeline configurations as trees and uses Monte Carlo Tree Search (MCTS) to guide intelligent, iterative experimentation based on experimental feedback. Across evaluations on 20 machine learning datasets, SELA achieves a 65% to 80% win rate over each baseline, demonstrating more effective exploration of the machine learning solution space and higher-quality results than traditional and agent-based AutoML approaches.","arXiv :2410 . 17238v 1 [ cs .AI] 22 Oct 2024  \nSELA: TREE-SEARCH ENHANCED LLM AGENTS FOR AUTOMATED MACHINE LEARNING  \nYizhou Chi 1 ,2 Yizhang Lin 1 Sirui Hong 1 , Duyi Pan3 , Yaying Fei, Guanghao Mei4 , Bangbang Liu 1 , Tianqi Pang5 , Jacky Kwok6 , Ceyao Zhang7 , Bang Liu8†, Chenglin Wu 1†  \n1DeepWisdom, 2University of California, Berkeley,  \n3The Hong Kong University of Science and Technology (Guangzhou),  \n4University of California, San Diego, 5 South China Normal University,  \n6 Stanford University, 7The Chinese University of Hong Kong, Shenzhen,  \n8Universit de Montral & Mila  \nABSTRACT  \nAutomated Machine Learning (AutoML) approaches encompass traditional methods that optimize fixed pipelines for model selection and ensembling, as well as newer LLM-based frameworks that autonomously build pipelines. While LLM-based agents have shown promise in automating machine learning tasks, they often generate low-diversity and suboptimal code, even after multiple iterations. To overcome these limitations, we introduce Tree-Search Enhanced LLM Agents (SELA), an innovative agent-based system that leverages Monte Carlo Tree Search (MCTS) to optimize the AutoML process. By representing pipeline configurations as trees, our framework enables agents to conduct experiments intelligently and iteratively refine their strategies, facilitating a more effective exploration of the machine learning solution space. This novel approach allows SELA to discover optimal pathways based on experimental feedback, improving the overall quality of the solutions. In an extensive evaluation across 20 machine learning datasets, we compare the performance of traditional and agent-based AutoML methods, demonstrating that SELA achieves a win rate of 65% to 80% against each baseline across all datasets. These results underscore the significant potential of agent-based strategies in AutoML, offering a fresh perspective on tackling complex machine learning challenges 1.  \n1 INTRODUCTION  \nAutomated Machine Learning (AutoML) is a rapidly evolving field that seeks to automate the process of designing reliable machine learning solutions with minimal human intervention. Traditional AutoML frameworks, such as Auto-WEKA (Thornton et al., 2013), Auto-Sklearn (Feurer et al., 2015; 2020), AutoGluon (Tang et al., 2024b), and H2O AutoML (LeDell & Poirier, 2020), rely on predefined search spaces and routines. These frameworks primarily focus on optimizing hyperparameters and model ensembling to find the best model configuration. However, this fixed and static approach often lacks the adaptability needed to handle diverse and dynamic data scenarios, resulting in suboptimal performance in more complex settings. Additionally, the traditional focus on model training leaves other crucial stages of the machine learning pipeline, such as data preprocessing and feature engineering, underexplored, thereby limiting the overall effectiveness of these systems.  \nRecently, large language model (LLM)-based agents have emerged as promising tools for automating machine learning tasks by leveraging natural language processing capabilities to generate code. These systems typically begin with a natural language prompt describing the dataset and the problem, after which an LLM generates an end-to-end solution. Early efforts, such as Zhang et al.  \n∗These authors contributed equally to this work.  \n†Bang Liu (E-mail: [bang.liu@umontreal.ca](bang.liu@umontreal.ca)) and Chenglin Wu (E-mail: [alexanderwu@deepwisdom.ai](alexanderwu@deepwisdom.ai)) are  \nthe corresponding authors.  \n1The code is available at [https://github.com/geekan/MetaGPT](https://github.com/geekan/MetaGPT)  \n(2024), experimented with prompting LLMs to generate machine learning solutions, while Hong et al. (2024a) introduced agents equipped with Hierarchical Graph Modeling and Programmable Node Generation to address complex and dynamic workflows. Despite these advances, LLM-based solutions often fall short in generating diverse and ","cbCaikphgBVUrIU2","https://ap.wps.com/l/cbCaikphgBVUrIU2","pdf",687265,1,21,"English","en",105,"# Abstract\n# Introduction\n## Problem with traditional AutoML\n## Limitations of LLM-based agents\n## Expert-like iterative refinement idea\n# SELA Method\n## Multi-step generation approach\n## One-step generation with iterative refinement\n# Experimental evaluation","[{\"question\":\"What problem does SELA address in Automated Machine Learning?\",\"answer\":\"SELA targets the low adaptability of fixed traditional AutoML pipelines and the limited search and low-diversity code generation of LLM-based agents, which often lead to suboptimal workflows.\"},{\"question\":\"How does SELA use MCTS to optimize the AutoML process?\",\"answer\":\"SELA represents pipeline configurations as trees and applies Monte Carlo Tree Search to explore and select more promising pipeline strategies through iterative experimentation and feedback.\"},{\"question\":\"What performance does SELA achieve compared with baselines?\",\"answer\":\"In evaluations across 20 machine learning datasets, SELA achieves a win rate of 65% to 80% against each baseline.\"}]","SELA - TREE-SEARCH ENHANCED LLM AGENTS FOR AUTOMATED MACHINE LEARNING | PDF",1785677287,53,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"sela-tree-search-enhanced-llm-agents-for-automated-machine-learning","",{"@graph":36,"@context":85},[37,54,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/sela-tree-search-enhanced-llm-agents-for-automated-machine-learning/117614/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":62,"encodingFormat":61,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-02",true,{"@type":65,"interactionType":66,"userInteractionCount":4},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What problem does SELA address in Automated Machine Learning?","Question",{"text":75,"@type":76},"SELA targets the low adaptability of fixed traditional AutoML pipelines and the limited search and low-diversity code generation of LLM-based agents, which often lead to suboptimal workflows.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"How does SELA use MCTS to optimize the AutoML process?",{"text":80,"@type":76},"SELA represents pipeline configurations as trees and applies Monte Carlo Tree Search to explore and select more promising pipeline strategies through iterative experimentation and feedback.",{"name":82,"@type":73,"acceptedAnswer":83},"What performance does SELA achieve compared with baselines?",{"text":84,"@type":76},"In evaluations across 20 machine learning datasets, SELA achieves a win rate of 65% to 80% against each baseline.","https://schema.org",{"og:url":52,"og:type":87,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":89,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":46,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":46,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":46,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":46,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]