[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-117734-en":3,"doc-seo-117734-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},117734,1099513958607,"Jiven","https://ap-avatar.wpscdn.com/avatar/100002390cf8733938c?x-image-process=image/resize,m_fixed,w_180,h_180&k=1778829742770036399",8,"Research & Report","AutoMLBench - Comprehensive Experimental Evaluation of Automated Machine Learning Frameworks","AutoMLBench presents a comprehensive evaluation of six widely used automated machine learning (AutoML) frameworks: Auto-Weka, AutoSKlearn, TPOT, Recipe, ATM, and SmartML. The study compares their performance across 100 datasets from established AutoML benchmark suites, focusing on how key design decisions affect outcomes. Experimental analysis covers factors such as time budget, search space size, meta-learning strategies, and ensemble construction, yielding insights intended to guide the design of future AutoML frameworks.","arXiv :2204 .08358v1 [ cs .LG] 18 Apr 2022  \nAutoMLBench: A Comprehensive Experimental Evaluation of Automated Machine Learning Frameworks  \nHassan Eldeeb 1 , Mohamed Maher 1y, Oleh Matsuk 1y, Abdelrahman Aldallal 1 , Radwa Elshawi 1* and Sherif  \nSakr 1*  \n1Institute of Computer Science, University of Tartu, Tartu, Estonia.  \n*Corresponding author(s). E-mail(s): [radwa.elshawi@ut.ee](radwa.elshawi@ut.ee) ; [sherif.sakr@ut.ee](sherif.sakr@ut.ee) ;  \nContributing authors: hassan.eldeeb@ut.ee; [mohamed.maher@ut.ee](mohamed.maher@ut.ee) ;  \n[oleh.matsuk@ut.ee](oleh.matsuk@ut.ee) ; [abdelrahman.aldallal@ut.ee](abdelrahman.aldallal@ut.ee) ;  \nyThese authors contributed equally to this work.  \nAbstract  \nNowadays, machine learning is playing a crucial role in harnessing the power of the massive amounts of data that we are currently producing every day in our digital world. With the booming demand for machine learning applications, it has been recognized that the number of knowledgeable data scientists can not scale with the growing data volumes and application needs in our digital world. In response to this demand, several automated machine learning (AutoML) techniques and frameworks have been developed to fill the gap of human expertise by automating the process of building machine learning pipelines. In this study, we present a comprehensive evaluation and comparison of the performance characteristics of six popular AutoML frameworks, namely, Auto-Weka, AutoSKlearn, TPOT, Recipe, ATM and SmartML across 100 data sets from established AutoML benchmark suites. Our experimental evaluation considers different aspects for its comparison including the performance impact of several design decisions including time budget, size of search space, meta-learning and ensemble construction. The results of our study reveal various interesting insights that can significantly guide and impact the design of AutoML frameworks.  \nKeywords: AutoML, optimization techniques, meta-learning, ensemble construction, hyperparameter tuning  \n2 AutoMLBench  \n1 Introduction  \nNowadays, we are witnessing tremendous interest in artificial intelligence applications across governments, industries and research communities with a yearly cost of around 12.5 billion US dollars report (2017) . The driver for this interest is the advent and increasing popularity of machine learning (ML) and deep learning (DL) techniques. The rise of generated data from different sources, processing capabilities, and ML algorithms opened the way for adopting ML in a wide range of real-world applications Zomaya and Sakr (2017) . This situation is increasingly contributing towards a potential data science crisis, similar to the software crisis Fitzgerald (2012), due to the crucial need of having an increasing number of data scientists with solid knowledge and good experience so that they can keep up with harnessing the power of the massive amounts of data produced daily. Thus, we are witnessing a growing interest in automating the process of building machine learning pipelines where the presence of a human in the loop can be dramatically reduced. Research in Automated machine learning (AutoML) aims to alleviate both the computational cost and human expertise required for developing machine learning pipelines through automation with efficient algorithms. In particular, AutoML techniques enable the widespread use of machine learning techniques by domain experts and non-technical users.  \nApplying machine learning to real-world problems is a multi-stage process and highly iterative exploratory process. Therefore, several frameworks were designed to support automating the Combined Algorithm Selection and hyperparameter tuning (CASH) problem Shawi et al (2019); Zller and Huber (2021); He et al (2019) . These techniques have commonly formulated the problem as an optimization problem that many techniques can solve. Let A = fA(1) ; :::; A (R)g be a set of machine learning algorithms, and let the hyperpar","cbCaijOIV2fjeNsx","https://ap.wps.com/l/cbCaijOIV2fjeNsx","pdf",2505912,1,37,"English","en",105,"# Abstract\n# Introduction\n## AutoML and the CASH problem\n## Time budget and design decisions\n# AutoMLBench","[{\"question\":\"AutoMLBench研究比较了哪些AutoML框架？\",\"answer\":\"研究比较了六个常用AutoML框架：Auto-Weka、AutoSKlearn、TPOT、Recipe、ATM和SmartML。\"},{\"question\":\"实验评估覆盖了哪些数据集规模与来源？\",\"answer\":\"实验在来自既有AutoML benchmark套件的100个数据集上进行对比评估。\"},{\"question\":\"研究重点分析哪些设计决策对AutoML性能的影响？\",\"answer\":\"评估重点包括时间预算、搜索空间大小、元学习（meta-learning）策略以及集成构建（ensemble construction）等因素。\"}]","AutoMLBench - Comprehensive Experimental Evaluation of Automated Machine Learning Frameworks | PDF",1785679271,93,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"automlbench-comprehensive-experimental-evaluation-of-automated-machine-learning-frameworks","",{"@graph":36,"@context":85},[37,54,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/automlbench-comprehensive-experimental-evaluation-of-automated-machine-learning-frameworks/117734/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":62,"encodingFormat":61,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-02",true,{"@type":65,"interactionType":66,"userInteractionCount":4},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"AutoMLBench研究比较了哪些AutoML框架？","Question",{"text":75,"@type":76},"研究比较了六个常用AutoML框架：Auto-Weka、AutoSKlearn、TPOT、Recipe、ATM和SmartML。","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"实验评估覆盖了哪些数据集规模与来源？",{"text":80,"@type":76},"实验在来自既有AutoML benchmark套件的100个数据集上进行对比评估。",{"name":82,"@type":73,"acceptedAnswer":83},"研究重点分析哪些设计决策对AutoML性能的影响？",{"text":84,"@type":76},"评估重点包括时间预算、搜索空间大小、元学习（meta-learning）策略以及集成构建（ensemble construction）等因素。","https://schema.org",{"og:url":52,"og:type":87,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":89,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":46,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":46,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":46,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":46,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]