[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-83943-en":3,"doc-seo-83943-105":29,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},83943,687197207639,"Asher","https://ap-avatar.wpscdn.com/davatar_a8503ba1806abce46bf441b54a3ca4cd",8,"Research & Report","TabPack Efficient Hyperparameter Ensembles for Tabular Deep Learning","TabPack targets supervised deep learning on tabular data by improving multilayer perceptron (MLP) ensembles. Instead of using identical hyperparameters across ensemble members, TabPack runs a single training procedure that efficiently samples and trains many MLPs with varied hyperparameters in parallel, then selects strong members on the fly. Performance comes from reducing dependency on precise tuning: only sampling ranges are needed. On medium-to-large public datasets, default TabPack matches extensively tuned methods while cutting effort and compute resources.","TabPack: Efficient Hyperparameter Ensembles for Tabular Deep Learning  \nYury Gorishniy 1 Akim Kotelnikov 2 1 Ivan Rubachev 1 2 Artem Babenko 1 2  \narXiv :2607 .05380v 1 [ cs .LG] 6 Jul 2026  \nAbstract  \nIn deep learning for tabular data, efficient ensembles of multilayer perceptrons (MLPs) have recently emerged as effective and practical architectures. Existing methods of this kind use the same hyperparameters for all underlying MLPs, which requires hyperparameter tuning for achieving the best performance. In this work, we introduce TabPack, an efficient MLP ensemble with strong out-of-the-box performance and reduced reliance on traditional tuning. In a single run, TabPack samples and trains many MLPs with different hyperparameters efficiently in parallel and selects ensemble members on the fly during training. Thus, TabPack only requires specifying ranges from which to sample MLP hyperparameter rather than exact hyperparameter values, which naturally demands less precision for good performance. In experiments on medium-to-large public datasets, TabPack with default settings performs on par with extensively tuned prior methods, thus substantially reducing effort and compute resources needed to achieve competitive results on tabular tasks. Notably, running the default TabPack configuration on a modern MacBook took less time than tuning some baselines on an industry-grade GPU. The source code is available at this URL.  \n1. Introduction  \nSupervised learning on tabular data is a common machine learning (ML) task in real-world applications. For a longtime, the dominant approach to such tasks was gradientboosted decision trees (GBDTs) (Chen & Guestrin, 2016 ; Prokhorenkova et al., 2018 ; Ke et al., 2017) . Due to the rapid progress over recent years, modern tabular deep learning (DL) models now also demonstrate strong performance and continue to improve (Holzm¨uller et al., 2024 ; Gorishniy  \n1Yandex 2HSE University. Correspondence to: Yury Gorishniy \u003C[yurygorishniy@gmail.com](yurygorishniy@gmail.com) >.  \nProceedings of the 43 rd International Conference on Machine Learning, Seoul, South Korea. PMLR 306, 2026 . Copyright 2026 by the author(s) .  \nMean rank  \n7  \n6  \n5  \n4  \n3  \n2  \n1  \nPerformance∗ vs. Runtime∗∗  \n\n|  |  | Non-foundatio Foundation models TabPack | n model | s |  | \u003Cbr>MLP |  |  |\n| --- | --- | --- | --- | --- | --- | --- | --- | --- |\n|  |  |  |  |  |  |  |  |  |\n|  |  |  |  |  |  |  |  |  |\n| Ta |  |  |  |  |  |  | \u003Cbr>Moder | nNCAy |\n|  |  | \u003Cbr>bICLv2 |  |  |  |  |  |  |\n| \u003Cbr>TabP |  | FN-3 |  |  |  | \u003Cbr>XGBo | ost |  |\n|  |  |  |  |  |  |  |  | \u003Cbr>RealMLP |\n| T |  | \u003Cbr>abPacky | TabP | \u003Cbr>ackyMac |  | \u003Cbr>Tab Book | My |  |\n| Be\u003Cbr> |  | tter |  |  |  |  |  |  |\n\n1h 2h 8h 16h 32h 64h 128h  \nTotal runtime across the benchmark  \nFigure 1. A summary of the results from Figure 6, obtained on 17 tabular datasets spanning classification and regression tasks with up to 700K+ training samples and up to 900+ features. TabPack†MacBook is evaluated on Apple M4 Pro chip with 20 GPU cores. Other methods are evaluated on NVIDIA A100 GPU.(∗ ) Task performance varies across benchmarks. For a more complete picture, see Figure 6 (our benchmark), subsection A.1 (largescale industrial datasets with temporal splits) and subsection A.2 (small-to-medium datasets with IID splits) .  \n(∗∗ ) For non-foundation models, runtime is the hyperparameter tuning time. For foundation models, runtime is the inference time with default hyperparameters. For TabPack, runtime is the training time of a single run with default hyperparameters.  \net al., 2025 ; Ye et al., 2024 ; Erickson et al., 2025 ; Qu et al., 2026 ; Grinsztajn et al., 2026) .  \nA common view in ML is that ensembles (mixtures of multiple models) outperform single models (Fort et al., 2020), and tabular ML is no exception (Erickson et al., 2025) . In tabular DL architectures, this idea has recently been realized in an efficient and practical form with efficient ensembles of mult","cbCaihCcznYqOoEx","https://ap.wps.com/l/cbCaihCcznYqOoEx","pdf",1201763,1,22,"English","en",105,"# Introduction\n## Efficient MLP ensembles for tabular data\n## Reducing hyperparameter tuning effort","[{\"question\":\"What problem does TabPack address in tabular deep learning?\",\"answer\":\"TabPack addresses the need for hyperparameter tuning in MLP ensembles for tabular tasks, which otherwise requires additional time and compute to reach strong performance.\"},{\"question\":\"How does TabPack differ from existing efficient MLP ensemble methods?\",\"answer\":\"Existing methods typically keep the same hyperparameters across ensemble members and rely on diversity from initialization, dropout, or batch order. TabPack instead samples different hyperparameters for many MLPs and trains them efficiently in parallel.\"},{\"question\":\"What inputs does TabPack require to achieve good results?\",\"answer\":\"TabPack requires specifying ranges for the most impactful model and optimizer hyperparameters rather than providing exact hyperparameter values, reducing the precision needed for competitive performance.\"}]",1784191582,55,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":27},"tabpack-efficient-hyperparameter-ensembles-for-tabular-deep-learning","",{"@graph":35,"@context":85},[36,53,68],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/tabpack-efficient-hyperparameter-ensembles-for-tabular-deep-learning/83943/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-17","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What problem does TabPack address in tabular deep learning?","Question",{"text":75,"@type":76},"TabPack addresses the need for hyperparameter tuning in MLP ensembles for tabular tasks, which otherwise requires additional time and compute to reach strong performance.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"How does TabPack differ from existing efficient MLP ensemble methods?",{"text":80,"@type":76},"Existing methods typically keep the same hyperparameters across ensemble members and rely on diversity from initialization, dropout, or batch order. TabPack instead samples different hyperparameters for many MLPs and trains them efficiently in parallel.",{"name":82,"@type":73,"acceptedAnswer":83},"What inputs does TabPack require to achieve good results?",{"text":84,"@type":76},"TabPack requires specifying ranges for the most impactful model and optimizer hyperparameters rather than providing exact hyperparameter values, reducing the precision needed for competitive performance.","https://schema.org",{"og:url":51,"og:type":87,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":89,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":45,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":45,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":45,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":45,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":45,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":45,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":45,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]