[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-120159-en":3,"doc-seo-120159-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},120159,8796095461610,"Oliver","https://ap-avatar.wpscdn.com/davatar_276721f389ce27ea32af1340a28f341c",6,"Technology","HYPPO - Using Equivalences to Optimize Pipelines in Exploratory Machine Learning","HYPPO is a system for optimizing machine-learning pipelines used in exploratory machine learning by leveraging equivalences found across artifacts produced by past executions. It reuses materialized artifacts while introducing alternative computational paths, which improves execution-plan generation. The approach models workload using directed hypergraphs and formulates optimal plan discovery as a search problem, while artifact materialization is posed as an optimization problem. Experiments show plans that are up to two orders of magnitude faster and cheaper than non-optimized pipelines, and up to one order better than state of the art, with cost reductions of 3–4 even when materialization is limited.","HYPPO: Using Equivalences to Optimize Pipelinesin Exploratory Machine Learning  \nAntonios Kontaxakis  \nULB, Brussels, Belgium UPC, Barcelona, Spain [antonios.kontaxakis@ulb.be](antonios.kontaxakis@ulb.be)  \nDimitris Sacharidis  \nUniversit´e Libre de Bruxelles Brussels, Belgium [dimitris.sacharidis@ulb.be](dimitris.sacharidis@ulb.be)  \nAlkis Simitsis  \nAthena Research Center Athena, Greece [alkis@athenarc.gr](alkis@athenarc.gr)  \nAlberto Abell´o  \nUniversitat Polite`cnica de Catalunya Barcelona, Spain [alberto.abello@upc.edu](alberto.abello@upc.edu)  \nSergi Nadal  \nUniversitat Polite`cnica de Catalunya Barcelona, Spain [sergi.nadal@upc.edu](sergi.nadal@upc.edu)  \nAbstract—We present HYPPO, a novel system to optimize pipelines encountered in exploratory machine learning. HYPPO exploits alternative computational paths of artifacts from past executions to derive better execution plans while reusing materialized artifacts. Adding alternative computations introduces new challenges for exploratory machine learning regarding workload representation, system architecture, and optimal execution plan generation. To this end, we present a novel workload representation based on directed hypergraphs, and we formulate the problem of discovering the optimal execution plan as a search problem over directed hypergraphs and that of selecting artifacts to materialize as an optimization problem. A thorough experimental evaluation shows that HYPPO results in plans that are typically one order (up to two orders) of magnitude faster and cheaper than the non-optimized pipeline and considerably (up to one order of magnitude) faster and cheaper than plans generated by the state of the art when materializing artifacts is possible. Lastly, our evaluation reveals that HYPPO reduces the cost by 3–4 ˆ even when materialization cannot be exploited.  \nI. INTRODUCTION  \nExploratory Machine Learning. Developing Machine Learning (ML) solutions involves an exploration phase with multiple ML pipelines, each having different combinations of data preprocessing, feature selection, model selection, or evaluation metrics. Exploratory ML (EML) describes the iterative process of trial and error, where an ML pipeline is revised and refined until a satisfactory level of performance is achieved.  \nExploratory search in ML is typically bound by time or cost concerns. Recent studies argue that the main impediment to the adoption of ML in many organizations is the high costs incurred by running ML pipelines [1], [2] . One direction to reduce such costs is to employ AutoML [3] techniques to reduce the number of pipelines that need to be investigated. A different, orthogonal approach is to reduce the cost of pipelines by applying optimization techniques, such as reuse and materialization [4], [5], [6], [7] . The latter approach seems very promising in exploratory scenarios, where an ML engineer executes pipelines that construct or require artifacts computed in past iterations, uncovering within-experiment reuse opportunities. Moreover, in large organizations, multiple data scientists work on the same data and perform similar ML tasks, thus presenting across-experiments reuse opportunities.  \nOptimizing ML Pipelines. An ML pipeline is a collection of computational tasks that produce and consume artifacts. Sharing computations, a.k.a. common subexpression elimination, is a widespread practice for optimization that merges multiple executions into one to remove redundant tasks. It identifies exact sequences of tasks applied to the same data. Reuse is a technique that builds on the idea of common subexpression elimination. It identifies artifacts computed in the past and, if stored, decides if it is beneficial to load or recompute them.  \nThe capacity to reuse highly depends on which previously computed artifacts are stored. As a result, reuse is always accompanied by a materialization strategy, which focuses on which of the produced artifacts should be stored. State-ofthe-art research","cbCaidduFoQpU3it","https://ap.wps.com/l/cbCaidduFoQpU3it","pdf",2457481,1,14,"English","en",105,"# Abstract\n# Introduction\n## Exploratory Machine Learning\n## Optimizing ML Pipelines\n## Task and Artifact Equivalence\n## Our Contributions","[{\"question\":\"What problem does HYPPO address in exploratory machine learning?\",\"answer\":\"HYPPO targets the high time and cost of repeatedly running many ML pipelines by optimizing execution plans through reuse and by exploiting equivalences between tasks and artifacts across past executions.\"},{\"question\":\"How does HYPPO represent the ML workload and dependencies?\",\"answer\":\"It uses directed hypergraphs to represent multi-input, multi-output tasks as hyperedges and artifacts as nodes, including operator states such as preprocessing steps and ML models.\"},{\"question\":\"What optimization strategies does HYPPO use for execution plans and artifact materialization?\",\"answer\":\"It formulates execution-plan discovery as a search problem over directed hypergraphs, while selecting which artifacts to materialize is posed as an optimization problem.\"}]","HYPPO - Using Equivalences to Optimize Pipelines in Exploratory Machine Learning | PDF",1785728493,35,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"hyppo-using-equivalences-to-optimize-pipelines-in-exploratory-machine-learning","",{"@graph":36,"@context":85},[37,54,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/technology/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/hyppo-using-equivalences-to-optimize-pipelines-in-exploratory-machine-learning/120159/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":62,"encodingFormat":61,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-03",true,{"@type":65,"interactionType":66,"userInteractionCount":4},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What problem does HYPPO address in exploratory machine learning?","Question",{"text":75,"@type":76},"HYPPO targets the high time and cost of repeatedly running many ML pipelines by optimizing execution plans through reuse and by exploiting equivalences between tasks and artifacts across past executions.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"How does HYPPO represent the ML workload and dependencies?",{"text":80,"@type":76},"It uses directed hypergraphs to represent multi-input, multi-output tasks as hyperedges and artifacts as nodes, including operator states such as preprocessing steps and ML models.",{"name":82,"@type":73,"acceptedAnswer":83},"What optimization strategies does HYPPO use for execution plans and artifact materialization?",{"text":84,"@type":76},"It formulates execution-plan discovery as a search problem over directed hypergraphs, while selecting which artifacts to materialize is posed as an optimization problem.","https://schema.org",{"og:url":52,"og:type":87,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":89,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,113,118,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":111,"slug":112},50,"technology",{"id":114,"doc_module":4,"doc_module_name":46,"category_name":115,"show_sort_weight":116,"slug":117},7,"Healthcare",40,"healthcare",{"id":119,"doc_module":4,"doc_module_name":46,"category_name":120,"show_sort_weight":121,"slug":122},8,"Research & Report",30,"research-report",{"id":124,"doc_module":4,"doc_module_name":46,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":46,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":46,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":46,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]