[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-123651-en":3,"doc-seo-123651-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},123651,549758146520,"Patrick","https://ap-avatar.wpscdn.com/avatar/80002397d8c0411e94?_k=1775819394049821470",8,"Research & Report","Is Your Learned Query Optimizer Behaving As You Expect? - A Machine Learning Perspective","Learned query optimizers gain momentum by reframing query optimization as a machine learning problem, often replacing dynamic programming with reinforcement learning–style approaches. Rapid adoption, however, disrupts the ML pipeline beyond model development: training-data generation restrictions, non-standard train/validation splits, and inconsistent benchmark-query evaluations reduce comparability. This paper standardizes an end-to-end benchmarking framework, details each ML stage for LQOs, and delivers rigorous evaluation results showing PostgreSQL outperforms existing LQOs across most train/test splits.","Is Your Learned Query Optimizer Behaving As You Expect? A Machine Learning Perspective  \nClaude Lehmann∗ [Claude.Lehmann@zhaw.ch](Claude.Lehmann@zhaw.ch)[ ](Claude.Lehmann@zhaw.ch)Zurich University of Applied Sciences Winterthur, Switzerland  \nPavel Sulimov∗ [Pavel.Sulimov@zhaw.ch](Pavel.Sulimov@zhaw.ch)[ ](Pavel.Sulimov@zhaw.ch)Zurich University of Applied Sciences Winterthur, Switzerland  \nKurt Stockinger  \n[Kurt.Stockinger@zhaw.ch](Kurt.Stockinger@zhaw.ch)[ ](Kurt.Stockinger@zhaw.ch)Zurich University of Applied Sciences Winterthur, Switzerland  \narXiv :2309 .0 155 1v 1 [ cs .DB] 4 Sep 2023  \nABSTRACT  \nThe current boom of learned query optimizers (LQO) can be explained not only by the general continuous improvement of deep learning (DL) methods but also by the straightforward formulation of a query optimization problem (QOP) as a machine learning (ML) one. The idea is often to replace dynamic programming approaches, widespread for solving QOP, with more powerful methods such as reinforcement learning. However, such a rapid \"game change\"in the field of QOP could not pass without consequences-other parts of the ML pipeline, except for predictive model development, have large improvement potential. For instance, different LQOs introduce their own restrictions on training data generation from queries, use an arbitrary train/validation approach, and evaluate on a voluntary split of benchmark queries.  \nIn this paper, we attempt to standardize the ML pipeline for evaluating LQOs by introducing a new end-to-end benchmarking framework. Additionally, we guide the reader through each data science stage in the ML pipeline and provide novel insights from the machine learning perspective, considering the specifics of QOP. Finally, we perform a rigorous evaluation of existing LQOs, showing that PostgreSQL outperforms these LQOs in almost all experiments depending on the train/test splits.  \n1 INTRODUCTION  \nOver the last decade, machine learning (ML) approaches have heavily dominated classical query optimization methods. This trend could be explained by the increased spread of deep learning (DL) applications and the nature of the query optimization problem (QOP) itself. Having in total 􀀤 (􀀽!) possible logical plans in the worst case for queries where the join graph is a clique with 􀀽 tables, the problem is classified as NP-hard [37] . This implies that exhaustive methods cannot solve the problem for a higher order of joins1 , thus demanding the need for heuristical approaches.  \nIn Figure 1, we compare typical pipelines for classical and learned query optimizers. The classical approach, implemented inside database management systems (DBMS), has the stages of query representation via logical and physical plans, with a follow-up search of an optimal plan using cardinality-based cost model estimations. In addition to dynamic programming-based methods, genetic algorithms [34] are also used since they are proven to be more efficient for queries with a high number of joins [28] .  \nThe bottom part of Figure 1 shows learned query optimizers (LQO), the most recent trend for end-to-end query optimization. These approaches require a more complicated pipeline because of the use of ML methods. Looking at it from the ML perspective, the  \n∗ Both authors contributed equally to this work.  \n1PostgreSQL abandons exhaustive methods for queries with 12 or more FROM items.  \nFigure 1: Comparison of classical and learned query optimizers (LQO) -see top and bottom halves, respectively. The stages (1) Training Data Generation,(3) LQO Training, and (4) LQO Evaluation are the primary components of our End-to-End Benchmarking Framework. Together with the (2) Query & Plan Encoding stage, they form the typical machine learning pipeline for a LQO.  \npipeline should consist of several stages, namely (1) training data generation,(2) query & plan encoding,(3) ML model training, and (4) ML model evaluation. The violation of theoretical ML principles [30] at each stage ","cbCaiuLvlbzbaytH","https://ap.wps.com/l/cbCaiuLvlbzbaytH","pdf",991063,1,13,"English","en",105,"# Abstract\n# Introduction\n## Training Data Generation\n## Query & Plan Encoding\n## LQO Training\n## LQO Evaluation\n# Conclusion","[{\"question\":\"Why is it difficult to compare learned query optimizers fairly?\",\"answer\":\"Different LQOs impose distinct rules for generating training data, use arbitrary train/validation splits, and evaluate on voluntary benchmark-query splits, breaking comparability.\"},{\"question\":\"What does the paper contribute to improve evaluation of LQOs?\",\"answer\":\"It proposes a new end-to-end benchmarking framework and walks through the ML pipeline stages specifically for learned query optimization.\"},{\"question\":\"What overall finding does the evaluation report?\",\"answer\":\"The evaluation shows PostgreSQL outperforms the learned query optimizers in almost all experiments across different train/test splits.\"}]","Is Your Learned Query Optimizer Behaving As You Expect? - A Machine Learning Perspective | PDF",1785817832,33,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"is-your-learned-query-optimizer-behaving-as-you-expect-a-machine-learning-perspective","",{"@graph":36,"@context":85},[37,54,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/is-your-learned-query-optimizer-behaving-as-you-expect-a-machine-learning-perspective/123651/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":62,"encodingFormat":61,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-04",true,{"@type":65,"interactionType":66,"userInteractionCount":4},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"Why is it difficult to compare learned query optimizers fairly?","Question",{"text":75,"@type":76},"Different LQOs impose distinct rules for generating training data, use arbitrary train/validation splits, and evaluate on voluntary benchmark-query splits, breaking comparability.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"What does the paper contribute to improve evaluation of LQOs?",{"text":80,"@type":76},"It proposes a new end-to-end benchmarking framework and walks through the ML pipeline stages specifically for learned query optimization.",{"name":82,"@type":73,"acceptedAnswer":83},"What overall finding does the evaluation report?",{"text":84,"@type":76},"The evaluation shows PostgreSQL outperforms the learned query optimizers in almost all experiments across different train/test splits.","https://schema.org",{"og:url":52,"og:type":87,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":89,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":46,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":46,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":46,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":46,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]