[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-125871-en":3,"doc-seo-125871-105":31,"detail-sidebar-cat-0-en-105":92},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":21,"is_downloadable":21,"audit_status":21,"page_count":22,"language":23,"language_code":24,"site_id":25,"html_lang":24,"table_of_contents":26,"faqs":27,"seo_title":28,"seo_description":14,"update_tm":29,"read_time":30},125871,1099523885336,"Violet","https://ap-avatar.wpscdn.com/davatar_276721f389ce27ea32af1340a28f341c",8,"Research & Report","Machine Learning for Columnar High Energy Physics Analysis - AI integration into IRIS-HEP Analysis Grand Challenge","Machine learning has become a core element of high energy physics data analyses and is expected to expand with higher luminosity conditions at the LHC. The work surveys how physicists apply supervised and unsupervised learning for tasks such as jet classification and searches beyond the Standard Model. It demonstrates integration of ML training and inference inside the IRISHEP Analysis Grand Challenge pipeline, leveraging open data for community uptake. Different inference approaches, including external server execution, are evaluated and compared to support scalable analysis workflows.","Machine Learning for Columnar High Energy Physics Analysis  \nElliott Kauffman 1 , ∗ , Alexander Held2 ,, and Oksana Shadura3 ,  \n1Princeton University  \n2University of Wisconsin-Madison  \n3University of Nebraska-Lincoln  \nAbstract. Machine learning (ML) has become an integral component of high energy physics data analyses and is likely to continue to grow in prevalence.  \nPhysicists are incorporating ML into many aspects of analysis, from using boosted decision trees to classify particle jets to using unsupervised learning to search for physics beyond the Standard Model. Since ML methods have become so widespread in analysis and these analyses need to be scaled up for HL-LHC data, neatly integrating ML training and inference into scalable analysis workflows will improve the user experience of analysis in the HL-LHC era.  \nWe present the integration of ML training and inference into the IRISHEP Analysis Grand Challenge pipeline to provide an example of how this integration can look like in a realistic analysis environment. We also utilize Open Data to ensure the project’s reach to the broader community. Different approaches for performing ML inference at analysis facilities are investigated and compared, including performing inference through external servers.  \nSince ML techniques are applied for many different types of tasks in physics analyses, we showcase options for ML integration that can be applied to various inference needs.  \n1 Introduction  \nMachine learning has increased in prevalence across many areas in high energy physics, a trend which is likely to continue through the high luminosity era of the LHC (HL-LHC) . The large integrated luminosity increase initiated by this upgrade will introduce much higher data volume and event size. Machine learning is well-positioned to help with many of the resulting challenges [1] . Analysis in high energy physics is one area where machine learning promises to assist, including tasks such as event classification, jet tagging, and unsupervised anomaly detection.  \nTraditionally, high energy physicists have depended on compiled languages such as C++ due to the need for fast processing speed and efficient memory usage. Analyses have been typically structured in \"event-loop\" format, in which event-level quantities are calculated and handled in a for-loop over events. An alternative to this approach is \"columnar  \n∗ e-mail: [ek8842@princeton.edu](ek8842@princeton.edu)  \n© The Authors, published by EDP Sciences. This is an open access article distributed under the terms of the Creative Commons Attribution License 4.0 ([https://creativecommons.org/licenses/by/4.0/](https://creativecommons.org/licenses/by/4.0/)).  \nanalysis\". In practice, there are many different definitions of this term, but for the purpose of the following discussion, columnar analysis refers to array programming methods and user interfaces that focus on handling arrays-at-a-time instead of users needing to handle each event separately. This allows one to avoid writing explicit for-loops and exploit parallel computations. Many users find the structure of columnar analysis pipelines more readable and enjoy the level of interactivity experienced when developing analysis. These features combine to offer a faster time-to-insight for physicists performing analysis [2] [3] .  \nColumnar analyses are also popular in other fields, allowing HEP columnar analyses to utilize well-maintained tools that are broadly used outside of HEP. This may be especially useful when it comes to machine learning, since Python offers an extensive and well-maintained collection of machine learning tools. We present a columnar high energy physics analysis pipeline with integrated machine learning inference in the context of the IRIS-HEP Analysis Grand Challenge.  \n2 The Analysis Grand Challenge  \nThe Institute for Research and Innovation in Software for High Energy Physics (IRISHEP) [4] develops software and software infrastructure in anticipation o","cbCaiiY7Leu7TCEH","https://ap.wps.com/l/cbCaiiY7Leu7TCEH","pdf",2061953,4,1,7,"English","en",105,"# Introduction\n## Columnar analysis and motivation\n# The Analysis Grand Challenge\n## CMS Open Data Task Description\n## Workflow steps and tooling","[{\"question\":\"Why is machine learning increasingly important for high energy physics analyses?\",\"answer\":\"Higher luminosity at the HL-LHC increases data volume and event size, and ML is positioned to address challenges in tasks like event classification, jet tagging, and anomaly detection.\"},{\"question\":\"What does the paper mean by “columnar analysis”?\",\"answer\":\"Columnar analysis refers to array programming methods and interfaces that process arrays-at-a-time, avoiding explicit event-by-event loops and enabling parallel computations and faster development.\"},{\"question\":\"How is machine learning integrated in the IRIS-HEP Analysis Grand Challenge pipeline?\",\"answer\":\"The paper presents an example integration of ML training and inference into the IRISHEP AGC workflow, including investigation of inference strategies such as running inference through external servers.\"}]","Machine Learning for Columnar High Energy Physics Analysis - AI integration into IRIS-HEP Analysis Grand Challenge | PDF",1785901736,18,{"code":4,"msg":32,"data":33},"ok",{"site_id":25,"language":24,"slug":34,"title":13,"keywords":35,"description":14,"schema_data":36,"social_meta":87,"head_meta":89,"extra_data":91,"updated_unix":29},"machine-learning-for-columnar-high-energy-physics-analysis-ai-integration-into-iris-hep-analysis-grand-challenge","",{"@graph":37,"@context":86},[38,54,69],{"@type":39,"itemListElement":40},"BreadcrumbList",[41,45,49,52],{"item":42,"name":43,"@type":44,"position":21},"https://docshare.wps.com","Home","ListItem",{"item":46,"name":47,"@type":44,"position":48},"https://docshare.wps.com/document/","Document",2,{"item":50,"name":12,"@type":44,"position":51},"https://docshare.wps.com/document/research-report/",3,{"item":53,"name":13,"@type":44,"position":20},"https://docshare.wps.com/document/machine-learning-for-columnar-high-energy-physics-analysis-ai-integration-into-iris-hep-analysis-grand-challenge/125871/",{"url":53,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":24,"description":14,"dateModified":62,"datePublished":63,"encodingFormat":61,"isAccessibleForFree":64,"interactionStatistic":65},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":42,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-23","2026-08-05",true,{"@type":66,"interactionType":67,"userInteractionCount":20},"InteractionCounter",{"@type":68},"ViewAction",{"@type":70,"mainEntity":71},"FAQPage",[72,78,82],{"name":73,"@type":74,"acceptedAnswer":75},"Why is machine learning increasingly important for high energy physics analyses?","Question",{"text":76,"@type":77},"Higher luminosity at the HL-LHC increases data volume and event size, and ML is positioned to address challenges in tasks like event classification, jet tagging, and anomaly detection.","Answer",{"name":79,"@type":74,"acceptedAnswer":80},"What does the paper mean by “columnar analysis”?",{"text":81,"@type":77},"Columnar analysis refers to array programming methods and interfaces that process arrays-at-a-time, avoiding explicit event-by-event loops and enabling parallel computations and faster development.",{"name":83,"@type":74,"acceptedAnswer":84},"How is machine learning integrated in the IRIS-HEP Analysis Grand Challenge pipeline?",{"text":85,"@type":77},"The paper presents an example integration of ML training and inference into the IRISHEP AGC workflow, including investigation of inference strategies such as running inference through external servers.","https://schema.org",{"og:url":53,"og:type":88,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":90,"canonical":53},"index,follow",{"doc_id":7,"site_id":25},{"code":4,"msg":5,"data":93},[94,98,102,106,111,116,120,123,128,131,135],{"id":21,"doc_module":4,"doc_module_name":47,"category_name":95,"show_sort_weight":96,"slug":97},"Story & Novel",90,"story-novel",{"id":48,"doc_module":4,"doc_module_name":47,"category_name":99,"show_sort_weight":100,"slug":101},"Literature",80,"literature",{"id":20,"doc_module":4,"doc_module_name":47,"category_name":103,"show_sort_weight":104,"slug":105},"Exam",70,"exam",{"id":107,"doc_module":4,"doc_module_name":47,"category_name":108,"show_sort_weight":109,"slug":110},5,"Comic",60,"comic",{"id":112,"doc_module":4,"doc_module_name":47,"category_name":113,"show_sort_weight":114,"slug":115},6,"Technology",50,"technology",{"id":22,"doc_module":4,"doc_module_name":47,"category_name":117,"show_sort_weight":118,"slug":119},"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":47,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":47,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":47,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":47,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":47,"category_name":137,"show_sort_weight":107,"slug":138},19,"General","general"]