[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-127068-en":3,"doc-seo-127068-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},127068,962084931830,"Theodore","https://ap-avatar.wpscdn.com/davatar_a8503ba1806abce46bf441b54a3ca4cd",8,"Research & Report","Exploring the Use of Software Product Lines for the Combination of Machine Learning Models","Large Language Models (LLMs) and other machine learning models deliver high-quality responses, yet their training and inference costs remain substantial. To address this, the paper focuses on model merging and Mixture of Experts (MoE) architectures that build larger models by combining existing ones. It proposes a Software Product Lines (SPL) approach to democratize the specification and training of combined ML architectures. Starting from an initial feature model, users can define model types, combination strategies, and MoE task-to-expert mappings. A code generation workflow transforms feature configurations into deployable merged LLM models.","Exploring the Use of Software Product Lines for the Combination  \nof Machine Learning Models  \nMarcos Gomez-Vazquez  \n[marcos.gomez@list.lu](marcos.gomez@list.lu)  \nLuxembourg Institute of Science and Technology Esch-sur-Alzette, Luxembourg  \nJordi Cabot  \n[jordi.cabot@list.lu](jordi.cabot@list.lu)  \nLuxembourg Institute of Science and Technology Esch-sur-Alzette, Luxembourg University of Luxembourg Esch-sur-Alzette, Luxembourg  \nABSTRACT  \nThe size of Large Language Models (LLMs), and Machine Learning (ML) models in general, is a key factor of their capacity and quality of their responses. But it comes with a high cost, both during the training and the model execution phase. Recently, various model merging techniques and Mixture of Experts (MoE) architectures are gaining popularity as they enable the creation of large models by combining other existing ones (the \"experts\" in the MoE approach) . Creating these combinations remains a deep technical task with many possible configurations to consider. In this sense, this paper aims to democratize the creation of combined ML models by presenting a product line approach to the specification and training of this type of ML architectures from an initial feature model that helps users define, among other aspects, the type of models they want to combine, the combination strategy and even, for the MoE approach, the tasks that should be associated to each expert.  \nCCS CONCEPTS  \n• Software and its engineering → Software product lines; Model-driven software engineering; • Computing methodologies → Machine learning approaches.  \nKEYWORDS  \nSoftware Product Line, Feature Model, Machine Learning, Large Language Model, Model Merging, Mixture of Experts  \nACM Reference Format:  \nMarcos Gomez-Vazquez and Jordi Cabot. 2024. Exploring the Use of Software Product Lines for the Combination of Machine Learning Models. In 28th ACM International Systems and Software Product Line Conference (SPLC ’24), September 2–6, 2024, Dommeldange, Luxembourg. ACM, New York, NY, USA, 4 pages. [https://doi.org/10.1145/3646548.3676599](https://doi.org/10.1145/3646548.3676599)  \n1 INTRODUCTION  \nMany of the new Large Language Models (LLMs) topping the LLM leaderboards are not trained from scratch but created by combining other preexisting LLMs. This is not only cheaper (both, in terms  \nPublication rights licensed to ACM. ACM acknowledges that this contribution was authored or co-authored by an employee, contractor or affiliate of a national government. As such, the Government retains a nonexclusive, royalty-free right to publish or reproduce this article, or to allow others to do so, for Government purposes only. Request permissions from owner/author(s) .  \nSPLC’24, September 2–6, 2024, Dommeldange, Luxembourg  \n© 2024 Copyright held by the owner/author(s) . Publication rights licensed to ACM. ACM ISBN 979-8-4007-0593-9/24/09  \n[https://doi.org/10.1145/3646548.3676599](https://doi.org/10.1145/3646548.3676599)  \nof time and money) but it also allows to push the limits of state-ofthe-art architectures by building models that excel at specific tasks and domains by combining the knowledge of the merged models.  \nWe are now witnessing an explosion of model composition strategies and techniques but there is a lack of well-defined methods to guide users interested in creating them. As with any other kind of emerging software, it is necessary to define precise requirements and properties of composite AI models in order to abstract its development process from the underlying technology and to enable non-technical users to create it. This paper advocates for the use of a Software Product Lines (SPLs) to define and generate LLM compositions by means of different merging algorithms.  \nIndeed, Software Modeling and Generative Software Development is key to defining the properties, requirements, commonalities and variabilities in system families. Machine Learning (ML) is establishing a new paradigm in Computer Science but we stil","cbCaimS7Aw4KpyxP","https://ap.wps.com/l/cbCaimS7Aw4KpyxP","pdf",724623,1,4,"English","en",105,"# Introduction\n# Background\n# Feature Model for LLM Combinations\n# Transforming Feature Configurations into Merged Models\n# Tool Support\n# Related Work\n# Conclusions and Further Work","[{\"question\":\"Why are model-merging and Mixture of Experts approaches needed for machine learning models?\",\"answer\":\"They enable building larger models by combining existing ones, improving the ability to target specific tasks and domains while helping manage costs compared with training from scratch.\"},{\"question\":\"How does the paper use a Software Product Lines approach?\",\"answer\":\"It proposes an SPL method for LLM combinations by introducing a feature model that captures dimensions and variability, enabling users to configure what should be combined and how.\"},{\"question\":\"What is the role of feature configurations in producing an actual merged model?\",\"answer\":\"A generated workflow transforms a feature configuration into a combined, ready-to-use LLM, using Mergekit for the generation phase and additional tool support for the process.\"}]","Exploring the Use of Software Product Lines for the Combination of Machine Learning Models | PDF",1785936647,10,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"exploring-the-use-of-software-product-lines-for-the-combination-of-machine-learning-models","",{"@graph":36,"@context":85},[37,53,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":21},"https://docshare.wps.com/document/exploring-the-use-of-software-product-lines-for-the-combination-of-machine-learning-models/127068/",{"url":52,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":41,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-08-22","2026-08-05",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"Why are model-merging and Mixture of Experts approaches needed for machine learning models?","Question",{"text":75,"@type":76},"They enable building larger models by combining existing ones, improving the ability to target specific tasks and domains while helping manage costs compared with training from scratch.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"How does the paper use a Software Product Lines approach?",{"text":80,"@type":76},"It proposes an SPL method for LLM combinations by introducing a feature model that captures dimensions and variability, enabling users to configure what should be combined and how.",{"name":82,"@type":73,"acceptedAnswer":83},"What is the role of feature configurations in producing an actual merged model?",{"text":84,"@type":76},"A generated workflow transforms a feature configuration into a combined, ready-to-use LLM, using Mergekit for the generation phase and additional tool support for the process.","https://schema.org",{"og:url":52,"og:type":87,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":89,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,134],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":21,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":46,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":46,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":29,"doc_module":4,"doc_module_name":46,"category_name":132,"show_sort_weight":29,"slug":133},"Lifestyle","lifestyle",{"id":135,"doc_module":4,"doc_module_name":46,"category_name":136,"show_sort_weight":106,"slug":137},19,"General","general"]