[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-seo-141198-105":3,"detail-sidebar-cat-0-en-105":81,"doc-detail-141198-en":130},{"code":4,"msg":5,"data":6},0,"ok",{"site_id":7,"language":8,"slug":9,"title":10,"keywords":11,"description":12,"schema_data":13,"social_meta":74,"head_meta":76,"extra_data":78,"updated_unix":80},105,"en","seki-self-evolution-and-knowledge-inspiration-based-neural-architecture-search-via-large-language-models","SEKI - Self-Evolution and Knowledge Inspiration based Neural Architecture Search via Large Language Models","","SEKI proposes an LLM-based neural architecture search approach that follows a chain-of-thought inspired workflow. The method operates in two stages: self-evolution and knowledge distillation. During self-evolution, an iterative refinement mechanism improves architectures using performance feedback, gradually building a repository of high-performing designs. In knowledge distillation, the LLM extracts common patterns from validated architectures to generate optimized new designs. Experiments report state-of-the-art results across datasets and search spaces with only 0.05 GPU-days and strong generalization on multiple tasks.",{"@graph":14,"@context":73},[15,34,56],{"@type":16,"itemListElement":17},"BreadcrumbList",[18,23,27,31],{"item":19,"name":20,"@type":21,"position":22},"https://docshare.wps.com","Home","ListItem",1,{"item":24,"name":25,"@type":21,"position":26},"https://docshare.wps.com/document/","Document",2,{"item":28,"name":29,"@type":21,"position":30},"https://docshare.wps.com/document/research-report/","Research & Report",3,{"item":32,"name":10,"@type":21,"position":33},"https://docshare.wps.com/document/seki-self-evolution-and-knowledge-inspiration-based-neural-architecture-search-via-large-language-models/141198/",4,{"url":32,"name":10,"@type":35,"image":36,"author":41,"headline":10,"publisher":44,"fileFormat":47,"inLanguage":8,"description":12,"dateModified":48,"datePublished":49,"encodingFormat":47,"isAccessibleForFree":50,"interactionStatistic":51},"DigitalDocument",{"url":37,"@type":38,"width":39,"height":40},"https://docshare.wps.com/thumbnails/seki-self-evolution-and-knowledge-inspiration-based-neural-architecture-search-via-large-language-models/141198.png","ImageObject",300,407,{"name":42,"@type":43},"\tWilliam","Person",{"url":19,"name":45,"@type":46},"DocShare","Organization","application/pdf","2026-10-29","2026-08-25",true,{"@type":52,"interactionType":53,"userInteractionCount":55},"InteractionCounter",{"@type":54},"ViewAction",9,{"@type":57,"mainEntity":58},"FAQPage",[59,65,69],{"name":60,"@type":61,"acceptedAnswer":62},"What is SEKI’s core idea for neural architecture search?","Question",{"text":63,"@type":64},"SEKI uses large language models to drive NAS through two stages: self-evolution and knowledge distillation, extracting and refining architecture designs based on feedback and patterns.","Answer",{"name":66,"@type":61,"acceptedAnswer":67},"How does the self-evolution stage work?",{"text":68,"@type":64},"It iteratively refines architectures by having the LLM generate optimization strategies from the current architecture and its performance metrics, gradually improving results and building a repository.",{"name":70,"@type":61,"acceptedAnswer":71},"What is the role of the knowledge distillation stage?",{"text":72,"@type":64},"It summarizes validated high-quality architectures to distill common design patterns, enabling the LLM to directly generate new optimized designs.","https://schema.org",{"og:url":32,"og:type":75,"og:title":10,"og:site_name":45,"og:description":12},"article",{"robots":77,"canonical":32},"index,follow",{"doc_id":79,"site_id":7},141198,1787650644,{"code":4,"msg":82,"data":83},"success",[84,88,92,96,101,106,111,115,119,122,126],{"id":22,"doc_module":4,"doc_module_name":25,"category_name":85,"show_sort_weight":86,"slug":87},"Story & Novel",90,"story-novel",{"id":26,"doc_module":4,"doc_module_name":25,"category_name":89,"show_sort_weight":90,"slug":91},"Literature",80,"literature",{"id":33,"doc_module":4,"doc_module_name":25,"category_name":93,"show_sort_weight":94,"slug":95},"Exam",70,"exam",{"id":97,"doc_module":4,"doc_module_name":25,"category_name":98,"show_sort_weight":99,"slug":100},5,"Comic",60,"comic",{"id":102,"doc_module":4,"doc_module_name":25,"category_name":103,"show_sort_weight":104,"slug":105},6,"Technology",50,"technology",{"id":107,"doc_module":4,"doc_module_name":25,"category_name":108,"show_sort_weight":109,"slug":110},7,"Healthcare",40,"healthcare",{"id":112,"doc_module":4,"doc_module_name":25,"category_name":29,"show_sort_weight":113,"slug":114},8,30,"research-report",{"id":55,"doc_module":4,"doc_module_name":25,"category_name":116,"show_sort_weight":117,"slug":118},"Religion & Spirituality",20,"religion-spirituality",{"id":117,"doc_module":4,"doc_module_name":25,"category_name":120,"show_sort_weight":117,"slug":121},"World Cup","world-cup",{"id":123,"doc_module":4,"doc_module_name":25,"category_name":124,"show_sort_weight":123,"slug":125},10,"Lifestyle","lifestyle",{"id":127,"doc_module":4,"doc_module_name":25,"category_name":128,"show_sort_weight":97,"slug":129},19,"General","general",{"code":4,"msg":82,"data":131},{"doc_id":79,"user_id":132,"nickname":42,"user_avatar":133,"doc_module":4,"category_id":112,"category_name":29,"doc_title":10,"doc_description":12,"doc_content":134,"file_id":135,"file_url":136,"file_type":137,"file_size":138,"view_count":55,"is_deleted":4,"is_public":22,"is_downloadable":22,"audit_status":22,"page_count":139,"language":140,"language_code":8,"site_id":7,"html_lang":8,"table_of_contents":141,"faqs":142,"seo_title":143,"seo_description":12,"update_tm":80,"read_time":144,"preview_page_count":4},5909887254083,"https://ap-avatar.wpscdn.com/davatar_276721f389ce27ea32af1340a28f341c","arXiv :2502 .20422v1 [ cs .CL] 27 Feb 2025  \nSEKI: Self-Evolution and Knowledge Inspiration based Neural Architecture Search via Large Language  \nModels  \nZicheng Cai∗ Yaohua Tang∗ Yutao Lai Hua Wang  \nZhi Chen † Hao Chen  \nMoore Threads AI  \nGuangDong University of Technology  \nAbstract  \nWe introduce SEKI, a novel large language model (LLM)-based neural architecture search (NAS) method. Inspired by the chain-of-thought (CoT) paradigm in modern LLMs, SEKI operates in two key stages: self-evolution and knowledge distillation.  \nIn the self-evolution stage, LLMs initially lack sufficient reference examples, so we implement an iterative refinement mechanism that enhances architectures based on performance feedback. Over time, this process accumulates a repository of high-performance architectures. In the knowledge distillation stage, LLMs analyze common patterns among these architectures to generate new, optimized designs.  \nCombining these two stages, SEKI greatly leverages the capacity of LLMs on NAS and without requiring any domain-specific data. Experimental results show that SEKI achieves state-of-the-art (SOTA) performance across various datasets and search spaces while requiring only 0.05 GPU-days, outperforming existing methods in both efficiency and accuracy. Furthermore, SEKI demonstrates strong generalization capabilities, achieving SOTA-competitive results across multiple tasks.  \n1 Introduction  \nDesigning high-performance deep neural network architectures requires significant human efforts and extensive experimentation. To accelerate the development of neural networks, Neural Architecture Search (NAS) has been introduced as an automated approach to efficiently and cost-effectively identify optimal network designs. In the early stages, methods like NASNet-A Zoph et al. [2018] and AmoebaNet-B Real et al. [2018] brought the concept of automated architecture search to life, despite their high computational costs of 3150 and 1800 GPU-days, respectively. Later, gradient-based methods Liu et al. [2018], Xu et al. [2019], Xiao et al. [2022] set a new trend by leveraging weight sharing and continuous relaxation techniques, reducing the search cost to as low as 0.4 GPU-days. However, the applicability of these methods was hindered by issues like unfair operation selection, which often resulted in performance collapse. At the same time, evolution-based methods such as EPCNAS-C Huang et al. [2022] and EAEPSO Yuan et al. [2023] achieved significant improvements inefficiency but continued to face challenges in delivering high performance. Additionally, training-free approaches like PINAT Lu et al. [2023] and SWAP-NAS Peng et al. [2024] struck a promising balance between search efficiency and performance. Nevertheless, these methods still face concerns regarding lack of theoretical guarantees and the disparity between proxy metrics and actual performance.  \n∗ Equal [contribution. tangyaohua28@gmail.com](contribution. tangyaohua28@gmail.com)[ ](contribution. tangyaohua28@gmail.com)†[Corresponding author. zhic@mthreads.com](Corresponding author. zhic@mthreads.com)  \nPreprint. Under review.  \nFigure 1: Speed-performance comparison of our proposed SEKI with other NAS methods on CIFAR- 10 (methods over 1 GPU day are not included) .  \nRecently, with the growing capabilities of large language models (LLMs), several studies have started exploring the use of LLMs for NAS Chen et al. [2023], Zhang et al. [2023], Nasir et al. [2024], Qin et al. [2024], Wang et al. [2024], Dong et al. [2023] . By enabling LLMs to output design principles, these methods significantly enhances interpretability, offering insights into the architecture design process. However, most of these studies are still in the early exploratory stages and have yet to achieve competitive results. LAPT-NAS Zhou et al. [2024] is one of the most successful recent works in this area. It introduced an innovative approach that uses LLMs to learn design principles from existing neu","cbCaioCXP6RRTmQ1","https://ap.wps.com/l/cbCaioCXP6RRTmQ1","pdf",724477,18,"English","# Introduction\n## Motivation and limitations of existing NAS methods\n## LLM-based NAS research direction\n# Proposed method\n## Self-evolution stage\n## Knowledge distillation stage\n# Experimental claims and results","[{\"question\":\"What is SEKI’s core idea for neural architecture search?\",\"answer\":\"SEKI uses large language models to drive NAS through two stages: self-evolution and knowledge distillation, extracting and refining architecture designs based on feedback and patterns.\"},{\"question\":\"How does the self-evolution stage work?\",\"answer\":\"It iteratively refines architectures by having the LLM generate optimization strategies from the current architecture and its performance metrics, gradually improving results and building a repository.\"},{\"question\":\"What is the role of the knowledge distillation stage?\",\"answer\":\"It summarizes validated high-quality architectures to distill common design patterns, enabling the LLM to directly generate new optimized designs.\"}]","SEKI - Self-Evolution and Knowledge Inspiration based Neural Architecture Search via Large Language Models | PDF",45]