LLM-Driven Neural Network Generation with Same-Family Architecture Guidance Disentangling Transfer and Adaptation

Jul 16, 2026

Upload by Levi

Large language models can propose neural-network modifications, yet unconstrained generation frequently yields invalid or unsafe code. This paper studies a controlled setting: improving a weak target model using a stronger same-family source model from a neural-network database. A source-guided candidate-generation protocol uses non-source controls, source-conditioned candidates, and an equal-budget no-LLM hp-copy ablation. Results separate validity from accuracy and choose the best valid candidate only when it improves the target, showing consistent gains across multiple datasets and families.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages10

Reading Time25 min

Abstract

Introduction

Related Work

LLM-Driven Neural Network Generation with Same-Family Architecture Guidance: Disentangling Transfer and Adaptation Kabir Dev Paul Baghel Radu Timofte Dmitry Ignatov Computer Vision Lab, CAIDAS, University ofW¨urzburg, Germany arXiv :2607 .05704v 1 [ cs .LG] 6 Jul 2026 Abstract Large language models (LLMs) can generate neuralnetwork modifications, but unrestricted generation is often invalid or harmful. This paper studies a narrower setting: improving a weak target model using a stronger same-family source model from a neural-network database. We propose a source-guided candidate-generation protocol with non-source controls, source-conditioned candidates, and a no-LLM hp copy ablation under equal evaluation budgets. The protocol reports validity separately from accuracy and selects the best valid candidate only when it improves the target. On CIFAR-10, the strongest source-guided candidate reaches 0.5049 accuracy versus 0.2398 for the best non-source candidate, a +0.2651 advantage, while improving a weak target originally at 0.1254; a five-epoch check preserves the gain at 0.7686 versus 0.4839. On SVHN AlexNet with DeepSeek-Coder-6.7B, source-guided transfer reaches 0.7880 versus 0.2254, a +0. 5626 advantage; a fresh repeat reaches 0.8069 versus 0.2509, a +0.5560 advantage. Direct source-recipe copy produces 0. 1959 on SVHN AlexNet, matching the original target, while hp transfer reaches 0.7880, showing that the LLM adapts rather than copies the source recipe. Family-level analysis shows the clearest positive signals for AlexNet, with 6/8 wins across SVHN, Imagenette, and CelebA-Gender, and alt nn1, with 8/10 wins on CIFAR- 10. 1. Introduction LLM-based neural-network generation is attractive because it searches directly over executable model code and training recipes. Instead of designing a hand-written search space, one can ask a model to propose candidate modifications and then train the result. Prior work in the ABrain/LEMUR ecosystem has explored LLMs for architecture generation, prompt design, hyperparameter tuning, and iterative NAS with feedback memory [5, 8, 10, 11] . The practical failure mode is clear: unrestricted generation often produces invalid or low-quality code. This paper asks the following scientific question: Given a weak target model, does a stronger samefamily source model help an LLM generate better valid candidates than a non-source baseline under the same evaluation budget? This question is useful because it separates two different effects. A weak model may improve simply because the LLM changes the learning rate, batch size, or transform. That is a non-source baseline effect. The source-guided effect only exists if access to a strong source model improves the outcome beyond the non-source controls. The method is treated as a controlled candidategeneration algorithm. Validity, best valid accuracy, and mean valid accuracy are reported separately so that codegeneration reliability and model quality are not conflated. Contributions. This paper makes the following contributions: • It defines a source-guided weak-target candidategeneration protocol with non-source controls, sourceconditioned candidates, and a no-LLM hp copy ablation baseline. • It provides evidence for two regimes, recipe-transfer and recipe-adaptation, characterized by the hp copy decomposition. • It reports architecture-family-level win rates showing where source guidance works reliably and where it does not. 2. Related Work This work sits at the intersection of neural architecture search, hyperparameter optimization, LLM code generation, and retrieval-conditioned generation. Neural architecture search and AutoML. Classical neural architecture search (NAS) studies how to automate network design through a search space, a search strategy, and an evaluation strategy [6] . Early work used reinforcement learning controllers to generate architectures [26], while later methods used evolutionary searc

FAQs

What is the central problem this paper addresses for LLM-based neural network generation?

Unrestricted LLM generation often produces invalid or low-quality executable model code. The paper examines how to improve a weak target model under a controlled, source-guided setup while keeping evaluation budgets equal.

How does the proposed method distinguish transfer from adaptation?

It introduces a no-LLM hp-copy ablation and reports outcomes under regimes consistent with recipe-transfer and recipe-adaptation. The paper argues that the LLM adapts rather than directly copying the source recipe when gains match transfer/adaptation signals.

What evidence shows the benefit of same-family source guidance over non-source baselines?

On CIFAR-10, the strongest source-guided candidate achieves 0.5049 accuracy versus 0.2398 for the best non-source candidate, and similar advantages are reported on SVHN AlexNet with DeepSeek-Coder-6.7B. A repeated run further confirms the improvement.

Jul 16, 2026

Upload by Levi