[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-86170-en":3,"doc-seo-86170-105":29,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},86170,962075114101,"Seraphina","https://ap-avatar.wpscdn.com/avatar/e000253a75eb197efd?x-image-process=image/resize,m_fixed,w_180,h_180&k=1780044092746381165",8,"Research & Report","REPTRAN: Search-Based Repair of Transformer Models","REPTRAN provides a search-based method to repair misbehaving Transformer models used in AI-enabled software. It focuses on feed-forward networks (FFNs) and detects suspicious weights by combining a variance-based neuron score with an existing bidirectional score. Selected weights are iteratively optimized using differential evolution to maximize repaired misbehaviors while limiting disruption to correct behaviors. Experiments on 18 fault benchmarks from CIFAR-100 and Tiny-ImageNet show a 74.7% average repair rate, outperforming random selection and ARACHNE.","REPTRAN: Search-Based Repair of  \nTransformer Models  \nYuta Ishimoto* , Paolo Arcaini†, Fuyuki Ishikawa†, Masanari Kondo‡, Naoyasu Ubayashi§ , and Yasutaka Kamei‡  \n*The University of Osaka, Japan, †National Institute of Informatics, Japan,  \n‡Kyushu University, Japan, §Waseda University, Japan  \narXiv :2607 . 1 1 193v 1 [ cs . SE] 13 Jul 2026  \nAbstract—To ensure the overall quality of AI-enabled software, not only traditional software components but also AI components need to be tested and repaired. Among AI components, Transformer models are increasingly integrated into software systems, which makes their misbehaviors critical. Although prior work in the software engineering community has proposed deep neural network (DNN) repair methods, most overlook Transformerspecific structures. We propose REPTRAN, a search-based repair method for Transformer models. It targets their feed-forward networks (FFNs), which play a central role in the architecture. REPTRAN identifies suspicious weights by combining two types of scores: a variance-based neuron score and an existing bidirectional score. It then iteratively optimizes these weights using differential evolution. Our evaluation includes 18 fault benchmarks constructed from CIFAR-100 and Tiny-ImageNet. We compare REPTRAN against three baselines: random weight selection, ARACHNE (a state-of-the-art DNN repair method), and ARACHNEW, which enables ARACHNE to control the number of selected weights. REPTRAN achieved an average repair rate of 74.7%, statistically outperforming random selection and ARACHNE across all benchmarks. Effect size analysis revealed that REPTRAN achieved higher repair rates than ARACHNEW regardless of the number of selected weights. These results suggest that REPTRAN is effective for enhancing the reliability of AI-enabled software.  \nIndex Terms—AI-enabled software, Transformer, Repair, Reliability  \nI. INTRODUCTION  \nThe Transformer [1] has become a critical neural network architecture underlying foundation models such as large language models (LLMs) . Since Transformer models are increasingly used in a wide range of AI-enabled software (e.g., autonomous driving [2]), misbehaviors of these models can significantly compromise the reliability of the entire system. Indeed, the OECD AI Incidents and Hazards Monitor has documented numerous cases of failures in autonomous driving systems [3], many of which resulted in severe injuries, fatalities, and significant financial losses.  \nOne promising direction for addressing such misbehaviorsis deep neural network (DNN) repair [4]–[11] . These methods are designed to complement full retraining: while full retraining aims to improve overall performance (e.g., accuracy), DNN repair targets specific types of misbehavior. It identifies and updates only the components of the model (e.g., neurons or weights) that are responsible for the misbehavior. This selective update seeks to correct the targeted misbehavior while preserving the existing correct behaviors.  \nGiven this background, there is a growing need for repair methods tailored to Transformer models; however, most existing DNN repair methods have focused on conventional networks (e.g., convolutional neural networks (CNNs) [5]–[7], [9]), with limited attention to Transformer models [10] . Unlike such conventional networks, Transformer models rely on unique architectural components such as multi-head selfattention (MSA) and feed-forward networks (FFNs) consisting of two fully connected layers with a non-linear activation in between [1] . Existing methods do not account for these components, indicating the need for repair methods designed for Transformer models.  \nTo this end, we propose REPTRAN, a novel repair approach for Transformer models. To identify which weights should be modified, REPTRAN computes a weight suspiciousness score, which reflects the contribution of each weight to themisbehavior. This score consists of two components: (1) a neuron score and (2) a bidir","cbCaia3nvQoBdjjB","https://ap.wps.com/l/cbCaia3nvQoBdjjB","pdf",503036,1,12,"English","en",105,"# Introduction\n# Background and Related Work","[{\"question\":\"What problem does REPTRAN address in Transformer-based AI software?\",\"answer\":\"REPTRAN targets misbehaviors in Transformer models deployed within AI-enabled software, where incorrect model outputs can undermine overall system reliability.\"},{\"question\":\"How does REPTRAN decide which weights to repair?\",\"answer\":\"It computes a weight suspiciousness score composed of a variance-based neuron score (based on intermediate FFN neuron activations) and a bidirectional score derived from ARACHNE, then selects high-suspiciousness weights.\"},{\"question\":\"What evaluation results demonstrate REPTRAN’s effectiveness?\",\"answer\":\"On 18 fault benchmarks from CIFAR-100 and Tiny-ImageNet using ViT, REPTRAN reaches an average repair rate of 74.7%, outperforming random selection and ARACHNE, and remains competitive in runtime.\"}]",1784209077,30,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":27},"reptran-search-based-repair-of-transformer-models","",{"@graph":35,"@context":85},[36,53,68],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/reptran-search-based-repair-of-transformer-models/86170/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-27","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What problem does REPTRAN address in Transformer-based AI software?","Question",{"text":75,"@type":76},"REPTRAN targets misbehaviors in Transformer models deployed within AI-enabled software, where incorrect model outputs can undermine overall system reliability.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"How does REPTRAN decide which weights to repair?",{"text":80,"@type":76},"It computes a weight suspiciousness score composed of a variance-based neuron score (based on intermediate FFN neuron activations) and a bidirectional score derived from ARACHNE, then selects high-suspiciousness weights.",{"name":82,"@type":73,"acceptedAnswer":83},"What evaluation results demonstrate REPTRAN’s effectiveness?",{"text":84,"@type":76},"On 18 fault benchmarks from CIFAR-100 and Tiny-ImageNet using ViT, REPTRAN reaches an average repair rate of 74.7%, outperforming random selection and ARACHNE, and remains competitive in runtime.","https://schema.org",{"og:url":51,"og:type":87,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":89,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,122,127,130,134],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":45,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":45,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":45,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":28,"slug":121},"research-report",{"id":123,"doc_module":4,"doc_module_name":45,"category_name":124,"show_sort_weight":125,"slug":126},9,"Religion & Spirituality",20,"religion-spirituality",{"id":125,"doc_module":4,"doc_module_name":45,"category_name":128,"show_sort_weight":125,"slug":129},"World Cup","world-cup",{"id":131,"doc_module":4,"doc_module_name":45,"category_name":132,"show_sort_weight":131,"slug":133},10,"Lifestyle","lifestyle",{"id":135,"doc_module":4,"doc_module_name":45,"category_name":136,"show_sort_weight":106,"slug":137},19,"General","general"]