[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-84067-en":3,"doc-seo-84067-105":29,"detail-sidebar-cat-0-en-105":90},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},84067,1099514067415,"Rowan","https://ap-avatar.wpscdn.com/avatar/100002539d78ffe74a7?x-image-process=image/resize,m_fixed,w_180,h_180&k=1779092875211072502",8,"Research & Report","Evaluating Fine-Tuning and Metrics for Neural Decompilation of Dart AOT Binaries","Neural decompilation is treated as a code-generation problem, yet evaluation methods lag for modern languages. The study performs a systematic empirical analysis of fine-tuning effectiveness and metric validity for Dart Ahead-of-Time (AOT) decompilation. Six fine-tuned model variants are tested across three base architectures (4B–8B) using CodeBLEU, compile@k, and pass@k on a new 154-task HumanEval-Dart benchmark, supported by paired statistical tests and confidence intervals.","arXiv :2607 .06 125v 1 [ cs . SE] 7 Jul 2026  \nEvaluating Fine-Tuning and Metrics for Neural Decompilation of Dart AOT Binaries  \nRAAFAT ABUALAZM, Cairo University, Egypt AYMAN ABOELHASSAN, Cairo University, Egypt AMR G. WASSAL, Cairo University, Egypt  \nNeural decompilation is increasingly studied as a code-generation problem, yet its evaluation methodology remains underdeveloped for modern languages. We present a systematic empirical study of fine-tuning effectiveness and metric validity for Dart Ahead-of-Time (AOT) neural decompilation, evaluating six finetuned model variants across three base architectures (4B–8B parameters) with three complementary metrics:  \nCodeBLEU for semantic similarity, compile@k for syntactic validity, and pass@k for functional correctness on a new 154-task HumanEval-Dart benchmark.  \nOur study yields three principal findings, grounded in paired task-level statistical tests (McNemar’s exact test and bootstrap confidence intervals) applied to each of the six fine-tuned variants.  \nFirst, no fine-tuning configuration produces a statistically significant pass@k improvement. The sole directionally positive case (decompiler-v1 at 4B) yields +0 . 71 pp with a 95% CI of [−1 .17, +2 . 60] pp (McNemar 􀀿 =0. 21) . By contrast, fine-tuning the strongest base (Qwen3-8B) causes a highly significant regression of −5 . 65 pp (95% CI [−8 .25, −3 . 38], 􀀿 \u003C0. 001), with 0 tasks gained and 22 tasks lost. This capacity-dependent trend is consistent across the three architectures tested, but cannot be confirmed as a general mechanism without broader scale sweeps.  \nSecond, cross-lingual interference from Swift training is highly significant at 4B (Δpass@1 = −2 .66 pp,􀀿 \u003C0. 001) but statistically indistinguishable from zero at 8B. This pattern is consistent with the scaling hypothesis that larger models better accommodate multi-language training. The optimization mismatch between Dart (AOT) and Swift (-O0) remains a confounding factor we cannot fully isolate.  \nThird, we demonstrate metric divergence: CodeBLEU and compile@k can improve significantly while pass@k moves in the opposite direction. For v3, for example, ΔCodeBLEU=+0 .051 (􀀿 =0. 001) and Δcompile@1=+16pp, yet pass@k regresses. This finding has implications beyond decompilation for any LLM code generation task where fine-tuning targets superficial similarity to reference implementations.  \nError analysis reveals that assembly sequence length is the strongest predictor of task difficulty among the factors examined (􀀿 =0. 001), with a capability cliff at approximately 200 instructions. Traditional code complexity metrics show negligible predictive power. We contribute the HumanEval-Dart benchmark, a Dart-adapted CodeBLEU implementation, and empirical evidence that pass@k must be adopted as the primary evaluation metric for neural decompilation.  \nAdditional Key Words and Phrases: neural decompilation, reverse engineering, empirical evaluation, large language models, Dart, functional correctness, code evaluation  \narXiv Notice. This manuscript is a preprint submitted to ACM Transactions on Software Engineering and Methodology (TOSEM) and is currently under peer review. It has not yet been accepted for publication by ACM, and this version may differ from any final accepted Version of Record. The authors retain copyright to this preprint. If accepted, the arXiv record will be updated with the ACM citation and DOI.  \nAuthors’ Contact Information: Raafat Abualazm, Cairo University, Computer Engineering Department, Faculty of Engineering, Giza, Egypt, [raafat.202210476@eng-st.cu.edu.eg](raafat.202210476@eng-st.cu.edu.eg); Ayman AboElhassan, Cairo University, Computer Engineering Department, Faculty of Engineering, Giza, Egypt, [ayman.abo.elmaaty@eng.cu.edu.eg](ayman.abo.elmaaty@eng.cu.edu.eg); Amr G. Wassal, Cairo University, Computer Engineering Department, Faculty of Engineering, Giza, Egypt, [wassal@eng.cu.edu.eg](wassal@eng.cu.edu.eg).  \n0:2 Abualazm et al.  \n1 I","cbCaisSiLRVEMt1A","https://ap.wps.com/l/cbCaisSiLRVEMt1A","pdf",737414,1,23,"English","en",105,"# Introduction\n## The Decompilation Challenge for Modern Languages\n## Limitations of Existing Neural Decompilation Approaches\n# Experimental Study\n## Fine-tuning Variants and Model Architectures\n## Metrics and Benchmark Design\n# Empirical Findings\n## Pass@k Impact\n## Cross-lingual Interference\n## Metric Divergence and Implications\n# Error Analysis and Contributions","[{\"question\":\"What is the document’s main objective?\",\"answer\":\"It evaluates how fine-tuning affects neural decompilation for Dart AOT binaries and tests whether common metrics are valid indicators of functional correctness.\"},{\"question\":\"Which metrics and benchmark are used in the study?\",\"answer\":\"The work uses CodeBLEU for semantic similarity, compile@k for syntactic validity, and pass@k for functional correctness on a new 154-task HumanEval-Dart benchmark.\"},{\"question\":\"What does the study conclude about pass@k after fine-tuning?\",\"answer\":\"No fine-tuning configuration yields a statistically significant pass@k improvement; one small directional increase is not significant, while fine-tuning the strongest base shows a highly significant regression.\"}]",1784192384,58,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":85,"head_meta":87,"extra_data":89,"updated_unix":27},"evaluating-fine-tuning-and-metrics-for-neural-decompilation-of-dart-aot-binaries","",{"@graph":35,"@context":84},[36,53,67],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/evaluating-fine-tuning-and-metrics-for-neural-decompilation-of-dart-aot-binaries/84067/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":61,"encodingFormat":60,"isAccessibleForFree":62,"interactionStatistic":63},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-16",true,{"@type":64,"interactionType":65,"userInteractionCount":4},"InteractionCounter",{"@type":66},"ViewAction",{"@type":68,"mainEntity":69},"FAQPage",[70,76,80],{"name":71,"@type":72,"acceptedAnswer":73},"What is the document’s main objective?","Question",{"text":74,"@type":75},"It evaluates how fine-tuning affects neural decompilation for Dart AOT binaries and tests whether common metrics are valid indicators of functional correctness.","Answer",{"name":77,"@type":72,"acceptedAnswer":78},"Which metrics and benchmark are used in the study?",{"text":79,"@type":75},"The work uses CodeBLEU for semantic similarity, compile@k for syntactic validity, and pass@k for functional correctness on a new 154-task HumanEval-Dart benchmark.",{"name":81,"@type":72,"acceptedAnswer":82},"What does the study conclude about pass@k after fine-tuning?",{"text":83,"@type":75},"No fine-tuning configuration yields a statistically significant pass@k improvement; one small directional increase is not significant, while fine-tuning the strongest base shows a highly significant regression.","https://schema.org",{"og:url":51,"og:type":86,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":88,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":91},[92,96,100,104,109,114,119,122,127,130,134],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":93,"show_sort_weight":94,"slug":95},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":97,"show_sort_weight":98,"slug":99},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":101,"show_sort_weight":102,"slug":103},"Exam",70,"exam",{"id":105,"doc_module":4,"doc_module_name":45,"category_name":106,"show_sort_weight":107,"slug":108},5,"Comic",60,"comic",{"id":110,"doc_module":4,"doc_module_name":45,"category_name":111,"show_sort_weight":112,"slug":113},6,"Technology",50,"technology",{"id":115,"doc_module":4,"doc_module_name":45,"category_name":116,"show_sort_weight":117,"slug":118},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":120,"slug":121},30,"research-report",{"id":123,"doc_module":4,"doc_module_name":45,"category_name":124,"show_sort_weight":125,"slug":126},9,"Religion & Spirituality",20,"religion-spirituality",{"id":125,"doc_module":4,"doc_module_name":45,"category_name":128,"show_sort_weight":125,"slug":129},"World Cup","world-cup",{"id":131,"doc_module":4,"doc_module_name":45,"category_name":132,"show_sort_weight":131,"slug":133},10,"Lifestyle","lifestyle",{"id":135,"doc_module":4,"doc_module_name":45,"category_name":136,"show_sort_weight":105,"slug":137},19,"General","general"]