[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-82347-en":3,"doc-seo-82347-105":29,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},82347,1374391974585,"Genevieve","https://ap-avatar.wpscdn.com/davatar_276721f389ce27ea32af1340a28f341c",8,"Research & Report","How Do Software Professionals Evaluate AI-Generated Code (Registered Report)","Generative AI tools such as GitHub Copilot, ChatGPT, and Claude are increasingly integrated into everyday software workflows, yet the evaluation practices used by software professionals for AI-generated code remain unclear. The registered report proposes a constructivist grounded theory study combining an initial survey, semi-structured interviews, and laddering interviews. Interviews will continue with 20–50 professionals iteratively until theoretical saturation, building a theory grounded in accounts of evaluative practices, perceptions, and preferences.","arXiv :2607 .09434v 1 [ cs . SE] 10 Jul 2026  \nHow Do Software Professionals Evaluate AI-Generated Code? (Registered Report)  \nSamuli Määttä \\#   \nSoftware Engineering Research Group (M3S), University of Oulu, Finland Hera Arif \\#   \nFaculty of Computer Science, Dalhousie University, Canada Burak Turhan \\#   \nSoftware Engineering Research Group (M3S), University of Oulu, Finland Paul Ralph \\#   \nFaculty of Computer Science, Dalhousie University, Canada Markus Kelanti \\#   \nSoftware Engineering Research Group (M3S), University of Oulu, Finland  \n~~ Abstract ~~  \nRecent advances in generative AI tools have significantly changed how software professionals write, evaluate, and interact with code. Generative AI tools such as GitHub Copilot, ChatGPT, and Claude are increasingly being integrated into everyday workflows. Despite the growing adoption of and reliance on these tools, it remains unclear as to how software professionals evaluate the code they generate. To explore this topic, we will conduct a constructivist grounded theory study that incorporates a survey, semi-structured interviews, and laddering interviews. With the initial survey data collection complete, we aim to interview 20–50 software professionals iteratively until theoretical saturation is achieved. This research aims to build a theory of how software professionals evaluate AI-generated code, grounded in their accounts of evaluative practices, perceptions, and preferences.  \n2012 ACM Subject Classification Software and its engineering → Software verification and validation; Software and its engineering → Automatic programming; Human-centered computing → HCI theory, concepts and models  \nKeywords and phrases Generative AI, software engineering, programming, grounded theory Digital Object Identifier 10.4230/LIPIcs...  \nFunding Samuli Määttä: FAST, the Finnish Software Engineering Doctoral Research Network, funded by the Ministry of Education and Culture, Finland  \nHera Arif: NSERC Discovery Grant RGPIN-2020-05001  \nBurak Turhan: Business Finland project MAISA (Decision number 3398/31/2024)  \nPaul Ralph: NSERC Discovery Grant RGPIN-2020-05001  \nAcknowledgements We thank Carolyn Seaman for advice on developing our positionality and reflexivity statement, and Tuure Tuunanen and Juuli Lumivalo for advice on laddering interviews.  \n 1  Introduction  \nProfessional software developers are increasingly adopting and relying on generative AI-based tools and agents in their work. The early literature on the topic suggested a shift from writing to reading, evaluating, and repairing generated code (e.g., [4, 16 , 21]) . These tasks are not always easy:  \n not being the author of the code may make it more difficult to understand [29]  \n it can contain bugs that are hidden or different from those authored by humans [21, 27]  \n it can be more difficult and time-consuming to debug and repair [3, 14, 21, 29]  \n the expectations of users do not always align with the tool’s capabilities [3, 8]  \n© Samuli Määttä, Hera Arif, Burak Turhan, Paul Ralph, and Markus Kelanti; licensed under Creative Commons License CC-BY 4.0  \nLeibniz International Proceedings in Informatics  \nSchloss Dagstuhl – Leibniz-Zentrum für Informatik, Dagstuhl Publishing, Germany  \nXX:2 How Do Software Professionals Evaluate AI-Generated Code? (Registered Report)  \nThere are also broader issues such as tendencies to outsource critical thinking and comprehension effort, and the risks of automation bias, over-reliance, skill decay, and impaired learning [2, 3 , 12 , 14 , 22 , 24 , 25 , 29] . Reading code requires time and mental effort. Direct forms of evaluation require investing time and effort in code comprehension, which developers tend to avoid whenever task completion is a priority [15] . Generative AI provides ways to use more indirect forms of evaluation (e.g., asking AI to critique or test code) and to take shortcuts in comprehension and evaluation by offloading them to AI. In particular, the issue of over-rel","cbCail4NUMg7ORvP","https://ap.wps.com/l/cbCail4NUMg7ORvP","pdf",527726,1,10,"English","en",105,"# Introduction\n## Motivation and challenges in evaluating AI-generated code\n## Research objective and study approach","[{\"question\":\"Why is evaluating AI-generated code challenging for software professionals?\",\"answer\":\"Professionals may struggle to understand code they did not author, encounter hidden or human-different bugs, and face time-consuming debugging and repair. User expectations may also not match tool capabilities.\"},{\"question\":\"What is the goal of the registered report study?\",\"answer\":\"To develop understanding of challenges in evaluating AI-generated code from software professionals’ perspectives, focusing on perceptions, preferences, and practical conditions shaping evaluative practices in AI-assisted programming.\"},{\"question\":\"How will the study collect and refine evidence?\",\"answer\":\"It uses a constructivist grounded theory approach with an initial survey, semi-structured interviews, and laddering interviews. Interviews with 20–50 professionals will proceed iteratively until theoretical saturation is reached.\"}]",1784179797,25,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":27},"how-do-software-professionals-evaluate-ai-generated-code-registered-report","",{"@graph":35,"@context":85},[36,53,68],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/how-do-software-professionals-evaluate-ai-generated-code-registered-report/82347/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-17","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"Why is evaluating AI-generated code challenging for software professionals?","Question",{"text":75,"@type":76},"Professionals may struggle to understand code they did not author, encounter hidden or human-different bugs, and face time-consuming debugging and repair. User expectations may also not match tool capabilities.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"What is the goal of the registered report study?",{"text":80,"@type":76},"To develop understanding of challenges in evaluating AI-generated code from software professionals’ perspectives, focusing on perceptions, preferences, and practical conditions shaping evaluative practices in AI-assisted programming.",{"name":82,"@type":73,"acceptedAnswer":83},"How will the study collect and refine evidence?",{"text":84,"@type":76},"It uses a constructivist grounded theory approach with an initial survey, semi-structured interviews, and laddering interviews. Interviews with 20–50 professionals will proceed iteratively until theoretical saturation is reached.","https://schema.org",{"og:url":51,"og:type":87,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":89,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,134],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":45,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":45,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":45,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":45,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":45,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":21,"doc_module":4,"doc_module_name":45,"category_name":132,"show_sort_weight":21,"slug":133},"Lifestyle","lifestyle",{"id":135,"doc_module":4,"doc_module_name":45,"category_name":136,"show_sort_weight":106,"slug":137},19,"General","general"]