[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-128498-en":3,"doc-seo-128498-105":31,"detail-sidebar-cat-0-en-105":92},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":21,"is_downloadable":21,"audit_status":21,"page_count":22,"language":23,"language_code":24,"site_id":25,"html_lang":24,"table_of_contents":26,"faqs":27,"seo_title":28,"seo_description":14,"update_tm":29,"read_time":30},128498,687207017582,"Himbo","https://ap-avatar.wpscdn.com/davatar_994ba38a5ba835b3df7d355c54d3ed8d",8,"Research & Report","The Work of Students and ChatGPT Compared - Using Machine Learning to Detect and Characterize AI-Generated Text","ChatGPT enables question answering across many knowledge domains, raising concerns that students may misuse it to undermine learning via cheating. An experiment was conducted using a typical college assignment to detect and compare genuine student responses with responses generated by ChatGPT. A text classification scheme discriminated student work from AI output both when general knowledge was used and when AI was prompted with the same course material, and authorship-style differences were identified. Results show formal, “frozen” AI language and provide a machine learning detector plus stylistic cues for human identification.","Association for Information Systems  \nAIS Electronic Library (AISeL)  \n\n| AMCIS 2023 Proceedings | SIG-Artificial Intelligence and Autonomous Applications |\n| --- | --- |\n| Aug 10th, 12:00 AM\u003Cbr>The Work of Students and ChatGPT Compared: Using Machine Learning to Detect and Characterize AI-Generated Text\u003Cbr>Roozmehr Safi\u003Cbr>University of Missouri-Kansas City, [safir@umkc.edu](safir@umkc.edu)\u003Cbr>Azadeh Jalali Naini\u003Cbr>Independent Researcher, [a.jalali@ku.edu](a.jalali@ku.edu)\u003Cbr>Follow this and additional works at: [https://aisel.aisnet.org/amcis2023](https://aisel.aisnet.org/amcis2023) |  |\n\nRecommended Citation  \nSafi, Roozmehr and Jalali Naini, Azadeh, \"The Work of Students and ChatGPT Compared: Using Machine Learning to Detect and Characterize AI-Generated Text\" (2023) . AMCIS 2023 Proceedings. 17.  \n[https://aisel.aisnet.org/amcis2023/sig_aiaa/sig_aiaa/17](https://aisel.aisnet.org/amcis2023/sig_aiaa/sig_aiaa/17)  \nThis material is brought to you by the Americas Conference on Information Systems (AMCIS) at AIS Electronic Library (AISeL) . It has been accepted for inclusion in AMCIS 2023 Proceedings by an authorized administrator of AIS Electronic Library (AISeL) . For more information, please [contact](contact elibrary@aisnet.org)[ elibrary@aisnet.org](contact elibrary@aisnet.org).  \nThe Work ofStudents and ChatGPT Compared: Using Machine Learning to Detect and Characterize AI-Generated Text  \nEmergent Research Forum (ERF) Paper  \nRoozmehr Safi  \nUniversity of Missouri – Kansas City [safir@umkc.edu](safir@umkc.edu)  \nAzadeh Jalali Naini  \nIndependent Researcher [azadeh.jalali@yahoo.com](azadeh.jalali@yahoo.com)  \nAbstract  \nChatGPT, an AI agent capable of answering questions from virtually all domains of knowledge, has created renewed concerns about the adverse effects of AI on students’ learning through cheating. We conducted an experiment involving a typical college course assignment to detect and compare genuine student responses to responses generated by ChatGPT. Using a text classification scheme, we showed that student responses are fairly accurately discriminable from AI’s, not only when AI uses its general knowledge to answer questions, but also when it is prompted to use the same material used by students. In addition, we identify authorship styles that set students’ and AI’s work apart. Specifically, we find AI’s language to be more formal and frozen. Together, results provide a machine learning model for detecting text generated by AI, as well as some stylistic cues that can help humans detect such text.  \nKeywords  \nChatGPT, Education, Cheating, Text classification, Authorship attribution, Authorship style, NBC, TF-IDF  \nIntroduction  \nIn November 2022, OpenAI, a technology research consortium with the goal of creating “friendly artificial intelligence”, publicly launched ChatGPT, a large scale, mainstream implementation of generative pretrained transformer (GPT) artificial intelligence (AI) . As an instance of “strong AI” (Searle, 1980), this type of software can “think” and “converse” with its users much like a human. It achieves this by collecting, weighing, and synthesizing massive amounts of information from the Internet. By answering questions from virtually all domains of human knowledge, AI agents such as ChatGPT have opened a world of new possibilities in all areas of life (Dwivedi et al., 2023) . Nonetheless, like other technological innovations, they have also created new practical, societal, and ethical challenges and dilemmas.  \nOne case in point is how AI agents like ChatGPT affect education. Academic dishonesty such as cheating and plagiarism is prevalent in educational institutions (DiPaulo, 2022), with more that 60% of college students freely admitting to it (ICAI, 2020) . “All-knowing”, fairly accurate, and easily accessible agents such as ChatGPT now make cheating easier than ever. This practice can suppress students’ creativity, expose them to inaccurate or biased information, and create a ho","cbCaiha2VUKdXnu1","https://ap.wps.com/l/cbCaiha2VUKdXnu1","pdf",205675,3,1,6,"English","en",105,"# Abstract\n# Keywords\n# Introduction\n## Research questions\n# The Experiment\n## Material and Student Participants","[{\"question\":\"How did the study compare students’ responses with ChatGPT-generated answers?\",\"answer\":\"The researchers used a typical college course assignment and applied a text classification scheme to distinguish genuine student responses from responses generated by ChatGPT.\"},{\"question\":\"Does the detection approach work only when ChatGPT uses general knowledge?\",\"answer\":\"No. The results also showed discrimination accuracy when ChatGPT was prompted to use the same material that students used.\"},{\"question\":\"What stylistic cues differentiate AI-generated text from students’ work?\",\"answer\":\"The paper reports that AI language tends to be more formal and “frozen,” and these authorship-style cues help separate AI output from student writing.\"}]","The Work of Students and ChatGPT Compared - Using Machine Learning to Detect and Characterize AI-Generated Text | PDF",1786001397,15,{"code":4,"msg":32,"data":33},"ok",{"site_id":25,"language":24,"slug":34,"title":13,"keywords":35,"description":14,"schema_data":36,"social_meta":87,"head_meta":89,"extra_data":91,"updated_unix":29},"the-work-of-students-and-chatgpt-compared-using-machine-learning-to-detect-and-characterize-ai-generated-text","",{"@graph":37,"@context":86},[38,54,69],{"@type":39,"itemListElement":40},"BreadcrumbList",[41,45,49,51],{"item":42,"name":43,"@type":44,"position":21},"https://docshare.wps.com","Home","ListItem",{"item":46,"name":47,"@type":44,"position":48},"https://docshare.wps.com/document/","Document",2,{"item":50,"name":12,"@type":44,"position":20},"https://docshare.wps.com/document/research-report/",{"item":52,"name":13,"@type":44,"position":53},"https://docshare.wps.com/document/the-work-of-students-and-chatgpt-compared-using-machine-learning-to-detect-and-characterize-ai-generated-text/128498/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":24,"description":14,"dateModified":62,"datePublished":63,"encodingFormat":61,"isAccessibleForFree":64,"interactionStatistic":65},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":42,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-23","2026-08-06",true,{"@type":66,"interactionType":67,"userInteractionCount":20},"InteractionCounter",{"@type":68},"ViewAction",{"@type":70,"mainEntity":71},"FAQPage",[72,78,82],{"name":73,"@type":74,"acceptedAnswer":75},"How did the study compare students’ responses with ChatGPT-generated answers?","Question",{"text":76,"@type":77},"The researchers used a typical college course assignment and applied a text classification scheme to distinguish genuine student responses from responses generated by ChatGPT.","Answer",{"name":79,"@type":74,"acceptedAnswer":80},"Does the detection approach work only when ChatGPT uses general knowledge?",{"text":81,"@type":77},"No. The results also showed discrimination accuracy when ChatGPT was prompted to use the same material that students used.",{"name":83,"@type":74,"acceptedAnswer":84},"What stylistic cues differentiate AI-generated text from students’ work?",{"text":85,"@type":77},"The paper reports that AI language tends to be more formal and “frozen,” and these authorship-style cues help separate AI output from student writing.","https://schema.org",{"og:url":52,"og:type":88,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":90,"canonical":52},"index,follow",{"doc_id":7,"site_id":25},{"code":4,"msg":5,"data":93},[94,98,102,106,111,115,120,123,128,131,135],{"id":21,"doc_module":4,"doc_module_name":47,"category_name":95,"show_sort_weight":96,"slug":97},"Story & Novel",90,"story-novel",{"id":48,"doc_module":4,"doc_module_name":47,"category_name":99,"show_sort_weight":100,"slug":101},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":47,"category_name":103,"show_sort_weight":104,"slug":105},"Exam",70,"exam",{"id":107,"doc_module":4,"doc_module_name":47,"category_name":108,"show_sort_weight":109,"slug":110},5,"Comic",60,"comic",{"id":22,"doc_module":4,"doc_module_name":47,"category_name":112,"show_sort_weight":113,"slug":114},"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":47,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":47,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":47,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":47,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":47,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":47,"category_name":137,"show_sort_weight":107,"slug":138},19,"General","general"]