[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-seo-577880-105":3,"detail-sidebar-cat-0-en-105":81,"doc-detail-577880-en":131},{"code":4,"msg":5,"data":6},0,"ok",{"site_id":7,"language":8,"slug":9,"title":10,"keywords":11,"description":12,"schema_data":13,"social_meta":74,"head_meta":76,"extra_data":78,"updated_unix":80},105,"en","chatgpt-4o-as-an-automated-scoring-tool-for-writing-assessment-strengths-and-weaknesses","ChatGPT-4o as an Automated Scoring Tool for Writing Assessment - Strengths and Weaknesses","","ChatGPT is widely used for educational purposes such as content generation and language translation, yet its effectiveness as an automated scoring tool needs stronger empirical evidence. This mixed-method study examines ChatGPT-4o for assessing English as a Foreign Language (EFL) learners’ written output. It compares ChatGPT-4o scoring against human raters on argumentative essays, using Quadratic Weighted Kappa for reliability, and interviews with experienced raters to analyze strengths and weaknesses.",{"@graph":14,"@context":73},[15,34,56],{"@type":16,"itemListElement":17},"BreadcrumbList",[18,23,27,31],{"item":19,"name":20,"@type":21,"position":22},"https://docshare.wps.com","Home","ListItem",1,{"item":24,"name":25,"@type":21,"position":26},"https://docshare.wps.com/document/","Document",2,{"item":28,"name":29,"@type":21,"position":30},"https://docshare.wps.com/document/research-report/","Research & Report",3,{"item":32,"name":10,"@type":21,"position":33},"https://docshare.wps.com/document/chatgpt-4o-as-an-automated-scoring-tool-for-writing-assessment-strengths-and-weaknesses/577880/",4,{"url":32,"name":10,"@type":35,"image":36,"author":41,"headline":10,"publisher":44,"fileFormat":47,"inLanguage":8,"description":12,"dateModified":48,"datePublished":49,"encodingFormat":47,"isAccessibleForFree":50,"interactionStatistic":51},"DigitalDocument",{"url":37,"@type":38,"width":39,"height":40},"https://docshare.wps.com/thumbnails/chatgpt-4o-as-an-automated-scoring-tool-for-writing-assessment-strengths-and-weaknesses/577880.png","ImageObject",300,407,{"name":42,"@type":43},"Emma Wilson","Person",{"url":19,"name":45,"@type":46},"DocShare","Organization","application/pdf","2026-11-15","2026-10-15",true,{"@type":52,"interactionType":53,"userInteractionCount":55},"InteractionCounter",{"@type":54},"ViewAction",15,{"@type":57,"mainEntity":58},"FAQPage",[59,65,69],{"name":60,"@type":61,"acceptedAnswer":62},"How was ChatGPT-4o evaluated as an automated scoring tool in the study?","Question",{"text":63,"@type":64},"The study first assigned scores to 240 argumentative essays by 13 human raters in pairs. Then, 28 model essays were selected, and the remaining 212 essays were scored by ChatGPT-4o only.","Answer",{"name":66,"@type":61,"acceptedAnswer":67},"What metric was used to measure agreement between human raters and ChatGPT-4o?",{"text":68,"@type":64},"Quantitative analysis used the Quadratic Weighted Kappa statistic to assess inter-rater reliability, focusing on agreement between human raters and ChatGPT-4o.",{"name":70,"@type":61,"acceptedAnswer":71},"What were the main strengths and weaknesses of ChatGPT-4o for writing assessment?",{"text":72,"@type":64},"Results indicate only fair agreement with human raters, with significantly lower and inconsistent scores. Interviews highlight practical advantages such as less manual effort and faster feedback, while also stressing the need for constant human supervision to improve nuance and rubric adherence.","https://schema.org",{"og:url":32,"og:type":75,"og:title":10,"og:site_name":45,"og:description":12},"article",{"robots":77,"canonical":32},"index,follow",{"doc_id":79,"site_id":7},577880,1792167299,{"code":4,"msg":82,"data":83},"success",[84,88,92,96,101,106,111,115,120,123,127],{"id":22,"doc_module":4,"doc_module_name":25,"category_name":85,"show_sort_weight":86,"slug":87},"Story & Novel",90,"story-novel",{"id":26,"doc_module":4,"doc_module_name":25,"category_name":89,"show_sort_weight":90,"slug":91},"Literature",80,"literature",{"id":33,"doc_module":4,"doc_module_name":25,"category_name":93,"show_sort_weight":94,"slug":95},"Exam",70,"exam",{"id":97,"doc_module":4,"doc_module_name":25,"category_name":98,"show_sort_weight":99,"slug":100},5,"Comic",60,"comic",{"id":102,"doc_module":4,"doc_module_name":25,"category_name":103,"show_sort_weight":104,"slug":105},6,"Technology",50,"technology",{"id":107,"doc_module":4,"doc_module_name":25,"category_name":108,"show_sort_weight":109,"slug":110},7,"Healthcare",40,"healthcare",{"id":112,"doc_module":4,"doc_module_name":25,"category_name":29,"show_sort_weight":113,"slug":114},8,30,"research-report",{"id":116,"doc_module":4,"doc_module_name":25,"category_name":117,"show_sort_weight":118,"slug":119},9,"Religion & Spirituality",20,"religion-spirituality",{"id":118,"doc_module":4,"doc_module_name":25,"category_name":121,"show_sort_weight":118,"slug":122},"World Cup","world-cup",{"id":124,"doc_module":4,"doc_module_name":25,"category_name":125,"show_sort_weight":124,"slug":126},10,"Lifestyle","lifestyle",{"id":128,"doc_module":4,"doc_module_name":25,"category_name":129,"show_sort_weight":97,"slug":130},19,"General","general",{"code":4,"msg":82,"data":132},{"doc_id":79,"user_id":133,"nickname":42,"user_avatar":134,"doc_module":4,"category_id":112,"category_name":29,"doc_title":10,"doc_description":12,"doc_content":135,"file_id":136,"file_url":137,"file_type":138,"file_size":139,"view_count":55,"is_deleted":4,"is_public":22,"is_downloadable":22,"audit_status":22,"page_count":140,"language":141,"language_code":8,"site_id":7,"html_lang":8,"table_of_contents":142,"faqs":143,"seo_title":144,"seo_description":12,"update_tm":145,"read_time":146,"preview_page_count":4},3848291630094,"https://eur-avatar.wpscdn.com/davatar_085a072bc5b1113ac321206ff7593b45","ChatGPT-4o as an automated scoring tool for writing assessment: Strengthsand weaknesses  \nRabia Damla Karaçeper1, Gülay Kıray2*  \n1İstanbul University-Cerrahpasa, Institute of Graduate Studies, English Language Teaching, Doctorate Program,İstanbul, Türkiye  \n2İstanbul University - Cerrahpasa, Hasan Ali Yücel Faculty of Education, Department of English Language Teaching, İstanbul, Türkiye  \nARTICLE HISTORY  \nReceived: May 23, 2025  \nAccepted: Sep. 27, 2025  \nKeywords:  \nAutomated scoring tool, ChatGPT-4o,  \nWriting assessment.  \nAbstract: ChatGPT is widely used for many educational purposes such as content generation and language translation, however, its role as an automated scoring tool requires further empirical investigation. This mixed-method study explores the effectiveness of ChatGPT-4o as an automated scoring tool for English as a Foreign Language (EFL) learners’ written output. It particularly aims to discover to what extent ChatGPT-4o can produce reliable and accurate scores in writing assessment and whether it can serve as an alternative scoring tool to the traditional human scoring. 240 argumentative essays were first scored by 13 human raters working in pairs. 28 of them were selected as model essays while the remaining 212 essays were thereafter scored by ChatGPT-4o only. Quantitative analysis employed the Quadratic Weighted Kappa statistic to measure inter-rater reliability, focusing on the agreement among human raters and ChatGPT-4o. Findings suggest that ChatGPT-4o demonstrates only fair agreement with human raters, producing significantly lower and inconsistent scores. Regarding this discrepancy, five experienced human raters were interviewed about the strengths and weaknesses of ChatGPT as a scoring tool, with their perspectives and practices thematically analyzed to triangulate the quantitative findings. The key differences were classified under the themes such as rubric adherence, scoring bias and sensitivity to nuances. Due to AI-enabled automation, ChatGPT exhibits pragmatic dualities in practicality, providing feedback and linguistic capacity. The remarkable strengths involve less manual effort, faster detailed scoring feedback and broader linguistic dataset. However, human-driven optimization through constant supervision, care and pedagogical expertise is essential for a more nuanced scoring.  \nINTRODUCTION  \nIn recent decades, Automated Writing Evaluation (AWE) systems have been of great importance in developing language instruction and assessment for many language-testing organizations (Shermis & Burstein, 2013). Dating from the 1960s, AWE is now a generic term, referring to both Automated Essay Scoring (AES) and Automated Essay Grading (AEG) . While AES and AEG primarily target the scoring and grading of written responses in both high-and low-stakes tests (Dikli, 2006), AWE places these within a more formative, feedback-rich  \n*Corresponding author at: Gülay KIRAY 􀀍 [gkiray@iuc.edu.tr](gkiray@iuc.edu.tr) 􀁽 İstanbul University-Cerrahpasa, Hasan Ali Yücel Faculty of Education, Department of English Language Teaching, İstanbul, Türkiye  \nThe copyright of the published article belongs to its author under CC BY 4.0 license. To view a copy of this licence, visit [http://creativecommons.org/licenses/by/4.0/](http://creativecommons.org/licenses/by/4.0/)  \n[e-ISSN: 2148-7456](e-ISSN: 2148-7456)  \nintegration of the technology into language learning (Nguyen & Hegelheimer, 2022) . These systems rely heavily on a combination of statistical techniques and linguistic strategies from a variety of fields such as linguistics, computer science, and educational measurement. The mechanisms behind these systems investigate related linguistic evidence in test taker’s writing and then create an algorithm that enables mapping the prepackaged collection of linguistic input to produce a comparable score to that of human raters. Project Essay Grade (PEG) is the earliest known AES system (Page, 2003), which started the journey ","cbCaio3fOjIFsuw8","https://ap.wps.com/l/cbCaio3fOjIFsuw8","pdf",706009,29,"English","# Abstract\n# Introduction","[{\"question\":\"How was ChatGPT-4o evaluated as an automated scoring tool in the study?\",\"answer\":\"The study first assigned scores to 240 argumentative essays by 13 human raters in pairs. Then, 28 model essays were selected, and the remaining 212 essays were scored by ChatGPT-4o only.\"},{\"question\":\"What metric was used to measure agreement between human raters and ChatGPT-4o?\",\"answer\":\"Quantitative analysis used the Quadratic Weighted Kappa statistic to assess inter-rater reliability, focusing on agreement between human raters and ChatGPT-4o.\"},{\"question\":\"What were the main strengths and weaknesses of ChatGPT-4o for writing assessment?\",\"answer\":\"Results indicate only fair agreement with human raters, with significantly lower and inconsistent scores. Interviews highlight practical advantages such as less manual effort and faster feedback, while also stressing the need for constant human supervision to improve nuance and rubric adherence.\"}]","ChatGPT-4o as an Automated Scoring Tool for Writing Assessment - Strengths and Weaknesses | PDF",1792032594,73]