[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-seo-156283-105":3,"detail-sidebar-cat-0-en-105":81,"doc-detail-156283-en":130},{"code":4,"msg":5,"data":6},0,"ok",{"site_id":7,"language":8,"slug":9,"title":10,"keywords":11,"description":12,"schema_data":13,"social_meta":74,"head_meta":76,"extra_data":78,"updated_unix":80},105,"en","examining-chat-gpt-with-nonwords-and-machine-psycholinguistic-techniques","Examining Chat GPT with nonwords and machine psycholinguistic techniques","","Nonwords—letter or sound strings lacking meaning—have long supported cognitive psychology and psycholinguistics research on human language processing and representation. This study set applies a machine psycholinguistic approach by using nonword stimuli and psycholinguistic-style tasks to evaluate ChatGPT relative to human performance. Study 1 tests definitions for extinct English words, Study 2 uses Spanish-derived nonwords to elicit sound-alike outputs, Study 3 collects ratings of wordlikeness and “buyability,” and Study 4 prompts generation of new English words. Results outline complementary strengths and weaknesses and suggest AI development that augments human abilities.",{"@graph":14,"@context":73},[15,34,56],{"@type":16,"itemListElement":17},"BreadcrumbList",[18,23,27,31],{"item":19,"name":20,"@type":21,"position":22},"https://docshare.wps.com","Home","ListItem",1,{"item":24,"name":25,"@type":21,"position":26},"https://docshare.wps.com/document/","Document",2,{"item":28,"name":29,"@type":21,"position":30},"https://docshare.wps.com/document/research-report/","Research & Report",3,{"item":32,"name":10,"@type":21,"position":33},"https://docshare.wps.com/document/examining-chat-gpt-with-nonwords-and-machine-psycholinguistic-techniques/156283/",4,{"url":32,"name":10,"@type":35,"image":36,"author":41,"headline":10,"publisher":44,"fileFormat":47,"inLanguage":8,"description":12,"dateModified":48,"datePublished":49,"encodingFormat":47,"isAccessibleForFree":50,"interactionStatistic":51},"DigitalDocument",{"url":37,"@type":38,"width":39,"height":40},"https://docshare.wps.com/thumbnails/examining-chat-gpt-with-nonwords-and-machine-psycholinguistic-techniques/156283.png","ImageObject",300,407,{"name":42,"@type":43},"Xiajie","Person",{"url":19,"name":45,"@type":46},"DocShare","Organization","application/pdf","2026-10-17","2026-08-28",true,{"@type":52,"interactionType":53,"userInteractionCount":55},"InteractionCounter",{"@type":54},"ViewAction",10,{"@type":57,"mainEntity":58},"FAQPage",[59,65,69],{"name":60,"@type":61,"acceptedAnswer":62},"What is the main approach used to evaluate ChatGPT in the studies?","Question",{"text":63,"@type":64},"The studies use a machine psycholinguistic approach, presenting nonwords and using tasks similar to those applied to humans to analyze LLM language-related performance.","Answer",{"name":66,"@type":61,"acceptedAnswer":67},"How do the studies test ChatGPT’s handling of nonwords?",{"text":68,"@type":64},"Study 1 asks for definitions of extinct words, Study 2 prompts for sound-alike words from Spanish-based nonwords, Study 3 collects wordlikeness and “buyability” ratings, and Study 4 prompts generation of a new English word for a novel concept.",{"name":70,"@type":61,"acceptedAnswer":71},"What do the results suggest about comparing human and machine language performance?",{"text":72,"@type":64},"The findings highlight both strengths and weaknesses in human and machine performance, supporting future work focused on AI that complements rather than duplicates human abilities.","https://schema.org",{"og:url":32,"og:type":75,"og:title":10,"og:site_name":45,"og:description":12},"article",{"robots":77,"canonical":32},"index,follow",{"doc_id":79,"site_id":7},156283,1787955686,{"code":4,"msg":82,"data":83},"success",[84,88,92,96,101,106,111,115,120,123,126],{"id":22,"doc_module":4,"doc_module_name":25,"category_name":85,"show_sort_weight":86,"slug":87},"Story & Novel",90,"story-novel",{"id":26,"doc_module":4,"doc_module_name":25,"category_name":89,"show_sort_weight":90,"slug":91},"Literature",80,"literature",{"id":33,"doc_module":4,"doc_module_name":25,"category_name":93,"show_sort_weight":94,"slug":95},"Exam",70,"exam",{"id":97,"doc_module":4,"doc_module_name":25,"category_name":98,"show_sort_weight":99,"slug":100},5,"Comic",60,"comic",{"id":102,"doc_module":4,"doc_module_name":25,"category_name":103,"show_sort_weight":104,"slug":105},6,"Technology",50,"technology",{"id":107,"doc_module":4,"doc_module_name":25,"category_name":108,"show_sort_weight":109,"slug":110},7,"Healthcare",40,"healthcare",{"id":112,"doc_module":4,"doc_module_name":25,"category_name":29,"show_sort_weight":113,"slug":114},8,30,"research-report",{"id":116,"doc_module":4,"doc_module_name":25,"category_name":117,"show_sort_weight":118,"slug":119},9,"Religion & Spirituality",20,"religion-spirituality",{"id":118,"doc_module":4,"doc_module_name":25,"category_name":121,"show_sort_weight":118,"slug":122},"World Cup","world-cup",{"id":55,"doc_module":4,"doc_module_name":25,"category_name":124,"show_sort_weight":55,"slug":125},"Lifestyle","lifestyle",{"id":127,"doc_module":4,"doc_module_name":25,"category_name":128,"show_sort_weight":97,"slug":129},19,"General","general",{"code":4,"msg":82,"data":131},{"doc_id":79,"user_id":132,"nickname":42,"user_avatar":133,"doc_module":4,"category_id":112,"category_name":29,"doc_title":10,"doc_description":12,"doc_content":134,"file_id":135,"file_url":136,"file_type":137,"file_size":138,"view_count":55,"is_deleted":4,"is_public":22,"is_downloadable":22,"audit_status":22,"page_count":118,"language":139,"language_code":8,"site_id":7,"html_lang":8,"table_of_contents":140,"faqs":141,"seo_title":142,"seo_description":12,"update_tm":80,"read_time":104,"preview_page_count":4},8814010472675,"https://avatar.qwps.com/avatar/WGlhamll","OPEN ACCESS  \nCitation: Vitevitch MS (2025) Examining Chat GPT with nonwords and machine psycholinguistic techniques. PLoS One 20(6):  \ne0325612. [https://doi.org/10.1371/journal](https://doi.org/10.1371/journal). pone.0325612  \nEditor: Yiu-Kei Tsang, Hong Kong Baptist  \nUniversity, HONG KONG Received: August 30, 2024  \nAccepted: May 15, 2025  \nPublished: June 6, 2025  \nCopyright: © 2025 Michael S. Vitevitch. This is an open access article distributed under the terms of the Creative Commons Attribution  License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.  \nData availability statement: All relevant data are within the manuscript and its Supporting  Information files.  \nFunding: The author(s) received no specific funding for this work.  \nRESEARCH ARTICLE  \nExamining Chat GPT with nonwords and machine psycholinguistic techniques  \nMichael S. Vitevitch*  \nSpoken Language Laboratory, Department of Speech-Language-Hearing, Sciences & Disorders, University of Kansas, Lawrence, Kansas, USA  \n* [mvitevit@ku.edu](mvitevit@ku.edu)  \nAbstract  \nStrings of letters or sounds that lack meaning (i.e. , nonwords) have been used in cognitive psychology and psycholinguistics to provide foundational knowledge of human processing and representation, and insights into language-related performance. The present set of studies used the machine psycholinguistic approach (i.e. , using nonword stimuli and tasks similar to those used with humans) to gain insight into the performance of Chat GPT in comparison to human performance. In Study 1, Chat GPT was able to provide correct definitions to many extinct words (i.e. , real English words that are no longer used) . In Study 2 the nonwords were real words in Spanish, and Chat GPT was prompted to provide a word that sounded similar to thenonword. Responses tended to be Spanish words unless the prompt specified that the similar sounding word should be an English word. In Study 3 Chat GPT provided subjective ratings of wordlikeness (and buyability) that correlated with ratings provided by humans, and with the phonotactic probabilities of the nonwords. In Study 4, Chat GPT was prompted to generate a new English word for a novel concept. The results of these studies highlight certain strengths and weaknesses in human and machine performance. Future work should focus on developing AI that complements or extends rather than duplicates or competes with human abilities. The machine psycholinguistic approach may help to discover additional strengths and weaknesses of human and artificial intelligences.  \nIntroduction  \nThe methodological techniques commonly used in Cognitive Psychology to examine the inner workings of the “black box” that leads to human behavior have been applied in the emerging disciplines of machine psychology [ 1] and artificial cognition [2–5] to examine the black box-like behavior of artificial intelligence. In the present set of studies methodological techniques commonly used in psycholinguistics were used in an approach that might be described as machine psycholinguistics to examine  \nPLOS One | [https://doi.org/10.1371/journal.pone.0325612](https://doi.org/10.1371/journal.pone.0325612) June 6, 2025 1 / 20  \nCompeting interests: The author declared that no competing interests exist.  \nthe language-related behavior (as opposed to decision making or problem-solving behaviors) of a widely-used LLM, Chat-GPT. Specifically, various types of nonwords were presented as stimuli, and the responses generated by the LLM were examined in various ways. The goal of the present studies was not to determine if the LLMused representations and processes that were similar to or different from those used by humans to perform the psycholinguistic tasks (because most assuredly they are different) . Rather, the goal of the present studies was to examine the ways in which performance differed in LLM and humans in order to highlig","cbCaig92x3fQbeW5","https://ap.wps.com/l/cbCaig92x3fQbeW5","pdf",631524,"English","# Abstract\n# Introduction\n## Machine psycholinguistics and LLM evaluation\n## Historical and methodological background of nonwords\n## Wug-test and morphology with nonwords","[{\"question\":\"What is the main approach used to evaluate ChatGPT in the studies?\",\"answer\":\"The studies use a machine psycholinguistic approach, presenting nonwords and using tasks similar to those applied to humans to analyze LLM language-related performance.\"},{\"question\":\"How do the studies test ChatGPT’s handling of nonwords?\",\"answer\":\"Study 1 asks for definitions of extinct words, Study 2 prompts for sound-alike words from Spanish-based nonwords, Study 3 collects wordlikeness and “buyability” ratings, and Study 4 prompts generation of a new English word for a novel concept.\"},{\"question\":\"What do the results suggest about comparing human and machine language performance?\",\"answer\":\"The findings highlight both strengths and weaknesses in human and machine performance, supporting future work focused on AI that complements rather than duplicates human abilities.\"}]","Examining Chat GPT with nonwords and machine psycholinguistic techniques | PDF"]