[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-118577-en":3,"doc-seo-118577-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},118577,1374391974564,"Clementine","https://ap-avatar.wpscdn.com/avatar/14000253aa45c000a9e?x-image-process=image/resize,m_fixed,w_180,h_180&k=1779874745381141002",8,"Research & Report","Evaluating SMS Spam Classification: Human Judgement vs Machine Learning Models","Spam messages in SMS networks increasingly evolve in content and sophistication, driven by active adaptation from senders. Because machine learning classifiers require time and data to retrain, manual human intervention becomes necessary when filters struggle. This study evaluates how detection rates vary across multiple machine learning algorithms and different groups of human participants. Incorrect classification can cause missed information and enable hacking-related harm, so the work emphasizes a collaborative defensive approach that combines human judgment with algorithmic strengths.","Dakota State University  \nBeadle Scholar  \n\n| Honors | Masters Thesis & Doctoral Dissertations |\n| --- | --- |\n\n2025  \nEvaluating SMS Spam Classification: Human Judgement vs Machine Learning Models  \nGabe Miller  \nFollow this and additional works at: [https://scholar.dsu.edu/honors](https://scholar.dsu.edu/honors)  \nEvaluating SMS Spam Classification: Human Judgement vs  \nMachine Learning Models  \nGabe Miller  \nDepartment of Cyber Operations  \nDakota State University  \n[gabe.miller@trojans.dsu.edu](gabe.miller@trojans.dsu.edu)  \nMay 9, 2025  \nAbstract—In the age of artificial intelligence, spam messages have become increasingly widespread and sophisticated. Their rapid evolution is driven by ongoing efforts to filter and block them, prompting spammers to constantly adapt their tactics. Since machine learning algorithms require time and data to retrain and adjust, it becomes essential for humans to step in and help classify messages manually when needed. This layered approach,referred to as defensive-indepth, adds multiple barriers through which spam and smishing messages must pass, reducing the likelihood of them reaching the end user.  \nThis case study explores the detection rates of various machine learning algorithms compared to different groups of human participants. Messages that are not correctly identified can result in missed information or, worse, users falling victim to hacking attempts. By examining human and machine learning performance in spam detection, this study underlines the importance of having a collaborative approach that leverages each group’s strengths.  \nI. INTRODUCTION  \nThe classification of spam and ham messages in SMS filtering presents an intriguing challenge, as spam content continually evolves to evade detection by both Nave Bayes and Neural Network algorithms [1] . Spam messages are unsolicited, unwanted messages sent in bulk to users in an attempt to elicit a response or promote a product. In contrast, ham refers to legitimate messages that the user would want to receive. This study seeks to determine which method, machine learning algorithms or human judgment, more effectively detects and classifies SMS spam messages, and whether their combined use can enhance protection against smishing attacks. Most SMS filters rely on machine learning to classify ham and spam, which results in performance differences based on the underlying classification technique.  \nTo learn which technique performs the best, this project will examine open-source filters to better understand differences in classification techniques. The filters must be open source for thorough analysis and adjustments, ensuring they meet testing criteria. A rigid testing criterion ensures that results are as accurate and specific as possible which reduces changes for outliers in testing results.  \nThis project assumes an important role due to the rising number of scam victims, as understanding SMS scams stands as the best way to decreasescam victims [2] . This research provides insight into the optimal algorithms to use; however, no algorithm is perfect because of human error or improper optimization. Algorithms are only as smart as the people using them because algorithms do as told or trained to do by humans.  \nII. LITERATURE REVIEW  \nScholars in technology and cellular devices see Short Message Service (SMS) as a massive leap in technological communications in the modern era. Almost all scholars agree that SMS spam keeps increasing drastically [1], which leads to increased smishing, a phishing attack done over SMS systems, attacks, or unwanted advertisements. Scholars’ opinions differ on the most effective method for reducing and filtering smishing attacks.  \nResearch shows that the most significant difference lies in whether SMS spam filters alone can stop all spam or if implementing safe practices is necessary to prevent smishing attacks. Scholars such as Mishra and Soni believe that mindfulness and awareness of smishing messages sig","cbCaiiy5Gk4mKpC8","https://ap.wps.com/l/cbCaiiy5Gk4mKpC8","pdf",538161,1,11,"English","en",105,"# Introduction\n# Literature Review","[{\"question\":\"What problem does the thesis address in SMS spam filtering?\",\"answer\":\"The thesis examines how to classify SMS spam and ham messages as spam content evolves to evade detection and as smishing risk increases.\"},{\"question\":\"Why does the study consider human judgment alongside machine learning?\",\"answer\":\"Machine learning needs time and new data to retrain, so humans can step in manually when needed and provide an additional layer of defense.\"},{\"question\":\"What does the thesis aim to compare between humans and algorithms?\",\"answer\":\"It compares detection rates of various machine learning algorithms against different groups of human participants to determine which approach detects SMS spam more effectively and how combining them could improve protection.\"}]","Evaluating SMS Spam Classification: Human Judgement vs Machine Learning Models | PDF",1785684362,28,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"evaluating-sms-spam-classification-human-judgement-vs-machine-learning-models","",{"@graph":36,"@context":85},[37,54,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/evaluating-sms-spam-classification-human-judgement-vs-machine-learning-models/118577/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":62,"encodingFormat":61,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-02",true,{"@type":65,"interactionType":66,"userInteractionCount":4},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What problem does the thesis address in SMS spam filtering?","Question",{"text":75,"@type":76},"The thesis examines how to classify SMS spam and ham messages as spam content evolves to evade detection and as smishing risk increases.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"Why does the study consider human judgment alongside machine learning?",{"text":80,"@type":76},"Machine learning needs time and new data to retrain, so humans can step in manually when needed and provide an additional layer of defense.",{"name":82,"@type":73,"acceptedAnswer":83},"What does the thesis aim to compare between humans and algorithms?",{"text":84,"@type":76},"It compares detection rates of various machine learning algorithms against different groups of human participants to determine which approach detects SMS spam more effectively and how combining them could improve protection.","https://schema.org",{"og:url":52,"og:type":87,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":89,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":46,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":46,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":46,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":46,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]