[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-121005-en":3,"doc-seo-121005-105":30,"detail-sidebar-cat-0-en-105":92},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},121005,4398048950312,"Violet","https://ap-avatar.wpscdn.com/avatar/400002538284de19e3c?_k=1778320343897328908",8,"Research & Report","Early Dementia Detection in Speech Transcripts Using Machine Learning and Large Language Models - Master’s Thesis - June 2024","Dementia is a prevalent neurodegenerative disorder that creates urgent demands for earlier detection and timely intervention. This thesis investigates dementia identification by analyzing linguistic patterns in speech transcripts with advanced machine learning, combining datasets from DementiaBank including the Pitt Corpus and ADReSS challenge. It applies data preprocessing, feature extraction from the Cookie Theft picture description task, and model fine-tuning. The study evaluates classic classifiers and modern Large Language Models, highlighting strong performance gains when using LoRA and QLoRA.","SHAIMA AHMAD FREJA AND YEGANEH HALLAJ  \nDEPARTMENT OF ELECTRICAL ENGINEERING AND COMPUTER SCIENCE  \n\n| Early Dementia Detection in Speech Transcripts Using Machine Learning and Language Models | Large |\n| --- | --- |\n\nMaster's Thesis-Data Science-June 2024  \nI, Shaima Ahmad Freja and Yeganeh Hallaj, declare that this thesis titled,“ Early Dementia Detection in Speech Transcripts Using Machine Learning and Large Language Models” and the work presented in it are my own. I confirm that:  \n■ This work was done wholly or mainly while in candidature for a master’s degree at the University of Stavanger.  \n■ Where I have consulted the published work of others, this is always clearly attributed.  \n■ Where I have quoted from the work of others, the source is always given. With the exception of such quotations, this thesis is entirely my own work.  \n■ I have acknowledged all main sources of help.  \nAbstract  \nDementia, a prevalent neurodegenerative disorder, poses significant challenges in early detection and intervention. Timely diagnosis is crucial for providing affected individuals and their families with the necessary support and resources, ultimately improving the quality of life.  \nThis thesis explores innovative approaches for detecting dementia through the analysis of linguistic patterns in speech transcripts using advanced machine learning techniques. Leveraging datasets from DementiaBank, specifically the Pitt Corpus and ADReSS challenge datasets, we employ a combination of traditional machine learning models and cutting-edge Large Language Models (LLMs) to analyze cognitive and linguistic features.  \nOur methodology includes comprehensive data preprocessing, feature extraction from the ’Cookie Theft’ picture description test, and model fine-tuning. We investigate the effectiveness of various classic machine learning models such as Logistic Regression, Random Forest, Support Vector Machines, and transfer learning techniques with pre-trained models like BERT, DistilBERT, RoBERTa, Mistral, and Llama. The study highlights the significant potential of LLMs, especially when enhanced with Low-Rank Adaptation (LoRA) and Quantization (QLoRA) methods, in accurately detecting dementia.  \nAcknowledgements  \nWe would like to express our utmost appreciation to our supervisor, Professor Mina Farmanbar, for her exceptional support and mentorship. Her organized approach and constructive feedback have been instrumental in the development of our thesis. Her extensive expertise and insightful advice have greatly shaped the direction of our research. We are truly grateful for her availability and willingness to provide technical guidance whenever needed.  \nwe are extremely grateful toArezo Shakeri, a PhD student, whose support and expertise in Natural Language Processing have been invaluable to our research. We sincerely appreciate her contributions and continuous support throughout this project.  \nLastly, we would like to sincerely thank our families for being our greatest supporters. Their continuous encouragement and immense kindness throughout our master’s program have provided us with constant motivation and energy.  \nContents  \nAbstract ii  \nAcknowledgements iii  \n1 Introduction 1  \n1.1 Background and Motivation ..................... 1  \n1.2 Objectives ................................ 4  \n1.3 Approach and Contributions ..................... 4  \n1.4 Outline ................................. 5  \n2 Related Work 6  \n3 Approach 10  \n3.1 Dataset ................................. 10  \n3.1.1 Pitt Corpus Dataset ...................... 11  \n3.1.2 ADReSS Dataset ........................ 12  \n3.2 Data Collection ............................. 12  \n3.3 Data Preprocessing .......................... 14  \n3.4 Feature Extraction ........................... 16  \n3.5 Proposed Method ........................... 17  \n4 Classic Machine Learning Models 19  \n4.1 Machine Learning Models ....................... 19  \n4.1.1 Logistic Regression (LR) ................... 1","cbCaidBIWPIN2SqX","https://ap.wps.com/l/cbCaidBIWPIN2SqX","pdf",5483952,1,108,"English","en",105,"# Abstract\n# Acknowledgements\n# Introduction\n## Background and Motivation\n## Objectives\n## Approach and Contributions\n## Outline\n# Related Work\n# Approach\n## Dataset\n### Pitt Corpus Dataset\n### ADReSS Dataset\n## Data Collection\n## Data Preprocessing\n## Feature Extraction\n## Proposed Method\n# Classic Machine Learning Models\n## Machine Learning Models\n### Logistic Regression (LR)\n### Support Vector Machine (SVM)\n### K-Nearest Neighbors (KNN)\n### Gradient Boosting Classifier (GPT)\n### Random Forest\n### Naive Bayes\n### Results and Discussion\n## Cross Validation\n# Improving Classic Machine Learning Models\n## Comparison results for the ADReSS\n## Comparison of the results for the Pitt-Corpus\n# Large Language Models (LLMs)\n## Transfer Learning in LLMs\n## Architectural of LLMs\n## Fine-Tuning LLM and Evaluation\n## Fine Tuning Encoder-style Models (Bert, DistilBert, RoBERTa)\n## Fine Tuning Decoder-style Models (Llama2 7B, Mistral 7B)\n# Discussion\n## LLM Encoder Model\n## LLM Decoder Model\n# Conclusions and Future Work\n# Instructions to Compile and Run System","[{\"question\":\"What problem does the thesis address in dementia care?\",\"answer\":\"It addresses the challenge of early dementia detection, where timely diagnosis enables earlier support, better symptom management, and improved quality of life for affected individuals and their families.\"},{\"question\":\"Which datasets and speech task are used to build the detection system?\",\"answer\":\"The work uses DementiaBank resources, including the Pitt Corpus and ADReSS challenge datasets, and performs feature extraction from the Cookie Theft picture description test.\"},{\"question\":\"How do classic machine learning models and Large Language Models differ in the study’s approach?\",\"answer\":\"The thesis evaluates classic models like Logistic Regression, Random Forest, SVM, and other methods, then compares them with LLM-based approaches using transfer learning and fine-tuning strategies. It emphasizes that LLMs can provide significant accuracy improvements, particularly with LoRA and quantization methods (QLoRA).\"}]","Early Dementia Detection in Speech Transcripts Using Machine Learning and Large Language Models - Master’s Thesis - June 2024 | PDF",1785733277,272,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":87,"head_meta":89,"extra_data":91,"updated_unix":28},"early-dementia-detection-in-speech-transcripts-using-machine-learning-and-large-language-models-masters-thesis-june-2024","",{"@graph":36,"@context":86},[37,54,69],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/early-dementia-detection-in-speech-transcripts-using-machine-learning-and-large-language-models-masters-thesis-june-2024/121005/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":63,"encodingFormat":61,"isAccessibleForFree":64,"interactionStatistic":65},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-04","2026-08-03",true,{"@type":66,"interactionType":67,"userInteractionCount":20},"InteractionCounter",{"@type":68},"ViewAction",{"@type":70,"mainEntity":71},"FAQPage",[72,78,82],{"name":73,"@type":74,"acceptedAnswer":75},"What problem does the thesis address in dementia care?","Question",{"text":76,"@type":77},"It addresses the challenge of early dementia detection, where timely diagnosis enables earlier support, better symptom management, and improved quality of life for affected individuals and their families.","Answer",{"name":79,"@type":74,"acceptedAnswer":80},"Which datasets and speech task are used to build the detection system?",{"text":81,"@type":77},"The work uses DementiaBank resources, including the Pitt Corpus and ADReSS challenge datasets, and performs feature extraction from the Cookie Theft picture description test.",{"name":83,"@type":74,"acceptedAnswer":84},"How do classic machine learning models and Large Language Models differ in the study’s approach?",{"text":85,"@type":77},"The thesis evaluates classic models like Logistic Regression, Random Forest, SVM, and other methods, then compares them with LLM-based approaches using transfer learning and fine-tuning strategies. It emphasizes that LLMs can provide significant accuracy improvements, particularly with LoRA and quantization methods (QLoRA).","https://schema.org",{"og:url":52,"og:type":88,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":90,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":93},[94,98,102,106,111,116,121,124,129,132,136],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":95,"show_sort_weight":96,"slug":97},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":99,"show_sort_weight":100,"slug":101},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":103,"show_sort_weight":104,"slug":105},"Exam",70,"exam",{"id":107,"doc_module":4,"doc_module_name":46,"category_name":108,"show_sort_weight":109,"slug":110},5,"Comic",60,"comic",{"id":112,"doc_module":4,"doc_module_name":46,"category_name":113,"show_sort_weight":114,"slug":115},6,"Technology",50,"technology",{"id":117,"doc_module":4,"doc_module_name":46,"category_name":118,"show_sort_weight":119,"slug":120},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":122,"slug":123},30,"research-report",{"id":125,"doc_module":4,"doc_module_name":46,"category_name":126,"show_sort_weight":127,"slug":128},9,"Religion & Spirituality",20,"religion-spirituality",{"id":127,"doc_module":4,"doc_module_name":46,"category_name":130,"show_sort_weight":127,"slug":131},"World Cup","world-cup",{"id":133,"doc_module":4,"doc_module_name":46,"category_name":134,"show_sort_weight":133,"slug":135},10,"Lifestyle","lifestyle",{"id":137,"doc_module":4,"doc_module_name":46,"category_name":138,"show_sort_weight":107,"slug":139},19,"General","general"]