[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-128319-en":3,"doc-seo-128319-105":31,"detail-sidebar-cat-0-en-105":92},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":21,"is_downloadable":21,"audit_status":21,"page_count":22,"language":23,"language_code":24,"site_id":25,"html_lang":24,"table_of_contents":26,"faqs":27,"seo_title":28,"seo_description":14,"update_tm":29,"read_time":30},128319,962085564381,"Clementine","https://ap-avatar.wpscdn.com/davatar_6f874abed73319feea01a86fa6f0fab8",8,"Research & Report","Evaluating Interaction with Machine Learning Text Classifiers and Interpretability Techniques - Dissertation","Machine learning systems are increasingly embedded in society, making it essential to understand how users perceive and engage with ML-driven text classifiers. This Ph.D. thesis examines user experience, usability, interpretability, and cognitive biases through three studies. The Thematic Analysis Coding Assistant (TACA) supports iterative thematic coding with an offline gradient boosting model, revealing critical reflection, new thematic insights, and misconceptions about ML concepts. An autoethnography further analyzes re-labeling, model inspection strategies, and user positionality. A third study compares LIME and SHAP with a global LLM-summary approach, showing differential feature focus but no accuracy improvement.","Evaluating Interaction with Machine Learning Text Classifiers and Interpretability Techniques  \nFederico Milana  \nA dissertation submitted in partial fulfillment of the requirements for the degree of  \nDoctor of Philosophy  \nof  \nUniversity College London.  \nUCL Interaction Centre  \nUniversity College London  \nMarch 13, 2025  \n2  \nI, Federico Milana, confirm that the work presented in this thesis is my own. Where information has been derived from other sources, I confirm that this has been indicated in the work.  \nAbstract  \nAs Machine Learning (ML) becomes increasingly integrated into society and more users interact with ML-driven systems, understanding how they perceive and engage with these technologies grows increasingly important. This Ph.D. thesis explores user experience, usability, interpretability, and cognitive biases of ML text classifiers through two research projects based on a desktop application developed to support thematic analysis, and a separate user evaluation of interpretability techniques.  \nThe Thematic Analysis Coding Assistant (TACA) enables users to import an initial thematic analysis, extracts labelled sentences, trains an offline gradient boosting classifier, and generates coding suggestions. Users can iteratively re-train the model after re-labelling individual or batches of sentences. A user study run with 20 non-ML expert participants revealed that participants critically reflected on their analysis, gained new thematic insights, and adapted their interpretative stance, while also showing misconceptions about ML concepts, positivist views, and self-blame for poor model performance.  \nA second study reports on an autoethnography of the use of TACA, revealing different re-labeling and model inspection strategies, reflecting on potential structural changes to the analysis, and examinining the positionality of the user as a developer, a researcher, and a participant. The findings provide complementary insights into how ML can support and challenge analytical processes, personal reflections, and perceptions of the model.  \nBuilding on the findings of the first two studies, a third study evaluates two popular local interpretability methods in text classification, LIME and SHAP, and  \nAbstract 4  \na proposed global method using LLM-generated summaries based on LIME importance weights. Among 128 participants, those without explanations identified broader topics, while those using LIME and SHAP focused on individual terms, and those using summaries identified more features overall. However, none of the methods significantly improved model prediction accuracy.  \nTogether, these studies contribute to seminal research on understanding the perception and interaction with ML and the implications of system and interaction design to improve the understanding of ML concepts.  \nImpact Statement  \nRecent advances in ML algorithms, computational power, and data availability have driven what is now considered an ongoing AI spring. ML is increasingly used in more aspects of society and is transforming industries, ranging from healthcare and finance to education and entertainment, reshaping how decisions are made and services are delivered.  \nAs the number of applications for ML continues to grow, so does the number of users exposed to these systems. Although the underlying theory behind ML is highly advanced, users are typically not required to understand the technical details to interact even with extremely large and complex models. This is significant considering the particularly high stakes in critical application areas where some of these models are used, such as medical diagnosis, legal decision-making, and autonomous driving. Consequently, understanding how users perceive and engage with ML is arguably just as important as advances and breakthroughs in AI itself.  \nThis thesis contributes to existing research on human-AI interaction by specifically evaluating user experience with ML text classifiers. ML in text ","cbCaisxlkxdcVHdL","https://ap.wps.com/l/cbCaisxlkxdcVHdL","pdf",6033282,3,1,233,"English","en",105,"# Abstract\n## Thematic Analysis Coding Assistant (TACA)\n## User study and findings\n## Autoethnography of TACA use\n## Interpretability methods evaluation (LIME, SHAP, LLM summaries)\n## Impact statement\n# Acknowledgements","[{\"question\":\"What research questions does the thesis address about machine learning text classifiers?\",\"answer\":\"It investigates user experience, usability, interpretability, and cognitive biases when people interact with ML text classifiers.\"},{\"question\":\"How does TACA support thematic analysis and model interaction?\",\"answer\":\"TACA lets users import initial thematic analysis, extracts labelled sentences, trains an offline gradient boosting classifier, and provides coding suggestions with iterative re-training after re-labeling.\"},{\"question\":\"What do the interpretability experiments (LIME, SHAP, and LLM summaries) show?\",\"answer\":\"Participants using explanations tended to focus on individual terms, while those without explanations identified broader topics; using summaries led to more features overall, but none significantly improved model prediction accuracy.\"}]","Evaluating Interaction with Machine Learning Text Classifiers and Interpretability Techniques - Dissertation | PDF",1785946812,587,{"code":4,"msg":32,"data":33},"ok",{"site_id":25,"language":24,"slug":34,"title":13,"keywords":35,"description":14,"schema_data":36,"social_meta":87,"head_meta":89,"extra_data":91,"updated_unix":29},"evaluating-interaction-with-machine-learning-text-classifiers-and-interpretability-techniques-dissertation","",{"@graph":37,"@context":86},[38,54,69],{"@type":39,"itemListElement":40},"BreadcrumbList",[41,45,49,51],{"item":42,"name":43,"@type":44,"position":21},"https://docshare.wps.com","Home","ListItem",{"item":46,"name":47,"@type":44,"position":48},"https://docshare.wps.com/document/","Document",2,{"item":50,"name":12,"@type":44,"position":20},"https://docshare.wps.com/document/research-report/",{"item":52,"name":13,"@type":44,"position":53},"https://docshare.wps.com/document/evaluating-interaction-with-machine-learning-text-classifiers-and-interpretability-techniques-dissertation/128319/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":24,"description":14,"dateModified":62,"datePublished":63,"encodingFormat":61,"isAccessibleForFree":64,"interactionStatistic":65},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":42,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-28","2026-08-05",true,{"@type":66,"interactionType":67,"userInteractionCount":20},"InteractionCounter",{"@type":68},"ViewAction",{"@type":70,"mainEntity":71},"FAQPage",[72,78,82],{"name":73,"@type":74,"acceptedAnswer":75},"What research questions does the thesis address about machine learning text classifiers?","Question",{"text":76,"@type":77},"It investigates user experience, usability, interpretability, and cognitive biases when people interact with ML text classifiers.","Answer",{"name":79,"@type":74,"acceptedAnswer":80},"How does TACA support thematic analysis and model interaction?",{"text":81,"@type":77},"TACA lets users import initial thematic analysis, extracts labelled sentences, trains an offline gradient boosting classifier, and provides coding suggestions with iterative re-training after re-labeling.",{"name":83,"@type":74,"acceptedAnswer":84},"What do the interpretability experiments (LIME, SHAP, and LLM summaries) show?",{"text":85,"@type":77},"Participants using explanations tended to focus on individual terms, while those without explanations identified broader topics; using summaries led to more features overall, but none significantly improved model prediction accuracy.","https://schema.org",{"og:url":52,"og:type":88,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":90,"canonical":52},"index,follow",{"doc_id":7,"site_id":25},{"code":4,"msg":5,"data":93},[94,98,102,106,111,116,121,124,129,132,136],{"id":21,"doc_module":4,"doc_module_name":47,"category_name":95,"show_sort_weight":96,"slug":97},"Story & Novel",90,"story-novel",{"id":48,"doc_module":4,"doc_module_name":47,"category_name":99,"show_sort_weight":100,"slug":101},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":47,"category_name":103,"show_sort_weight":104,"slug":105},"Exam",70,"exam",{"id":107,"doc_module":4,"doc_module_name":47,"category_name":108,"show_sort_weight":109,"slug":110},5,"Comic",60,"comic",{"id":112,"doc_module":4,"doc_module_name":47,"category_name":113,"show_sort_weight":114,"slug":115},6,"Technology",50,"technology",{"id":117,"doc_module":4,"doc_module_name":47,"category_name":118,"show_sort_weight":119,"slug":120},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":47,"category_name":12,"show_sort_weight":122,"slug":123},30,"research-report",{"id":125,"doc_module":4,"doc_module_name":47,"category_name":126,"show_sort_weight":127,"slug":128},9,"Religion & Spirituality",20,"religion-spirituality",{"id":127,"doc_module":4,"doc_module_name":47,"category_name":130,"show_sort_weight":127,"slug":131},"World Cup","world-cup",{"id":133,"doc_module":4,"doc_module_name":47,"category_name":134,"show_sort_weight":133,"slug":135},10,"Lifestyle","lifestyle",{"id":137,"doc_module":4,"doc_module_name":47,"category_name":138,"show_sort_weight":107,"slug":139},19,"General","general"]