[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-seo-135890-105":3,"detail-sidebar-cat-0-en-105":73,"doc-detail-135890-en":122},{"code":4,"msg":5,"data":6},0,"ok",{"site_id":7,"language":8,"slug":9,"title":10,"keywords":11,"description":12,"schema_data":13,"social_meta":66,"head_meta":68,"extra_data":70,"updated_unix":72},105,"en","lexpansion-evaluating-dictionary-based-lexicon-expansion-for-social-media","Lexpansion - Evaluating Dictionary Based Lexicon Expansion for Social Media","","Lexicons are indispensable tools for textual analysis through labeled term associations, but they remain static and demand manual curation. Language evolution and social media can introduce semantic shifts and neologisms, making timely updates essential. This work studies supervised learning for expanding categorical lexicons using dictionary definitions as transformer (BERT) input features, analyzing impacts of noise, size, labels, structure, and curation. Evaluations use English dictionaries to measure accuracy, coverage, and label consistency, then apply expanded lexicons to downstream tasks.",{"@graph":14,"@context":65},[15,34,56],{"@type":16,"itemListElement":17},"BreadcrumbList",[18,23,27,31],{"item":19,"name":20,"@type":21,"position":22},"https://docshare.wps.com","Home","ListItem",1,{"item":24,"name":25,"@type":21,"position":26},"https://docshare.wps.com/document/","Document",2,{"item":28,"name":29,"@type":21,"position":30},"https://docshare.wps.com/document/research-report/","Research & Report",3,{"item":32,"name":10,"@type":21,"position":33},"https://docshare.wps.com/document/lexpansion-evaluating-dictionary-based-lexicon-expansion-for-social-media/135890/",4,{"url":32,"name":10,"@type":35,"image":36,"author":41,"headline":10,"publisher":44,"fileFormat":47,"inLanguage":8,"description":12,"dateModified":48,"datePublished":49,"encodingFormat":47,"isAccessibleForFree":50,"interactionStatistic":51},"DigitalDocument",{"url":37,"@type":38,"width":39,"height":40},"https://docshare.wps.com/thumbnails/lexpansion-evaluating-dictionary-based-lexicon-expansion-for-social-media/135890.png","ImageObject",300,407,{"name":42,"@type":43},"Ophelia","Person",{"url":19,"name":45,"@type":46},"DocShare","Organization","application/pdf","2026-10-17","2026-08-21",true,{"@type":52,"interactionType":53,"userInteractionCount":55},"InteractionCounter",{"@type":54},"ViewAction",6,{"@type":57,"mainEntity":58},"FAQPage",[59],{"name":60,"@type":61,"acceptedAnswer":62},"Which factors are evaluated for dictionary-based lexicon expansion?","Question",{"text":63,"@type":64},"The work studies dictionary properties including noise, size, labels, structure, and curation method, and evaluates resulting expansions by accuracy, coverage, and labeling consistency before testing downstream impact.","Answer","https://schema.org",{"og:url":32,"og:type":67,"og:title":10,"og:site_name":45,"og:description":12},"article",{"robots":69,"canonical":32},"index,follow",{"doc_id":71,"site_id":7},135890,1787328019,{"code":4,"msg":74,"data":75},"success",[76,80,84,88,93,97,102,106,111,114,118],{"id":22,"doc_module":4,"doc_module_name":25,"category_name":77,"show_sort_weight":78,"slug":79},"Story & Novel",90,"story-novel",{"id":26,"doc_module":4,"doc_module_name":25,"category_name":81,"show_sort_weight":82,"slug":83},"Literature",80,"literature",{"id":33,"doc_module":4,"doc_module_name":25,"category_name":85,"show_sort_weight":86,"slug":87},"Exam",70,"exam",{"id":89,"doc_module":4,"doc_module_name":25,"category_name":90,"show_sort_weight":91,"slug":92},5,"Comic",60,"comic",{"id":55,"doc_module":4,"doc_module_name":25,"category_name":94,"show_sort_weight":95,"slug":96},"Technology",50,"technology",{"id":98,"doc_module":4,"doc_module_name":25,"category_name":99,"show_sort_weight":100,"slug":101},7,"Healthcare",40,"healthcare",{"id":103,"doc_module":4,"doc_module_name":25,"category_name":29,"show_sort_weight":104,"slug":105},8,30,"research-report",{"id":107,"doc_module":4,"doc_module_name":25,"category_name":108,"show_sort_weight":109,"slug":110},9,"Religion & Spirituality",20,"religion-spirituality",{"id":109,"doc_module":4,"doc_module_name":25,"category_name":112,"show_sort_weight":109,"slug":113},"World Cup","world-cup",{"id":115,"doc_module":4,"doc_module_name":25,"category_name":116,"show_sort_weight":115,"slug":117},10,"Lifestyle","lifestyle",{"id":119,"doc_module":4,"doc_module_name":25,"category_name":120,"show_sort_weight":89,"slug":121},19,"General","general",{"code":4,"msg":74,"data":123},{"doc_id":71,"user_id":124,"nickname":42,"user_avatar":125,"doc_module":4,"category_id":103,"category_name":29,"doc_title":10,"doc_description":12,"doc_content":126,"file_id":127,"file_url":128,"file_type":129,"file_size":130,"view_count":55,"is_deleted":4,"is_public":22,"is_downloadable":22,"audit_status":22,"page_count":131,"language":132,"language_code":8,"site_id":7,"html_lang":8,"table_of_contents":133,"faqs":134,"seo_title":135,"seo_description":12,"update_tm":72,"read_time":136},7971461741311,"https://ap-avatar.wpscdn.com/avatar/74000253aff267980c6?x-image-process=image/resize,m_fixed,w_180,h_180&k=1779345379180704826","Lexpansion: Evaluating Dictionary Based Lexicon Expansion for Social Media  \nAnalysis  \nMohamed Bahgat 1 , Steven R Wilson2 , Walid Magdy 1  \n1University of Edinburgh  \n2University of Michigan-Flint  \n[m.bahgat@ed.ac.uk](m.bahgat@ed.ac.uk), [steverw@umich.edu](steverw@umich.edu), [wmagdy@inf.ed.ac.uk](wmagdy@inf.ed.ac.uk)  \nAbstract  \nLexicons are indispensable tools for textual analysis through labelled term associations. Despite their utility, lexicons are static and require manual effort to curate and maintain. Regular updates are essential to stay relevant amid semantic shiftsand neologisms during language evolution. In this work, we explore the potential of supervised learning for expanding lexicons using dictionaries. We study the effect of using dictionaries with varying properties such as noise, size, labels, structure and curation method. Definitions are used as input features to a transformer model (BERT) that assigns categories to terms. We analyse the expansions using varying English dictionaries and lexicons for estimated accuracy, coverage and labelling consistency and apply the expanded versions to a downstream task. Our analyses show dictionary based expansion is a robust approach. We release our expanded lexicons, code, and pretrained models.  \n1 Introduction  \nLexicons have been broadly used for text analysis in domains such as politics (Elbagir and Yang 2019), social attitudes (Gao and Huang 2017), mental health (Zanwar et al. 2023) and more. Even in the era of LLMs, lexicons are useful to interpret results, build lightweight models or add emphasis on relevant features (Li et al. 2020) .  \nHowever, lexicons are static, yet language evolves overtime. Various societal and demographic factors were shown to affect language change (Labov 2011; Milroy and Milroy 2017) . These changes are amplified by social media both syntactically (Maity et al. 2016) and semantically (McGillivray et al. 2022) . Only a limited number of lexicons are updated to cope with language changes, like LIWC,(Pennebaker, Francis, and Booth 2001; Pennebaker et al. 2007, 2015; Boyd et al. 2022) which has been revised at 7-year intervals. An alternative is using resources such as dictionaries to expand lexicons (Bahgat, Wilson, and Magdy 2022) . Dictionaries are updated frequently and capture language changes (Nguyen, McGillivray, and Yasseri 2018) . For example, in March 2023, Oxford English Dictionary (OED) added 700 new terms like “deepfake” and  \n“groomzilla” while revising 1,400 1. Other dictionaries are Copyright © 2025, Association for the Advancement of Artificial Intelligence ([www.aaai.org](www.aaai.org)). All rights reserved.  \n1[https://www.oed.com/information/updates/march-2023/](https://www.oed.com/information/updates/march-2023/)  \neven more sensitive. In Urban Dictionary (UD), a crowdsourced dictionary, “selfie” was first defined in September 2009 versus August 2013 for OED (Nguyen, McGillivray, and Yasseri 2018) . UD contains a variety of slang terms and is useful in relevant applications (Wilson et al. 2020a) .  \nDownstream tasks that analyse content from social media would benefit largely from such increased coverage especially for terms that commonly surface on these forums.  \nIn this work, we present “Lexpansion”, a framework for adding new terms to lexicons by obtaining candidates from dictionaries and analysing results. We specifically address categorical lexicons which comprise lists of terms and their corresponding categories. Dictionaries used in our work are lists of words and corresponding senses. We describe senses as definitions composed of meaning and any examples provided for a given term. Dictionary definitions that correspond to lexicon terms are used to train a model to classify other unseen dictionary terms into one of the categories. Models are evaluated on held-out lexicon terms and their corresponding dictionary definitions.  \nOur work focuses on understanding the effect lexicon and dictionary properties h","cbCaiseUdS96hrMU","https://ap.wps.com/l/cbCaiseUdS96hrMU","pdf",648712,17,"English","# Abstract\n# 1 Introduction\n# 2 Related Work\n## 2.1 Early Automated Lexicon Creation and Expansion","[{\"question\":\"Which factors are evaluated for dictionary-based lexicon expansion?\",\"answer\":\"The work studies dictionary properties including noise, size, labels, structure, and curation method, and evaluates resulting expansions by accuracy, coverage, and labeling consistency before testing downstream impact.\"}]","Lexpansion - Evaluating Dictionary Based Lexicon Expansion for Social Media | PDF",43]