[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-seo-153779-105":3,"detail-sidebar-cat-0-en-105":81,"doc-detail-153779-en":130},{"code":4,"msg":5,"data":6},0,"ok",{"site_id":7,"language":8,"slug":9,"title":10,"keywords":11,"description":12,"schema_data":13,"social_meta":74,"head_meta":76,"extra_data":78,"updated_unix":80},105,"en","detecting-transliterated-orthographic-variants-via-two-similarity-metrics","Detecting Transliterated Orthographic Variants via Two Similarity Metrics","","A robust detection method identifies orthographic variants of foreign words caused by transliteration in large corpora. The approach combines two complementary similarity signals: string similarity based on edit distance and contextual similarity produced by a vector space model. Evaluation on an open test shows strong effectiveness, achieving a 0.889 F-measure. The method targets transliteration-driven spelling divergence where conventional rule-based back-transliteration is difficult and misspellings can still match dictionary entries.",{"@graph":14,"@context":73},[15,34,56],{"@type":16,"itemListElement":17},"BreadcrumbList",[18,23,27,31],{"item":19,"name":20,"@type":21,"position":22},"https://docshare.wps.com","Home","ListItem",1,{"item":24,"name":25,"@type":21,"position":26},"https://docshare.wps.com/document/","Document",2,{"item":28,"name":29,"@type":21,"position":30},"https://docshare.wps.com/document/research-report/","Research & Report",3,{"item":32,"name":10,"@type":21,"position":33},"https://docshare.wps.com/document/detecting-transliterated-orthographic-variants-via-two-similarity-metrics/153779/",4,{"url":32,"name":10,"@type":35,"image":36,"author":41,"headline":10,"publisher":44,"fileFormat":47,"inLanguage":8,"description":12,"dateModified":48,"datePublished":49,"encodingFormat":47,"isAccessibleForFree":50,"interactionStatistic":51},"DigitalDocument",{"url":37,"@type":38,"width":39,"height":40},"https://docshare.wps.com/thumbnails/detecting-transliterated-orthographic-variants-via-two-similarity-metrics/153779.png","ImageObject",300,407,{"name":42,"@type":43},"Aria","Person",{"url":19,"name":45,"@type":46},"DocShare","Organization","application/pdf","2026-10-17","2026-08-28",true,{"@type":52,"interactionType":53,"userInteractionCount":55},"InteractionCounter",{"@type":54},"ViewAction",8,{"@type":57,"mainEntity":58},"FAQPage",[59,65,69],{"name":60,"@type":61,"acceptedAnswer":62},"What problem does the paper address?","Question",{"text":63,"@type":64},"The paper addresses detecting orthographic variants of foreign words that arise from transliteration, especially in large corpora with many mismatches and misspelled forms.","Answer",{"name":66,"@type":61,"acceptedAnswer":67},"How does the proposed method detect transliteration-induced variants?",{"text":68,"@type":64},"It combines two similarity metrics: string similarity via edit distance and contextual similarity using a vector space model.",{"name":70,"@type":61,"acceptedAnswer":71},"How well does the method perform in evaluation?",{"text":72,"@type":64},"Experiments on an open test report a 0.889 F-measure, indicating effective detection of orthographic variants.","https://schema.org",{"og:url":32,"og:type":75,"og:title":10,"og:site_name":45,"og:description":12},"article",{"robots":77,"canonical":32},"index,follow",{"doc_id":79,"site_id":7},153779,1787882294,{"code":4,"msg":82,"data":83},"success",[84,88,92,96,101,106,111,114,119,122,126],{"id":22,"doc_module":4,"doc_module_name":25,"category_name":85,"show_sort_weight":86,"slug":87},"Story & Novel",90,"story-novel",{"id":26,"doc_module":4,"doc_module_name":25,"category_name":89,"show_sort_weight":90,"slug":91},"Literature",80,"literature",{"id":33,"doc_module":4,"doc_module_name":25,"category_name":93,"show_sort_weight":94,"slug":95},"Exam",70,"exam",{"id":97,"doc_module":4,"doc_module_name":25,"category_name":98,"show_sort_weight":99,"slug":100},5,"Comic",60,"comic",{"id":102,"doc_module":4,"doc_module_name":25,"category_name":103,"show_sort_weight":104,"slug":105},6,"Technology",50,"technology",{"id":107,"doc_module":4,"doc_module_name":25,"category_name":108,"show_sort_weight":109,"slug":110},7,"Healthcare",40,"healthcare",{"id":55,"doc_module":4,"doc_module_name":25,"category_name":29,"show_sort_weight":112,"slug":113},30,"research-report",{"id":115,"doc_module":4,"doc_module_name":25,"category_name":116,"show_sort_weight":117,"slug":118},9,"Religion & Spirituality",20,"religion-spirituality",{"id":117,"doc_module":4,"doc_module_name":25,"category_name":120,"show_sort_weight":117,"slug":121},"World Cup","world-cup",{"id":123,"doc_module":4,"doc_module_name":25,"category_name":124,"show_sort_weight":123,"slug":125},10,"Lifestyle","lifestyle",{"id":127,"doc_module":4,"doc_module_name":25,"category_name":128,"show_sort_weight":97,"slug":129},19,"General","general",{"code":4,"msg":82,"data":131},{"doc_id":79,"user_id":132,"nickname":42,"user_avatar":133,"doc_module":4,"category_id":55,"category_name":29,"doc_title":10,"doc_description":12,"doc_content":134,"file_id":135,"file_url":136,"file_type":137,"file_size":138,"view_count":55,"is_deleted":4,"is_public":22,"is_downloadable":22,"audit_status":22,"page_count":107,"language":139,"language_code":8,"site_id":7,"html_lang":8,"table_of_contents":140,"faqs":141,"seo_title":142,"seo_description":12,"update_tm":80,"read_time":143,"preview_page_count":4},2336464648322,"https://ap-avatar.wpscdn.com/avatar/2200025388227c56fec?_k=1778556882303663488","Detecting Transliterated Orthographic Variants via Two Similarity Metrics  \nKiyonori Ohtake  \nATR SLT  \nKeihanna Science City Kyoto 619-0288, Japan  \n[kiyonori.ohtake@atr.jp](kiyonori.ohtake@atr.jp)  \nYouichi Sekiguchi  \nNagaoka Univ. of Tech. Nagaoka City, Niigata 940-2188, Japan  \n[sekiguti@nlp.nagaokaut.ac.jp](sekiguti@nlp.nagaokaut.ac.jp)  \nKazuhide Yamamoto  \nNagaoka Univ. of Tech. Nagaoka City, Niigata 940-2188, Japan  \n[yamamoto@fw.ipsj.or.jp](yamamoto@fw.ipsj.or.jp)  \nAbstract  \nWe propose a detection method for orthographic variants caused by transliteration in a large corpus. The method employs two similarities. One is string similarity based on edit distance. The other is contextual similarity by a vector space model. Experimental results show that the method performed a 0.889 F-measure in an open test.  \n1 Introduction  \nThis paper discusses a detection method for transliterated orthographic variants of foreign words. Transliteration of foreign words causes orthographic variants because there are several conditions required for transliterating. One may person transliterate to approximate pronunciation, whereas another one may conduct transliteration based on spelling. For example, the English word report can be transliterated into two Japanese words, 􀂓リポート(ripooto)􀂔 and 􀂓レポー ト(repooto) .􀂔 The former 􀂓ripooto􀂔 is based on an approximation of its pronunciation, while 􀂓repooto􀂔 is transliterated from its spelling.  \nIn addition, several source languages can be transliterated. For instance, the English word virus corresponds to the Japanese words: 􀂓ウイルス(uirusu)􀂔 from Latin, 􀂓ビールス(biirusu)􀂔 and 􀂓ヴィールス(viirusu)􀂔 from German, while 􀂓バイラス(bairasu)􀂔 or 􀂓ヴァイラス(vairasu)􀂔 are also possible as transliterations that approximate the English pronunciation. Moreover, some foreign words end up in different forms in Japanese because of variation in English pronunciation; e.g., between British and American. For example, the  \nEnglish word body corresponds to two words: 􀂓ボディ(bodi)􀂔 from British and 􀂓バディ(badi)􀂔 from American.  \nOne may think that if back-transliteration were done precisely, those variants would be backtransliterated into one word, and they would be recognized as variants. However, backtransliteration is known to be a very dif􀀂cult task(Knight and Graehl, 1997) .  \nNot only Japanese but any language that has a phonetic spelling has this problem of transliterated orthographic variants. For example, English has variants for a Chinese proper noun as 􀂓Shanhaiguan,􀂔 􀂓Shanhaikwan,􀂔 or 􀂓Shanhaikuan.􀂔  \nNowadays, it is well recognized that orthographic variant correction is an important processing step for achieving high performance in natural language processing. In order to achieve robust and reliable processing, we have to use many language resources: many types of corpora, dictionaries, thesauri, and so on. Orthographic variants cause many mismatches at any stage of natural language processing. In addition, not only orthographic variants, but also misspelled words tend to slip into a corpus. These words boost the perplexity of the corpus, and worsen the data sparseness problem.  \nTo date, several studies have tried to cope with this orthographic variant problem; however, they considered the problem in a relatively clean corpus that was well organized by natives ofthe target language. As with orthographic variants, misspelled words cause mismatches, and we have to detect not only predictable orthographic variants but also misspelled variants. In addition, it is very hard to detect orthographic variants caused by misspelling with ordinary rule-based methods, because preparing such rules for misspellings that might be writ-  \nten is an unrealistic approach.  \nIf a corpus includes texts that were written by non-natives of the language, orthographic variants that are misspelled will likely be increased because non-natives have a limited vocabulary in that language.  \nWe propose a robust detection method for transliterated orthogr","cbCaijypPUSWTkO2","https://ap.wps.com/l/cbCaijypPUSWTkO2","pdf",144712,"English","# Introduction\n# Transliteration for foreign words in Japanese: katakana\n## Romanization","[{\"question\":\"What problem does the paper address?\",\"answer\":\"The paper addresses detecting orthographic variants of foreign words that arise from transliteration, especially in large corpora with many mismatches and misspelled forms.\"},{\"question\":\"How does the proposed method detect transliteration-induced variants?\",\"answer\":\"It combines two similarity metrics: string similarity via edit distance and contextual similarity using a vector space model.\"},{\"question\":\"How well does the method perform in evaluation?\",\"answer\":\"Experiments on an open test report a 0.889 F-measure, indicating effective detection of orthographic variants.\"}]","Detecting Transliterated Orthographic Variants via Two Similarity Metrics | PDF",18]