[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-127434-en":3,"doc-seo-127434-105":31,"detail-sidebar-cat-0-en-105":92},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":21,"is_downloadable":21,"audit_status":21,"page_count":22,"language":23,"language_code":24,"site_id":25,"html_lang":24,"table_of_contents":26,"faqs":27,"seo_title":28,"seo_description":14,"update_tm":29,"read_time":30},127434,8796095027276,"Valentina","https://avatar.qwps.com/avatar/d3BzX2FwX3Rlc3RfMjUxMTI2XzAxODA=",8,"Research & Report","UEM24 at the NTCIR-18 MedNLP-CHAT - A Machine Learning Approach to Multilingual Healthcare Risk Prediction","Risk prediction in the medical, ethical, and legal domains is essential for safety and informed decisions. This study develops machine learning solutions for the MedNLP-CHAT task using English-translated data from Japanese and German subtasks. Text is processed via tokenization, n-gram extraction, and lemmatization, then modeled with Logistic Regression, Nu-SVC, Gradient Boosting, and XGB Regressor. Objective risks use binary classification, while subjective labels are handled through regression aligned to human annotations. Evaluation includes accuracy, precision, recall, F1-score, and Earth Mover’s Distance, highlighting class-imbalance and overfitting considerations.","NTCIR-18: Proceedings of the 18th NTCIR Conference on Evaluation ofInformation Access Technologies, June 10-13, 2025, Tokyo, Japan  \nDOI: [https://doi.org/10.20736/0002002053](https://doi.org/10.20736/0002002053)  \nUEM24 at the NTCIR-18 MedNLP-CHAT: A Machine Learning Approach to Multilingual Healthcare Risk Prediction  \nAyantika Das  \nBrainware University University of Engineering & Management West Bengal, India [ayd.cse@brainwareuniversity.ac.in](ayd.cse@brainwareuniversity.ac.in)  \nAnupam Mondal  \nInstitute of Engineering and Management University of Engineering & Management West Bengal, India [anupam.mondal@iem.edu.in](anupam.mondal@iem.edu.in)  \nAbstract  \nRisk prediction in the context of medical, ethical, and legal is crucial for ensuring safety and informed decision-making. This study explores machine learning approaches for the MedNLP-CHAT task, utilizing English-translated datasets from Japanese and German subtasks. The textual data underwent preprocessing, including tokenization, n-gram extraction, and lemmatization, before being modeled using Logistic Regression, Nu-SVC (nu=0.1) [3], Gradient Boosting, and XGB Regressor. Objective risks were framed as a binary classification task, while subjective labels were predicted via regression, ensuring alignment with human-annotated distributions. Performance was evaluated using accuracy, precision, recall, F1-score, and Earth Mover’s Distance (EMD) . The findings indicate the model’s strengths and weaknesses, emphasizing the need to enhance how class imbalances and potential overfitting are addressed. This work increases AI-driven risk assessment with applications in regulatory compliance, healthcare, and ethical AI development.  \nKeywords  \nMedical Natural Language Processing (MedNLP), Count Vectorization, n-gram, medical ethical and legal risks  \nTeam Name  \nUEM24  \nSubtasks  \nJapanese subtask (EN) German subtask (EN)  \n1 Introduction  \nRisk assessment is significant in healthcare because it makes an impact on legal compliance, ethical considerations, and medical decision-making. Accurately analyzing medical, ethical, and legal risks from textual data has become difficult with the increasing use of AI-driven systems. By creating models that can categorize these risks and assess subjective factors like fluency, helpfulness, and harmlessness, the MedNLP-CHAT task [2] seeks to overcome this difficulty. To provide a linguistically diverse basis for training and assessment, this study uses a dataset that includes English translations from Japanese (Task1) and German (Task2) subtasks. This study advances our knowledge of AI-based risk assessment in healthcare and related domains by utilizing both objective and subjective risk indicators.  \nTo improve the reliability and consistency of risk prediction, we implemented a robust text preprocessing pipeline, including tokenization, n-gram extraction, and lemmatization. Risk classification  \nwas framed as a binary classification task, distinguishing between high-risk and low-risk responses, while subjective label estimation was approached using regression models trained in linguistic characteristics. Performance was rigorously evaluated using multiple metrics, including accuracy, precision, recall[5], F1-score, and Earth Mover’s Distance (EMD), ensuring that the models effectively capture both risk patterns and subjective response distributions. By analyzing various machine learning approaches, this study provides valuable insights into the effectiveness of AI in risk classification and highlights its potential applications in automated decisionmaking systems for healthcare, regulatory frameworks, and ethical AI development.  \n2 Related Work  \nRecent developments in medical NLP have focused on various multilabel classification tasks. The \"NTCIR-12 MedNLPDoc Task\" is designed to classify disease names from Japanese medical records using the ICD-10 coding system, demonstrating the continued effectiveness of rule-based systems due to s","cbCaiahwM1PEdSfB","https://ap.wps.com/l/cbCaiahwM1PEdSfB","pdf",1651672,2,1,5,"English","en",105,"# Introduction\n## Risk assessment in healthcare\n# Related Work\n## Multilabel medical NLP tasks\n# Methodology\n## Dataset setup\n## Text preprocessing and modeling\n## Evaluation metrics","[{\"question\":\"What task does this study address in healthcare risk prediction?\",\"answer\":\"It addresses the MedNLP-CHAT task by predicting both objective medical/ethical/legal risks and subjective attributes such as fluency, helpfulness, and harmlessness from multilingual text translations.\"},{\"question\":\"How are objective and subjective risk labels modeled differently?\",\"answer\":\"Objective risks are framed as binary classification, while subjective labels are predicted using regression models trained to fit the distribution of human-annotated scores.\"},{\"question\":\"Which models and evaluation metrics are used to assess performance?\",\"answer\":\"The study uses Logistic Regression, Nu-SVC, Gradient Boosting, and XGB Regressor, and evaluates with accuracy, precision, recall, F1-score, and Earth Mover’s Distance (EMD).\"}]","UEM24 at the NTCIR-18 MedNLP-CHAT - A Machine Learning Approach to Multilingual Healthcare Risk Prediction | PDF",1785938847,13,{"code":4,"msg":32,"data":33},"ok",{"site_id":25,"language":24,"slug":34,"title":13,"keywords":35,"description":14,"schema_data":36,"social_meta":87,"head_meta":89,"extra_data":91,"updated_unix":29},"uem24-at-the-ntcir-18-mednlp-chat-a-machine-learning-approach-to-multilingual-healthcare-risk-prediction","",{"@graph":37,"@context":86},[38,54,69],{"@type":39,"itemListElement":40},"BreadcrumbList",[41,45,48,51],{"item":42,"name":43,"@type":44,"position":21},"https://docshare.wps.com","Home","ListItem",{"item":46,"name":47,"@type":44,"position":20},"https://docshare.wps.com/document/","Document",{"item":49,"name":12,"@type":44,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":44,"position":53},"https://docshare.wps.com/document/uem24-at-the-ntcir-18-mednlp-chat-a-machine-learning-approach-to-multilingual-healthcare-risk-prediction/127434/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":24,"description":14,"dateModified":62,"datePublished":63,"encodingFormat":61,"isAccessibleForFree":64,"interactionStatistic":65},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":42,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-23","2026-08-05",true,{"@type":66,"interactionType":67,"userInteractionCount":20},"InteractionCounter",{"@type":68},"ViewAction",{"@type":70,"mainEntity":71},"FAQPage",[72,78,82],{"name":73,"@type":74,"acceptedAnswer":75},"What task does this study address in healthcare risk prediction?","Question",{"text":76,"@type":77},"It addresses the MedNLP-CHAT task by predicting both objective medical/ethical/legal risks and subjective attributes such as fluency, helpfulness, and harmlessness from multilingual text translations.","Answer",{"name":79,"@type":74,"acceptedAnswer":80},"How are objective and subjective risk labels modeled differently?",{"text":81,"@type":77},"Objective risks are framed as binary classification, while subjective labels are predicted using regression models trained to fit the distribution of human-annotated scores.",{"name":83,"@type":74,"acceptedAnswer":84},"Which models and evaluation metrics are used to assess performance?",{"text":85,"@type":77},"The study uses Logistic Regression, Nu-SVC, Gradient Boosting, and XGB Regressor, and evaluates with accuracy, precision, recall, F1-score, and Earth Mover’s Distance (EMD).","https://schema.org",{"og:url":52,"og:type":88,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":90,"canonical":52},"index,follow",{"doc_id":7,"site_id":25},{"code":4,"msg":5,"data":93},[94,98,102,106,110,115,120,123,128,131,135],{"id":21,"doc_module":4,"doc_module_name":47,"category_name":95,"show_sort_weight":96,"slug":97},"Story & Novel",90,"story-novel",{"id":20,"doc_module":4,"doc_module_name":47,"category_name":99,"show_sort_weight":100,"slug":101},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":47,"category_name":103,"show_sort_weight":104,"slug":105},"Exam",70,"exam",{"id":22,"doc_module":4,"doc_module_name":47,"category_name":107,"show_sort_weight":108,"slug":109},"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":47,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":47,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":47,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":47,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":47,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":47,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":47,"category_name":137,"show_sort_weight":22,"slug":138},19,"General","general"]