[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-123338-en":3,"doc-seo-123338-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},123338,1374391974468,"Eden","https://ap-avatar.wpscdn.com/davatar_29158cc5080c5b710cf443261637dec0",8,"Research & Report","Evaluating the Performance of Agreement Metrics in a Delphi Study on Chemical, Biological, Radiological and Nuclear Major Incidents Preparedness Using Classical and Machine Learning Approaches","Delphi studies in disaster medicine often lack consensus on how expert agreement metrics perform. This study evaluates agreement measures within a Delphi process for chemical, biological, radiological, and nuclear (CBRN) major-incident preparedness in the Middle East and North Africa. Forty international experts rated 133 items across ten CBRN Preparedness Assessment Tool themes on a 5-point Likert scale. Kendall’s W, intraclass correlation coefficient, and Cohen’s kappa were compared using statistical and machine-learning methods. Overall agreement reached 4.91 ± 0.71, and kappa proved the most sensitive metric, varying by themes such as medical protocols, logistics, and infrastructure, supporting future refinement.","Journal of  \nContingencies and Crisis Management  \nORIGINAL ARTICLE   \nEvaluating the Performance of Agreement Metrics in a Delphi Study on Chemical, Biological, Radiological and Nuclear Major Incidents Preparedness Using Classical and Machine Learning Approaches  \nHassan Farhat1,2  | Alan M. Batt3,4  | Mariana Helou5,6  | Heejun Shin7,8,9, 10 | James Laughton2  | Carolyn Dumbeck11 | Arezoo Dehghani12, 13 | Fatemeh Rezaei13  | Nidaa Bajow14  | Luc Mortelmans15,16, 17  | Walid Abougalala18 | Roberto Mugavero19,20,21 | Gregory Ciottone10,22  | Guillaume Alinier2,23,24,25  | Mohamed Ben Dhiab1   \n1Faculty of Medicine “Ibn El Jazzar,”, University of Sousse, Sousse, Tunisia | 2Ambulance Service, Hamad Medical Corporation, Doha, Qatar | 3Queen's University, Kingston, Ontario, Canada | 4Monash University, Melbourne, Victoria, Australia | 5School of Medicine, Lebanese American University, Beirut, Lebanon | 6Lebanese American University‐Rizk Hospital, Beirut, Lebanon | 7Soonchunhyang Disaster Medicine Center, Bucheon, South  \nKorea | 8Soonchunhyang University Bucheon Hospital, Bucheon, South Korea | 9Shin's Disaster Medicine Academy, Seoul, South Korea | 10Harvard Medical School, Harvard University, Cambridge, Massachusetts, USA | 11Department of Disaster Management, Alberta Health Services | 12Safety Promotion and Injury Prevention Research Center, Shahid Beheshti University of Medical Sciences, Tehran, Iran | 13Department of Health in Emergencies and Disasters, Health Management and Economic Research Center, Isfahan University of Medical Sciences, Isfahan, Iran | 14Security Forces Hospital, Riyadh, Saudi Arabia | 15European Society for Emergency Medicine, Belgium | 16Catholic University of Leuven, Belgium | 17Free University Brussels, Belgium | 18Hamad Medical Corporation, Doha, Qatar | 19Department of Electronic Engineering – DIE, University of Rome “Tor Vergata, Rome, Italy | 20Centre for Security Studies – CUFS, University of the Republic of San Marino, San Marino | 21Observatory on Security and CBRNe Defense – OSDIFE | 22Harvard T.H. Chan School of Public Health, USA | 23School of Health and Social Work, University of Hertfordshire, Hatfield, UK | 24Weill Cornell Medicine‐Qatar, Doha,  \nQatar | 25Faculty of Health and Life Sciences, Northumbria University, Newcastle upon Tyne, UK Correspondence: Hassan Farhat ([hassen.farhat@gmail.com](hassen.farhat@gmail.com))  \nReceived: 9 December 2024 | Revised: 24 March 2025 | Accepted: 24 March 2025  \nFunding: The authors received no specific funding for this study.  \nKeywords: agreement analysis | Delphi study | disaster medicine | expert's opinion | MENA  \nABSTRACT  \nDelphi studies in disaster medicine lack consensus on expert agreement metrics. This study examined various metrics using a Delphi study on chemical, biological, radiological, and nuclear (CBRN) preparedness in the Middle East and North Africa region. Forty international disaster medicine experts evaluated 133 items across ten CBRN Preparedness Assessment Tool themes using a 5‐point Likert scale. Agreement was measured using Kendall's W, Intraclass Correlation Coefficient, and Cohen's Kappa. Statistical and machine learning techniques compared metric performance. The overall agreement mean score was 4.91 ± 0.71, with 89.21% average agreement. Kappa emerged as the most sensitive metric in statistical and machine learning analyses, with a feature importance score of 168.32 . The Kappa coefficient showed variations across CBRN PAT themes, including medical protocols, logistics, and infrastructure. The integrated statistical and machine learning approach provides a promising method for understanding expert consensus in disaster preparedness, with potential for future refinement by incorporating additional contextual factors.  \nThis is an open access article under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited.  \n© ","cbCaim8ty4mEDZ0N","https://ap.wps.com/l/cbCaim8ty4mEDZ0N","pdf",901981,1,20,"English","en",105,"# Introduction\n## Delphi studies in disaster medicine\n## Expert consensus, benefits, and criticisms\n# Methods and analysis\n## Delphi process and assessment items\n## Agreement metrics and comparison approaches\n# Results and interpretation\n## Overall agreement and metric sensitivity\n## Variations across CBRN themes\n# Discussion and implications\n## Integrated statistical and machine learning value","[{\"question\":\"What is the study evaluating in the Delphi process?\",\"answer\":\"It evaluates how different agreement metrics perform when experts assess chemical, biological, radiological, and nuclear preparedness items in a Delphi study.\"},{\"question\":\"How were expert agreement metrics measured?\",\"answer\":\"Agreement was measured using Kendall’s W, intraclass correlation coefficient, and Cohen’s kappa, then compared with statistical and machine-learning techniques.\"},{\"question\":\"Which agreement metric was most sensitive, and what did it show?\",\"answer\":\"Cohen’s kappa was the most sensitive metric, showing variations across themes including medical protocols, logistics, and infrastructure.\"}]","Evaluating the Performance of Agreement Metrics in a Delphi Study on Chemical, Biological, Radiological and Nuclear Major Incidents Preparedness Using Classical and Machine Learning Approaches | PDF",1785816007,50,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"evaluating-the-performance-of-agreement-metrics-in-a-delphi-study-on-chemical-biological-radiological-and-nuclear-major-incidents-preparedness-using-classical-and-machine-learning-approaches","",{"@graph":36,"@context":85},[37,54,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/evaluating-the-performance-of-agreement-metrics-in-a-delphi-study-on-chemical-biological-radiological-and-nuclear-major-incidents-preparedness-using-classical-and-machine-learning-approaches/123338/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":62,"encodingFormat":61,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-04",true,{"@type":65,"interactionType":66,"userInteractionCount":4},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What is the study evaluating in the Delphi process?","Question",{"text":75,"@type":76},"It evaluates how different agreement metrics perform when experts assess chemical, biological, radiological, and nuclear preparedness items in a Delphi study.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"How were expert agreement metrics measured?",{"text":80,"@type":76},"Agreement was measured using Kendall’s W, intraclass correlation coefficient, and Cohen’s kappa, then compared with statistical and machine-learning techniques.",{"name":82,"@type":73,"acceptedAnswer":83},"Which agreement metric was most sensitive, and what did it show?",{"text":84,"@type":76},"Cohen’s kappa was the most sensitive metric, showing variations across themes including medical protocols, logistics, and infrastructure.","https://schema.org",{"og:url":52,"og:type":87,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":89,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,114,119,122,126,129,133],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":29,"slug":113},6,"Technology","technology",{"id":115,"doc_module":4,"doc_module_name":46,"category_name":116,"show_sort_weight":117,"slug":118},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":120,"slug":121},30,"research-report",{"id":123,"doc_module":4,"doc_module_name":46,"category_name":124,"show_sort_weight":21,"slug":125},9,"Religion & Spirituality","religion-spirituality",{"id":21,"doc_module":4,"doc_module_name":46,"category_name":127,"show_sort_weight":21,"slug":128},"World Cup","world-cup",{"id":130,"doc_module":4,"doc_module_name":46,"category_name":131,"show_sort_weight":130,"slug":132},10,"Lifestyle","lifestyle",{"id":134,"doc_module":4,"doc_module_name":46,"category_name":135,"show_sort_weight":106,"slug":136},19,"General","general"]