[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-85547-en":3,"doc-seo-85547-105":29,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},85547,16904993612988,"Olivia Brown","https://ap-avatar.wpscdn.com/davatar_a8503ba1806abce46bf441b54a3ca4cd",8,"Research & Report","IslamicMMLU: A Benchmark for Evaluating LLMs on Islamic Knowledge","Large language models are increasingly used for Islamic knowledge, yet no comprehensive benchmark assesses performance across core Islamic disciplines. IslamicMMLU introduces 10,013 multiple-choice questions across three tracks—Quran (2,013), Hadith (4,000), and Fiqh/jurisprudence (4,000)—built from multiple task types per track. It powers a public leaderboard and an initial evaluation of 26 LLMs, with averaged accuracy spanning 39.8% to 93.8%. The benchmark releases evaluation code and results publicly via HuggingFace.","IslamicMMLU:  \nA Benchmark for Evaluating LLMs on Islamic Knowledge  \nAli Abdelaal, Mohammed Nader Al Haffar,  \nMahmoud Fawzi, Walid Magdy  \nThe University of Edinburgh  \n{A.Abdelaal, M.N.Al-haffar, [M.F.G.Ibrahim}@sms.ed.ac.uk](M.F.G.Ibrahim}@sms.ed.ac.uk)  \n[wmagdy@inf.ed.ac.uk](wmagdy@inf.ed.ac.uk)  \narXiv :2603 .23750v 3 [ cs .CL] 11 Jul 2026  \nAbstract  \nLarge language models are increasingly consulted for Islamic knowledge, yet no comprehensive benchmark evaluates their performance across core Islamic disciplines. We introduce IslamicMMLU, a benchmark of  \n10,013 multiple-choice questions spanning three tracks: Quran (2,013 questions), Hadith (4,000 questions), and Fiqh (jurisprudence, 4,000 questions) . Each track is formed of multiple types of questions to examine LLMs capabilities handling different aspects of Islamic knowledge. The benchmark is used to create the IslamicMMLU public leaderboard for evaluating LLMs, and we initially evaluate 26 LLMs, where their averaged accuracy across the three tracks varied between 39.8% to 93.8%(by Gemini 3 Flash) . The Quran track shows the widest span (99.3% to 32.4%), while the Fiqh track includes a novel madhab (Islamic school of jurisprudence) bias detection task revealing variable school-of-thought preferences across models. Arabic-specific models show mixed results, but they all underperform compared to frontier models. The evaluation code and leaderboard are made publicly available.  \n1 Introduction  \nLarge language models (LLMs) are increasingly consulted for information about Islamic topics by millions of users worldwide (Mubarak et al., 2025) . Islamic knowledge spans multiple interconnected disciplines (e.g., Quranic studies, Hadith sciences, and Islamic jurisprudence), each requiring distinct scholarly expertise and evaluation methodologies. Yet no comprehensive benchmark evaluates LLM performance across these core domains. This gap can lead to religious misinformation affecting daily practice for Muslims who may lack easy access to qualified scholars (Naous et al., 2024 ; Fawzi et al., 2026) .  \nSeveral recent efforts address aspects of Islamic NLP (e.g., hallucination detection, cultural com-  \npetence, and jurisprudential QA), but none provides comprehensive cross-discipline evaluation. The MMLU paradigm (Hendrycks et al., 2021) has been extended to legal (Hijazi et al., 2024), cultural (Huang et al., 2024), and general Arabic (Koto et al., 2024) domains, but to our knowledge no systematic Islamic knowledge evaluation benchmark exists.  \nIslamic knowledge evaluation faces unique challenges across all three disciplines. Quranic evaluation requires precise textual recall across 6,236 verses (Ayahs) . Hadith evaluation demands source authentication across authentic canonical collections, textual completion, and reliability grading of authenticity. Fiqh evaluation confronts the inherent pluralism of Islamic jurisprudence, where the same act may receive different rulings across multiple valid schools of thought (madhahib) .“Correct/Incorrect” evaluation is fundamentally insufficient for fiqh, as a model penalised for answering according to one school when the benchmark expected another is not exhibiting error but reflecting the genuine plurality of the tradition.  \nWe introduce IslamicMMLU with the following contributions:  \n1. IslamicMMLU benchmark: 10,013 multiple-choice questions spanning three tracks with 12 task types that evaluate different facets of Islamic knowledge.  \n2. Fiqh track with madhab bias detection: A novel evaluation task measuring implicit school-of-thought bias in LLMs.  \n3. Evaluation of 26 LLMs: Comprehensive evaluation across all three tracks with bootstrap confidence intervals and statistical significance testing.  \n4. IslamicMMLU leaderboard 1 : Evaluation code and results released publicly via Hug-  \n1 [https://huggingface.co/spaces/](https://huggingface.co/spaces/)[ ](https://huggingface.co/spaces/)islamicmmlu/leaderboard  \ngingFace, with a l","cbCaisUwqgvQzH2t","https://ap.wps.com/l/cbCaisUwqgvQzH2t","pdf",273661,1,13,"English","en",105,"# Abstract\n# 1 Introduction\n## Motivation and gap\n## Challenges in Islamic evaluation\n## Contributions\n# 2 Background and Related Work\n## Islamic Knowledge Domains","[{\"question\":\"What is IslamicMMLU and what does it benchmark?\",\"answer\":\"IslamicMMLU is a benchmark designed to evaluate large language models on core Islamic knowledge through multiple-choice questions spanning Quran, Hadith, and Fiqh.\"},{\"question\":\"How is the IslamicMMLU dataset structured across tracks and question counts?\",\"answer\":\"It contains 10,013 questions total: 2,013 for the Quran track, 4,000 for the Hadith track, and 4,000 for the Fiqh (jurisprudence) track, with multiple task types within each track.\"},{\"question\":\"What was learned from the initial evaluation of 26 LLMs?\",\"answer\":\"Averaged accuracy across the three tracks ranged from 39.8% to 93.8%, with the Quran track showing the widest span and the Fiqh track including a madhab bias detection task revealing model-dependent school-of-thought preferences.\"}]",1784204373,33,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":27},"islamicmmlu-a-benchmark-for-evaluating-llms-on-islamic-knowledge","",{"@graph":35,"@context":85},[36,53,68],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/islamicmmlu-a-benchmark-for-evaluating-llms-on-islamic-knowledge/85547/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-17","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What is IslamicMMLU and what does it benchmark?","Question",{"text":75,"@type":76},"IslamicMMLU is a benchmark designed to evaluate large language models on core Islamic knowledge through multiple-choice questions spanning Quran, Hadith, and Fiqh.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"How is the IslamicMMLU dataset structured across tracks and question counts?",{"text":80,"@type":76},"It contains 10,013 questions total: 2,013 for the Quran track, 4,000 for the Hadith track, and 4,000 for the Fiqh (jurisprudence) track, with multiple task types within each track.",{"name":82,"@type":73,"acceptedAnswer":83},"What was learned from the initial evaluation of 26 LLMs?",{"text":84,"@type":76},"Averaged accuracy across the three tracks ranged from 39.8% to 93.8%, with the Quran track showing the widest span and the Fiqh track including a madhab bias detection task revealing model-dependent school-of-thought preferences.","https://schema.org",{"og:url":51,"og:type":87,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":89,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":45,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":45,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":45,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":45,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":45,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":45,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":45,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]