[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-84085-en":3,"doc-seo-84085-105":29,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},84085,1099514067415,"Rowan","https://ap-avatar.wpscdn.com/avatar/100002539d78ffe74a7?x-image-process=image/resize,m_fixed,w_180,h_180&k=1779092875211072502",8,"Research & Report","Sample Complexity Bounds for the Jensen-Shannon Divergence","The Jensen-Shannon divergence (JSD) provides a symmetric, bounded measure of dissimilarity between two probability distributions used across statistics, information theory, and machine learning. This work links JSD values to the data required to distinguish two generating distributions at a target error rate. It derives sample-size scalings for two i.i.d. classifiers: a log-likelihood-ratio rule needing samples scaling as the inverse JSD, and a majority-vote rule scaling as the squared inverse JSD.","arXiv :2607 .06270v 1 [ cs .IT] 7 Jul 2026  \nSample complexity bounds for the Jensen-Shannon divergence  \nOren Richter 1 , Adi Ben-Ari 1 , Tom Talpir 1 , and Elad Schneidman 1,B  \n1 Department of Brain Sciences, Weizmann Institute of Science, Rehovot 76100, Israel  \n[B](Belad.schneidman@weizmann.ac.il)[elad.schneidman@weizmann.ac.il](Belad.schneidman@weizmann.ac.il)  \nAbstract  \nThe Jensen-Shannon divergence (JSD) is a symmetric and bounded measure of the dissimilarity of two probability distributions, which has become a standard tool in statistics, information theory, and machine learning. We complement the understanding of its mathematical properties by presenting an analysis of the amount of data that is needed to distinguish between two distributions, given the value of JSD between them. We find the number of independent and identically distributed samples that suffice for a classifier to determine which of two distributions generated observed data at a desired error rate, for two complementary classifiers: we show that for the log-likelihood-ratio classifier, a sample size that grows as the inverse JSD is sufficient, whereas for a majority-vote classifier assembled from independent single-sample decisions, the sufficient size grows as the squared inverse JSD. These distinct scalings offer operational readings of JSD values and their translation into distinguishability in different contexts.  \nIntroduction  \nThe Kullback-Leibler (KL) divergence between two probability distributions P (x) and Q (x) ,  \nDKL [P(x)||Q(x)] = Xx P (x)log2 PQ((xx)) (1)  \nhas been a prominent measure of the dissimilarity of probability distributions [1] due to its foundational role in information theory, and its coding-related interpretations as a measure of coding inefficiency or distinguishability of sources [2] . Notably, the utility of KL and its interpretation are limited by its asymmetric nature, and because it diverges if there is an x for which P (x) > 0 whereas Q (x) = 0 .  \nThe Jensen-Shannon divergence (JSD) is a symmetric and finite measure of the dissimilarity of probability distributions [3], which has become a popular  \ntool in many data-oriented applications [4–6], as well as an interesting measure from a theoretical perspective. JSD measures the dissimilarity of P(x) and Q(x), by weighting their respective KL dissimilarity to an intermediate distribution M (x), namely,  \nDJS [P(x)||Q(x)] = λDKL [P(x)||M(x)] + (1 − λ)DKL [Q(x)||M(x)] (2)  \nwhere M(x) = λP(x) + (1 − λ)Q (x), and λ is a fraction between 0 and 1 (commonly taken to be equal to ~~1~~2 , a convention we also use here) . Importantly, the value of DJS is bounded, ranging from 0 for identical distributions, to 1 bit for non-overlapping distributions with disjoint supports. Moreover, it belongs to the family of f-divergences, inheriting their information-monotonicity under coarse graining [7–9]; and pDJS [P(x)||Q(x)] is a proper metric, satisfying the triangle inequality [10] . These properties have made the JSD popular across different fields, from statistics and information theory to data science and machine learning, where it appears in two-sample testing, generative modeling, and representation learning, among many other settings.  \nDespite its widespread use, the “operational” meaning of a given JSD value is often left implicit. While there is a known bound on the Bayes classification error in distinguishing between two probability distributions from a single observation based on knowing that DJS [P(x)||Q(x)] = d [3], it is not immediately clear how JSD governs distinguishability from many samples.  \nWe therefore ask here how many independent and identically distributed (i.i.d.) samples are sufficient for a classifier to identify the source distribution at a desired classification error rate. The sample complexity of distinguishing two distributions from i.i.d. samples is a classical quantity in statistical decision theory, and under uniform prior over the two distribut","cbCaimQZvhNiHQ52","https://ap.wps.com/l/cbCaimQZvhNiHQ52","pdf",294358,1,10,"English","en",105,"# Abstract\n# Introduction\n# Results\n## Sample complexity of optimal binary detection scales as the inverse JSD","[{\"question\":\"What question does the paper address about Jensen-Shannon divergence?\",\"answer\":\"It asks how many i.i.d. samples are needed for a classifier to identify which of two distributions generated observed data, given the JSD value and a desired error rate.\"},{\"question\":\"How does the required sample size scale for the log-likelihood-ratio classifier?\",\"answer\":\"For the log-likelihood-ratio classifier, the sufficient sample size grows proportionally to the inverse of the JSD.\"},{\"question\":\"How does the scaling differ for the majority-vote classifier?\",\"answer\":\"For a majority-vote classifier built from independent single-sample decisions, the sufficient sample size grows proportionally to the squared inverse of the JSD.\"}]",1784192628,25,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":27},"sample-complexity-bounds-for-the-jensen-shannon-divergence","",{"@graph":35,"@context":85},[36,53,68],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/sample-complexity-bounds-for-the-jensen-shannon-divergence/84085/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-17","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What question does the paper address about Jensen-Shannon divergence?","Question",{"text":75,"@type":76},"It asks how many i.i.d. samples are needed for a classifier to identify which of two distributions generated observed data, given the JSD value and a desired error rate.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"How does the required sample size scale for the log-likelihood-ratio classifier?",{"text":80,"@type":76},"For the log-likelihood-ratio classifier, the sufficient sample size grows proportionally to the inverse of the JSD.",{"name":82,"@type":73,"acceptedAnswer":83},"How does the scaling differ for the majority-vote classifier?",{"text":84,"@type":76},"For a majority-vote classifier built from independent single-sample decisions, the sufficient sample size grows proportionally to the squared inverse of the JSD.","https://schema.org",{"og:url":51,"og:type":87,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":89,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,134],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":45,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":45,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":45,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":45,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":45,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":21,"doc_module":4,"doc_module_name":45,"category_name":132,"show_sort_weight":21,"slug":133},"Lifestyle","lifestyle",{"id":135,"doc_module":4,"doc_module_name":45,"category_name":136,"show_sort_weight":106,"slug":137},19,"General","general"]