[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-seo-203758-105":3,"detail-sidebar-cat-0-en-105":81,"doc-detail-203758-en":130},{"code":4,"msg":5,"data":6},0,"ok",{"site_id":7,"language":8,"slug":9,"title":10,"keywords":11,"description":12,"schema_data":13,"social_meta":74,"head_meta":76,"extra_data":78,"updated_unix":80},105,"en","estimating-the-unseen-an-nlogn-sample-estimator-for-entropy-and-support-size-shown-optimal-via-new-clts","Estimating the Unseen - An n/log(n)-Sample Estimator for Entropy and Support Size - Shown Optimal via New CLTs","","Presents a new characterization of the unobserved portion of a distribution, yielding sublinear-sample estimators with arbitrarily small additive constant error for properties including entropy and distribution support size. Provides matching lower bounds to resolve longstanding questions about sample complexity up to constant factors. Gives an algorithm using O(n/log n) samples, with linear time in the sample size, and proves Ω(n/log n) lower bounds. Uses new multivariate central limit theorems for generalized multinomial distributions.",{"@graph":14,"@context":73},[15,34,56],{"@type":16,"itemListElement":17},"BreadcrumbList",[18,23,27,31],{"item":19,"name":20,"@type":21,"position":22},"https://docshare.wps.com","Home","ListItem",1,{"item":24,"name":25,"@type":21,"position":26},"https://docshare.wps.com/document/","Document",2,{"item":28,"name":29,"@type":21,"position":30},"https://docshare.wps.com/document/research-report/","Research & Report",3,{"item":32,"name":10,"@type":21,"position":33},"https://docshare.wps.com/document/estimating-the-unseen-an-nlogn-sample-estimator-for-entropy-and-support-size-shown-optimal-via-new-clts/203758/",4,{"url":32,"name":10,"@type":35,"image":36,"author":41,"headline":10,"publisher":44,"fileFormat":47,"inLanguage":8,"description":12,"dateModified":48,"datePublished":49,"encodingFormat":47,"isAccessibleForFree":50,"interactionStatistic":51},"DigitalDocument",{"url":37,"@type":38,"width":39,"height":40},"https://docshare.wps.com/thumbnails/estimating-the-unseen-an-nlogn-sample-estimator-for-entropy-and-support-size-shown-optimal-via-new-clts/203758.png","ImageObject",300,407,{"name":42,"@type":43},"Himbo","Person",{"url":19,"name":45,"@type":46},"DocShare","Organization","application/pdf","2026-10-07","2026-09-04",true,{"@type":52,"interactionType":53,"userInteractionCount":55},"InteractionCounter",{"@type":54},"ViewAction",8,{"@type":57,"mainEntity":58},"FAQPage",[59,65,69],{"name":60,"@type":61,"acceptedAnswer":62},"What estimation problems does the paper address?","Question",{"text":63,"@type":64},"It focuses on estimating the entropy of a discrete distribution and estimating the distribution support size, emphasizing the unobserved portion of the distribution.","Answer",{"name":66,"@type":61,"acceptedAnswer":67},"How many samples does the proposed method use?",{"text":68,"@type":64},"The algorithm estimates entropy and support size within an arbitrarily small additive constant using O(n/log n) samples, where n bounds the support size.",{"name":70,"@type":61,"acceptedAnswer":71},"How are the matching lower bounds proved?",{"text":72,"@type":64},"The lower bound proof relies on new multivariate central limit theorems, using Stein’s method under the Wasserstein (earthmover) distance and applying them to generalized multinomial distributions.","https://schema.org",{"og:url":32,"og:type":75,"og:title":10,"og:site_name":45,"og:description":12},"article",{"robots":77,"canonical":32},"index,follow",{"doc_id":79,"site_id":7},203758,1788563267,{"code":4,"msg":82,"data":83},"success",[84,88,92,96,101,106,111,114,119,122,126],{"id":22,"doc_module":4,"doc_module_name":25,"category_name":85,"show_sort_weight":86,"slug":87},"Story & Novel",90,"story-novel",{"id":26,"doc_module":4,"doc_module_name":25,"category_name":89,"show_sort_weight":90,"slug":91},"Literature",80,"literature",{"id":33,"doc_module":4,"doc_module_name":25,"category_name":93,"show_sort_weight":94,"slug":95},"Exam",70,"exam",{"id":97,"doc_module":4,"doc_module_name":25,"category_name":98,"show_sort_weight":99,"slug":100},5,"Comic",60,"comic",{"id":102,"doc_module":4,"doc_module_name":25,"category_name":103,"show_sort_weight":104,"slug":105},6,"Technology",50,"technology",{"id":107,"doc_module":4,"doc_module_name":25,"category_name":108,"show_sort_weight":109,"slug":110},7,"Healthcare",40,"healthcare",{"id":55,"doc_module":4,"doc_module_name":25,"category_name":29,"show_sort_weight":112,"slug":113},30,"research-report",{"id":115,"doc_module":4,"doc_module_name":25,"category_name":116,"show_sort_weight":117,"slug":118},9,"Religion & Spirituality",20,"religion-spirituality",{"id":117,"doc_module":4,"doc_module_name":25,"category_name":120,"show_sort_weight":117,"slug":121},"World Cup","world-cup",{"id":123,"doc_module":4,"doc_module_name":25,"category_name":124,"show_sort_weight":123,"slug":125},10,"Lifestyle","lifestyle",{"id":127,"doc_module":4,"doc_module_name":25,"category_name":128,"show_sort_weight":97,"slug":129},19,"General","general",{"code":4,"msg":82,"data":131},{"doc_id":79,"user_id":132,"nickname":42,"user_avatar":133,"doc_module":4,"category_id":55,"category_name":29,"doc_title":10,"doc_description":12,"doc_content":134,"file_id":135,"file_url":136,"file_type":137,"file_size":138,"view_count":55,"is_deleted":4,"is_public":22,"is_downloadable":22,"audit_status":22,"page_count":123,"language":139,"language_code":8,"site_id":7,"html_lang":8,"table_of_contents":140,"faqs":141,"seo_title":142,"seo_description":12,"update_tm":80,"read_time":143},687197100911,"https://ap-avatar.wpscdn.com/avatar/a000239b6f1da00475?x-image-process=image/resize,m_fixed,w_180,h_180&k=1785132997149421697","Estimating the Unseen: An n/log(n)-sample Estimator for Entropy and Support Size, Shown Optimal via New CLTs∗  \n†  \nGregory Valiant  \nUC Berkeley Berkeley, California  \n[gregory.valiant@gmail.com](gregory.valiant@gmail.com)  \nPaul Valiant‡ UC Berkeley Berkeley, California  \n[pvaliant@gmail.com](pvaliant@gmail.com)  \nABSTRACT  \nWe introduce a new approach to characterizing the unobserved portion of a distribution, which provides sublinear– sample estimators achieving arbitrarily small additive constant error for a class of properties that includes entropy and distribution support size. Additionally, we show new matching lower bounds. Together, this settles the longstanding question of the sample complexities of these estimation problems, up to constant factors.  \nOur algorithm estimates these properties up to an arbitrarily small additive constant, using O (n/ log n) samples, where n is a bound on the support size, or in the case of estimating the support size, 1/n is a lower bound on the probability of any element of the domain. Previously, no explicit sublinear–sample algorithms for either of these problems were known. Our algorithm is also computationally extremely efficient, running in time linear in the number of samples used.  \nIn the second half of the paper, we provide a matching lower bound of Ω(n/ log n) samples for estimating entropy or distribution support size to within an additive constant. The previous lower-bounds on these sample complexities weren/2O ( √log n) .  \nTo show our lower bound, we prove two new and natural multivariate central limit theorems (CLTs); the first uses Stein’s method to relate the sum of independent distributions to the multivariate Gaussian of corresponding mean and covariance, under the earthmover distance metric (also known as the Wasserstein metric) . We leverage this central limit theorem to prove a stronger but more specific central limit theorem for “generalized multinomial”distributions—a large class of discrete distributions, parameterized by matri-  \n∗Preliminary full versions, [39] and [40], available at: [http://www.eccc.uni-trier.de/report/2010/179](http://www.eccc.uni-trier.de/report/2010/179) and [http://www.eccc.uni-trier.de/report/2010/180](http://www.eccc.uni-trier.de/report/2010/180)[ ](http://www.eccc.uni-trier.de/report/2010/180)†Supported by an NSF graduate research fellowship.‡Supported by an NSF postdoctoral research fellowship.  \nPermission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, torepublish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee.  \nSTOC’11, June 6–8, 2011, San Jose, California, USA.  \nCopyright 2011 ACM 978-1-4503-0691-1/11/06 ...$10.00 .  \nces, that represents sums of independent binomial or multinomial distributions, and describes many distributions encountered in computer science. Convergence here is in the strong sense of statistical distance, which immediately implies that any algorithm with input drawn from a generalized multinomial distribution behaves essentially as if the input were drawn from a discretized Gaussian with the same mean and covariance. Such tools in the multivariate setting are rare, and we hope this new tool will be of use to the community.  \nCategories and Subject Descriptors: F.2 [Analysis of Algorithms and Problem Complexity]: Miscellaneous General Terms: Algorithms, Theory.  \n1. INTRODUCTION  \nGiven samples from an unknown discrete distribution, what can we infer about the distribution? The empirical distribution of the samples roughly captures the portion of the distribution which we have observed, but what can we say about the unobserved portion of the distribution? Answers to this question are, at least implicitly, central to many estima","cbCaipuTQQC7AK8o","https://ap.wps.com/l/cbCaipuTQQC7AK8o","pdf",277431,"English","# Abstract\n# Introduction\n## Problem background: unobserved distribution mass\n## Main contributions: estimators and lower bounds","[{\"question\":\"What estimation problems does the paper address?\",\"answer\":\"It focuses on estimating the entropy of a discrete distribution and estimating the distribution support size, emphasizing the unobserved portion of the distribution.\"},{\"question\":\"How many samples does the proposed method use?\",\"answer\":\"The algorithm estimates entropy and support size within an arbitrarily small additive constant using O(n/log n) samples, where n bounds the support size.\"},{\"question\":\"How are the matching lower bounds proved?\",\"answer\":\"The lower bound proof relies on new multivariate central limit theorems, using Stein’s method under the Wasserstein (earthmover) distance and applying them to generalized multinomial distributions.\"}]","Estimating the Unseen - An n/log(n)-Sample Estimator for Entropy and Support Size - Shown Optimal via New CLTs | PDF",25]