[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-82338-en":3,"doc-seo-82338-105":29,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},82338,1374391974585,"Genevieve","https://ap-avatar.wpscdn.com/davatar_276721f389ce27ea32af1340a28f341c",8,"Research & Report","Similarity Search Generalisation in Contrastive Learning with InfoNCE Loss","Similarity search is a core use of embedding models learned through contrastive learning. For the widely used InfoNCE loss, the work proves that population risk with k negative samples is O(1/k) close to a cross-entropy measuring the deviation between (i) a softmax similarity search on unseen data using the learned embedding and (ii) an idealised softmax search on the same data where similarity is represented implicitly by the positive sample generator. It adds a new continuity bound via Gˆateaux differentiation and shows k-negative-sample averaging stabilises generalisation error growth.","arXiv :2607 .09405v 1 [ cs .LG] 10 Jul 2026  \nSimilarity search generalisation in contrastive learning with  \nInfoNCE loss  \nNick Whiteley  \nSchool of Mathematics, University of Bristol, U.K.  \nJuly 13, 2026  \nAbstract  \nSimilarity search is a primary application of embedding models trained by contrastive learning.  \nFor one of the most popular contrastive learning loss functions, InfoNCE, we show that the population risk with k negative samples is O(1/k) close to an expected cross-entropy which quantifies deviation between i) a softmax similarity search over unseen data using the learned embedding function, and  \nii) an idealised softmax search over the same data but using similarity implicitly represented in the positive sample generator. This complements existing interpretations of InfoNCE in the k → ∞ limit which are phrased in terms of mutual information, and alignment versus uniformity in embeddings.  \nTo quantify generalisation performance, we introduce a new continuity bound for the InfoNCE loss, obtained via Gˆateaux differentiation. The bound preserves the structure of averaging over negative samples present in the loss function and features an “inverse temperature” parameter which can be tuned to account for the algorithmic temperature. For embedding functions which are Lipschitz in a parameter, this yields a simple demonstration that the averaging effect of k negative samples in the InfoNCE loss carries over to stabilisation of the generalisation error as k grows.  \n1 Introduction  \nThe InfoNCE loss [van den Oord et al. , 2018] is one of the most popular loss functions in contrastive learning and is a fundamental ingredient in hugely impactful systems such as SimCLR [Chen et al., 2020], MoCo [He et al., 2020] and CLIP [Radford et al., 2021] . Similarity search is a primary application of these technologies and other embedding models trained using contrastive learning; the learned embedding is used to calculate cosine similarities and hence evaluate closeness among unseen data. The goal of the present work is to add clarity to our theoretical understanding of InfoNCE, similarity search and generalisation.  \nTo date, theoretical generalisation analysis of contrastive learning has largely focused on downstream classification. In that context, “generalisation” has the conventional meaning of a model’s ability to make accurate predictions on unseen data: a pioneering step forward was made by Saunshi et al. [2019], who showed that the InfoNCE population risk can contribute to bounding the population risk of a downstream linear classifier, thus quantifying classification accuracy when an unseen input data point, such as an image or text document, is represented by its embedding vector. Subsequent refinements and extensions, discussed in more detail later, have been made by [Lei et al. , 2023 , Ghanooni et al. , 2024 , Hieu and Ledent, 2025] .  \nIn the present work we also analyse InfoNCE risk but from a different perspective, which we call similarity search generalisation: the performance of similarity search on unseen data using the learned embedding, compared to an idealised search which we show is implicitly defined by the ingredients of contrastive learning. We draw inspiration not only from learning-theoretic analyses of e.g., [Saunshiet al. , 2019 , Lei et al. , 2023 , Ghanooni et al. , 2024 , Hieu and Ledent, 2025], but also from widely referenced interpretations of contrastive learning in terms of mutual information [van den Oord et al. , 2018],“alignment” versus “uniformity” in embeddings [Wang and Isola, 2020a], and cross-entropy between conditional probability densities [Zimmermann et al. , 2021] .  \n1.1 Interpretations of InfoNCE  \nEach training tuple or mini-batch in contrastive learning comprises “anchor”, “positive” and “negative”samples 1 . Pairs of anchor and positive samples are generated in order to convey some notion of semantic similarity which is specific to input data modality, such as text, images,","cbCaigDnV1oTrade","https://ap.wps.com/l/cbCaigDnV1oTrade","pdf",502775,1,25,"English","en",105,"# Abstract\n# Introduction\n## Interpretations of InfoNCE\n## Generalisation analysis of contrastive learning","[{\"question\":\"What does the document study about InfoNCE loss and similarity search?\",\"answer\":\"It studies how InfoNCE population risk relates to similarity search on unseen data, comparing a learned softmax search with an idealised softmax search defined implicitly by the positive sample generator.\"},{\"question\":\"How does the analysis depend on the number of negative samples k?\",\"answer\":\"With k negative samples, the population risk is shown to be O(1/k) close to an appropriate expected cross-entropy, and the averaging over negatives is shown to stabilise generalisation error as k grows.\"},{\"question\":\"What new contribution is made to quantify generalisation performance?\",\"answer\":\"The document introduces a new continuity bound for the InfoNCE loss, derived using Gˆateaux differentiation, preserving the negative-sample averaging structure and including an “inverse temperature” parameter to reflect algorithmic temperature.\"}]",1784179738,63,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":27},"similarity-search-generalisation-in-contrastive-learning-with-infonce-loss","",{"@graph":35,"@context":85},[36,53,68],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/similarity-search-generalisation-in-contrastive-learning-with-infonce-loss/82338/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-17","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What does the document study about InfoNCE loss and similarity search?","Question",{"text":75,"@type":76},"It studies how InfoNCE population risk relates to similarity search on unseen data, comparing a learned softmax search with an idealised softmax search defined implicitly by the positive sample generator.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"How does the analysis depend on the number of negative samples k?",{"text":80,"@type":76},"With k negative samples, the population risk is shown to be O(1/k) close to an appropriate expected cross-entropy, and the averaging over negatives is shown to stabilise generalisation error as k grows.",{"name":82,"@type":73,"acceptedAnswer":83},"What new contribution is made to quantify generalisation performance?",{"text":84,"@type":76},"The document introduces a new continuity bound for the InfoNCE loss, derived using Gˆateaux differentiation, preserving the negative-sample averaging structure and including an “inverse temperature” parameter to reflect algorithmic temperature.","https://schema.org",{"og:url":51,"og:type":87,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":89,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":45,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":45,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":45,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":45,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":45,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":45,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":45,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]