[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-126295-en":3,"doc-seo-126295-105":31,"detail-sidebar-cat-0-en-105":92},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":21,"is_downloadable":21,"audit_status":21,"page_count":22,"language":23,"language_code":24,"site_id":25,"html_lang":24,"table_of_contents":26,"faqs":27,"seo_title":28,"seo_description":14,"update_tm":29,"read_time":30},126295,2336475104736,"Quinn","https://ap-avatar.wpscdn.com/avatar/22000c4c5e0e5b17e70?x-image-process=image/resize,m_fixed,w_180,h_180&k=1786591360781797222",8,"Research & Report","On Stronger Computational Separations Between Multimodal and Unimodal Machine Learning - Research","Recently, multimodal machine learning has achieved remarkable empirical success, motivating a search for theoretical separation from unimodal learning. This work builds a stronger average-case computational separation: for typical learning-task instances, unimodal learning is computationally hard while multimodal learning is efficient. It further tests the “naturalness” of such separations by proving that, under basic conditions, any average-case computational separation implies a cryptographic key agreement protocol, suggesting strong multimodal advantages may arise mainly from pathological cryptographic distributions.","On Stronger Computational Separations Between Multimodal and Unimodal Machine Learning  \nAri Karchmer 1  \narXiv :2404 .02254v2 [ stat .ML] 17 Jul 2024  \nAbstract  \nRecently, multimodal machine learning has enjoyed huge empirical success (e.g. GPT-4) . Motivated to develop theoretical justi􀀂cation for this empirical success, Lu (NeurIPS ’23, ALT ’24) introduces a theory of multimodal learning, and considers possible separations between theoretical models of multimodal and unimodal learning. In particular, Lu (ALT ’24) shows a computational separation, which is relevant to worstcase instances of the learning task. In this paper, we give a stronger average-case computational separation, where for “typical” instances of the learning task, unimodal learning is computationally hard, but multimodal learning is easy.  \nWe then question how “natural” the average-case separation is. Would it be encountered in practice? To this end, we prove that under basic conditions, any given computational separation between average-case unimodal and multimodal learning tasks implies a corresponding cryptographic key agreement protocol. We suggest to interpret this as evidence that very strong computational advantages of multimodal learning mayarise infrequently in practice, since they exist only for the “pathological” case of inherently cryptographic distributions. However, this does not apply to possible (super-polynomial) statistical advantages.  \n1. Introduction  \nFor humans, multimodal perception—the ability to interpret the same or similar information expressed in multiple ways (e.g. text and image)—is absolutely critical to learning. We hold it as self-evident that access to multiple representations of the same idea can ease the process of forming  \n1Department of Computer Science, Boston University, Boston, MA, USA. Correspondence to: Ari Karchmer \u003C[arika@bu.edu](arika@bu.edu)>.  \nProceedings of the 41 st International Conference on Machine Learning, Vienna, Austria. PMLR 235, 2024 . Copyright 2024 by the author(s) .  \na mental model applicable to new situations (“when you put it that way...”) .  \nEmpirical triumphs of Machine Learning from multimodal data such as GPT-4 (Achiam et al., 2023), Gemini (Team et al., 2023), and Gato (Reed et al., 2022) suggest that multimodal perception is also really useful for some machine learning tasks. Lu (2023b;a) introduces a formal study of multimodal versus unimodal machine learning tasks, in order to develop theoretical justi􀀂cation for the empirical results (see also Huang et al. (2021) and others; we elaborate on related work in Section 1.2) . However, the theory of multimodal learning is still in its infancy. The main theoretical question is:  \nIs multimodal data truly (provably) more useful than unimodal data, or is it a mirage?  \nTo attack this question, Lu (2023b) 􀀂rst shows a statistical separation: that there exist machine learning tasks that do require asymptotically more samples to complete when the data is expressed unimodally as opposed to multimodally. Second, Lu (2023a) shows that not only is there a statistical separation, but there also exist machine learning tasks that might be computationally easier when given access to bimodal data (two modes), rather than just unimodal data. The computational separation of Lu (2023a) identi-􀀂es a machine learning task that is possible in polynomial time with bimodal data, but not with unimodal data, for it’sworst-case instance. This means that the unimodal learning task could still possibly be easy on most or “typical”instances. Of course, Lu’s separation requires a relatively weak assumption of computational hardness: that a certain NP-hard problem is not also in P.  \nIn this work, we continue to develop a theory of multimodal learning, in pursuit of the truth about how useful multimodal data is (when compared to unimodal data) . In particular, we study the existence of stronger computational separations, which apply to the average-case instances of","cbCaiuSjqQA8ecLJ","https://ap.wps.com/l/cbCaiuSjqQA8ecLJ","pdf",262402,2,1,14,"English","en",105,"# Introduction\n## Motivation: multimodal vs. unimodal usefulness\n## Average-case computational separation\n# LPN assumption and main result\n## Low-noise LPN and bimodal/unimodal tasks","[{\"question\":\"What computational separation does the paper establish between multimodal and unimodal learning?\",\"answer\":\"It gives a stronger average-case separation where typical unimodal instances are computationally hard, while typical multimodal instances can be learned in polynomial time.\"},{\"question\":\"Why does the paper question whether the average-case separation is “natural”?\",\"answer\":\"It asks whether such separations would appear in real practice or only under specially constructed, non-representative conditions.\"},{\"question\":\"What connection does the paper prove between computational separation and cryptography?\",\"answer\":\"Under basic conditions, any computational separation between average-case unimodal and multimodal learning tasks implies a cryptographic key agreement protocol.\"}]","On Stronger Computational Separations Between Multimodal and Unimodal Machine Learning - Research | PDF",1785904304,35,{"code":4,"msg":32,"data":33},"ok",{"site_id":25,"language":24,"slug":34,"title":13,"keywords":35,"description":14,"schema_data":36,"social_meta":87,"head_meta":89,"extra_data":91,"updated_unix":29},"on-stronger-computational-separations-between-multimodal-and-unimodal-machine-learning-research","",{"@graph":37,"@context":86},[38,54,69],{"@type":39,"itemListElement":40},"BreadcrumbList",[41,45,48,51],{"item":42,"name":43,"@type":44,"position":21},"https://docshare.wps.com","Home","ListItem",{"item":46,"name":47,"@type":44,"position":20},"https://docshare.wps.com/document/","Document",{"item":49,"name":12,"@type":44,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":44,"position":53},"https://docshare.wps.com/document/on-stronger-computational-separations-between-multimodal-and-unimodal-machine-learning-research/126295/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":24,"description":14,"dateModified":62,"datePublished":63,"encodingFormat":61,"isAccessibleForFree":64,"interactionStatistic":65},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":42,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-15","2026-08-05",true,{"@type":66,"interactionType":67,"userInteractionCount":20},"InteractionCounter",{"@type":68},"ViewAction",{"@type":70,"mainEntity":71},"FAQPage",[72,78,82],{"name":73,"@type":74,"acceptedAnswer":75},"What computational separation does the paper establish between multimodal and unimodal learning?","Question",{"text":76,"@type":77},"It gives a stronger average-case separation where typical unimodal instances are computationally hard, while typical multimodal instances can be learned in polynomial time.","Answer",{"name":79,"@type":74,"acceptedAnswer":80},"Why does the paper question whether the average-case separation is “natural”?",{"text":81,"@type":77},"It asks whether such separations would appear in real practice or only under specially constructed, non-representative conditions.",{"name":83,"@type":74,"acceptedAnswer":84},"What connection does the paper prove between computational separation and cryptography?",{"text":85,"@type":77},"Under basic conditions, any computational separation between average-case unimodal and multimodal learning tasks implies a cryptographic key agreement protocol.","https://schema.org",{"og:url":52,"og:type":88,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":90,"canonical":52},"index,follow",{"doc_id":7,"site_id":25},{"code":4,"msg":5,"data":93},[94,98,102,106,111,116,121,124,129,132,136],{"id":21,"doc_module":4,"doc_module_name":47,"category_name":95,"show_sort_weight":96,"slug":97},"Story & Novel",90,"story-novel",{"id":20,"doc_module":4,"doc_module_name":47,"category_name":99,"show_sort_weight":100,"slug":101},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":47,"category_name":103,"show_sort_weight":104,"slug":105},"Exam",70,"exam",{"id":107,"doc_module":4,"doc_module_name":47,"category_name":108,"show_sort_weight":109,"slug":110},5,"Comic",60,"comic",{"id":112,"doc_module":4,"doc_module_name":47,"category_name":113,"show_sort_weight":114,"slug":115},6,"Technology",50,"technology",{"id":117,"doc_module":4,"doc_module_name":47,"category_name":118,"show_sort_weight":119,"slug":120},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":47,"category_name":12,"show_sort_weight":122,"slug":123},30,"research-report",{"id":125,"doc_module":4,"doc_module_name":47,"category_name":126,"show_sort_weight":127,"slug":128},9,"Religion & Spirituality",20,"religion-spirituality",{"id":127,"doc_module":4,"doc_module_name":47,"category_name":130,"show_sort_weight":127,"slug":131},"World Cup","world-cup",{"id":133,"doc_module":4,"doc_module_name":47,"category_name":134,"show_sort_weight":133,"slug":135},10,"Lifestyle","lifestyle",{"id":137,"doc_module":4,"doc_module_name":47,"category_name":138,"show_sort_weight":107,"slug":139},19,"General","general"]