[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-127007-en":3,"doc-seo-127007-105":29,"detail-sidebar-cat-0-en-105":89},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":11},127007,2336474466412,"Ezra","https://ap-avatar.wpscdn.com/davatar_155a257f0dc6eb9ab79c44ca47cae57d",8,"Research & Report","The Multimodal Universe - 100 TB of Machine Learning Ready Astronomical Data","The Multimodal Universe presents a unified framework aggregating over 100 TB of multimodal astronomical data for its first release, spanning images, spectra, time series, tabular, and hyperspectral modalities. It consolidates observations from multiple surveys, facilities, and wavelength regimes through standardized formats and access patterns, while documenting selection effects and biases. Designed to simplify cross-dataset machine learning workflows for observational astronomy, the framework is actively supported and intended to be extended with minimal, self-consistent conventions.","Draft version December 6, 2024  \nTypeset using LATEX default style in AASTeX631  \nThe Multimodal Universe:  \n100 TB of Machine Learning Ready Astronomical Data  \nThe Multimodal Universe Collaboration  \nEirini Angeloudi,1, 2 Jeroen Audenaert,3 Micah Bowles,4, 5 Benjamin M. Boyd,6 David Chemaly,6 Brian Cherinka,7 Ioana Ciucă,8, 9, 10 Miles Cranmer,6, 5 Aaron Do,6 Matthew Grayling,6 Erin E. Hayes,6 Tom Hehir,6, 5 Shirley Ho,11, 12, 13, 5 Marc Huertas-Company,1, 2, 9 Kartheik G. Iyer,14, 11, 9 Maja Jablonska,10, 9 Francois Lanusse,11, 5, 15 Henry W. Leung,16 Kaisey Mandel,6 Juan Rafael Martínez-Galarza, 17, 18 Peter Melchior,13 Lucas Meyer,11, 5 Liam H. Parker,11, 5, 19 Helen Qu,20 Jeff Shen,13 Michael J. Smith,21, 9  \nMike Walmsley,16 John F. Wu,7, 22  \n1 Instituto de Astrofisica de Canarias  \n2 Universidad de La Laguna  \n3 Massachusetts Institute of Technology  \n4 University of Oxford  \n5 Polymathic AI  \n6 University of Cambridge  \n7 Space Telescope Science Institute  \n8 Stanford University  \n9 Universe TBD  \n10 Australian National University  \n11 Flatiron Institute  \n12 New York University  \n13 Princeton University  \n14 Columbia University  \n15 Université Paris-Saclay, Université Paris Cité, CEA, CNRS, AIM  \n16 University of Toronto  \n17 Center for Astrophysics, Harvard & Smithsonian  \n18 AstroAI  \n19 University of California, Berkeley  \n20 University of Pennsylvania  \n21 Aspia Space  \n22 Johns Hopkins University  \nABSTRACT  \nWe present the Multimodal Universe, a new framework collating over 100 TB of multimodal astronomical data for its first release, spanning images, spectra, time series, tabular and hyper-spectral data. This unified collection enables a wide variety of machine learning applications and research across astronomical domains. The dataset brings together observations from multiple surveys, facilities, and wavelength regimes, providing standardized access to diverse data types. By providing uniform access to this diverse data, the Multimodal Universe aims to accelerate the development of machine learning methods for observational astronomy that can work across the large differences in astronomical datasets. The framework is actively supported and is designed to be extended whilst enforcing minimal self consistent conventions making contributing data as simple and practical as possible.  \nKeywords: Astronomical databases: miscellaneous—Methods: data analysis—Methods: statistical  \nThe increasing volume and complexity of astronomical data has driven significant adoption of machine learning (ML) methods in astronomy. However, most applications are tailored to specific datasets, surveys, or instruments, requiring significant domain expertise and custom implementations. We introduce the Multimodal Universe, a curated collection  \n2  \nof multimodal data designed to accelerate research in fields related to astronomy, astrophysics and machine learning. A full report on the dataset has been published by NeurIPS 2024 as The Multimodal Universe Collaboration et al.(2024). The most up to date version is documented at [https://github.com/MultimodalUniverse/MultimodalUniverse/](https://github.com/MultimodalUniverse/MultimodalUniverse/) .  \nThe Multimodal Universe combines data from major astronomical surveys, summarized by Table 1. The dataset is designed with several key principles, all of which are designed to make using the collated data in a machine learning context simpler than ever before.  \n• Multimodal alignment through careful cross-matching between surveys,  \n• Standardized data formats and access patterns,  \n• Comprehensive documentation of selection effects and biases,  \n• Public availability of all data download, collection and processing scripts.  \nTable 1 . Adapted from Table 1 of The Multimodal Universe Collaboration et al. (2024), this is a summary of data included in the Multimodal Universe. Details of all samples, including full citations, are provided in the full paper. Nc indicates the number of channels of a given","cbCaiah11NBZQ3Pv","https://ap.wps.com/l/cbCaiah11NBZQ3Pv","pdf",61133,1,3,"English","en",105,"# Abstract\n# Dataset Overview and Goals\n## Core Principles for ML Usability\n# Data Modalities and Included Surveys\n## Images\n## Spectra\n## Hyperspectral Image\n## Time Series\n## Tabular\n# Benchmarks and Dataset Utility","[{\"question\":\"What data modalities are included in The Multimodal Universe?\",\"answer\":\"The release includes images, spectra, time series, tabular data, and hyperspectral data.\"},{\"question\":\"How does the framework make datasets easier to use for machine learning?\",\"answer\":\"It emphasizes multimodal alignment via cross-matching, standardized data formats and access patterns, comprehensive documentation of selection effects and biases, and public availability of download, collection, and processing scripts.\"},{\"question\":\"Is the dataset restricted to a single survey or instrument?\",\"answer\":\"No. It combines data from multiple major astronomical surveys and facilities across different wavelength regimes.\"}]","The Multimodal Universe - 100 TB of Machine Learning Ready Astronomical Data | PDF",1785936243,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":84,"head_meta":86,"extra_data":88,"updated_unix":28},"the-multimodal-universe-100-tb-of-machine-learning-ready-astronomical-data","",{"@graph":35,"@context":83},[36,52,66],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,49],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":21},"https://docshare.wps.com/document/research-report/",{"item":50,"name":13,"@type":42,"position":51},"https://docshare.wps.com/document/the-multimodal-universe-100-tb-of-machine-learning-ready-astronomical-data/127007/",4,{"url":50,"name":13,"@type":53,"author":54,"headline":13,"publisher":56,"fileFormat":59,"inLanguage":23,"description":14,"dateModified":60,"datePublished":60,"encodingFormat":59,"isAccessibleForFree":61,"interactionStatistic":62},"DigitalDocument",{"name":9,"@type":55},"Person",{"url":40,"name":57,"@type":58},"DocShare","Organization","application/pdf","2026-08-05",true,{"@type":63,"interactionType":64,"userInteractionCount":4},"InteractionCounter",{"@type":65},"ViewAction",{"@type":67,"mainEntity":68},"FAQPage",[69,75,79],{"name":70,"@type":71,"acceptedAnswer":72},"What data modalities are included in The Multimodal Universe?","Question",{"text":73,"@type":74},"The release includes images, spectra, time series, tabular data, and hyperspectral data.","Answer",{"name":76,"@type":71,"acceptedAnswer":77},"How does the framework make datasets easier to use for machine learning?",{"text":78,"@type":74},"It emphasizes multimodal alignment via cross-matching, standardized data formats and access patterns, comprehensive documentation of selection effects and biases, and public availability of download, collection, and processing scripts.",{"name":80,"@type":71,"acceptedAnswer":81},"Is the dataset restricted to a single survey or instrument?",{"text":82,"@type":74},"No. It combines data from multiple major astronomical surveys and facilities across different wavelength regimes.","https://schema.org",{"og:url":50,"og:type":85,"og:title":13,"og:site_name":57,"og:description":14},"article",{"robots":87,"canonical":50},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":90},[91,95,99,103,108,113,118,121,126,129,133],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":92,"show_sort_weight":93,"slug":94},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":96,"show_sort_weight":97,"slug":98},"Literature",80,"literature",{"id":51,"doc_module":4,"doc_module_name":45,"category_name":100,"show_sort_weight":101,"slug":102},"Exam",70,"exam",{"id":104,"doc_module":4,"doc_module_name":45,"category_name":105,"show_sort_weight":106,"slug":107},5,"Comic",60,"comic",{"id":109,"doc_module":4,"doc_module_name":45,"category_name":110,"show_sort_weight":111,"slug":112},6,"Technology",50,"technology",{"id":114,"doc_module":4,"doc_module_name":45,"category_name":115,"show_sort_weight":116,"slug":117},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":119,"slug":120},30,"research-report",{"id":122,"doc_module":4,"doc_module_name":45,"category_name":123,"show_sort_weight":124,"slug":125},9,"Religion & Spirituality",20,"religion-spirituality",{"id":124,"doc_module":4,"doc_module_name":45,"category_name":127,"show_sort_weight":124,"slug":128},"World Cup","world-cup",{"id":130,"doc_module":4,"doc_module_name":45,"category_name":131,"show_sort_weight":130,"slug":132},10,"Lifestyle","lifestyle",{"id":134,"doc_module":4,"doc_module_name":45,"category_name":135,"show_sort_weight":104,"slug":136},19,"General","general"]