[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-116854-en":3,"doc-seo-116854-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},116854,1099513958762,"Logic","https://ap-avatar.wpscdn.com/avatar/1000023916a998db790?x-image-process=image/resize,m_fixed,w_180,h_180&k=1784791008015729253",8,"Research & Report","Machine Learning for Observational Cosmology","An array of large observational programs using ground-based and spaceborne telescopes will generate unprecedented astronomical data volumes in the coming decade. Wide-field sky surveys are expected to produce datasets exceeding an exabyte, making multiplex data processing technically difficult. Fully automated machine-learning and AI-based methods are urgently needed to maximize scientific returns. The review summarizes recent advances in applying ML to observational cosmology and highlights essential high-performance computing issues for data processing and statistical analysis.","arXiv :2303 . 15794v1 [ astro-ph .IM] 28 Mar 2023  \nMachine Learning for Observational Cosmology  \nKana Moriwaki 1 , Takahiro Nishimichi2 ;3 , Naoki Yoshida3 ;4  \n1 Research Center for the Early Universe, The University of Tokyo, 7-3-1 Hongo, Bunkyo, Tokyo 113-0033, Japan  \n2 Center for Gravitational Physics and Quantum Information, Yukawa Institute for Theoretical Physics, Kyoto University, Kitashirakawa Oiwakecho, Sakyo-ku, Kyoto 606-8502 Japan  \n3 Kavli Institute for the Physics and Mathematics of the Universe (WPI), The University of Tokyo Institutes for Advanced Study (UTIAS), The University of Tokyo, Kashiwa, Chiba 277-8583, Japan  \n4 Department of Physics, The University of Tokyo, 7-3-1 Hongo, Bunkyo, Tokyo 113-0033, Japan  \nE-mail: [kana.moriwaki@phys.s.u-tokyo.ac.jp](kana.moriwaki@phys.s.u-tokyo.ac.jp) ,  \n[takahiro.nishimichi@yukawa.kyoto-u.ac.jp](takahiro.nishimichi@yukawa.kyoto-u.ac.jp) ,[naoki.yoshida@ipmu.jp](naoki.yoshida@ipmu.jp)  \nMarch 2023  \nAbstract. An array of large observational programs using ground-based and spaceborne telescopes is planned in the next decade. The forthcoming wide-􀀌eld sky surveys are expected to deliver a sheer volume of data exceeding an exabyte. Processing the large amount of multiplex astronomical data is technically challenging, and fully automated technologies based on machine learning and arti􀀌cial intelligence are urgently needed. Maximizing scienti􀀌c returns from the big data requires communitywide e􀀋orts. We summarize recent progress in machine learning applications in observational cosmology. We also address crucial issues in high-performance computing that are needed for the data processing and statistical analysis.  \n1. Cosmology in the big data era  \nThe last decade witnessed an extremely rapid increase of observational data in astronomy. Sky survey is a commonly adopted mode of observation that runs a telescope to scan over a wide area of the sky, instead of pointing to speci􀀌c celestial objects. Modern imaging devices such as Charge Coupled Devices (CCDs) and Complementary Metal Oxide Semiconductor (CMOS) sensors can generate a large amount of data ina short time. For instance, Subaru Hyper-Suprime Cam (HSC) has 104 CCDs on its focal plane, and a single snapshot generates a billion-pixel image [1] . Typically, onenight observation by HSC generates a few hundred gigabytes of data. The Vera C. Rubin observatory LSST Camera has a greater capability of generating a 3.2 billion-pixel image per one snapshot. As a designated survey telescope, it operates continuously for many years, and is expected to deliver over 500 petabyte of imaging data per year [2] . There  \nMachine Learning for Observational Cosmology 2  \nare a variety of exciting scienti􀀌c returns from such wide-􀀌eld, multi-epoch surveys. Discovering distant supernovae and new types of transient objects, mapping the universe with nearby and distant galaxies, and probing the nature of mysterious dark matter and dark energy, are noted key scienti􀀌c cases among many others. All these science goals can be achieved through a sequence of fairly complex data analysis processes. E􀀎cient data processing is a central issue, but remains technically challenging.  \nSimilar situations can also be found in other research domains, from life science to engineering, where new experiments and sensor technologies boost production and acquisition of data of impressive quality and quantity. Naturally, machine learning (ML) applications have become increasingly popular in virtually all these research domains. In astronomy, massive amount of data have already been obtained by ongoing surveys such as Dark Energy Survey (DES) [3], Kilo-Degree Survey (KiDS) [4], and Subaru HSC Survey [5] . It is taking over years to produce major science results after the completion or occasional data release of each of these observations. The situation may get even harder with upcoming surveys. Just as an example, the estimated data production rate by Square K","cbCaigGqk8DNitpt","https://ap.wps.com/l/cbCaigGqk8DNitpt","pdf",5953373,1,55,"English","en",105,"# Cosmology in the big data era\n## Detection and classification of transients","[{\"question\":\"Why is machine learning urgently needed in observational cosmology?\",\"answer\":\"Upcoming wide-field sky surveys will produce data volumes exceeding an exabyte, making fully automated processing necessary. Machine learning and AI methods are required to handle multiplex astronomical data efficiently.\"},{\"question\":\"What are the key observational programs and surveys mentioned?\",\"answer\":\"The text cites existing and ongoing surveys such as Dark Energy Survey (DES), Kilo-Degree Survey (KiDS), and Subaru HSC, as well as upcoming high-rate instruments like the Square Kilometre Array (SKA).\"},{\"question\":\"Which scientific topics does the review plan to cover?\",\"answer\":\"The review focuses on time domain astronomy, cosmology with galaxy surveys, and emulation technologies, with examples applied to real observational data.\"}]","Machine Learning for Observational Cosmology | PDF",1785672092,139,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"machine-learning-for-observational-cosmology","",{"@graph":36,"@context":85},[37,54,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/machine-learning-for-observational-cosmology/116854/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":62,"encodingFormat":61,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-02",true,{"@type":65,"interactionType":66,"userInteractionCount":4},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"Why is machine learning urgently needed in observational cosmology?","Question",{"text":75,"@type":76},"Upcoming wide-field sky surveys will produce data volumes exceeding an exabyte, making fully automated processing necessary. Machine learning and AI methods are required to handle multiplex astronomical data efficiently.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"What are the key observational programs and surveys mentioned?",{"text":80,"@type":76},"The text cites existing and ongoing surveys such as Dark Energy Survey (DES), Kilo-Degree Survey (KiDS), and Subaru HSC, as well as upcoming high-rate instruments like the Square Kilometre Array (SKA).",{"name":82,"@type":73,"acceptedAnswer":83},"Which scientific topics does the review plan to cover?",{"text":84,"@type":76},"The review focuses on time domain astronomy, cosmology with galaxy surveys, and emulation technologies, with examples applied to real observational data.","https://schema.org",{"og:url":52,"og:type":87,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":89,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":46,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":46,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":46,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":46,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]