[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-120765-en":3,"doc-seo-120765-105":30,"detail-sidebar-cat-0-en-105":83},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},120765,34359740700684,"Finn","https://ap-avatar.wpscdn.com/avatar/1f400023980c374ae676?_k=1777273430885731487",8,"Research & Report","Multimodal machine learning for materials science - composition-structure bimodal learning for experimentally measured properties","Multimodal machine learning can combine information from multiple data modalities, yet materials informatics still lacks broad adoption compared with vision and NLP. This work proposes COmposition-Structure Bimodal Network (COSNet) to predict experimentally measured materials properties when structural information is incomplete, using composition–structure bimodal learning. The model reduces prediction errors across Li conductivity, band gap, refractive index, dielectric constant, energy, and magnetic moment, outperforming composition-only learning. Results highlight modal-availability-based data augmentation as pivotal for success.","Multimodal machine learning for materials science: composition-structure bimodal learning for experimentally measured properties  \nSheng Gong 1 *, Shuo Wang 1 *, Taishan Zhu1, Yang Shao-Horn1,2, and Jeffrey C. Grossman 1  \n1Department of Materials Science and Engineering, Massachusetts Institute of Technology, Cambridge, MA 02139, USA  \n2Department of Mechanical Engineering, Massachusetts Institute of Technology, Cambridge, MA 02139, USA * These authors contribute equally  \nAbstract  \nThe widespread application of multimodal machine learning models like GPT-4 has revolutionized various research fields including computer vision and natural language processing. However, its implementation in materials informatics remains underexplored, despite the presence of materials data across diverse modalities, such as composition and structure. The effectiveness of machine learning models trained on large calculated datasets depends on the accuracy of calculations, while experimental datasets often have limited data availability and incomplete information. This paper introduces a novel approach to multimodal machine learning in materials science via compositionstructure bimodal learning. The proposed COmposition-Structure Bimodal Network (COSNet) is designed to enhance learning and predictions of experimentally measured materials properties that have incomplete structure information. Bimodal learning significantly reduces prediction errors across distinct materials properties including Li conductivity in solid electrolyte, band gap, refractive index, dielectric constant, energy, and magnetic moment, surpassing composition-only learning methods. Furthermore, we identified that data augmentation based on modal availability plays a pivotal role in the success ofbimodal learning.  \nIntroduction  \nRecently, multimodal machine learning models, such as GPT-4(1), have profoundly transformed the influence of artificial intelligence on society(2) . By definition, multimodal machine learning obtains information from different modalities, such as text, image, and audio(3), and fuses all the information for downstream tasks. Multimodal machine learning has attracted increasing attention in many research communities such as computer vision and natural language processing, because in many cases multimodal machine learning outperforms the single modal learning(4) . There also exist many modalities in materials science, such as composition, structure, spectrum, image, and text(5), which in principle can be simultaneously incorporated into multimodal machine learning models that potentially outperform machine learning models trained on single modality. Despite the potential, however, there is still a lack of widespread application of multimodal machine learning in the field of materials science. Therefore, it is important to demonstrate the effectiveness of multimodal machine learning in materials science.  \nOne of the ultimate goals of materials informatics is to predict materials properties that are close to experimental measurements (6). Although most of current machine learning models applied to materials are trained on theoretical datasets (7-16) due to the high availability, the inherit gap between theoretical calculations and experiments indicates that the usefulness of such machine learning models largely depends on the accuracy of the theoretical calculations (15, 17-19) . To bypass the dependency of accuracy of training data, recently experimentally measured properties, the “ground truth”, have been used as the training set for machine learning models(12, 20, 21) .  \nHowever, learning experimental data faces many challenges, such as limited number of data points, conflicted results in different literatures, and incomplete information recorded in literatures. While data cleaning has been used to assess the issue of conflicts(22), and machine learning techniques such as transfer learning(12, 21, 23, 24) and multi-fidelity learning(20) have been dev","cbCaiaMrWp58eefT","https://ap.wps.com/l/cbCaiaMrWp58eefT","pdf",523569,1,19,"English","en",105,"# Abstract\n# Introduction\n## Multimodal learning background\n## Materials informatics and the theory–experiment gap\n## Challenges of experimental data and incomplete information\n# Proposed approach: composition–structure bimodal learning\n## COSNet framework\n## Performance on multiple experimentally measured properties\n## Modal availability-based data augmentation","[{\"question\":\"Why is modal availability-based data augmentation important?\",\"answer\":\"Modal availability-based data augmentation ensures the composition network is effectively trained using all available data, making the bimodal learning benefit achievable.\"}]","Multimodal machine learning for materials science - composition-structure bimodal learning for experimentally measured properties | PDF",1785731918,48,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":78,"head_meta":80,"extra_data":82,"updated_unix":28},"multimodal-machine-learning-for-materials-science-composition-structure-bimodal-learning-for-experimentally-measured-properties","",{"@graph":36,"@context":77},[37,54,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/multimodal-machine-learning-for-materials-science-composition-structure-bimodal-learning-for-experimentally-measured-properties/120765/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":62,"encodingFormat":61,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-03",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71],{"name":72,"@type":73,"acceptedAnswer":74},"Why is modal availability-based data augmentation important?","Question",{"text":75,"@type":76},"Modal availability-based data augmentation ensures the composition network is effectively trained using all available data, making the bimodal learning benefit achievable.","Answer","https://schema.org",{"og:url":52,"og:type":79,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":81,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":84},[85,89,93,97,102,107,112,115,120,123,127],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":86,"show_sort_weight":87,"slug":88},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":90,"show_sort_weight":91,"slug":92},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Exam",70,"exam",{"id":98,"doc_module":4,"doc_module_name":46,"category_name":99,"show_sort_weight":100,"slug":101},5,"Comic",60,"comic",{"id":103,"doc_module":4,"doc_module_name":46,"category_name":104,"show_sort_weight":105,"slug":106},6,"Technology",50,"technology",{"id":108,"doc_module":4,"doc_module_name":46,"category_name":109,"show_sort_weight":110,"slug":111},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":113,"slug":114},30,"research-report",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},9,"Religion & Spirituality",20,"religion-spirituality",{"id":118,"doc_module":4,"doc_module_name":46,"category_name":121,"show_sort_weight":118,"slug":122},"World Cup","world-cup",{"id":124,"doc_module":4,"doc_module_name":46,"category_name":125,"show_sort_weight":124,"slug":126},10,"Lifestyle","lifestyle",{"id":21,"doc_module":4,"doc_module_name":46,"category_name":128,"show_sort_weight":98,"slug":129},"General","general"]