[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-84047-en":3,"doc-seo-84047-105":29,"detail-sidebar-cat-0-en-105":83},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},84047,1374391975076,"Riley","https://ap-avatar.wpscdn.com/avatar/14000253ca4ec9f6853?x-image-process=image/resize,m_fixed,w_180,h_180&k=1783305029341752051",8,"Research & Report","Music I Care About Automated Multimodal Benchmarking of LLM Music Perception Skills","Music represents a cornerstone of human culture, existing digitally across audio, symbolic encodings such as MIDI/MusicXML, and sheet music images. Existing multimodal music benchmarks limit transferability, often test “music understanding” without true perception, and rarely enable systematic cross-modality comparisons. MusICA-MetaBench introduces an on-demand meta-benchmark that generates music-perception question–answer pairs from user-provided data using symbolic representations and templates aligned with music pedagogy. Experiments with ChoraleBricks determine dataset sizes for reliable comparisons and confirm measurement of perception via text-only and noise baselines.","Music I Care About: Automated Multimodal Benchmarking of LLM Music  \nPerception Skills on (Almost) Any Music  \nTomáš Sourada Katia Vendrame Jan Hajijr.  \nCharles University Brno University of Technology Charles University  \n[sourada@ufal.mff.cuni.cz](sourada@ufal.mff.cuni.cz) [ivendrame@fit.vut.cz](ivendrame@fit.vut.cz) [hajicj@ufal.mff.cuni.cz](hajicj@ufal.mff.cuni.cz)  \narXiv :2607 .060 15v 1 [ cs . SD] 7 Jul 2026  \nABSTRACT  \nMusic represents a cornerstone of human culture, existing digitally across diverse modalities, including audio, symbolic encodings (e.g., MIDI, MusicXML), and sheet music. Despite the advancement of Multimodal Large Language Models (MLLMs), current music benchmarks face three major limitations. First, large static benchmarks are resource-intensive to evaluate, and it remains unclear how their results transfer to diverse kinds of music beyond those included in the benchmark. Second, benchmarks claiming to measure “music understanding” often fail to require music perception. Third, they do not support systematic performance comparisons across musical modalities. To overcome these issues, we introduce the Music I Care About Meta-Benchmark (MusICA-MetaBench), a framework that automatically derives on-demand benchmarks directly from user-provided data. By leveraging structured symbolic representations (e.g., MusicXML) and our pre-defined question templates, we build multiple-choice question-answer pairs that probe music perception competencies, aligned with music pedagogy, across audio, music notation images, and symbolic files. We demonstrate our framework with the ChoraleBricks dataset, and experimentally determine benchmark sizes that ensure statistically reliable model comparisons for this setup. By comparing against text-only and white-noise baselines, we show our questions do measure music perception. Ultimately, MusICA-MetaBench represents a significant advancement in the cross-modal assessment of music perception for MLLMs. By proposing a dataset-specific benchmarking paradigm, it enables efficient on-demand evaluation of music perception capabilities.  \n1. INTRODUCTION  \nMusic exists digitally across three modalities: audio recordings, symbolic encodings (e.g., MIDI, MusicXML, ABC notation), and sheet music images. Multimodal Large Language Models (MLLMs) have demonstrated strong capabilities in audio and visual understanding [1–3],  \n © T. Sourada, K. Vendrame, and J. Hajijr.. Licensed under a Creative Commons Attribution 4 .0 International License (CC BY 4 .0) . Attribution: T. Sourada, K. Vendrame, and J. Haji jr.,“Music I Care About: Automated Multimodal Benchmarking of LLM Music Perception Skills on (Almost) Any Music”, preprint, 2026 .  \nand their application to music is an active area of MIR research [4, 5] . Evaluating how well these models handle musical tasks is both timely and consequential: it determines how much of our field’s research agenda can be delegated to general-purpose models, and where purpose-built MIR systems remain indispensable. Yet rigorous evaluation across diverse repertoires, modalities, and research contexts remains an open methodological challenge.  \nExisting benchmarks for evaluating MLLMs on music suffer from three major limitations. First, large static benchmarks are impractical: they are resource-intensive to evaluate; accuracy values age quickly as new models are released; results may not transfer beyond the styles and genres included; benchmark data may leak into future training corpora, invalidating comparisons; human annotation of ground truth is extremely costly while LLM-assisted annotation raises quality concerns; no single benchmark can cover music’s diversity, and new models arrive faster than evaluation can keep pace; and copyrighted material creates legal constraints on what data can be included.  \nSecond, some benchmarks claiming to evaluate music understanding (e.g., by asking questions about audio) do not require actual perception of musical conten","cbCaij77fu8GS6IH","https://ap.wps.com/l/cbCaij77fu8GS6IH","pdf",1548206,1,9,"English","en",105,"# Abstract\n# Introduction\n## Digital Modalities of Music\n## Limitations of Existing Benchmarks\n## MusICA-MetaBench Overview\n## MusICA-MetaBench Design Principles","[{\"question\":\"Why are piece-aligned multimodal inputs important for cross-modal evaluation?\",\"answer\":\"When the same piece is represented consistently across audio, sheet music images, and symbolic files, identical questions can be asked across modalities with the same ground-truth answers. This enables direct cross-modal comparison on the same underlying musical content.\"}]",1784192229,23,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":78,"head_meta":80,"extra_data":82,"updated_unix":27},"music-i-care-about-automated-multimodal-benchmarking-of-llm-music-perception-skills","",{"@graph":35,"@context":77},[36,53,68],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/music-i-care-about-automated-multimodal-benchmarking-of-llm-music-perception-skills/84047/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-17","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71],{"name":72,"@type":73,"acceptedAnswer":74},"Why are piece-aligned multimodal inputs important for cross-modal evaluation?","Question",{"text":75,"@type":76},"When the same piece is represented consistently across audio, sheet music images, and symbolic files, identical questions can be asked across modalities with the same ground-truth answers. This enables direct cross-modal comparison on the same underlying musical content.","Answer","https://schema.org",{"og:url":51,"og:type":79,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":81,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":84},[85,89,93,97,102,107,112,115,119,122,126],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":86,"show_sort_weight":87,"slug":88},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":90,"show_sort_weight":91,"slug":92},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":94,"show_sort_weight":95,"slug":96},"Exam",70,"exam",{"id":98,"doc_module":4,"doc_module_name":45,"category_name":99,"show_sort_weight":100,"slug":101},5,"Comic",60,"comic",{"id":103,"doc_module":4,"doc_module_name":45,"category_name":104,"show_sort_weight":105,"slug":106},6,"Technology",50,"technology",{"id":108,"doc_module":4,"doc_module_name":45,"category_name":109,"show_sort_weight":110,"slug":111},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":113,"slug":114},30,"research-report",{"id":21,"doc_module":4,"doc_module_name":45,"category_name":116,"show_sort_weight":117,"slug":118},"Religion & Spirituality",20,"religion-spirituality",{"id":117,"doc_module":4,"doc_module_name":45,"category_name":120,"show_sort_weight":117,"slug":121},"World Cup","world-cup",{"id":123,"doc_module":4,"doc_module_name":45,"category_name":124,"show_sort_weight":123,"slug":125},10,"Lifestyle","lifestyle",{"id":127,"doc_module":4,"doc_module_name":45,"category_name":128,"show_sort_weight":98,"slug":129},19,"General","general"]