[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-122280-en":3,"doc-seo-122280-105":29,"detail-sidebar-cat-0-en-105":90},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":11,"language":21,"language_code":22,"site_id":23,"html_lang":22,"table_of_contents":24,"faqs":25,"seo_title":26,"seo_description":14,"update_tm":27,"read_time":28},122280,1099514067438,"River Wang","https://ap-avatar.wpscdn.com/avatar/100002539ee87300030?x-image-process=image/resize,m_fixed,w_180,h_180&k=1780474512215547542",6,"Technology","Audio-Video Synchronization Using Machine Learning - Technical Disclosure Commons","Audio and video tracks in a video can become misaligned due to separate recording, encoding or packaging errors, or failures during re-encoding. Out-of-sync playback reduces viewing quality and manual synchronization is costly, especially with multiple audio/video streams. This disclosure presents a machine learning approach that takes paired audio and video inputs into a multimodal model to infer synchronization timestamps. By matching audio/video landmark events, the generated timing data can be used by editing or playback tools to trim, offset, and restore synchronization efficiently.","Technical Disclosure Commons  \nDefensive Publications Series  \n08 May 2025  \nAudio-Video Synchronization Using Machine Learning Avichal Rakesh  \nMozart Alexander Louis  \nFollow this and additional works at: [https://www.tdcommons.org/dpubs_series](https://www.tdcommons.org/dpubs_series)  \nRecommended Citation  \nRakesh, Avichal and Louis, Mozart Alexander, \"Audio-Video Synchronization Using Machine Learning\", Technical Disclosure Commons,(May 08, 2025)  \n[https://www.tdcommons.org/dpubs_series/8100](https://www.tdcommons.org/dpubs_series/8100)  \nThis work is licensed under a Creative Commons Attribution 4.0 License.  \nThis Article is brought to you for free and open access by Technical Disclosure Commons. It has been accepted for inclusion in Defensive Publications Series by an authorized administrator of Technical Disclosure Commons.  \nAudio-Video Synchronization Using Machine Learning  \nABSTRACT  \nAudio and video components ofa video can fall out of alignment for a variety of reasons, e.g., since they are recorded separately, due to encoding errors, etc. The viewing  \nexperience is degraded when audio and video are out of sync. Manual synchronization is  \nexpensive. This disclosure describes the use of a machine learning model to determine  \nsynchronization information for audio and video streams that are to be synchronized. The audio  \nand video streams are provided as input to a multimodal model, e.g., a large language model  \nthat can handle both audio and video. The model is tasked with outputting the synchronization information, e.g., a set of timestamps corresponding to the different streams that are to be lined  \nup for synchronization. The multimodal model performs this task via audio and video  \nunderstanding by picking out specific landmark points in both the audio and video streams that  \ncorrespond to each other. Synchronization information from the multimodal model can be  \nprovided to video editing tools or playback software to perform the necessary action, e.g., trim  \nand offset the streams, to provide a synchronized video.  \nKEYWORDS  \n● Audio-video synchronization  \n● Audio delay  \n● Misaligned audio  \n● Video understanding  \n● Audio understanding  \n● Synchronization timestamp  \n● Video encoding  \nPublished by Technical Disclosure Commons, 2025 2  \nBACKGROUND  \nFor satisfactory video playback, it is essential that the audio and video are synchronized, such that the audio at any instance matches the visuals. Since audio and video components in video content are typically recorded as separate streams, the streams must be synchronized before being encoded together. This process is often performed manually. Complexity of this process increases with multiple audio and video streams, increasing the possibility of  \ndesynchronization.  \nEven when the audio and video streams are synchronized, it is possible for the encoding (packaging the audio and video) to fail partially and break the synchronization. To correct such errors, video and audio streams need to be separated, resynchronized, and re-encoded. This process is time consuming, cannot be performed in real time, and often requires re-encoding the entire video, even if only a small portion is out of sync. Some media players provide features to add delay to audio or video streams at playback time which can alleviate some synchronization issues. However, this process is manual and may not support multi-stream synchronization. Audio/video editing software provides automated tools to bring different audio streams in sync with each other, e.g., using waveform analysis, but there are no such tools for bringing a set of video streams in sync with a set of audio streams. Generative machine learning techniques can generate audio for an existing video stream, but these techniques do not synchronize existing audio and video streams.  \nDESCRIPTION  \nThis disclosure describes the use of a machine learning model to determine synchronization information for audio and video st","cbCaitauLQ3UumAb","https://ap.wps.com/l/cbCaitauLQ3UumAb","pdf",329252,1,"English","en",105,"# Abstract\n# Background\n## Synchronization challenges\n## Limitations of existing tools\n# Description\n## Multimodal synchronization approach\n## Output as timestamps for editing","[{\"question\":\"Why do audio and video streams become out of sync?\",\"answer\":\"They can be misaligned when components are recorded separately or due to encoding/packaging errors that break synchronization.\"},{\"question\":\"What problem does manual synchronization create?\",\"answer\":\"Manual synchronization is expensive and becomes more complex as the number of audio and video streams increases, raising the risk of desynchronization.\"},{\"question\":\"How does the proposed method determine synchronization?\",\"answer\":\"A multimodal machine learning model processes the audio and video streams and outputs synchronization information such as timestamps by matching corresponding landmark points across both streams.\"}]","Audio-Video Synchronization Using Machine Learning - Technical Disclosure Commons | PDF",1785809805,15,{"code":4,"msg":30,"data":31},"ok",{"site_id":23,"language":22,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":85,"head_meta":87,"extra_data":89,"updated_unix":27},"audio-video-synchronization-using-machine-learning-technical-disclosure-commons","",{"@graph":35,"@context":84},[36,53,67],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/technology/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/audio-video-synchronization-using-machine-learning-technical-disclosure-commons/122280/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":22,"description":14,"dateModified":61,"datePublished":61,"encodingFormat":60,"isAccessibleForFree":62,"interactionStatistic":63},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-08-04",true,{"@type":64,"interactionType":65,"userInteractionCount":4},"InteractionCounter",{"@type":66},"ViewAction",{"@type":68,"mainEntity":69},"FAQPage",[70,76,80],{"name":71,"@type":72,"acceptedAnswer":73},"Why do audio and video streams become out of sync?","Question",{"text":74,"@type":75},"They can be misaligned when components are recorded separately or due to encoding/packaging errors that break synchronization.","Answer",{"name":77,"@type":72,"acceptedAnswer":78},"What problem does manual synchronization create?",{"text":79,"@type":75},"Manual synchronization is expensive and becomes more complex as the number of audio and video streams increases, raising the risk of desynchronization.",{"name":81,"@type":72,"acceptedAnswer":82},"How does the proposed method determine synchronization?",{"text":83,"@type":75},"A multimodal machine learning model processes the audio and video streams and outputs synchronization information such as timestamps by matching corresponding landmark points across both streams.","https://schema.org",{"og:url":51,"og:type":86,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":88,"canonical":51},"index,follow",{"doc_id":7,"site_id":23},{"code":4,"msg":5,"data":91},[92,96,100,104,109,112,117,122,127,130,134],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":93,"show_sort_weight":94,"slug":95},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":97,"show_sort_weight":98,"slug":99},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":101,"show_sort_weight":102,"slug":103},"Exam",70,"exam",{"id":105,"doc_module":4,"doc_module_name":45,"category_name":106,"show_sort_weight":107,"slug":108},5,"Comic",60,"comic",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":110,"slug":111},50,"technology",{"id":113,"doc_module":4,"doc_module_name":45,"category_name":114,"show_sort_weight":115,"slug":116},7,"Healthcare",40,"healthcare",{"id":118,"doc_module":4,"doc_module_name":45,"category_name":119,"show_sort_weight":120,"slug":121},8,"Research & Report",30,"research-report",{"id":123,"doc_module":4,"doc_module_name":45,"category_name":124,"show_sort_weight":125,"slug":126},9,"Religion & Spirituality",20,"religion-spirituality",{"id":125,"doc_module":4,"doc_module_name":45,"category_name":128,"show_sort_weight":125,"slug":129},"World Cup","world-cup",{"id":131,"doc_module":4,"doc_module_name":45,"category_name":132,"show_sort_weight":131,"slug":133},10,"Lifestyle","lifestyle",{"id":135,"doc_module":4,"doc_module_name":45,"category_name":136,"show_sort_weight":105,"slug":137},19,"General","general"]