[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-84313-en":3,"doc-seo-84313-105":29,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},84313,1374391974585,"Genevieve","https://ap-avatar.wpscdn.com/davatar_276721f389ce27ea32af1340a28f341c",8,"Research & Report","LightCrafter PBR-Conditioned Video Diffusion Refinement for Controllable and Consistent Relighting","Video relighting balances long-form temporal consistency with a physically grounded understanding of light transport that relies on accurate intrinsic scene estimation, including materials, geometry, and illumination. Prior approaches either reconstruct photometric properties via inverse rendering and then relight with PBR/neural rendering, or treat relighting as conditional video-to-video diffusion. Both face noisy reconstructions, weak control, temporal instability, and limited paired training data. LightCrafter uses a hybrid proxy translation pipeline with PBR “baked” illumination targets and diffusion refinement.","arXiv :2607 .080 16v 1 [ cs .CV] 9 Jul 2026  \nLightCrafter: PBR-Conditioned Video Diffusion Refinement for Controllable and Consistent Relighting  \nZixin Guo1 Yehonathan Litman1 Yifeng He2  \nJohn Miller3 Chuhan Chen1 Deva Ramanan1  \n1 Carnegie Mellon University 2University of Toronto 3Bosch Research  \n[http://zixinguo.me/lightcrafter/](http://zixinguo.me/lightcrafter/)  \nAbstract  \nVideo relighting requires balancing long-form temporal consistency with a physically grounded understanding of light transport, which depends on accurate estimation of intrinsic scene properties such as materials, geometry, and illumination.  \nExisting methods follow two paradigms: (1) Given an input video, explicitly reconstruct its photometric properties via inverse rendering, and then relight thereconstructions to a target illumination via forward rendering, either via physicallybased rendering (PBR) or a neural rendering engine. Such methods suffer from noisy reconstructions and struggle to capture hard-to-model illumination effects such as global illumination. (2) Alternatively, frame the task as a generative videoto-video translation task that conditions on relighting targets (specified as target environment map or text) . However, such a framing limits relighting control and temporal stability since generative diffusion models struggle to translate long-form videos. Moreover, such data-driven methods are limited by the availability of training pairs of input videos and their relit targets. We propose LightCrafter, a hybrid pipeline that reformulates video relighting as video translation of a proxy video;  \nrather directly translating the input video to the target, we translate a PBR rendering (of the input video under the target illumination conditions) to the final target. This allows us to “bake\" illumination targets into the PBR-proxy rendering, removing the need to explicitly teach the diffusion model about illumination concepts like environment maps. We find PBR proxy-renderings allow for more intricate lighting control while naturally providing long-form temporal consistency. In fact, we show that PBR renders already outperform some prior art for relighting, but struggle to model intricate effects like global illumination. To capture such effects, we leverage photometric priors implicit in video generation models. Specifically, we post-train CogVideoX on synthetic video pairs and real-world unpaired videos. We outperform prior state-of-the-art on existing real-world relighting benchmarks and also contribute our own synthetic benchmark for further analysis. We will release our dataset, benchmark, metrics, and code.  \n1 Introduction  \nRelighting aims to edit scene appearance while preserving content, geometry, materials, and motion. In video, this requirement is especially demanding: shadows, highlights, and shading must remain temporally coherent and physically consistent with camera motion and dynamic objects. This difficulty stems from the entangled nature of scene appearance, where geometry, material properties, reflectance, visibility, and illumination jointly determine the observed video. A practical video relighting method therefore requires not only photorealism, but also faithful lighting control and stable scene appearance interaction over time.  \nPreprint.  \nInput  \nInverse Rendering (Reconstruction)  \nRelighting via Physically Based Rendering  \nDiffusion Refinement  \nFigure 1: Controllable Video Relighting as a Rendering Refinement Task. Given an input video, LightCrafter utilizes inverse-rendered photometric and geometric scene properties and refines a physically-based rendering (PBR) proxy for video relighting control. The PBR rendering captures most scene-light interaction, while a video diffusion model translates the rendering to a photorealistic relit video with coherent shadows, reflections, materials, and light sources.  \nRelighting via Inverse Rendering. Recent work [15] has shown that scene intrinsic properties, such ","cbCaivzKoZvZOSp0","https://ap.wps.com/l/cbCaivzKoZvZOSp0","pdf",45761403,1,20,"English","en",105,"# Introduction\n## Problem Definition: Controllable Video Relighting\n## Two Existing Paradigms and Their Limitations\n## Proposed Approach: LightCrafter Hybrid Pipeline\n## Inverse Rendering-Based Relighting\n## Video-to-Video Translation via Diffusion","[{\"question\":\"What core challenge does video relighting need to solve?\",\"answer\":\"It must edit scene appearance while preserving content, geometry, materials, and motion, keeping shadows, highlights, and shading temporally coherent and physically consistent with camera motion and dynamic objects.\"},{\"question\":\"Why do inverse-rendering-based relighting methods struggle?\",\"answer\":\"Recovering accurate intrinsic properties from monocular video is ill-posed and ambiguous, leading to degenerate solutions and errors that directly propagate into the relit results.\"},{\"question\":\"How does LightCrafter improve controllable and consistent relighting?\",\"answer\":\"It reformulates relighting as translation of a proxy video by converting PBR rendering under the target illumination into a video that a diffusion model refines, baking illumination targets into the proxy to improve lighting control and long-form temporal consistency.\"}]",1784194749,50,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":27},"lightcrafter-pbr-conditioned-video-diffusion-refinement-for-controllable-and-consistent-relighting","",{"@graph":35,"@context":85},[36,53,68],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/lightcrafter-pbr-conditioned-video-diffusion-refinement-for-controllable-and-consistent-relighting/84313/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-17","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What core challenge does video relighting need to solve?","Question",{"text":75,"@type":76},"It must edit scene appearance while preserving content, geometry, materials, and motion, keeping shadows, highlights, and shading temporally coherent and physically consistent with camera motion and dynamic objects.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"Why do inverse-rendering-based relighting methods struggle?",{"text":80,"@type":76},"Recovering accurate intrinsic properties from monocular video is ill-posed and ambiguous, leading to degenerate solutions and errors that directly propagate into the relit results.",{"name":82,"@type":73,"acceptedAnswer":83},"How does LightCrafter improve controllable and consistent relighting?",{"text":84,"@type":76},"It reformulates relighting as translation of a proxy video by converting PBR rendering under the target illumination into a video that a diffusion model refines, baking illumination targets into the proxy to improve lighting control and long-form temporal consistency.","https://schema.org",{"og:url":51,"og:type":87,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":89,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,114,119,122,126,129,133],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":45,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":45,"category_name":112,"show_sort_weight":28,"slug":113},6,"Technology","technology",{"id":115,"doc_module":4,"doc_module_name":45,"category_name":116,"show_sort_weight":117,"slug":118},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":120,"slug":121},30,"research-report",{"id":123,"doc_module":4,"doc_module_name":45,"category_name":124,"show_sort_weight":21,"slug":125},9,"Religion & Spirituality","religion-spirituality",{"id":21,"doc_module":4,"doc_module_name":45,"category_name":127,"show_sort_weight":21,"slug":128},"World Cup","world-cup",{"id":130,"doc_module":4,"doc_module_name":45,"category_name":131,"show_sort_weight":130,"slug":132},10,"Lifestyle","lifestyle",{"id":134,"doc_module":4,"doc_module_name":45,"category_name":135,"show_sort_weight":106,"slug":136},19,"General","general"]