[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-seo-142409-105":3,"detail-sidebar-cat-0-en-105":81,"doc-detail-142409-en":131},{"code":4,"msg":5,"data":6},0,"ok",{"site_id":7,"language":8,"slug":9,"title":10,"keywords":11,"description":12,"schema_data":13,"social_meta":74,"head_meta":76,"extra_data":78,"updated_unix":80},105,"en","relightablehands-efficient-neural-relighting-of-articulated-hand-models-cvpr-2023-paper","RelightableHands - Efficient Neural Relighting of Articulated Hand Models - CVPR 2023 paper","","This CVPR paper introduces an efficient neural relighting method for personalized, animatable hand models, enabling high-fidelity rendering in real time under novel illumination. A teacher-student framework is adopted: a teacher learns appearance from light-stage images under single point light inputs, then generates pseudo ground truth for training. The student predicts real-time appearance under natural lighting by conditioning on physics-inspired, geometry-aligned features including visibility, diffuse shading, and specular reflections, improving generalization to unseen poses and illuminations. Experiments show improved performance over baselines and photorealistic relighting of two interacting hands.",{"@graph":14,"@context":73},[15,34,56],{"@type":16,"itemListElement":17},"BreadcrumbList",[18,23,27,31],{"item":19,"name":20,"@type":21,"position":22},"https://docshare.wps.com","Home","ListItem",1,{"item":24,"name":25,"@type":21,"position":26},"https://docshare.wps.com/document/","Document",2,{"item":28,"name":29,"@type":21,"position":30},"https://docshare.wps.com/document/research-report/","Research & Report",3,{"item":32,"name":10,"@type":21,"position":33},"https://docshare.wps.com/document/relightablehands-efficient-neural-relighting-of-articulated-hand-models-cvpr-2023-paper/142409/",4,{"url":32,"name":10,"@type":35,"image":36,"author":41,"headline":10,"publisher":44,"fileFormat":47,"inLanguage":8,"description":12,"dateModified":48,"datePublished":49,"encodingFormat":47,"isAccessibleForFree":50,"interactionStatistic":51},"DigitalDocument",{"url":37,"@type":38,"width":39,"height":40},"https://docshare.wps.com/thumbnails/relightablehands-efficient-neural-relighting-of-articulated-hand-models-cvpr-2023-paper/142409.png","ImageObject",300,407,{"name":42,"@type":43},"Nguyễn Văn Học","Person",{"url":19,"name":45,"@type":46},"DocShare","Organization","application/pdf","2026-10-27","2026-08-25",true,{"@type":52,"interactionType":53,"userInteractionCount":55},"InteractionCounter",{"@type":54},"ViewAction",13,{"@type":57,"mainEntity":58},"FAQPage",[59,65,69],{"name":60,"@type":61,"acceptedAnswer":62},"What problem does RelightableHands address?","Question",{"text":63,"@type":64},"It enables photorealistic, personalized rendering of animatable hand models in real time under novel illumination while supporting two-hand interactions.","Answer",{"name":66,"@type":61,"acceptedAnswer":67},"How does the teacher-student framework work?",{"text":68,"@type":64},"The teacher learns radiance from light-stage images given point-light position, viewing direction, and light visibility, then renders pseudo ground truth to train an efficient student that predicts appearance under natural illuminations.",{"name":70,"@type":61,"acceptedAnswer":71},"Why use physics-inspired illumination features in the student model?",{"text":72,"@type":64},"Features such as visibility, diffuse shading, and specular reflections are computed from coarse proxy geometry and are spatially aligned, providing conditioning that correlates with global light transport and generalizes better than bottleneck illumination conditioning.","https://schema.org",{"og:url":32,"og:type":75,"og:title":10,"og:site_name":45,"og:description":12},"article",{"robots":77,"canonical":32},"index,follow",{"doc_id":79,"site_id":7},142409,1787680770,{"code":4,"msg":82,"data":83},"success",[84,88,92,96,101,106,111,115,120,123,127],{"id":22,"doc_module":4,"doc_module_name":25,"category_name":85,"show_sort_weight":86,"slug":87},"Story & Novel",90,"story-novel",{"id":26,"doc_module":4,"doc_module_name":25,"category_name":89,"show_sort_weight":90,"slug":91},"Literature",80,"literature",{"id":33,"doc_module":4,"doc_module_name":25,"category_name":93,"show_sort_weight":94,"slug":95},"Exam",70,"exam",{"id":97,"doc_module":4,"doc_module_name":25,"category_name":98,"show_sort_weight":99,"slug":100},5,"Comic",60,"comic",{"id":102,"doc_module":4,"doc_module_name":25,"category_name":103,"show_sort_weight":104,"slug":105},6,"Technology",50,"technology",{"id":107,"doc_module":4,"doc_module_name":25,"category_name":108,"show_sort_weight":109,"slug":110},7,"Healthcare",40,"healthcare",{"id":112,"doc_module":4,"doc_module_name":25,"category_name":29,"show_sort_weight":113,"slug":114},8,30,"research-report",{"id":116,"doc_module":4,"doc_module_name":25,"category_name":117,"show_sort_weight":118,"slug":119},9,"Religion & Spirituality",20,"religion-spirituality",{"id":118,"doc_module":4,"doc_module_name":25,"category_name":121,"show_sort_weight":118,"slug":122},"World Cup","world-cup",{"id":124,"doc_module":4,"doc_module_name":25,"category_name":125,"show_sort_weight":124,"slug":126},10,"Lifestyle","lifestyle",{"id":128,"doc_module":4,"doc_module_name":25,"category_name":129,"show_sort_weight":97,"slug":130},19,"General","general",{"code":4,"msg":82,"data":132},{"doc_id":79,"user_id":133,"nickname":42,"user_avatar":134,"doc_module":4,"category_id":112,"category_name":29,"doc_title":10,"doc_description":12,"doc_content":135,"file_id":136,"file_url":137,"file_type":138,"file_size":139,"view_count":55,"is_deleted":4,"is_public":22,"is_downloadable":22,"audit_status":22,"page_count":140,"language":141,"language_code":8,"site_id":7,"html_lang":8,"table_of_contents":142,"faqs":143,"seo_title":144,"seo_description":12,"update_tm":80,"read_time":145,"preview_page_count":4},1374402739827,"https://ap-avatar.wpscdn.com/avatar/14000c97e7351f1a627?x-image-process=image/resize,m_fixed,w_180,h_180&k=1787885694763230660","This CVPR paper is the Open Access version, provided by the Computer Vision Foundation.  \nExcept for this watermark, it is identical to the accepted version; the final published version of the proceedings is available on IEEE Xplore.  \nRelightableHands: Efficient Neural Relighting of Articulated Hand Models  \nShun Iwase 1 ,2⇤ Shunsuke Saito2 Tomas Simon2  \nStephen Lombardi2 Timur Bagautdinov2 Rohan Joshi2 Fabian Prada2 Takaaki Shiratori2 Yaser Sheikh2 Jason Saragih2  \n1 Carnegie Mellon University 2Reality Labs Research  \nEnvironment Map  \nStudent Model Rendering (21ms)  \nTwo Hand Rendering (32ms)  \nFigure 1 . Neural Relighting of Animatable Hands. Our model-based neural rendering approach enables high-fidelity rendering of hands with various poses, views, and illuminations. Our student model is highly efficient enough to render in real-time.  \nAbstract  \nWe present the first neural relighting approach for rendering high-fidelity personalized hands that can be animated in real-time under novel illumination. Our approach adopts a teacher-student framework, where the teacher learns appearance under a single point light from images captured in a light-stage, allowing us to synthesize hands in arbitrary illuminations but with heavy compute. Using images rendered by the teacher model as training data, an efficient student model directly predicts appearance under natural illuminations in real-time. To achieve generalization, we condition the student model with physics-inspired illumination features such as visibility, diffuse shading, and specular reflections computed on a coarse proxy geometry, maintaining a small computational overhead. Our key insight is that these features have strong correlation with subsequent global light transport effects, which proves suffi cient as conditioning data for the neural relighting network. Moreover, in contrast to bottleneck illumination conditioning, these features are spatially aligned based on underlying geometry, leading to better generalization to unseen illuminations and poses. In our experiments, we demonstrate the efficacy of our illumination feature representations, outperforming baseline approaches. We also show that our approach can photorealistically relight two interacting hands  \n⇤This work was done during an internship at Meta  \nat real-time speeds. [https://sh8.io/\\#/relightable](https://sh8.io/#/relightable) hands  \n1. Introduction  \nNeural rendering approaches have significantly advanced photorealistic face rendering [42, 55, 66] in recent years. These methods use deep neural networks to model the light transport on human skin [11, 14, 31, 63], directly reproducing physical effects such as subsurface scattering by reconstructing real images. However, despite the success of neural relighting, extending this approach to animatable hand models poses a unique challenge: generalization across articulations.  \nUnlike faces, hands have many joints, and the state of a single joint affects all child joints. This leads to extremely diverse shape variations even within a single subject. Changes in pose drastically affect the appearance of hands, creating wrinkles, casting shadows, and interreflecting across topologically distant regions. Rendering these effects is challenging because sufficiently accurate geometry and material properties required for photorealism are difficult to obtain, and even then, path tracing to sufficient accuracy is computationally expensive. The use ofsimplified geometric and appearance models (such as linear blend skinning and reduced material models) allow faster computation but come at a noticeable degradation in rendering fidelity. So far, photorealistic rendering of animatable  \nhands with global illumination effects in real-time remains an open problem.  \nIn this work, we aim to enable photorealistic rendering of a personalized hand model that can be animated with novel poses, in novel lighting environments, and supports rendering two-hand interactions. To this end, we pres","cbCainEuvwHcMxXn","https://ap.wps.com/l/cbCainEuvwHcMxXn","pdf",6108975,11,"English","# Abstract\n# Introduction\n## Challenge of animatable hand relighting\n## Proposed teacher-student framework\n## Physics-inspired illumination conditioning","[{\"question\":\"What problem does RelightableHands address?\",\"answer\":\"It enables photorealistic, personalized rendering of animatable hand models in real time under novel illumination while supporting two-hand interactions.\"},{\"question\":\"How does the teacher-student framework work?\",\"answer\":\"The teacher learns radiance from light-stage images given point-light position, viewing direction, and light visibility, then renders pseudo ground truth to train an efficient student that predicts appearance under natural illuminations.\"},{\"question\":\"Why use physics-inspired illumination features in the student model?\",\"answer\":\"Features such as visibility, diffuse shading, and specular reflections are computed from coarse proxy geometry and are spatially aligned, providing conditioning that correlates with global light transport and generalizes better than bottleneck illumination conditioning.\"}]","RelightableHands - Efficient Neural Relighting of Articulated Hand Models - CVPR 2023 paper | PDF",28]