[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-86111-en":3,"doc-seo-86111-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":21,"is_downloadable":21,"audit_status":21,"page_count":22,"language":23,"language_code":24,"site_id":25,"html_lang":24,"table_of_contents":26,"faqs":27,"seo_title":13,"seo_description":14,"update_tm":28,"read_time":29},86111,1374391974468,"Eden","https://ap-avatar.wpscdn.com/davatar_29158cc5080c5b710cf443261637dec0",8,"Research & Report","Reference-Based Face Super-Resolution Using the Spatial Transformer","Face super-resolution increases the resolution of face images while adding fine-grained visual detail, a common need in computer vision where users often do not realize the enhancement occurs. The problem is ill-posed, so the paper introduces a reference-based approach that leverages higher-resolution images of the same person. A spatial-transformer alignment module is presented for stable correspondence, along with an aggregation function that uses reference quality when available and suppresses it otherwise. Experiments show a smaller model reaching state-of-the-art results on multiple datasets.","arXiv :2607 . 11025v1 [ cs .CV] 13 Jul 2026  \nReference-Based Face Super-Resolution Using the Spatial Transformer  \nVarun Ramesh Jois, Antonella DiLillo, James Storer  \nBrandeis University, Waltham MA 02453, USA  \n{vjois,dilant,[storer}@brandeis.edu](storer}@brandeis.edu)  \nAbstract. Face super-resolution is the task of increasing the resolution of an image containing a face thereby adding finer detail. It is a ubiquitous task in many computer vision applications and quite often the user isn’t even aware that it is being performed. However, doing it with high fidelity is challenging as it is an ill-posed problem. In this paper we present a reference-based solution for face super-resolution that uses higher resolution reference images to aid in the task. We show an alignment module based on the spatial transformer that is considerably more stable than the popular deformable convolutions. We also show an aggregation function that can take good quality information from the reference images when available or suppress the function when such information is unavailable. Finally, we show that our relatively smaller model can achieve state of the art results on multiple datasets. The source code is available at [https://github.com/varun-jois/FSRST](https://github.com/varun-jois/FSRST).  \nKeywords: Reference-Based Super-Resolution · Face Super-Resolution  \n· Image Alignment  \n1 Introduction  \nSuper-resolution is the task of taking an image and increasing its resolution. For instance, if we have a 100×100 pixel image, and we convert it to a 400×400 pixel image, what we have done is perform 4× super-resolution. This has the effect of increasing the finer details in an image leading to a more visually pleasing image. Super-resolution is a fundamental task in low-level computer vision and most image and video applications have some functionality for it. In fact, it is so universal a task that oftentimes, the user isn’t even aware that it is being performed in the background. The super-resolution task is challenging, especially when upsampling by a large factor such as 4× and 8 × . The main issue being that it is an ill-posed problem where a single input could potentially map to different outputs.  \nTo counteract the ill-posedness of super-resolution, many techniques have been suggested to constrain the output of the model. One of these methods is to use one or more high-resolution reference images that are similar in content and texture to the image that is being super-resolved, thereby aiding the task. This is the study known as reference-based super-resolution. Another method to tighten  \nThis is a post-peer-review, pre-copyedit version of an article published in ACCV 2024 . The final authenticated version is  \navailable online at [https://doi.org/10.1007/978-981-96-0911-6_24](https://doi.org/10.1007/978-981-96-0911-6_24) .  \n2 Jois et al.  \nthe definition of the task is to put constraints on the data the model is being shown. This happens naturally when we train for a particular type of data such as medical images or satellite images. When we constrain the data to images of faces, we call this face super-resolution. In this paper we perform face superresolution with the help of high-resolution reference images where the reference images are of the same person. This idea is intriguing for a number of reasons. First, we reduce the difficulty of the problem by constraining the output to textures and shapes found in the reference images and those commonly found in faces. Second, high-resolution images of faces are in abundance whether they be on social media or stored in a user’s device making it a practical approach. Third, this idea can directly be applied to compressing video in video conferencing applications that have seen a surge in usage in recent years [13,19,21] .  \nHowever, using high-resolution reference images introduces many new challenges: How do we find similar shapes and concepts in the reference i. e. the correspondence prob","cbCaioYkS2xYrWnL","https://ap.wps.com/l/cbCaioYkS2xYrWnL","pdf",1800490,4,1,17,"English","en",105,"# Introduction\n## Problem of super-resolution and ill-posedness\n## Reference-based face super-resolution concept\n## Correspondence via alignment\n## Spatial transformer alignment module\n## Reference information aggregation","[{\"question\":\"What is reference-based face super-resolution and why is it needed?\",\"answer\":\"It is a super-resolution method that increases face image resolution using higher-resolution reference images. It addresses the ill-posed nature of super-resolution by constraining the output to shapes and textures consistent with the reference faces.\"},{\"question\":\"How does the paper perform image alignment compared with deformable convolutions?\",\"answer\":\"The approach builds an alignment module based on the spatial transformer. It aims to provide stable alignment results and avoids the instability and training-collapse issues often associated with deformable convolution-based alignment.\"},{\"question\":\"How does the method combine information from the input and the reference images?\",\"answer\":\"It uses a fast aggregation function that weights reference images according to their L2-distance to the input, giving higher weight to references more similar to the low-resolution input. When high-quality reference information is unavailable, the aggregation suppresses its influence.\"}]",1784208592,43,{"code":4,"msg":31,"data":32},"ok",{"site_id":25,"language":24,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"reference-based-face-super-resolution-using-the-spatial-transformer","",{"@graph":36,"@context":85},[37,53,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":21},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":20},"https://docshare.wps.com/document/reference-based-face-super-resolution-using-the-spatial-transformer/86111/",{"url":52,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":24,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":41,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-24","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What is reference-based face super-resolution and why is it needed?","Question",{"text":75,"@type":76},"It is a super-resolution method that increases face image resolution using higher-resolution reference images. It addresses the ill-posed nature of super-resolution by constraining the output to shapes and textures consistent with the reference faces.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"How does the paper perform image alignment compared with deformable convolutions?",{"text":80,"@type":76},"The approach builds an alignment module based on the spatial transformer. It aims to provide stable alignment results and avoids the instability and training-collapse issues often associated with deformable convolution-based alignment.",{"name":82,"@type":73,"acceptedAnswer":83},"How does the method combine information from the input and the reference images?",{"text":84,"@type":76},"It uses a fast aggregation function that weights reference images according to their L2-distance to the input, giving higher weight to references more similar to the low-resolution input. When high-quality reference information is unavailable, the aggregation suppresses its influence.","https://schema.org",{"og:url":52,"og:type":87,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":89,"canonical":52},"index,follow",{"doc_id":7,"site_id":25},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":21,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":20,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":46,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":46,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":46,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":46,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]