Exploiting Deep Generative Prior for Versatile Image Restoration and Manipulation

Aug 22, 2026

Learning a strong image prior drives progress in image restoration and manipulation. Existing approaches such as deep image prior focus on low-level statistics, leaving a gap for priors that model semantics like color, spatial coherence, textures, and higher-level concepts. This work leverages a generative adversarial network trained on large-scale natural images to exploit a deep generative prior, enabling restoration of missing semantics and flexible manipulation while relaxing strict generator-fixing assumptions in prior GAN inversion methods.

LanguageEnglish

CategoryTechnology

FormatPDF

Pages16

Reading Time40 min

AccessFree to download

Introduction

Image prior for restoration and manipulation

Deep image prior and GAN-based priors

Motivation: general priors from large-scale GANs

Problem setup and challenges

GAN inversion methods and limitations

Exploiting Deep Generative Prior for Versatile Image Restoration and Manipulation Xingang Pan 1 , Xiaohang Zhan 1 , Bo Dai 1 , Dahua Lin 1 , Chen Change Loy2 , and Ping Luo3 1 The Chinese University of Hong Kong fpx117,zx017,bdai,[dhlin](dhling@ie.cuhk.edu.hk)[g](dhling@ie.cuhk.edu.hk)[@ie.cuhk.edu.hk](dhling@ie.cuhk.edu.hk)[ ](dhling@ie.cuhk.edu.hk)2 Nanyang Technological University 3 The University of Hong Kong [ccloy@ntu.edu.sg](ccloy@ntu.edu.sg) pluo@cs.hku.hk Abstract. Learning a good image prior is a long-term goal for image restoration and manipulation. While existing methods like deep image prior (DIP) capture low-level image statistics, there are still gaps toward an image prior that captures rich image semantics including color, spatial coherence, textures, and high-level concepts. This work presents ane􀀋ective way to exploit the image prior captured by a generative adversarial network (GAN) trained on large-scale natural images. As shown in Fig. 1, the deep generative prior (DGP) provides compelling results to restore missing semantics, e.g. , color, patch, resolution, of various degraded images. It also enables diverse image manipulation including random jittering, image morphing, and category transfer. Such highly 􀀍exible e􀀋ects are made possible through relaxing the assumption of existing GAN-inversion methods, which tend to 􀀌x the generator. Notably, we allow the generator to be 􀀌ne-tuned on-the-􀀍y in a progressive manner regularized by feature distance obtained by the discriminator in GAN. We show that these easy-to-implement and practical changes help preserve the reconstruction to remain in the manifold of nature image, and thus lead to more precise and faithful reconstruction for real images. Code is at [https://github.com/XingangPan/deep-generative-prior](https://github.com/XingangPan/deep-generative-prior). 1 Introduction Learning image prior models is important to solve various tasks of image restoration and manipulation, such as image colorization [21, 36], image inpainting [35], super-resolution [12, 22], and adversarial defense [27] . In the past decades, many image priors [25, 40, 13, 16, 26] have been proposed to capture certain statistics of natural images. Despite their successes, these priors often serve a dedicated purpose. For instance, markov random 􀀌eld [25, 40, 13] is often used to model the correlation among neighboring pixels, while dark channel prior [16] and total variation [26] are developed for dehazing and denoising respectively. There is a surge of interest to seek for more general priors that capture richer statistics of images through deep learning models. For instance, the seminal work on deep image prior (DIP) [30] showed that the structure of a randomly initialized Convolutional Neural Network (CNN) implicitly captures texture-level 2 X. Pan et al. (a) Colorization (b) Inpainting (c) Super-resolution (e) Random jittering (f) Category transfer (d) Adversarial defense jigsaw puzzle × oystercatcher √ target reconstruction jittering effects target reconstruction transfer to other categories (g) Image morphing target A reconstruction A interpolation reconstruction B target B Fig. 1. These image restoration(a)(b)(c)(d) and manipulation(e)(f)(g) e􀀋ects are achieved by leveraging the rich generative prior of a GAN. The GAN does not see these images during training image prior, thus can be used for restoration by 􀀌ne-tuning it to reconstruct a corrupted image. SinGAN [28] further shows that a randomly-initialized generative adversarial network (GAN) model is able to capture rich patch statistics after training from a single image. These priors have shown impressive resultson some low-level image restoration and manipulation tasks like super-resolution and harmonizing. In both the representative works, the CNN and GAN are trained from a single image of interest from scratch. In this study, we are interested to go one step further, examining how

FAQs

What problem does this work address in image restoration and manipulation?

It targets the lack of image priors that capture rich semantics such as color, textures, and high-level concepts, beyond low-level statistics used by prior methods.

How does the proposed approach exploit a GAN as an image prior?

It uses a deep generative prior from a GAN trained on large-scale natural images, treating a degraded image as a partial observation and reconstructing missing semantics in the observation space.

What key change is made compared with typical GAN inversion methods?

Instead of keeping the GAN generator fixed, the method allows progressive fine-tuning of the generator on-the-fly, regularized by a feature distance derived from the discriminator.

Aug 22, 2026

Upload by Chumphorn