Pooling-Based Context Modeling for Convolution-Free Deep Image Prior - arXiv 2607.02952v1 - research paper style summary

Sep 21, 2026

Uploaded by Aria

Convolutional Neural Networks restore images effectively by exploiting spatial context from neighboring pixels, and Deep Image Prior (DIP) leverages this idea from a single noisy input without large datasets. Yet DIP’s over-parameterization often causes noise fitting during optimization. This paper introduces Pool-DIP, a convolution-free architecture using pooling-based contrast modeling to capture context efficiently. It reduces parameters and computation, improves denoising across datasets, stabilizes high-frequency evolution, suppresses erroneous signals, and generalizes to super-resolution and inpainting.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages16

Reading Time40 min

AccessFree to download

Introduction

Background: CNN context and image denoising

Deep Image Prior and the noise-fitting issue

Related non-convolutional DIP variants

Proposed Pool-DIP and Multi-Scale Local Contrast Pooling

arXiv :2607 .02952v 1 [ cs .CV] 3 Jul 2026 Pooling-Based Context Modeling for Convolution-Free Deep Image Prior Gihyun Kim and Jong-Seok Lee Yonsei University Republic of Korea [kkh9314@yonsei.ac.kr](kkh9314@yonsei.ac.kr) , [jong-seok.lee@yonsei.ac.kr](jong-seok.lee@yonsei.ac.kr) Abstract. Convolutional Neural Networks (CNNs) achieve strong denoising performance by exploiting spatial context from neighboring pixels. Deep Image Prior (DIP) leverages this property to restore images from a single noisy input without requiring large datasets. However, the over-parameterized architecture of DIP often leads to noise fitting during optimization. In this paper, we propose Pool-DIP, a convolution-free architecture that incorporates pooling-based contrast modeling to capture spatial context efficiently. Pool-DIP improves denoising performance while significantly reducing the number of parameters and computational complexity compared to convolution-based DIP models. Experimental results show that Pool-DIP achieves competitive performance across multiple datasets, including a real-world benchmark. Spectral analysis further reveals that Pool-DIP stabilizes the evolution of high-frequency components during optimization and suppresses erroneous high-frequency signals. The proposed architecture also generalizes well to other image restoration tasks such as super-resolution and inpainting. 1 Introduction Convolution Neural Networks (CNNs) are widely used in computer vision due to their strong performance. A key factor contributing of CNNs is their capability to model context, which captures relationships among neighboring pixels. This characteristic makes CNNs particularly effective for image restoration tasks, especially image denoising [8, 28] . However, as in other areas of computer vision, CNNs typically require large training datasets to achieve satisfactory denoising performance. Deep Image Prior (DIP) [23] reveals that the CNN architecture itself inherently possesses a powerful prior, allowing it to achieve competitive denoising results using only a single noisy image, without the need for extensive datasets. In the absence of large training datasets, the architectural design of DIP becomes particularly important [2, 5 , 7 , 9 , 14], as it strongly influences the effectiveness of the resulting prior. An early study [23] used convolution architectures that have a large number of parameters in order to ensure flexibility in the achievable representation space. Best Final DIP 32.11 / 0.907 27.21 / 0.646 DeepDecoder 29.80 / 0.831 28.46 / 0.694 ConvDecoder 32.49 / 0.914 24.87 / 0.662 Pool-DIP (Ours) 32.61 / 0.916 30.85 / 0.812 Best Final 31.66 / 0.880 26.75 / 0.623 29.90 / 0.815 28.30 / 0.685 31.35 / 0.862 23.60 / 0.593 32.32 / 0.898 31.96 / 0.886 Fig. 1: Denoising results of DIP, DeepDecoder, ConvDecoder, and our Pool-DIP. The best achievable results during training and the results at the end of training are shown. PSNR (dB) and SSIM are shown below each image. The images are kodim03 and kodim12 of the Set9 dataset [4] . However, such over-parameterized architectures may lead to fitting noise during optimization [9, 10] . DeepDecoder [9] demonstrates that non-convolutional networks can also serve as DIP architectures when equipped with upsampling layers that enable interactions among neighboring pixels. While this approach reduces noise-fitting by limiting the number of parameters, it also tends to degrade denoising performance since it relies solely on fixed upsampling operations to capture context information, which constrains its expressivity. To address this limitation, ConvDecoder [5] introduces convolution layers into the DeepDecoder architecture. Although this modification improves performance compared to DeepDecoder, it still does not fully match the denoising performance of convolution-based DIP models [23], and the reduced parameter constraint may weaken its

FAQs

What problem does the paper address in Deep Image Prior (DIP)?

DIP’s over-parameterized architecture can fit noise during optimization, degrading restoration quality.

How does Pool-DIP differ from convolution-based DIP models?

Pool-DIP uses a convolution-free design with pooling-based contrast modeling, aiming to capture spatial context with fewer parameters and lower computation.

What do the experiments and spectral analysis show about Pool-DIP?

Pool-DIP achieves competitive denoising across multiple datasets and spectral analysis indicates it stabilizes high-frequency component evolution while suppressing erroneous high-frequency signals.

Sep 21, 2026

Uploaded by Aria