Uncertainty Learning in Kernel Estimation for Multi-Stage

Blind Image Super-Resolution

Zhenxuan Fang¹ Weisheng Dong¹*    Xin Li² Jinjian Wu¹ Leida Li¹ Guangming Shi¹

¹School of Artificial Intelligence, Xidian University      ² West Virginia University, Morgantown WV, USA

Figure 1. Overview of the proposed KULNet for blind SR. The architectures of (a) the uncertain kernel estimation network,

(b) the layer A, which contains a convolution layer with the estimated kernel and a downsampling layer, (c) the multi-stage SR network.

Abstract

Conventional wisdom in blind super-resolution (SR) first estimates the unknown degradation from the low-resolution image and then exploits the degradation information for image reconstruction. Such sequential approaches suffer from two fundamental weaknesses - i.e., the lack of robustness (the performance drops when the estimated degradation is inaccurate) and the lack of transparency (network architectures are heuristic without incorporating domain knowledge). To address these issues, we propose a joint Maximum a Posteriori (MAP) approach for estimating the unknown kernel and high-resolution image simultaneously. Our method first introduces uncertainty learning in the latent space when estimating the blur kernel, aiming at improving the robustness to the estimation error. Then we propose a novel SR network by unfolding the joint MAP estimator with a learned Laplacian Scale Mixture (LSM) prior and the estimated kernel. We have also developed a novel approach of estimating both the scale prior coefficient and the local means of the LSM model through a deep convolutional neural network (DCNN). All parameters of the MAP estimation algorithm and the DCNN parameters are jointly optimized through end-to-end training. Extensive experiments on both synthetic and real-world images show that our method achieves state-of-the-art performance for the task of blind image SR.

Paper

ECCV 2022 Supplementary Material

Citation

Zhenxuan Fang, Weisheng Dong, Xin Li, Jinjian Wu, Leida Li, and Guangming Shi. Uncertainty learning in kernel estimation for multi-stage blind image super-resolution. In
European Conference on Computer Vision, pages 144–161. Springer, 2022.

Bibtex

@inproceedings{fang2022uncertainty,

   title={Uncertainty Learning in Kernel Estimation for Multi-stage Blind Image Super-Resolution},

   author={Fang, Zhenxuan and Dong, Weisheng and Li, Xin and Wu, Jinjian and Li, Leida and Shi, Guangming},

   booktitle={European Conference on Computer Vision},

   pages={144--161},

   year={2022},

   organization={Springer}

}

Download

Code

Results

Quantitative Results:

Table 1. Quantitative comparison of the SOTA blind SR methods and the proposed method on various datasets and noise levels.

Visualization Results:

Figure 2. Visual comparison to other methods. The blur kernels are illustrated on the

top left. Noise levels are set to 0 and 10 for scale factor ×2 and ×4, respectively.

Real Image Result:

Figure 3. Visualization results of different methods on real-world images upscaled by ×4.

Contact

Zhenxuan Fang, Email: zxfang@stu.xidian.edu.cn

Weisheng Dong, Email: wsdong@mail.xidian.edu.cn

Xin Li, Email: xin.li@mail.wvu.edu

Leida Li, Email: ldli@xidian.edu.cn

Jinjian Wu, Email: jinjian.wu@mail.xidian.edu.cn

Guangming Shi, Email: gmshi@xidian.edu.cn