Image Restoration Using Joint Statistical Modeling in Space-Transform Domain

Jian Zhang1, Debin Zhao1, Ruiqin Xiong2, Siwei Ma2, Wen Gao2

1School of Computer Science and Technology, Harbin Institute of Technology
2National Engineering Laboratory for Video Technology, Peking University

Abstract—This paper presents a novel strategy for high-fidelity image restoration by characterizing both local smoothness and nonlocal self-similarity of natural images in a unified statistical manner. The main contributions are three-folds. First, from the perspective of image statistics, a joint statistical modeling (JSM) in an adaptive hybrid space-transform domain is established, which offers a powerful mechanism of combining local smoothness and nonlocal self-similarity simultaneously to ensure a more reliable and robust estimation. Second, a new form of minimization functional for solving image inverse problem is formulated using JSM under regularization-based framework. Finally, in order to make JSM tractable and robust, a new Split-Bregman based algorithm is developed to efficiently solve the above severely underdetermined inverse problem associated with theoretical proof of convergence. Extensive experiments on image inpainting, image deblurring and mixed Gaussian plus salt-and-pepper noise removal applications verify the effectiveness of the proposed algorithm.

Paper:

Image Restoration Using Joint Statistical Modeling in Space-Transform Domain
J. Zhang, D. Zhao, R. Xiong, S. Ma, W. Gao
IEEE Transactions on Circuits System and Video Technology 2014. (To appear)
[PDF] [Matlab Code]

Instructions: click on the thumbnail image to see the corresponding results from various algorithms.


Experimental Results:

1. Image Restoration from Partial Random Samples
2. Mixed Gaussian plus Salt-and-Pepper Impulse Noise Removal
3. Image Deblurring
4. Text Removal

1. Image Restoration from Partial Random Samples

In the case of Ratio=20%

Input

Ground Truth

 

Barbara in the case of Ratio=20%

Barbara20

SALSA

SKR

FoE

MCA

BPFA

Proposed


Input

Ground Truth

 

Foreman in the case of Ratio=20%

Foreman20

SALSA

SKR

FoE

MCA

BPFA

Proposed


Input

Ground Truth

 

House in the case of Ratio=20%

House20

SALSA

SKR

FoE

MCA

BPFA

Proposed

 

In the case of Ratio=30%

Input

Ground Truth

 

Barbara in the case of Ratio=30%

Barbara30

SALSA

SKR

FoE

MCA

BPFA

Proposed


Input

Ground Truth

 

Foreman in the case of Ratio=30%

Foreman30

SALSA

SKR

FoE

MCA

BPFA

Proposed



2. Mixed Gaussian plus Salt-and-Pepper Impulse Noise Removal

Input

Ground Truth

 

Barbara in the case of r=50% and σ=10

BarbaraMixed50

 

TV

IFASDA

Proposed


Input

Ground Truth

 

Lena in the case of r=50% and σ=10

LenaMixed50

 

TV

IFASDA

Proposed



3. Image Deblurring

Input

Ground Truth

Butterfly (9x9 Uniform Blur)

ButterflyDeblur

SALSA

SADCT

BM3D

Proposed


Input

Ground Truth

Leaves (Gaussian Blur [25 25] 1.6)

LeavesDeblur

SALSA

SADCT

BM3D

Proposed

4. Text Removal

Input

Ground Truth

Barbara

BarbaraText

SKR

BPFA

FoE

Proposed


Input

Ground Truth

Parthenon

ParthenonText

SKR

BPFA

FoE

Proposed



References

[1] SALSA: M. Afonso, J. Bioucas-Dias and M. Figueiredo, “Fast image recovery using variable splitting and constrained optimization,” IEEE Trans. on Image Process., vol. 19, no. 9, pp. 2345–2356, Sep. 2010.
[2] SKR:
H. Takeda, S. Farsiu, and P. Milanfar, “Kernel regression for image processing and reconstruction,” IEEE Trans. on Image Process., vol. 16, no. 2, pp. 349–366, Feb. 2007.
[3] FoE:
S. Roth and M. J. Black, “Fields of experts,” International Journal of Computer Vision, vol. 82, no. 2, pp. 205–229, 2009.
[4] MCA:
M. Elad, J. L. Starck, P. Querre, and D.L. Donoho, “Simultaneous cartoon and texture image inpainting using morphological component analysis (MCA),” Applied and Computational Harmonic Analysis, vol. 19, pp. 340–358, 2005.
[5] BPFA: M. Zhou, H. Chen, J. Paisley, L. Ren, L. Li, Z. Xing, D. Dunson, G. Sapiro and L. Carin, “Nonparametric Bayesian dictionary learning for analysis of noisy and incomplete images,” IEEE Trans. Image Processing, vol. 21, no. 1, pp. 130–144, Jan. 2012.
[6] TV:
Y. Huang, M. Ng and Y. Wen, “Fast image restoration methods for impulse and Gaussian noise removal,” IEEE Signal Process. Letters, pp. 457–460, 2009.
[7] IFASDA:
Y.-R. Li, L. Shen, D.-Q. Dai, and B.W. Suter, “Framelet algorithms for de-blurring images corrupted by impulse plus Gaussian noise,” IEEE Trans. Image Processing, vol. 20, no. 7, pp. 1822–1837, Jul. 2011.
[8] SADCT:
A. Foi, V. Katkovnik, and K. Egiazarian, “Pointwise shape-adaptive DCT for high-quality denoising and deblocking of grayscale and color images,” IEEE Trans. Image Process., vol. 16, no. 5, pp. 1395–1411, May 2007.

TOP